Message ID | 20221102195957.82871-1-stuart.w.hayes@gmail.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp114847wru; Wed, 2 Nov 2022 13:05:59 -0700 (PDT) X-Google-Smtp-Source: AMsMyM564UHrMadpXT1moSUaO6SENc16n2IXT49P3Djoy7e4JdH4sXsWsMEYuZjQKv2ULFKWp+EQ X-Received: by 2002:a05:6a00:234d:b0:561:f0c3:cde1 with SMTP id j13-20020a056a00234d00b00561f0c3cde1mr26959458pfj.34.1667419558922; Wed, 02 Nov 2022 13:05:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667419558; cv=none; d=google.com; s=arc-20160816; b=xDvNkM7wNgcV/0eurOuzy4QoCQpWhemS8zXHcL4kVFQ/RZnKEvfZoljrSw9CSQ5+Y4 a/opGfmSeVppVntBI2vF7wRqY07KkeS3fdudb4es/HPs+lHCyqvoDZsjEPMjQ0RO+H8h 7Gq7j/Y/9/eQ3UIU0ETHfzlwqVoMuvhrDsqoPtIEbcC+xkgn2vq6KCZGPNLYA9feeLq7 y77zXNAE6/Vl3hNbBXHc6ePd8BwPqlEYvebW0wqx/bj8m50uYB8IbLAfe5/KY82BTroE P9K0VHLw3KPWj0QKFqmygSvf6Yfdw9FSyjUd6wisCMQRd7Z7hQx+PNISR+b6aUDyZkGR uTaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=NdYxLy6u/2PaTIupg+k2bBdapnT54tEDA9Zj5rVwx9I=; b=zxM9FaWzpBcy97khsAN6K/niI64bBBjH1R91Ap1BhIwHahRmmCcuxl1bs4iCvKPHpO pnRAZPq3QzwmndzsoKCIgj4qzY8FZIl60PYuZJiDVTGJlFu3eiF+6+M0qIv2ahmXRPrT iIsvOa3TfeROrQ4eL3KXcuz8drr6wo2C9GVfp3Gp8ef6jFflj1tE7klJnRhi9eZFtRhw zWM7VeCUPSQWCDaqvw0GWnZM+BUIgFNooQlmdaCccfNMF2AMsqDZ+TxMmSoIzBXQAxlq JxrflD/Hw1KgReR4/9kpVVeq36PYGZH0oM8ZBgTN/T4TyNBaUyz59SHuy9wuPFeCCGFF tD8A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=bAVCp0iv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h185-20020a636cc2000000b0046fd4b828b4si9770410pgc.354.2022.11.02.13.05.33; Wed, 02 Nov 2022 13:05:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=bAVCp0iv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230188AbiKBUBP (ORCPT <rfc822;yves.mi.zy@gmail.com> + 99 others); Wed, 2 Nov 2022 16:01:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37204 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230374AbiKBUBL (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Wed, 2 Nov 2022 16:01:11 -0400 Received: from mail-oi1-x22d.google.com (mail-oi1-x22d.google.com [IPv6:2607:f8b0:4864:20::22d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DB296E53; Wed, 2 Nov 2022 13:01:08 -0700 (PDT) Received: by mail-oi1-x22d.google.com with SMTP id n83so20312191oif.11; Wed, 02 Nov 2022 13:01:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=NdYxLy6u/2PaTIupg+k2bBdapnT54tEDA9Zj5rVwx9I=; b=bAVCp0ivJaQO14A0QNklaSQIqSD0fnB3qCvfGZk24diiJXROl20vlLybQHTqWsCAmd G/OXrtfBqqVagHhnVljb3P4K1POpC+oMSjohUKIAU26KLaKRJc4r1uPq6NiktKorDDIa czZhkVpV7cjbJw10jQ6cLNEiH4zM80BpD4X2GUYYKDEKQZlQfVZlX8/S2HUzIS5GT1Mx UYarHeMq11u0lkibbOD8CWVpdyogUHObFPoBf83nIh6+PJWshohpERwc34q55ZiK2mYu lL2/t29Z1BzWqlfoblPJuC6Gua9PE/lVxDcwcijz3jyG78oDxRo+KiwAPxxl0R9Id/M3 E2jA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=NdYxLy6u/2PaTIupg+k2bBdapnT54tEDA9Zj5rVwx9I=; b=RLU6ucL1+CyQqg6Q/Nh3kB/YugtA0rmHQ8/syAyzteqOs6gmY4BidQMETJcdlRL0YD SrCPi0L2HbLb+1SgS1rsBEq6Uu7cqq2YkwsWvY0wNiQZwpB98SPrjkl/XvHOAIDmgGmX OZFfP0nCvIIYIf2/CMqRQ35k8OxoCl2qv1tbTV3IJAMuEeYOyTZVgdaAoTZlrNv22tAZ xAsqEtU+hGQJdVfzuzBogxl7+vNnyHJgJ662L88F167DyBOkVQbZpES5oB5Lo7La8aMw ma+qRHcetydQrjcXEfDHgHCdRzlAH2WtKnrBBZCVxVJWC0u3VcEE1iTg+NvdhsoVO9LL N66Q== X-Gm-Message-State: ACrzQf09e9aS/wASMcF2S/2r3rsKSVdAl9dnUfzI1Nn/JFBSe0swD0ot hnVCmnHSMqxmWDyJvrGioIs= X-Received: by 2002:a05:6808:1442:b0:35a:1542:a1a6 with SMTP id x2-20020a056808144200b0035a1542a1a6mr8584650oiv.243.1667419267979; Wed, 02 Nov 2022 13:01:07 -0700 (PDT) Received: from auvcetillem1m1.corp.emc.net ([143.166.81.254]) by smtp.gmail.com with ESMTPSA id n29-20020a0568080a1d00b003549397fde4sm4873182oij.54.2022.11.02.13.01.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Nov 2022 13:01:07 -0700 (PDT) From: Stuart Hayes <stuart.w.hayes@gmail.com> To: "Rafael J . Wysocki" <rafael@kernel.org>, Viresh Kumar <viresh.kumar@linaro.org>, Stuart Hayes <stuart.w.hayes@gmail.com>, Kyle Meyer <kyle.meyer@hpe.com> Cc: linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH] cpufreq: acpi: Defer setting boost MSRs Date: Wed, 2 Nov 2022 14:59:57 -0500 Message-Id: <20221102195957.82871-1-stuart.w.hayes@gmail.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748416131062887057?= X-GMAIL-MSGID: =?utf-8?q?1748416131062887057?= |
Series |
cpufreq: acpi: Defer setting boost MSRs
|
|
Commit Message
stuart hayes
Nov. 2, 2022, 7:59 p.m. UTC
When acpi-cpufreq is loaded, boost is enabled on every CPU (by setting an
MSR) before the driver is registered with cpufreq. This can be very time
consuming, because it is done with a CPU hotplug startup callback, and
cpuhp_setup_state() schedules the callback (cpufreq_boost_online()) to run
on each CPU one at a time, waiting for each to run before calling the next.
If cpufreq_register_driver() fails--if, for example, there are no ACPI
P-states present--this is wasted time.
Since cpufreq already sets up a CPU hotplug startup callback if and when
acpi-cpufreq is registered, set the boost MSRs in acpi_cpufreq_cpu_init(),
which is called by the cpufreq cpuhp callback. This allows acpi-cpufreq to
exit quickly if it is loaded but not needed.
On one system with 192 CPUs, this patch speeds up boot by about 30 seconds.
Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com>
---
drivers/cpufreq/acpi-cpufreq.c | 31 +++----------------------------
1 file changed, 3 insertions(+), 28 deletions(-)
Comments
On Wed, Nov 2, 2022 at 9:01 PM Stuart Hayes <stuart.w.hayes@gmail.com> wrote: > > When acpi-cpufreq is loaded, boost is enabled on every CPU (by setting an > MSR) before the driver is registered with cpufreq. This can be very time > consuming, because it is done with a CPU hotplug startup callback, and > cpuhp_setup_state() schedules the callback (cpufreq_boost_online()) to run > on each CPU one at a time, waiting for each to run before calling the next. > > If cpufreq_register_driver() fails--if, for example, there are no ACPI > P-states present--this is wasted time. > > Since cpufreq already sets up a CPU hotplug startup callback if and when > acpi-cpufreq is registered, set the boost MSRs in acpi_cpufreq_cpu_init(), > which is called by the cpufreq cpuhp callback. This allows acpi-cpufreq to > exit quickly if it is loaded but not needed. > > On one system with 192 CPUs, this patch speeds up boot by about 30 seconds. > > Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com> > --- > drivers/cpufreq/acpi-cpufreq.c | 31 +++---------------------------- > 1 file changed, 3 insertions(+), 28 deletions(-) > > diff --git a/drivers/cpufreq/acpi-cpufreq.c b/drivers/cpufreq/acpi-cpufreq.c > index 1bb2b90ebb21..cb167263de72 100644 > --- a/drivers/cpufreq/acpi-cpufreq.c > +++ b/drivers/cpufreq/acpi-cpufreq.c > @@ -535,15 +535,6 @@ static void free_acpi_perf_data(void) > free_percpu(acpi_perf_data); > } > > -static int cpufreq_boost_online(unsigned int cpu) > -{ > - /* > - * On the CPU_UP path we simply keep the boost-disable flag > - * in sync with the current global state. > - */ > - return boost_set_msr(acpi_cpufreq_driver.boost_enabled); > -} > - > static int cpufreq_boost_down_prep(unsigned int cpu) > { > /* > @@ -897,6 +888,8 @@ static int acpi_cpufreq_cpu_init(struct cpufreq_policy *policy) > if (perf->states[0].core_frequency * 1000 != freq_table[0].frequency) > pr_warn(FW_WARN "P-state 0 is not max freq\n"); > > + set_boost(policy, acpi_cpufreq_driver.boost_enabled); > + > return result; > > err_unreg: > @@ -916,6 +909,7 @@ static int acpi_cpufreq_cpu_exit(struct cpufreq_policy *policy) > > pr_debug("%s\n", __func__); > > + cpufreq_boost_down_prep(policy->cpu); > policy->fast_switch_possible = false; > policy->driver_data = NULL; > acpi_processor_unregister_performance(data->acpi_perf_cpu); > @@ -972,25 +966,9 @@ static void __init acpi_cpufreq_boost_init(void) > acpi_cpufreq_driver.set_boost = set_boost; > acpi_cpufreq_driver.boost_enabled = boost_state(0); > > - /* > - * This calls the online callback on all online cpu and forces all > - * MSRs to the same value. > - */ > - ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "cpufreq/acpi:online", > - cpufreq_boost_online, cpufreq_boost_down_prep); > - if (ret < 0) { > - pr_err("acpi_cpufreq: failed to register hotplug callbacks\n"); > - return; > - } > acpi_cpufreq_online = ret; > } > > -static void acpi_cpufreq_boost_exit(void) > -{ > - if (acpi_cpufreq_online > 0) > - cpuhp_remove_state_nocalls(acpi_cpufreq_online); > -} > - > static int __init acpi_cpufreq_init(void) > { > int ret; > @@ -1032,7 +1010,6 @@ static int __init acpi_cpufreq_init(void) > ret = cpufreq_register_driver(&acpi_cpufreq_driver); > if (ret) { > free_acpi_perf_data(); > - acpi_cpufreq_boost_exit(); > } > return ret; > } > @@ -1041,8 +1018,6 @@ static void __exit acpi_cpufreq_exit(void) > { > pr_debug("%s\n", __func__); > > - acpi_cpufreq_boost_exit(); > - > cpufreq_unregister_driver(&acpi_cpufreq_driver); > > free_acpi_perf_data(); > -- Applied as 6.2 material, thanks!
On Thu, Nov 03, 2022 at 07:19:47PM +0100, Rafael J. Wysocki wrote: > On Wed, Nov 2, 2022 at 9:01 PM Stuart Hayes <stuart.w.hayes@gmail.com> wrote: > > > > When acpi-cpufreq is loaded, boost is enabled on every CPU (by setting an > > MSR) before the driver is registered with cpufreq. This can be very time > > consuming, because it is done with a CPU hotplug startup callback, and > > cpuhp_setup_state() schedules the callback (cpufreq_boost_online()) to run > > on each CPU one at a time, waiting for each to run before calling the next. > > > > If cpufreq_register_driver() fails--if, for example, there are no ACPI > > P-states present--this is wasted time. > > > > Since cpufreq already sets up a CPU hotplug startup callback if and when > > acpi-cpufreq is registered, set the boost MSRs in acpi_cpufreq_cpu_init(), > > which is called by the cpufreq cpuhp callback. This allows acpi-cpufreq to > > exit quickly if it is loaded but not needed. > > > > On one system with 192 CPUs, this patch speeds up boot by about 30 seconds. > > > > Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com> > > --- > > drivers/cpufreq/acpi-cpufreq.c | 31 +++---------------------------- > > 1 file changed, 3 insertions(+), 28 deletions(-) ... > Applied as 6.2 material, thanks! My 32-bit Atom doesn't like this one. Reverting fixes it ofc. [ 22.780260] unchecked MSR access error: WRMSR to 0x1a0 (tried to write 0x0000004364950488) at rIP: 0xf80b37d4 (boost_set_msr.isra.0+0x9c/0x114 [acpi_cpufreq]) [ 22.781186] Call Trace: [ 22.781186] boost_set_msr_each+0x15/0x1c [acpi_cpufreq] [ 22.781186] __flush_smp_call_function_queue+0x132/0x1cc [ 22.781186] ? sysvec_call_function+0x30/0x30 [ 22.781186] generic_smp_call_function_single_interrupt+0x12/0x18 [ 22.781186] __sysvec_call_function_single.constprop.0+0x43/0x1d8 [ 22.781186] sysvec_call_function_single+0x18/0x30 [ 22.781186] handle_exception+0x133/0x133 [ 22.781186] EIP: finish_task_switch.isra.0+0x124/0x3a8 [ 22.781186] Code: d8 e8 8c 16 92 00 85 f6 75 e8 a1 04 24 6c c2 85 c0 0f 8f 9b 00 00 00 89 d8 e8 e4 02 92 00 e8 53 9e 0b 00 fb 64 a1 40 f9 69 c2 <8b> 80 5c 0f 00 00 85 c0 0f 85 72 01 00 00 a1 28 24 6c c2 64 8b 15 [ 22.781186] EAX: c32e2700 EBX: f748af40 ECX: 00000000 EDX: c1c3876e [ 22.781186] ESI: 00000000 EDI: 00000000 EBP: c3241f90 ESP: c3241f78 [ 22.781186] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 EFLAGS: 00000206 [ 22.781186] ? uevent_seqnum_show+0x1b/0x28 [ 22.781186] ? pid_list_refill_irq+0x128/0x1c0 [ 22.781186] ? sysvec_call_function+0x30/0x30 [ 22.781186] ? pid_list_refill_irq+0x128/0x1c0 [ 22.781186] ? sysvec_call_function+0x30/0x30 [ 22.781186] ? finish_task_switch.isra.0+0x124/0x3a8 [ 22.781186] schedule_tail+0x12/0x78 [ 22.781186] schedule_tail_wrapper+0x9/0x10 [ 22.781186] ret_from_fork+0x5/0x28 [ 22.781186] EIP: 0xb7fba549 [ 22.781186] Code: Unable to access opcode bytes at 0xb7fba51f. [ 22.781186] EAX: 00000000 EBX: 01200011 ECX: 00000000 EDX: 00000000 [ 22.781186] ESI: 00000000 EDI: b7bfe868 EBP: 00000000 ESP: bfcfedc0 [ 22.781186] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b EFLAGS: 00000246
On 12/4/2022 12:29 PM, Borislav Petkov wrote: > On Thu, Nov 03, 2022 at 07:19:47PM +0100, Rafael J. Wysocki wrote: >> On Wed, Nov 2, 2022 at 9:01 PM Stuart Hayes <stuart.w.hayes@gmail.com> wrote: >>> >>> When acpi-cpufreq is loaded, boost is enabled on every CPU (by setting an >>> MSR) before the driver is registered with cpufreq. This can be very time >>> consuming, because it is done with a CPU hotplug startup callback, and >>> cpuhp_setup_state() schedules the callback (cpufreq_boost_online()) to run >>> on each CPU one at a time, waiting for each to run before calling the next. >>> >>> If cpufreq_register_driver() fails--if, for example, there are no ACPI >>> P-states present--this is wasted time. >>> >>> Since cpufreq already sets up a CPU hotplug startup callback if and when >>> acpi-cpufreq is registered, set the boost MSRs in acpi_cpufreq_cpu_init(), >>> which is called by the cpufreq cpuhp callback. This allows acpi-cpufreq to >>> exit quickly if it is loaded but not needed. >>> >>> On one system with 192 CPUs, this patch speeds up boot by about 30 seconds. >>> >>> Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com> >>> --- >>> drivers/cpufreq/acpi-cpufreq.c | 31 +++---------------------------- >>> 1 file changed, 3 insertions(+), 28 deletions(-) > > ... > >> Applied as 6.2 material, thanks! > > My 32-bit Atom doesn't like this one. Reverting fixes it ofc. > > [ 22.780260] unchecked MSR access error: WRMSR to 0x1a0 (tried to write 0x0000004364950488) at rIP: 0xf80b37d4 (boost_set_msr.isra.0+0x9c/0x114 [acpi_cpufreq]) > [ 22.781186] Call Trace: > [ 22.781186] boost_set_msr_each+0x15/0x1c [acpi_cpufreq] > [ 22.781186] __flush_smp_call_function_queue+0x132/0x1cc > [ 22.781186] ? sysvec_call_function+0x30/0x30 > [ 22.781186] generic_smp_call_function_single_interrupt+0x12/0x18 > [ 22.781186] __sysvec_call_function_single.constprop.0+0x43/0x1d8 > [ 22.781186] sysvec_call_function_single+0x18/0x30 > [ 22.781186] handle_exception+0x133/0x133 > [ 22.781186] EIP: finish_task_switch.isra.0+0x124/0x3a8 > [ 22.781186] Code: d8 e8 8c 16 92 00 85 f6 75 e8 a1 04 24 6c c2 85 c0 0f 8f 9b 00 00 00 89 d8 e8 e4 02 92 00 e8 53 9e 0b 00 fb 64 a1 40 f9 69 c2 <8b> 80 5c 0f 00 00 85 c0 0f 85 72 01 00 00 a1 28 24 6c c2 64 8b 15 > [ 22.781186] EAX: c32e2700 EBX: f748af40 ECX: 00000000 EDX: c1c3876e > [ 22.781186] ESI: 00000000 EDI: 00000000 EBP: c3241f90 ESP: c3241f78 > [ 22.781186] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 EFLAGS: 00000206 > [ 22.781186] ? uevent_seqnum_show+0x1b/0x28 > [ 22.781186] ? pid_list_refill_irq+0x128/0x1c0 > [ 22.781186] ? sysvec_call_function+0x30/0x30 > [ 22.781186] ? pid_list_refill_irq+0x128/0x1c0 > [ 22.781186] ? sysvec_call_function+0x30/0x30 > [ 22.781186] ? finish_task_switch.isra.0+0x124/0x3a8 > [ 22.781186] schedule_tail+0x12/0x78 > [ 22.781186] schedule_tail_wrapper+0x9/0x10 > [ 22.781186] ret_from_fork+0x5/0x28 > [ 22.781186] EIP: 0xb7fba549 > [ 22.781186] Code: Unable to access opcode bytes at 0xb7fba51f. > [ 22.781186] EAX: 00000000 EBX: 01200011 ECX: 00000000 EDX: 00000000 > [ 22.781186] ESI: 00000000 EDI: b7bfe868 EBP: 00000000 ESP: bfcfedc0 > [ 22.781186] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b EFLAGS: 00000246 > I believe I see the problem... acpi_cpufreq_cpu_init is calling set_boost() directly without checking to see if acpi_cpufreq_driver.set_boost was set, so it is trying to set the MSR on CPUs that don't support it. Thanks, I can submit a patch to fix this.
On Sun, Dec 4, 2022 at 8:20 PM stuart hayes <stuart.w.hayes@gmail.com> wrote: > > > > On 12/4/2022 12:29 PM, Borislav Petkov wrote: > > On Thu, Nov 03, 2022 at 07:19:47PM +0100, Rafael J. Wysocki wrote: > >> On Wed, Nov 2, 2022 at 9:01 PM Stuart Hayes <stuart.w.hayes@gmail.com> wrote: > >>> > >>> When acpi-cpufreq is loaded, boost is enabled on every CPU (by setting an > >>> MSR) before the driver is registered with cpufreq. This can be very time > >>> consuming, because it is done with a CPU hotplug startup callback, and > >>> cpuhp_setup_state() schedules the callback (cpufreq_boost_online()) to run > >>> on each CPU one at a time, waiting for each to run before calling the next. > >>> > >>> If cpufreq_register_driver() fails--if, for example, there are no ACPI > >>> P-states present--this is wasted time. > >>> > >>> Since cpufreq already sets up a CPU hotplug startup callback if and when > >>> acpi-cpufreq is registered, set the boost MSRs in acpi_cpufreq_cpu_init(), > >>> which is called by the cpufreq cpuhp callback. This allows acpi-cpufreq to > >>> exit quickly if it is loaded but not needed. > >>> > >>> On one system with 192 CPUs, this patch speeds up boot by about 30 seconds. > >>> > >>> Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com> > >>> --- > >>> drivers/cpufreq/acpi-cpufreq.c | 31 +++---------------------------- > >>> 1 file changed, 3 insertions(+), 28 deletions(-) > > > > ... > > > >> Applied as 6.2 material, thanks! > > > > My 32-bit Atom doesn't like this one. Reverting fixes it ofc. > > > > [ 22.780260] unchecked MSR access error: WRMSR to 0x1a0 (tried to write 0x0000004364950488) at rIP: 0xf80b37d4 (boost_set_msr.isra.0+0x9c/0x114 [acpi_cpufreq]) > > [ 22.781186] Call Trace: > > [ 22.781186] boost_set_msr_each+0x15/0x1c [acpi_cpufreq] > > [ 22.781186] __flush_smp_call_function_queue+0x132/0x1cc > > [ 22.781186] ? sysvec_call_function+0x30/0x30 > > [ 22.781186] generic_smp_call_function_single_interrupt+0x12/0x18 > > [ 22.781186] __sysvec_call_function_single.constprop.0+0x43/0x1d8 > > [ 22.781186] sysvec_call_function_single+0x18/0x30 > > [ 22.781186] handle_exception+0x133/0x133 > > [ 22.781186] EIP: finish_task_switch.isra.0+0x124/0x3a8 > > [ 22.781186] Code: d8 e8 8c 16 92 00 85 f6 75 e8 a1 04 24 6c c2 85 c0 0f 8f 9b 00 00 00 89 d8 e8 e4 02 92 00 e8 53 9e 0b 00 fb 64 a1 40 f9 69 c2 <8b> 80 5c 0f 00 00 85 c0 0f 85 72 01 00 00 a1 28 24 6c c2 64 8b 15 > > [ 22.781186] EAX: c32e2700 EBX: f748af40 ECX: 00000000 EDX: c1c3876e > > [ 22.781186] ESI: 00000000 EDI: 00000000 EBP: c3241f90 ESP: c3241f78 > > [ 22.781186] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 EFLAGS: 00000206 > > [ 22.781186] ? uevent_seqnum_show+0x1b/0x28 > > [ 22.781186] ? pid_list_refill_irq+0x128/0x1c0 > > [ 22.781186] ? sysvec_call_function+0x30/0x30 > > [ 22.781186] ? pid_list_refill_irq+0x128/0x1c0 > > [ 22.781186] ? sysvec_call_function+0x30/0x30 > > [ 22.781186] ? finish_task_switch.isra.0+0x124/0x3a8 > > [ 22.781186] schedule_tail+0x12/0x78 > > [ 22.781186] schedule_tail_wrapper+0x9/0x10 > > [ 22.781186] ret_from_fork+0x5/0x28 > > [ 22.781186] EIP: 0xb7fba549 > > [ 22.781186] Code: Unable to access opcode bytes at 0xb7fba51f. > > [ 22.781186] EAX: 00000000 EBX: 01200011 ECX: 00000000 EDX: 00000000 > > [ 22.781186] ESI: 00000000 EDI: b7bfe868 EBP: 00000000 ESP: bfcfedc0 > > [ 22.781186] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b EFLAGS: 00000246 > > > > I believe I see the problem... acpi_cpufreq_cpu_init is calling set_boost() directly without checking to see if acpi_cpufreq_driver.set_boost was set, so it is trying to set the MSR on CPUs that don't support it. > > Thanks, I can submit a patch to fix this. Yes, please. Note that I need to get this fix shortly, though, or I will just revert the problemating commit before the 6.2 merge window opens.
On 12/5/2022 6:43 AM, Rafael J. Wysocki wrote: > On Sun, Dec 4, 2022 at 8:20 PM stuart hayes <stuart.w.hayes@gmail.com> wrote: >> >> >> >> On 12/4/2022 12:29 PM, Borislav Petkov wrote: >>> On Thu, Nov 03, 2022 at 07:19:47PM +0100, Rafael J. Wysocki wrote: >>>> On Wed, Nov 2, 2022 at 9:01 PM Stuart Hayes <stuart.w.hayes@gmail.com> wrote: >>>>> >>>>> When acpi-cpufreq is loaded, boost is enabled on every CPU (by setting an >>>>> MSR) before the driver is registered with cpufreq. This can be very time >>>>> consuming, because it is done with a CPU hotplug startup callback, and >>>>> cpuhp_setup_state() schedules the callback (cpufreq_boost_online()) to run >>>>> on each CPU one at a time, waiting for each to run before calling the next. >>>>> >>>>> If cpufreq_register_driver() fails--if, for example, there are no ACPI >>>>> P-states present--this is wasted time. >>>>> >>>>> Since cpufreq already sets up a CPU hotplug startup callback if and when >>>>> acpi-cpufreq is registered, set the boost MSRs in acpi_cpufreq_cpu_init(), >>>>> which is called by the cpufreq cpuhp callback. This allows acpi-cpufreq to >>>>> exit quickly if it is loaded but not needed. >>>>> >>>>> On one system with 192 CPUs, this patch speeds up boot by about 30 seconds. >>>>> >>>>> Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com> >>>>> --- >>>>> drivers/cpufreq/acpi-cpufreq.c | 31 +++---------------------------- >>>>> 1 file changed, 3 insertions(+), 28 deletions(-) >>> >>> ... >>> >>>> Applied as 6.2 material, thanks! >>> >>> My 32-bit Atom doesn't like this one. Reverting fixes it ofc. >>> >>> [ 22.780260] unchecked MSR access error: WRMSR to 0x1a0 (tried to write 0x0000004364950488) at rIP: 0xf80b37d4 (boost_set_msr.isra.0+0x9c/0x114 [acpi_cpufreq]) >>> [ 22.781186] Call Trace: >>> [ 22.781186] boost_set_msr_each+0x15/0x1c [acpi_cpufreq] >>> [ 22.781186] __flush_smp_call_function_queue+0x132/0x1cc >>> [ 22.781186] ? sysvec_call_function+0x30/0x30 >>> [ 22.781186] generic_smp_call_function_single_interrupt+0x12/0x18 >>> [ 22.781186] __sysvec_call_function_single.constprop.0+0x43/0x1d8 >>> [ 22.781186] sysvec_call_function_single+0x18/0x30 >>> [ 22.781186] handle_exception+0x133/0x133 >>> [ 22.781186] EIP: finish_task_switch.isra.0+0x124/0x3a8 >>> [ 22.781186] Code: d8 e8 8c 16 92 00 85 f6 75 e8 a1 04 24 6c c2 85 c0 0f 8f 9b 00 00 00 89 d8 e8 e4 02 92 00 e8 53 9e 0b 00 fb 64 a1 40 f9 69 c2 <8b> 80 5c 0f 00 00 85 c0 0f 85 72 01 00 00 a1 28 24 6c c2 64 8b 15 >>> [ 22.781186] EAX: c32e2700 EBX: f748af40 ECX: 00000000 EDX: c1c3876e >>> [ 22.781186] ESI: 00000000 EDI: 00000000 EBP: c3241f90 ESP: c3241f78 >>> [ 22.781186] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 EFLAGS: 00000206 >>> [ 22.781186] ? uevent_seqnum_show+0x1b/0x28 >>> [ 22.781186] ? pid_list_refill_irq+0x128/0x1c0 >>> [ 22.781186] ? sysvec_call_function+0x30/0x30 >>> [ 22.781186] ? pid_list_refill_irq+0x128/0x1c0 >>> [ 22.781186] ? sysvec_call_function+0x30/0x30 >>> [ 22.781186] ? finish_task_switch.isra.0+0x124/0x3a8 >>> [ 22.781186] schedule_tail+0x12/0x78 >>> [ 22.781186] schedule_tail_wrapper+0x9/0x10 >>> [ 22.781186] ret_from_fork+0x5/0x28 >>> [ 22.781186] EIP: 0xb7fba549 >>> [ 22.781186] Code: Unable to access opcode bytes at 0xb7fba51f. >>> [ 22.781186] EAX: 00000000 EBX: 01200011 ECX: 00000000 EDX: 00000000 >>> [ 22.781186] ESI: 00000000 EDI: b7bfe868 EBP: 00000000 ESP: bfcfedc0 >>> [ 22.781186] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b EFLAGS: 00000246 >>> >> >> I believe I see the problem... acpi_cpufreq_cpu_init is calling set_boost() directly without checking to see if acpi_cpufreq_driver.set_boost was set, so it is trying to set the MSR on CPUs that don't support it. >> >> Thanks, I can submit a patch to fix this. > > Yes, please. > > Note that I need to get this fix shortly, though, or I will just > revert the problemating commit before the 6.2 merge window opens. I sent it a few hours ago as "[PATCH] cpufreq: acpi: Only set boost MSRs on supported CPUs". Thanks!
diff --git a/drivers/cpufreq/acpi-cpufreq.c b/drivers/cpufreq/acpi-cpufreq.c index 1bb2b90ebb21..cb167263de72 100644 --- a/drivers/cpufreq/acpi-cpufreq.c +++ b/drivers/cpufreq/acpi-cpufreq.c @@ -535,15 +535,6 @@ static void free_acpi_perf_data(void) free_percpu(acpi_perf_data); } -static int cpufreq_boost_online(unsigned int cpu) -{ - /* - * On the CPU_UP path we simply keep the boost-disable flag - * in sync with the current global state. - */ - return boost_set_msr(acpi_cpufreq_driver.boost_enabled); -} - static int cpufreq_boost_down_prep(unsigned int cpu) { /* @@ -897,6 +888,8 @@ static int acpi_cpufreq_cpu_init(struct cpufreq_policy *policy) if (perf->states[0].core_frequency * 1000 != freq_table[0].frequency) pr_warn(FW_WARN "P-state 0 is not max freq\n"); + set_boost(policy, acpi_cpufreq_driver.boost_enabled); + return result; err_unreg: @@ -916,6 +909,7 @@ static int acpi_cpufreq_cpu_exit(struct cpufreq_policy *policy) pr_debug("%s\n", __func__); + cpufreq_boost_down_prep(policy->cpu); policy->fast_switch_possible = false; policy->driver_data = NULL; acpi_processor_unregister_performance(data->acpi_perf_cpu); @@ -972,25 +966,9 @@ static void __init acpi_cpufreq_boost_init(void) acpi_cpufreq_driver.set_boost = set_boost; acpi_cpufreq_driver.boost_enabled = boost_state(0); - /* - * This calls the online callback on all online cpu and forces all - * MSRs to the same value. - */ - ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "cpufreq/acpi:online", - cpufreq_boost_online, cpufreq_boost_down_prep); - if (ret < 0) { - pr_err("acpi_cpufreq: failed to register hotplug callbacks\n"); - return; - } acpi_cpufreq_online = ret; } -static void acpi_cpufreq_boost_exit(void) -{ - if (acpi_cpufreq_online > 0) - cpuhp_remove_state_nocalls(acpi_cpufreq_online); -} - static int __init acpi_cpufreq_init(void) { int ret; @@ -1032,7 +1010,6 @@ static int __init acpi_cpufreq_init(void) ret = cpufreq_register_driver(&acpi_cpufreq_driver); if (ret) { free_acpi_perf_data(); - acpi_cpufreq_boost_exit(); } return ret; } @@ -1041,8 +1018,6 @@ static void __exit acpi_cpufreq_exit(void) { pr_debug("%s\n", __func__); - acpi_cpufreq_boost_exit(); - cpufreq_unregister_driver(&acpi_cpufreq_driver); free_acpi_perf_data();