Message ID | 20231101111621.GC19106@noisy.programming.kicks-ass.net |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:abcd:0:b0:403:3b70:6f57 with SMTP id f13csp338401vqx; Wed, 1 Nov 2023 04:17:09 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFdQIDc3OE/H4Jr+XNSOPmklvAhFk9WTUu0boQjSMxR4yP3QJUgoPXiyC0AnHFOuuRp23a3 X-Received: by 2002:a05:6a20:3ca7:b0:14c:a2e1:65ec with SMTP id b39-20020a056a203ca700b0014ca2e165ecmr20115587pzj.38.1698837428904; Wed, 01 Nov 2023 04:17:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698837428; cv=none; d=google.com; s=arc-20160816; b=rKgUv3OPwqBOV6qyHAhLUwVXUY1r8A7qVNO39ejdsCnpZyALinidI6xPA+IMsDdUyB e2ZFKudltyMJAUWnR41H+LxeJYlbOGvBejluuHHF8XliTQnL9vCbUDH0NesgzYyB37zm 6y6wtoZMuY7cpFM2KK0moUp6ijB4lPiXEEKk7uemRUAtSE15jh/gdaPdL/RWNVEBQRAa DVuJ9gcx8vrr3ahjOAruylsFUrvQUO8RLPk+BxChqjhgR5y8Jn1u10rmR7qO6CQzKiJA evR/ZMm2n7Ar9CPES6qNhEIuMFy0eaoiSThCYIDbGD1+7JZYdzz8RgqS/9fWWGupjoKf zwrg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=zSh13B1ZZlhI2RrNckeOHFhnIe/fCZORE5Av6JJudls=; fh=AHVcDOsi0MmVlV8SGoJjlW7KUgtoIHOIv61fzJVkck8=; b=A3+HtRyifLnQaekTJqfdyxBy36dYULFQ2R8eqALCWT8bYJhMark1/8KjditOZpo3Gm 36kVuEJdnPUT0kAErTkzQ6vokvN1fGfb63qc3lDZp7pGDwWUBM6Oz9o+ePH7DzdIh1UN ICqXk6heYIy0bVz8P8e0olyH+kHeYdCFmIaXLFNl7GEQVHnvHpYUiDYthlHcckyNtmUa mJroGzJRebj1kOiDL1M74xGl7beL3vFBYtvXYUauuZY9oiIuT8z0UCPyggUCv87ZAS1s tf3Gvmx8ZuGYN1506Wxfy5DbEsBONPnHJUqELi6asOy/XQheIxUOv4mBwuxcQE7sGAAg rtjg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=RONL5jvb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id z126-20020a626584000000b006bd360e70edsi1327249pfb.103.2023.11.01.04.17.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Nov 2023 04:17:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=RONL5jvb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 13FE88039EF4; Wed, 1 Nov 2023 04:16:57 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232947AbjKALQk (ORCPT <rfc822;rbbytesnap@gmail.com> + 35 others); Wed, 1 Nov 2023 07:16:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38332 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229731AbjKALQj (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Wed, 1 Nov 2023 07:16:39 -0400 Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D1FCEF7 for <linux-kernel@vger.kernel.org>; Wed, 1 Nov 2023 04:16:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=zSh13B1ZZlhI2RrNckeOHFhnIe/fCZORE5Av6JJudls=; b=RONL5jvbv9t8WJohNIHc5jI0nG gl+wlFbu1+NtVPYCnZCmPwnto1RKN9oaAx5jlFC0pN7SoobS+JHPDJh+ULYoNOifQpiHjMT+zuWEi f7+LHJJmuXBXGlGWifK6z7jeUSTfeQe7c1Xp8R85WvigUBcwN+CitLRXDKsAOWVszcoPVOA4o9q4q Z8vP7A0g3gcSBy+CD4mWI94wd8pPHvZwFLMTv7ncLexZSOxNKkMLj5dou1MYl6RvEaeqOg4neAMXz vQXRPVYrN/fFRFwjEZmzG78KlkHrLmqDYVbocvbGq3x1hZeYQmF2f/lGE45HgaIDaeJpyIV324FL6 NzSMyskA==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qy9Cv-005fUV-1I; Wed, 01 Nov 2023 11:16:21 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 1000) id 108F23002AF; Wed, 1 Nov 2023 12:16:21 +0100 (CET) Date: Wed, 1 Nov 2023 12:16:21 +0100 From: Peter Zijlstra <peterz@infradead.org> To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, Thomas Gleixner <tglx@linutronix.de>, Arjan van de Ven <arjan@linux.intel.com>, feng.tang@intel.com Subject: [PATCH v2] x86/tsc: Have tsc=recalibrate override things Message-ID: <20231101111621.GC19106@noisy.programming.kicks-ass.net> References: <20231030160050.GA15024@noisy.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20231030160050.GA15024@noisy.programming.kicks-ass.net> X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Wed, 01 Nov 2023 04:16:57 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781196872813510817 X-GMAIL-MSGID: 1781360155818425565 |
Series |
[v2] x86/tsc: Have tsc=recalibrate override things
|
|
Commit Message
Peter Zijlstra
Nov. 1, 2023, 11:16 a.m. UTC
Subject: x86/tsc: Have tsc=recalibrate override things From: Peter Zijlstra <peterz@infradead.org> Date: Mon, 30 Oct 2023 17:00:50 +0100 My brand-spanking new SPR supermicro workstation was reporting NTP failures: Oct 30 13:00:26 spr ntpd[3517]: CLOCK: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized Oct 30 13:00:58 spr ntpd[3517]: CLOCK: time stepped by 32.316775 Oct 30 13:00:58 spr ntpd[3517]: CLOCK: frequency error 41699 PPM exceeds tolerance 500 PPM CPUID provides: Time Stamp Counter/Core Crystal Clock Information (0x15): TSC/clock ratio = 200/2 nominal core crystal clock = 25000000 Hz Processor Frequency Information (0x16): Core Base Frequency (MHz) = 0x9c4 (2500) Core Maximum Frequency (MHz) = 0x12c0 (4800) Bus (Reference) Frequency (MHz) = 0x64 (100) and the kernel believes this. Since commit a7ec817d5542 ("x86/tsc: Add option to force frequency recalibration with HW timer") there is the tsc=recalibrate option, which forces the recalibrate. This duely reports: Oct 30 12:42:39 spr kernel: tsc: Warning: TSC freq calibrated by CPUID/MSR differs from what is calibrated by HW timer, please check with vendor!! Oct 30 12:42:39 spr kernel: tsc: Previous calibrated TSC freq: 2500.000 MHz Oct 30 12:42:39 spr kernel: tsc: TSC freq recalibrated by [HPET]: 2399.967 MHz but then continues running at 2500, offering no solace and keeping NTP upset -- drifting ~30 seconds for every 15 minutes. Have tsc=recalibrate override the CPUID provided numbers. This makes the machine usable and keeps NTP 'happy': Oct 30 16:48:44 spr ntpd[2200]: CLOCK: time stepped by -0.768679 Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> --- arch/x86/kernel/tsc.c | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-)
Comments
Add Paul as he also met similar issue before. On Wed, Nov 01, 2023 at 12:16:21PM +0100, Peter Zijlstra wrote: > Subject: x86/tsc: Have tsc=recalibrate override things > From: Peter Zijlstra <peterz@infradead.org> > Date: Mon, 30 Oct 2023 17:00:50 +0100 > > My brand-spanking new SPR supermicro workstation was reporting NTP > failures: > > Oct 30 13:00:26 spr ntpd[3517]: CLOCK: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized > Oct 30 13:00:58 spr ntpd[3517]: CLOCK: time stepped by 32.316775 > Oct 30 13:00:58 spr ntpd[3517]: CLOCK: frequency error 41699 PPM exceeds tolerance 500 PPM > > CPUID provides: > > Time Stamp Counter/Core Crystal Clock Information (0x15): > TSC/clock ratio = 200/2 > nominal core crystal clock = 25000000 Hz > Processor Frequency Information (0x16): > Core Base Frequency (MHz) = 0x9c4 (2500) > Core Maximum Frequency (MHz) = 0x12c0 (4800) > Bus (Reference) Frequency (MHz) = 0x64 (100) > > and the kernel believes this. Since commit a7ec817d5542 ("x86/tsc: Add > option to force frequency recalibration with HW timer") there is the > tsc=recalibrate option, which forces the recalibrate. > > This duely reports: > > Oct 30 12:42:39 spr kernel: tsc: Warning: TSC freq calibrated by CPUID/MSR differs from what is calibrated by HW timer, please check with vendor!! > Oct 30 12:42:39 spr kernel: tsc: Previous calibrated TSC freq: 2500.000 MHz > Oct 30 12:42:39 spr kernel: tsc: TSC freq recalibrated by [HPET]: 2399.967 MHz > > but then continues running at 2500, offering no solace and keeping NTP > upset -- drifting ~30 seconds for every 15 minutes. > > Have tsc=recalibrate override the CPUID provided numbers. This makes the > machine usable and keeps NTP 'happy': > > Oct 30 16:48:44 spr ntpd[2200]: CLOCK: time stepped by -0.768679 Hi Peter, The patch looks fine to me. One thought is the original semantics of 'tsc=recalibrate' is just to recalibrate and print out the frequency, and with this change, it adds 'override' semantics, so the description for 'recalibrate' in kernel-parameters.txt may also need some update. Thanks, Feng > Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> > --- > arch/x86/kernel/tsc.c | 15 ++++++--------- > 1 file changed, 6 insertions(+), 9 deletions(-) > > --- a/arch/x86/kernel/tsc.c > +++ b/arch/x86/kernel/tsc.c > @@ -1430,14 +1430,13 @@ static void tsc_refine_calibration_work( > hpet ? "HPET" : "PM_TIMER", > (unsigned long)freq / 1000, > (unsigned long)freq % 1000); > + } else { > > - return; > + /* Make sure we're within 1% */ > + if (abs(tsc_khz - freq) > tsc_khz/100) > + goto out; > } > > - /* Make sure we're within 1% */ > - if (abs(tsc_khz - freq) > tsc_khz/100) > - goto out; > - > tsc_khz = freq; > pr_info("Refined TSC clocksource calibration: %lu.%03lu MHz\n", > (unsigned long)tsc_khz / 1000, > @@ -1479,14 +1478,12 @@ static int __init init_tsc_clocksource(v > * When TSC frequency is known (retrieved via MSR or CPUID), we skip > * the refined calibration and directly register it as a clocksource. > */ > - if (boot_cpu_has(X86_FEATURE_TSC_KNOWN_FREQ)) { > + if (boot_cpu_has(X86_FEATURE_TSC_KNOWN_FREQ) && !tsc_force_recalibrate) { > if (boot_cpu_has(X86_FEATURE_ART)) > art_related_clocksource = &clocksource_tsc; > clocksource_register_khz(&clocksource_tsc, tsc_khz); > clocksource_unregister(&clocksource_tsc_early); > - > - if (!tsc_force_recalibrate) > - return 0; > + return 0; > } > > schedule_delayed_work(&tsc_irqwork, 0);
On Wed, Nov 01, 2023 at 12:16:21PM +0100, Peter Zijlstra wrote: > Subject: x86/tsc: Have tsc=recalibrate override things > From: Peter Zijlstra <peterz@infradead.org> > Date: Mon, 30 Oct 2023 17:00:50 +0100 > > My brand-spanking new SPR supermicro workstation was reporting NTP > failures: > > Oct 30 13:00:26 spr ntpd[3517]: CLOCK: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized > Oct 30 13:00:58 spr ntpd[3517]: CLOCK: time stepped by 32.316775 > Oct 30 13:00:58 spr ntpd[3517]: CLOCK: frequency error 41699 PPM exceeds tolerance 500 PPM > > CPUID provides: > > Time Stamp Counter/Core Crystal Clock Information (0x15): > TSC/clock ratio = 200/2 > nominal core crystal clock = 25000000 Hz > Processor Frequency Information (0x16): > Core Base Frequency (MHz) = 0x9c4 (2500) > Core Maximum Frequency (MHz) = 0x12c0 (4800) > Bus (Reference) Frequency (MHz) = 0x64 (100) > > and the kernel believes this. Since commit a7ec817d5542 ("x86/tsc: Add > option to force frequency recalibration with HW timer") there is the > tsc=recalibrate option, which forces the recalibrate. > > This duely reports: > > Oct 30 12:42:39 spr kernel: tsc: Warning: TSC freq calibrated by CPUID/MSR differs from what is calibrated by HW timer, please check with vendor!! > Oct 30 12:42:39 spr kernel: tsc: Previous calibrated TSC freq: 2500.000 MHz > Oct 30 12:42:39 spr kernel: tsc: TSC freq recalibrated by [HPET]: 2399.967 MHz Additionally, we could consider something like the below. This makes the machine print: [ 0.000000] DMI: Supermicro SYS-531A-I/X13SRA-TF, BIOS 1.1b 08/01/2023 [ 0.000000] tsc: [Firmware Bug]: DMI based CPUID-15h crystal frequency override: 24000000 This way I can boot without additional parameters. --- Index: linux-2.6/arch/x86/kernel/tsc.c =================================================================== --- linux-2.6.orig/arch/x86/kernel/tsc.c +++ linux-2.6/arch/x86/kernel/tsc.c @@ -15,6 +15,7 @@ #include <linux/timex.h> #include <linux/static_key.h> #include <linux/static_call.h> +#include <linux/dmi.h> #include <asm/hpet.h> #include <asm/timer.h> @@ -651,6 +652,35 @@ success: return delta; } +static unsigned long dmi_crystal_hz; + +static int dmi_crystal_hz_override(const struct dmi_system_id *d) +{ + if (dmi_crystal_hz != (unsigned long)d->driver_data) { + dmi_crystal_hz = (unsigned long)d->driver_data; + pr_err(FW_BUG "DMI based CPUID-15h crystal frequency override: %lu\n", + dmi_crystal_hz); + } + return 1; +} + +/* + * List of systems that managed to screw up CPUID-15h :-( + */ +static const struct dmi_system_id tsc_dmi_table[] = { + { + .callback = dmi_crystal_hz_override, + .ident = "Supermicro X13SRA-TF", + .matches = { + DMI_MATCH(DMI_BOARD_VENDOR, "Supermicro"), + DMI_MATCH(DMI_BOARD_NAME, "X13SRA-TF"), + }, + /* The board advertises a 25MHz crystal, in reality it has 24MHz */ + .driver_data = (void *)24000000UL, + }, + {} +}; + /** * native_calibrate_tsc * Determine TSC frequency via CPUID, else return 0. @@ -671,6 +701,10 @@ unsigned long native_calibrate_tsc(void) /* CPUID 15H TSC/Crystal ratio, plus optionally Crystal Hz */ cpuid(0x15, &eax_denominator, &ebx_numerator, &ecx_hz, &edx); + dmi_crystal_hz = ecx_hz; + if (ecx_hz && dmi_check_system(tsc_dmi_table)) + ecx_hz = dmi_crystal_hz; + if (ebx_numerator == 0 || eax_denominator == 0) return 0;
--- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -1430,14 +1430,13 @@ static void tsc_refine_calibration_work( hpet ? "HPET" : "PM_TIMER", (unsigned long)freq / 1000, (unsigned long)freq % 1000); + } else { - return; + /* Make sure we're within 1% */ + if (abs(tsc_khz - freq) > tsc_khz/100) + goto out; } - /* Make sure we're within 1% */ - if (abs(tsc_khz - freq) > tsc_khz/100) - goto out; - tsc_khz = freq; pr_info("Refined TSC clocksource calibration: %lu.%03lu MHz\n", (unsigned long)tsc_khz / 1000, @@ -1479,14 +1478,12 @@ static int __init init_tsc_clocksource(v * When TSC frequency is known (retrieved via MSR or CPUID), we skip * the refined calibration and directly register it as a clocksource. */ - if (boot_cpu_has(X86_FEATURE_TSC_KNOWN_FREQ)) { + if (boot_cpu_has(X86_FEATURE_TSC_KNOWN_FREQ) && !tsc_force_recalibrate) { if (boot_cpu_has(X86_FEATURE_ART)) art_related_clocksource = &clocksource_tsc; clocksource_register_khz(&clocksource_tsc, tsc_khz); clocksource_unregister(&clocksource_tsc_early); - - if (!tsc_force_recalibrate) - return 0; + return 0; } schedule_delayed_work(&tsc_irqwork, 0);