From patchwork Fri Jun 16 18:27:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Luck, Tony" X-Patchwork-Id: 109299 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp1550262vqr; Fri, 16 Jun 2023 11:55:23 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7Qvo3q9Y352bNRFAJkinAR+h6Xb9uVa5KOI4mS4Gk0GOfsKIyKXt9zZGPufS5kAoWYF+xt X-Received: by 2002:a17:902:d345:b0:1af:e2e2:2bd9 with SMTP id l5-20020a170902d34500b001afe2e22bd9mr2300508plk.17.1686941723041; Fri, 16 Jun 2023 11:55:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686941723; cv=none; d=google.com; s=arc-20160816; b=ji+xC16/RX9AOE1f6qToeCYKXS6dw2tUVMcKZjfyW2xkgptQ/gAdA0SxnLhUiZVT8g o4F4juSTWcHnyq6Ow+fMxjEkBX/OKSnBQH0FjpYzSec/DTnTQL4dYuvNpE8TmNF8VToC rndbE6cCZR82u+s56Cc20OtnKMgCyIaenZ8B0N+xTiF3W0cpDGaY+2zdHbGqTXx5JX75 +co1Dyh0cV8lwuXwoMLLHnX5iKXhKzVVZA/bynToFjJvcactFlKoV4zUqUd5TzV5DMWv Ow4Zsza9TIox8emvevtwEqgwnzcFMElgMFH2MGYnHuCdRPko4NQ7xzTuZd9n2CF2GfwK 4waQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Iyy62ZC7e7fdO8vxiGdJMcg9T1WX0i5pcfWEmk/GFno=; b=VQzldATg+eoe+tiwLFHJUsCPfRaA9gCEkCvqYZN/cWs3nY38S9t7Ld/USKNhLTRn1w n/r2Fng1eZZqbLAvfaRxSE8WS19N0usfvFply9ga7ttsq8dD5lFBWYexjkeyJQDdJCUr BVvWMWMbcv3wD9EkJ9do+4KlXGy+VA6trmqt/9YZX6mOqPRi7gJRVPQX5hGKy3veG3sj RL6cOIV6Sw9njNTcyQuQRoMjaMYmxvbFzYfDdwAzT8A4hVYD0UfGy3y1FlT31zQ8UowL AZi0ZZ6DhH3IsSJrSs0yNU7qqoi9Pzp9phcOr8d9Hiv4buBPEvo8ffStso7Um0vt+vhx yn7w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=HzNs1fLS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b6-20020a170903228600b001b5395382a0si1155518plh.212.2023.06.16.11.55.10; Fri, 16 Jun 2023 11:55:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=HzNs1fLS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245082AbjFPS14 (ORCPT + 99 others); Fri, 16 Jun 2023 14:27:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52118 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232355AbjFPS1y (ORCPT ); Fri, 16 Jun 2023 14:27:54 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1991A35AC; Fri, 16 Jun 2023 11:27:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686940073; x=1718476073; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ucPC+PBVv9p/o/aEzjjq4yEhQH8Zxrop8RvU9O2oRok=; b=HzNs1fLSuocsAGf67kcrfZdRgtjAkJ2w7J7nP7WQWFw8X3BCuce57jd5 kHsESRWg85xp+vUmL2ZcWt+nic6YAVku+aGOYaOhpKbH2+iT5c34NRDbP 0jT7GyGfrOXQO8NzGY3iN1jDGgvJFXM37580huv5nE/8npE8MICvulh2F GOLA0BIFzgWgDANrOCvK/IFwMynQ0VFpyeQlk3VTFvKDF0j/VWTxcmkpZ Ysgk9iPot6HHDe7H7IX3bLZzK61WoKIhb99PBj0eZh3/DUu1wUE7y7rpo 9lhV+Cn4BmlZzNSh34cKvN4wSE9aAn9ZqI9fcFe8q9dwh9waDtlVp6E3K Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10743"; a="361815187" X-IronPort-AV: E=Sophos;i="6.00,248,1681196400"; d="scan'208";a="361815187" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jun 2023 11:27:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10743"; a="783018166" X-IronPort-AV: E=Sophos;i="6.00,248,1681196400"; d="scan'208";a="783018166" Received: from agluck-desk3.sc.intel.com ([172.25.222.74]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jun 2023 11:27:51 -0700 From: Tony Luck To: Borislav Petkov Cc: Yazen Ghannam , Smita.KoralahalliChannabasappa@amd.com, dave.hansen@linux.intel.com, x86@kernel.org, linux-edac@vger.kernel.org, linux-kernel@vger.kernel.org, patches@lists.linux.dev, Tony Luck Subject: [PATCH v6 1/4] x86/mce: Remove old CMCI storm mitigation code Date: Fri, 16 Jun 2023 11:27:41 -0700 Message-Id: <20230616182744.17632-2-tony.luck@intel.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616182744.17632-1-tony.luck@intel.com> References: <20230411173841.70491-1-tony.luck@intel.com> <20230616182744.17632-1-tony.luck@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762190929039148390?= X-GMAIL-MSGID: =?utf-8?q?1768886604193991917?= When a "storm" of CMCI is detected this code mitigates by disabling CMCI interrupt signalling from all of the banks owned by the CPU that saw the storm. There are problems with this approach: 1) It is very coarse grained. In all likelihood only one of the banks was generating the interrupts, but CMCI is disabled for all. This means Linux may delay seeing and processing errors logged from other banks. 2) Although CMCI stands for Corrected Machine Check Interrupt, it is also used to signal when an uncorrected error is logged. This is a problem because these errors should be handled in a timely manner. Delete all this code in preparation for a finer grained solution. Reviewed-by: Yazen Ghannam Tested-by: Yazen Ghannam Signed-off-by: Tony Luck --- arch/x86/kernel/cpu/mce/internal.h | 6 -- arch/x86/kernel/cpu/mce/core.c | 20 +--- arch/x86/kernel/cpu/mce/intel.c | 145 ----------------------------- 3 files changed, 1 insertion(+), 170 deletions(-) diff --git a/arch/x86/kernel/cpu/mce/internal.h b/arch/x86/kernel/cpu/mce/internal.h index d2412ce2d312..9dcad55835fa 100644 --- a/arch/x86/kernel/cpu/mce/internal.h +++ b/arch/x86/kernel/cpu/mce/internal.h @@ -41,18 +41,12 @@ struct dentry *mce_get_debugfs_dir(void); extern mce_banks_t mce_banks_ce_disabled; #ifdef CONFIG_X86_MCE_INTEL -unsigned long cmci_intel_adjust_timer(unsigned long interval); -bool mce_intel_cmci_poll(void); -void mce_intel_hcpu_update(unsigned long cpu); void cmci_disable_bank(int bank); void intel_init_cmci(void); void intel_init_lmce(void); void intel_clear_lmce(void); bool intel_filter_mce(struct mce *m); #else -# define cmci_intel_adjust_timer mce_adjust_timer_default -static inline bool mce_intel_cmci_poll(void) { return false; } -static inline void mce_intel_hcpu_update(unsigned long cpu) { } static inline void cmci_disable_bank(int bank) { } static inline void intel_init_cmci(void) { } static inline void intel_init_lmce(void) { } diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c index 2eec60f50057..e7936be84204 100644 --- a/arch/x86/kernel/cpu/mce/core.c +++ b/arch/x86/kernel/cpu/mce/core.c @@ -1588,13 +1588,6 @@ static unsigned long check_interval = INITIAL_CHECK_INTERVAL; static DEFINE_PER_CPU(unsigned long, mce_next_interval); /* in jiffies */ static DEFINE_PER_CPU(struct timer_list, mce_timer); -static unsigned long mce_adjust_timer_default(unsigned long interval) -{ - return interval; -} - -static unsigned long (*mce_adjust_timer)(unsigned long interval) = mce_adjust_timer_default; - static void __start_timer(struct timer_list *t, unsigned long interval) { unsigned long when = jiffies + interval; @@ -1617,15 +1610,9 @@ static void mce_timer_fn(struct timer_list *t) iv = __this_cpu_read(mce_next_interval); - if (mce_available(this_cpu_ptr(&cpu_info))) { + if (mce_available(this_cpu_ptr(&cpu_info))) machine_check_poll(0, this_cpu_ptr(&mce_poll_banks)); - if (mce_intel_cmci_poll()) { - iv = mce_adjust_timer(iv); - goto done; - } - } - /* * Alert userspace if needed. If we logged an MCE, reduce the polling * interval, otherwise increase the polling interval. @@ -1635,7 +1622,6 @@ static void mce_timer_fn(struct timer_list *t) else iv = min(iv * 2, round_jiffies_relative(check_interval * HZ)); -done: __this_cpu_write(mce_next_interval, iv); __start_timer(t, iv); } @@ -1972,7 +1958,6 @@ static void mce_zhaoxin_feature_init(struct cpuinfo_x86 *c) intel_init_cmci(); intel_init_lmce(); - mce_adjust_timer = cmci_intel_adjust_timer; } static void mce_zhaoxin_feature_clear(struct cpuinfo_x86 *c) @@ -1985,7 +1970,6 @@ static void __mcheck_cpu_init_vendor(struct cpuinfo_x86 *c) switch (c->x86_vendor) { case X86_VENDOR_INTEL: mce_intel_feature_init(c); - mce_adjust_timer = cmci_intel_adjust_timer; break; case X86_VENDOR_AMD: { @@ -2642,8 +2626,6 @@ static void mce_reenable_cpu(void) static int mce_cpu_dead(unsigned int cpu) { - mce_intel_hcpu_update(cpu); - /* intentionally ignoring frozen here */ if (!cpuhp_tasks_frozen) cmci_rediscover(); diff --git a/arch/x86/kernel/cpu/mce/intel.c b/arch/x86/kernel/cpu/mce/intel.c index 95275a5e57e0..052bf2708391 100644 --- a/arch/x86/kernel/cpu/mce/intel.c +++ b/arch/x86/kernel/cpu/mce/intel.c @@ -41,15 +41,6 @@ */ static DEFINE_PER_CPU(mce_banks_t, mce_banks_owned); -/* - * CMCI storm detection backoff counter - * - * During storm, we reset this counter to INITIAL_CHECK_INTERVAL in case we've - * encountered an error. If not, we decrement it by one. We signal the end of - * the CMCI storm when it reaches 0. - */ -static DEFINE_PER_CPU(int, cmci_backoff_cnt); - /* * cmci_discover_lock protects against parallel discovery attempts * which could race against each other. @@ -57,21 +48,6 @@ static DEFINE_PER_CPU(int, cmci_backoff_cnt); static DEFINE_RAW_SPINLOCK(cmci_discover_lock); #define CMCI_THRESHOLD 1 -#define CMCI_POLL_INTERVAL (30 * HZ) -#define CMCI_STORM_INTERVAL (HZ) -#define CMCI_STORM_THRESHOLD 15 - -static DEFINE_PER_CPU(unsigned long, cmci_time_stamp); -static DEFINE_PER_CPU(unsigned int, cmci_storm_cnt); -static DEFINE_PER_CPU(unsigned int, cmci_storm_state); - -enum { - CMCI_STORM_NONE, - CMCI_STORM_ACTIVE, - CMCI_STORM_SUBSIDED, -}; - -static atomic_t cmci_storm_on_cpus; static int cmci_supported(int *banks) { @@ -127,124 +103,6 @@ static bool lmce_supported(void) return tmp & FEAT_CTL_LMCE_ENABLED; } -bool mce_intel_cmci_poll(void) -{ - if (__this_cpu_read(cmci_storm_state) == CMCI_STORM_NONE) - return false; - - /* - * Reset the counter if we've logged an error in the last poll - * during the storm. - */ - if (machine_check_poll(0, this_cpu_ptr(&mce_banks_owned))) - this_cpu_write(cmci_backoff_cnt, INITIAL_CHECK_INTERVAL); - else - this_cpu_dec(cmci_backoff_cnt); - - return true; -} - -void mce_intel_hcpu_update(unsigned long cpu) -{ - if (per_cpu(cmci_storm_state, cpu) == CMCI_STORM_ACTIVE) - atomic_dec(&cmci_storm_on_cpus); - - per_cpu(cmci_storm_state, cpu) = CMCI_STORM_NONE; -} - -static void cmci_toggle_interrupt_mode(bool on) -{ - unsigned long flags, *owned; - int bank; - u64 val; - - raw_spin_lock_irqsave(&cmci_discover_lock, flags); - owned = this_cpu_ptr(mce_banks_owned); - for_each_set_bit(bank, owned, MAX_NR_BANKS) { - rdmsrl(MSR_IA32_MCx_CTL2(bank), val); - - if (on) - val |= MCI_CTL2_CMCI_EN; - else - val &= ~MCI_CTL2_CMCI_EN; - - wrmsrl(MSR_IA32_MCx_CTL2(bank), val); - } - raw_spin_unlock_irqrestore(&cmci_discover_lock, flags); -} - -unsigned long cmci_intel_adjust_timer(unsigned long interval) -{ - if ((this_cpu_read(cmci_backoff_cnt) > 0) && - (__this_cpu_read(cmci_storm_state) == CMCI_STORM_ACTIVE)) { - mce_notify_irq(); - return CMCI_STORM_INTERVAL; - } - - switch (__this_cpu_read(cmci_storm_state)) { - case CMCI_STORM_ACTIVE: - - /* - * We switch back to interrupt mode once the poll timer has - * silenced itself. That means no events recorded and the timer - * interval is back to our poll interval. - */ - __this_cpu_write(cmci_storm_state, CMCI_STORM_SUBSIDED); - if (!atomic_sub_return(1, &cmci_storm_on_cpus)) - pr_notice("CMCI storm subsided: switching to interrupt mode\n"); - - fallthrough; - - case CMCI_STORM_SUBSIDED: - /* - * We wait for all CPUs to go back to SUBSIDED state. When that - * happens we switch back to interrupt mode. - */ - if (!atomic_read(&cmci_storm_on_cpus)) { - __this_cpu_write(cmci_storm_state, CMCI_STORM_NONE); - cmci_toggle_interrupt_mode(true); - cmci_recheck(); - } - return CMCI_POLL_INTERVAL; - default: - - /* We have shiny weather. Let the poll do whatever it thinks. */ - return interval; - } -} - -static bool cmci_storm_detect(void) -{ - unsigned int cnt = __this_cpu_read(cmci_storm_cnt); - unsigned long ts = __this_cpu_read(cmci_time_stamp); - unsigned long now = jiffies; - int r; - - if (__this_cpu_read(cmci_storm_state) != CMCI_STORM_NONE) - return true; - - if (time_before_eq(now, ts + CMCI_STORM_INTERVAL)) { - cnt++; - } else { - cnt = 1; - __this_cpu_write(cmci_time_stamp, now); - } - __this_cpu_write(cmci_storm_cnt, cnt); - - if (cnt <= CMCI_STORM_THRESHOLD) - return false; - - cmci_toggle_interrupt_mode(false); - __this_cpu_write(cmci_storm_state, CMCI_STORM_ACTIVE); - r = atomic_add_return(1, &cmci_storm_on_cpus); - mce_timer_kick(CMCI_STORM_INTERVAL); - this_cpu_write(cmci_backoff_cnt, INITIAL_CHECK_INTERVAL); - - if (r == 1) - pr_notice("CMCI storm detected: switching to poll mode\n"); - return true; -} - /* * The interrupt handler. This is called on every event. * Just call the poller directly to log any events. @@ -253,9 +111,6 @@ static bool cmci_storm_detect(void) */ static void intel_threshold_interrupt(void) { - if (cmci_storm_detect()) - return; - machine_check_poll(MCP_TIMESTAMP, this_cpu_ptr(&mce_banks_owned)); } From patchwork Fri Jun 16 18:27:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Luck, Tony" X-Patchwork-Id: 109295 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp1540095vqr; Fri, 16 Jun 2023 11:33:31 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4nPLXVdzpJGIit/KucdwRu0k5fx3zQnqeVWJ0pJv1f1SNyw4KOKDikoqcH1rvjZ9XYTXao X-Received: by 2002:a05:6a00:1483:b0:665:7f27:dbab with SMTP id v3-20020a056a00148300b006657f27dbabmr3352529pfu.25.1686940410822; Fri, 16 Jun 2023 11:33:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686940410; cv=none; d=google.com; s=arc-20160816; b=t/F67cGl2e4ggf32QfFP9EesfTWKnDhJv/JyPyQ4quXYjqIKkLT+MUV9KFcYiklq1I EMF6UhFaD48iEqMFRXzgHCcyQ0UI8iXYG06JV01Lw8seLJu07WZe8lU48BZytkjXSlvm HgiuN8SWnBx1QibajbGNlTACyPyvkRYZdyXaWeJZgneqhuDhhn9/rI02FvdTeInCtsxP gn4IiaqfupX9Jn+YqimfNuzI3T195QBbjiRKDg+BgMGpxRnkMIOTbp+rhGWMhqPVqfeb BEcSuyWZmqQpgSjkZxOGkaD0cO7vMAQ2qO6CivKf8he9m1ex7H4WqEkGNvFDvT0XD+/g G91A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ytCnozJisHnW1IIzeoNPO1r7yafUBUV/rRj3A7YQf7Q=; b=ct+dTnv6ED8aM10T8XMzu8S3giwndm8c+4iyP4FkzivZqrQ4e82V0nJDDFO0jkSFxk 2rsYz67E7oKVaR4GK2hnHS4Y1E872j4Br50kFAElHISOWZguCbJ+XEMHFRNYMa6sx91Q uYojv1cKEw/YVq68W2SzSsl3YNVx52ratk8daDFhQZG8dXP0QMNunCtrqwALyrqPAQIW OzEejP1grA6V8XzQdwmGtBhRHvWv3B7FInHygZ8JVHUCaobtE01yabzvPcXxenj2FYgv MHw/dpeFN5DpMriIIYBfdWjHT1jpJW9vmvx3hEA8PQbF8bFTcTuPJBxtx+ZIUWQyiCbj 3rvQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Y6sK4ICd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l70-20020a639149000000b0055383df74c7si1043344pge.224.2023.06.16.11.33.17; Fri, 16 Jun 2023 11:33:30 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Y6sK4ICd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344544AbjFPS17 (ORCPT + 99 others); Fri, 16 Jun 2023 14:27:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52138 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231865AbjFPS1z (ORCPT ); Fri, 16 Jun 2023 14:27:55 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4F6EA35AD; Fri, 16 Jun 2023 11:27:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686940073; x=1718476073; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=saXdChlUx9A9/qRbX5PWkqsKLR3duIy530rHq98coDc=; b=Y6sK4ICd6XW01L2wrzmIFxMF6/MNZ4+W8DaCUku3BFLdOsVOTgppY9Se W/WaJrY/L4Ie6+X4DELhZf0ImyA3LjcFEaeKH2WWCm4so+Z7ODd1AWkiR w8DjL+DOKldGiRW5Vp3VwybJNQH226m4SCdq3yArkc2MkmQy+ocXSzvjh hXNh9k/Q3oXlWdSKNzkkyQte2TmlUWZlyxZAaC+M9Qht8Kr/RgKHFsbAL j/E4rNrOQMe/6/pYKSpD1yRJMdi3grPaZq2o+hkZZSyghJWRMfPRYrVd2 ryNgWW5IyI/1pF748/86347dQnvCfkUAKIWsJOChaxH1TgH7IHugD3Ix9 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10743"; a="361815192" X-IronPort-AV: E=Sophos;i="6.00,248,1681196400"; d="scan'208";a="361815192" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jun 2023 11:27:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10743"; a="783018172" X-IronPort-AV: E=Sophos;i="6.00,248,1681196400"; d="scan'208";a="783018172" Received: from agluck-desk3.sc.intel.com ([172.25.222.74]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jun 2023 11:27:51 -0700 From: Tony Luck To: Borislav Petkov Cc: Yazen Ghannam , Smita.KoralahalliChannabasappa@amd.com, dave.hansen@linux.intel.com, x86@kernel.org, linux-edac@vger.kernel.org, linux-kernel@vger.kernel.org, patches@lists.linux.dev, Tony Luck Subject: [PATCH v6 2/4] x86/mce: Add per-bank CMCI storm mitigation Date: Fri, 16 Jun 2023 11:27:42 -0700 Message-Id: <20230616182744.17632-3-tony.luck@intel.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616182744.17632-1-tony.luck@intel.com> References: <20230411173841.70491-1-tony.luck@intel.com> <20230616182744.17632-1-tony.luck@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762191557205294420?= X-GMAIL-MSGID: =?utf-8?q?1768885227786477233?= This is the core functionality to track CMCI storms at the machine check bank granularity. Subsequent patches will add the vendor specific hooks to supply input to the storm detection and take actions on the start/end of a storm. Maintain a bitmap history for each bank showing whether the bank logged an corrected error or not each time it is polled. In normal operation the interval between polls of this banks determines how far to shift the history. The 64 bit width corresponds to about one second. When a storm is observed a CPU vendor specific action is taken to reduce or stop CMCI from the bank that is the source of the storm. The bank is added to the bitmap of banks for this CPU to poll. The polling rate is increased to once per second. During a storm each bit in the history indicates the status of the bank each time it is polled. Thus the history covers just over a minute. Declare a storm for that bank if the number of corrected interrupts seen in that history is above some threshold (defined as 5 in this series, could be tuned later if there is data to suggest a better value). A storm on a bank ends if enough consecutive polls of the bank show no corrected errors (defined as 30, may also change). That calls the CPU vendor specific function to revert to normal operational mode, and changes the polling rate back to the default. Signed-off-by: Tony Luck --- arch/x86/kernel/cpu/mce/internal.h | 33 ++++++++- arch/x86/kernel/cpu/mce/core.c | 109 ++++++++++++++++++++++++++--- 2 files changed, 133 insertions(+), 9 deletions(-) diff --git a/arch/x86/kernel/cpu/mce/internal.h b/arch/x86/kernel/cpu/mce/internal.h index 9dcad55835fa..eae88a824d97 100644 --- a/arch/x86/kernel/cpu/mce/internal.h +++ b/arch/x86/kernel/cpu/mce/internal.h @@ -40,6 +40,8 @@ struct dentry *mce_get_debugfs_dir(void); extern mce_banks_t mce_banks_ce_disabled; +void track_cmci_storm(int bank, u64 status); + #ifdef CONFIG_X86_MCE_INTEL void cmci_disable_bank(int bank); void intel_init_cmci(void); @@ -54,7 +56,36 @@ static inline void intel_clear_lmce(void) { } static inline bool intel_filter_mce(struct mce *m) { return false; } #endif -void mce_timer_kick(unsigned long interval); +void mce_timer_kick(bool storm); +void mce_handle_storm(int bank, bool on); +void cmci_storm_begin(int bank); +void cmci_storm_end(int bank); + +/** + * struct mca_storm_desc - CMCI storm tracking data + * @stormy_bank_count: count of MC banks in storm state + * @bank_history: bitmask tracking of corrected errors seen in each bank + * @bank_storm: determines whether the bank is in storm mode + * @bank_time_stamp: last time (in jiffies) that each bank was polled + */ +struct mca_storm_desc { + int stormy_bank_count; + u64 bank_history[MAX_NR_BANKS]; + bool bank_storm[MAX_NR_BANKS]; + unsigned long bank_time_stamp[MAX_NR_BANKS]; +}; +DECLARE_PER_CPU(struct mca_storm_desc, storm_desc); + +/* + * How many errors within the history buffer mark the start of a storm + */ +#define STORM_BEGIN_THRESHOLD 5 + +/* + * How many polls of machine check bank without an error before declaring + * the storm is over + */ +#define STORM_END_POLL_THRESHOLD 30 #ifdef CONFIG_ACPI_APEI int apei_write_mce(struct mce *m); diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c index e7936be84204..cd9d9ea5bb0a 100644 --- a/arch/x86/kernel/cpu/mce/core.c +++ b/arch/x86/kernel/cpu/mce/core.c @@ -607,6 +607,83 @@ static struct notifier_block mce_default_nb = { .priority = MCE_PRIO_LOWEST, }; +DEFINE_PER_CPU(struct mca_storm_desc, storm_desc); + +void cmci_storm_begin(int bank) +{ + struct mca_storm_desc *storm = this_cpu_ptr(&storm_desc); + + __set_bit(bank, this_cpu_ptr(mce_poll_banks)); + storm->bank_storm[bank] = true; + + /* + * If this is the first bank on this CPU to enter storm mode + * start polling + */ + if (++storm->stormy_bank_count == 1) + mce_timer_kick(true); +} + +void cmci_storm_end(int bank) +{ + struct mca_storm_desc *storm = this_cpu_ptr(&storm_desc); + + __clear_bit(bank, this_cpu_ptr(mce_poll_banks)); + storm->bank_history[bank] = 0ull; + storm->bank_storm[bank] = false; + + /* If no banks left in storm mode, stop polling */ + if (!this_cpu_dec_return(storm_desc.stormy_bank_count)) + mce_timer_kick(false); +} + +void track_cmci_storm(int bank, u64 status) +{ + struct mca_storm_desc *storm = this_cpu_ptr(&storm_desc); + unsigned long now = jiffies, delta; + unsigned int shift = 1; + u64 history; + + /* + * When a bank is in storm mode it is polled once per second and + * the history mask will record about the last minute of poll results. + * If it is not in storm mode, then the bank is only checked when + * there is a CMCI interrupt. Check how long it has been since + * this bank was last checked, and adjust the amount of "shift" + * to apply to history. + */ + if (!storm->bank_storm[bank]) { + delta = now - storm->bank_time_stamp[bank]; + shift = (delta + HZ) / HZ; + } + + /* If has been a long time since the last poll, clear history */ + if (shift >= 64) + history = 0; + else + history = storm->bank_history[bank] << shift; + storm->bank_time_stamp[bank] = now; + + /* History keeps track of corrected errors. VAL=1 && UC=0 */ + if ((status & (MCI_STATUS_VAL | MCI_STATUS_UC)) == MCI_STATUS_VAL) + history |= 1; + storm->bank_history[bank] = history; + + if (storm->bank_storm[bank]) { + if (history & GENMASK_ULL(STORM_END_POLL_THRESHOLD - 1, 0)) + return; + pr_notice("CPU%d BANK%d CMCI storm subsided\n", smp_processor_id(), bank); + mce_handle_storm(bank, false); + cmci_storm_end(bank); + } else { + if (hweight64(history) < STORM_BEGIN_THRESHOLD) + return; + pr_notice("CPU%d BANK%d CMCI storm detected\n", smp_processor_id(), bank); + mce_handle_storm(bank, true); + cmci_storm_begin(bank); + } +} + /* * Read ADDR and MISC registers. */ @@ -680,6 +757,8 @@ bool machine_check_poll(enum mcp_flags flags, mce_banks_t *b) barrier(); m.status = mce_rdmsrl(mca_msr_reg(i, MCA_STATUS)); + track_cmci_storm(i, m.status); + /* If this entry is not valid, ignore it */ if (!(m.status & MCI_STATUS_VAL)) continue; @@ -1587,6 +1666,7 @@ static unsigned long check_interval = INITIAL_CHECK_INTERVAL; static DEFINE_PER_CPU(unsigned long, mce_next_interval); /* in jiffies */ static DEFINE_PER_CPU(struct timer_list, mce_timer); +static DEFINE_PER_CPU(bool, storm_poll_mode); static void __start_timer(struct timer_list *t, unsigned long interval) { @@ -1622,22 +1702,29 @@ static void mce_timer_fn(struct timer_list *t) else iv = min(iv * 2, round_jiffies_relative(check_interval * HZ)); - __this_cpu_write(mce_next_interval, iv); - __start_timer(t, iv); + if (__this_cpu_read(storm_poll_mode)) { + __start_timer(t, HZ); + } else { + __this_cpu_write(mce_next_interval, iv); + __start_timer(t, iv); + } } /* - * Ensure that the timer is firing in @interval from now. + * When a storm starts on any bank on this CPU, switch to polling + * once per second. When the storm ends, revert to the default + * polling interval. */ -void mce_timer_kick(unsigned long interval) +void mce_timer_kick(bool storm) { struct timer_list *t = this_cpu_ptr(&mce_timer); - unsigned long iv = __this_cpu_read(mce_next_interval); - __start_timer(t, interval); + __this_cpu_write(storm_poll_mode, storm); - if (interval < iv) - __this_cpu_write(mce_next_interval, interval); + if (storm) + __start_timer(t, HZ); + else + __this_cpu_write(mce_next_interval, check_interval * HZ); } /* Must not be called in IRQ context where del_timer_sync() can deadlock */ @@ -1965,6 +2052,12 @@ static void mce_zhaoxin_feature_clear(struct cpuinfo_x86 *c) intel_clear_lmce(); } +void mce_handle_storm(int bank, bool on) +{ + switch (boot_cpu_data.x86_vendor) { + } +} + static void __mcheck_cpu_init_vendor(struct cpuinfo_x86 *c) { switch (c->x86_vendor) { From patchwork Fri Jun 16 18:27:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Luck, Tony" X-Patchwork-Id: 109298 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp1550006vqr; Fri, 16 Jun 2023 11:54:48 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5PoGLAFNlhxf5AbRYjIH45DWq488umW3cQx1H3elRxAEO0k3gvPqtbfRCDMfxpEkdMV065 X-Received: by 2002:a05:6a00:2d07:b0:641:39cb:1716 with SMTP id fa7-20020a056a002d0700b0064139cb1716mr3576142pfb.20.1686941687885; Fri, 16 Jun 2023 11:54:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686941687; cv=none; d=google.com; s=arc-20160816; b=h4SfhxsIfbeXna3kS1NmkUj1vM73Ih9A2Gmzo4XRPdBweNj0osrs03xapSKRwuZ6qx U252SoqikO0uDrm0TYgmjNFWCOEdH4BF9Lf0yLggj32dsn/7nk4UxD7/KwKuR+H1uZ6U 65WtG5xpF7FKM3/Q79Zyfr/8RhdEqYpJ/wH/e5pgqxMcuLojdNhP48zb8R8oexd6F5C2 c11OnpCLJcsEkmhaxDRM9WvALpgbyNM1uIeVgB9rurWR6LIi4UjHos3CqQ1cn6HbNWJ/ gOLzH1s7CdYYoj2hEHiVc4PR0hQtqB/ASXzzZ0zlht7vNYEW4b+34jTOlwPFPPxfe3Ls n7TA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=WRyYdkfyfB/9ORwqKdXXDwPQP99PSDaWw5+88bchAgI=; b=gPB/cXGqBeQjFmOiodMbt+yEYpAXDoS2OvnCS5yTZIu2PFLsHl1YrmC0xWEfhXU9fe o4UJDAywDKN+etR9l9b4m5xS56kMcJg1GYlCBJc4NvubKwYcY5cwG1npjBHuM5CjDbbo zR3C+W0nOB+74n5XCfQOxZAE+wMjhkZNqXIXUa0yQIUYYh1kX3UMxIc69AVAtwURRUfj uqWqalzteIGIbW+st1m4Mu6Ijx7oysBmAG+igRKCKw6UW49Va7YsEXqrKalBvMv9WHwB ubmoNb9fl+77i9p8xGc52kry2bMQRlU0rf6LAsiHSh18XQTqA9hWrK6R/rQFEk1/U0gu dDew== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=VA5DXYVG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s6-20020a625e06000000b006562b3c3c33si12076629pfb.78.2023.06.16.11.54.35; Fri, 16 Jun 2023 11:54:47 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=VA5DXYVG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344607AbjFPS2E (ORCPT + 99 others); Fri, 16 Jun 2023 14:28:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52140 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229705AbjFPS1z (ORCPT ); Fri, 16 Jun 2023 14:27:55 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6EBFD35AB; Fri, 16 Jun 2023 11:27:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686940074; x=1718476074; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zxtEr+49Z8SAPXXw+VLSmKO/k0+mLZ8hYyIUwukrFTo=; b=VA5DXYVGxNPxlswh6cAFi17GZfVvbKo+t3G1CbZJHCYyx72uwd5zhFAP tx/TWcf7ry4AIHAFYHj6cJrunAg1Mk6lphQQDP5ESFU6DtRMfusrxi+sC R4ltbnhphJryIIepbe+yaoUgzhh835HyCF/VqjaMvNOBBDam2e48DYfre 4mxMFq9j5AomJaTcN9TUWk1/lHP9BMyVd4Kl6v1tEzMNV63cVP/SI+gT7 TLG6ki9ADlvsJh3ZxOpjMwpUblAcidD//olpHuPtAizEqtdQNcfH6PR5S a3jHXotkC5YNDgQKqHijmcRsj2LW+uI4IwHoDb8JahNZFD4/DzA82DOMN A==; X-IronPort-AV: E=McAfee;i="6600,9927,10743"; a="361815197" X-IronPort-AV: E=Sophos;i="6.00,248,1681196400"; d="scan'208";a="361815197" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jun 2023 11:27:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10743"; a="783018175" X-IronPort-AV: E=Sophos;i="6.00,248,1681196400"; d="scan'208";a="783018175" Received: from agluck-desk3.sc.intel.com ([172.25.222.74]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jun 2023 11:27:51 -0700 From: Tony Luck To: Borislav Petkov Cc: Yazen Ghannam , Smita.KoralahalliChannabasappa@amd.com, dave.hansen@linux.intel.com, x86@kernel.org, linux-edac@vger.kernel.org, linux-kernel@vger.kernel.org, patches@lists.linux.dev, Tony Luck Subject: [PATCH v6 3/4] x86/mce: Handle AMD threshold interrupt storms Date: Fri, 16 Jun 2023 11:27:43 -0700 Message-Id: <20230616182744.17632-4-tony.luck@intel.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616182744.17632-1-tony.luck@intel.com> References: <20230411173841.70491-1-tony.luck@intel.com> <20230616182744.17632-1-tony.luck@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762191595436347208?= X-GMAIL-MSGID: =?utf-8?q?1768886567612617117?= From: Smita Koralahalli Add hook into core storm handling code for AMD threshold interrupts. Disable the interrupt on the corresponding CPU and bank. Re-enable back the interrupts if enough consecutive polls of the bank show no corrected errors. Turning off the threshold interrupts is the best solution on AMD systems as other error severities will still be handled even if the threshold interrupts are disabled. [Tony: Updated places where storm tracking variables moved into a structure] Signed-off-by: Smita Koralahalli Signed-off-by: Tony Luck --- arch/x86/kernel/cpu/mce/internal.h | 2 ++ arch/x86/kernel/cpu/mce/amd.c | 49 ++++++++++++++++++++++++++++++ arch/x86/kernel/cpu/mce/core.c | 3 ++ 3 files changed, 54 insertions(+) diff --git a/arch/x86/kernel/cpu/mce/internal.h b/arch/x86/kernel/cpu/mce/internal.h index eae88a824d97..22899d28138f 100644 --- a/arch/x86/kernel/cpu/mce/internal.h +++ b/arch/x86/kernel/cpu/mce/internal.h @@ -232,6 +232,7 @@ extern bool filter_mce(struct mce *m); #ifdef CONFIG_X86_MCE_AMD extern bool amd_filter_mce(struct mce *m); +void mce_amd_handle_storm(int bank, bool on); /* * If MCA_CONFIG[McaLsbInStatusSupported] is set, extract ErrAddr in bits @@ -259,6 +260,7 @@ static __always_inline void smca_extract_err_addr(struct mce *m) #else static inline bool amd_filter_mce(struct mce *m) { return false; } +static inline void mce_amd_handle_storm(int bank, bool on) {} static inline void smca_extract_err_addr(struct mce *m) { } #endif diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c index 0b971f974096..b19f3eb70187 100644 --- a/arch/x86/kernel/cpu/mce/amd.c +++ b/arch/x86/kernel/cpu/mce/amd.c @@ -468,6 +468,47 @@ static void threshold_restart_bank(void *_tr) wrmsr(tr->b->address, lo, hi); } +static void _reset_block(struct threshold_block *block) +{ + struct thresh_restart tr; + + memset(&tr, 0, sizeof(tr)); + tr.b = block; + threshold_restart_bank(&tr); +} + +static void toggle_interrupt_reset_block(struct threshold_block *block, bool on) +{ + if (!block) + return; + + block->interrupt_enable = !!on; + _reset_block(block); +} + +void mce_amd_handle_storm(int bank, bool on) +{ + struct threshold_block *first_block = NULL, *block = NULL, *tmp = NULL; + struct threshold_bank **bp = this_cpu_read(threshold_banks); + unsigned long flags; + + if (!bp) + return; + + local_irq_save(flags); + + first_block = bp[bank]->blocks; + if (!first_block) + goto end; + + toggle_interrupt_reset_block(first_block, on); + + list_for_each_entry_safe(block, tmp, &first_block->miscj, miscj) + toggle_interrupt_reset_block(block, on); +end: + local_irq_restore(flags); +} + static void mce_threshold_block_init(struct threshold_block *b, int offset) { struct thresh_restart tr = { @@ -868,6 +909,7 @@ static void amd_threshold_interrupt(void) struct threshold_block *first_block = NULL, *block = NULL, *tmp = NULL; struct threshold_bank **bp = this_cpu_read(threshold_banks); unsigned int bank, cpu = smp_processor_id(); + u64 status; /* * Validate that the threshold bank has been initialized already. The @@ -881,6 +923,13 @@ static void amd_threshold_interrupt(void) if (!(per_cpu(bank_map, cpu) & BIT_ULL(bank))) continue; + rdmsrl(mca_msr_reg(bank, MCA_STATUS), status); + track_cmci_storm(bank, status); + + /* Return early on an interrupt storm */ + if (this_cpu_read(storm_desc.bank_storm[bank])) + return; + first_block = bp[bank]->blocks; if (!first_block) continue; diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c index cd9d9ea5bb0a..d4c9dc194d56 100644 --- a/arch/x86/kernel/cpu/mce/core.c +++ b/arch/x86/kernel/cpu/mce/core.c @@ -2055,6 +2055,9 @@ static void mce_zhaoxin_feature_clear(struct cpuinfo_x86 *c) void mce_handle_storm(int bank, bool on) { switch (boot_cpu_data.x86_vendor) { + case X86_VENDOR_AMD: + mce_amd_handle_storm(bank, on); + break; } } From patchwork Fri Jun 16 18:27:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Luck, Tony" X-Patchwork-Id: 109297 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp1549584vqr; Fri, 16 Jun 2023 11:53:50 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5SuY+n/HlHclgRQ8txUXwuEwJ70dvW0QcE70i/+bU1bXJ8I0BS1ToXouHa5Ww/i/0/P0/1 X-Received: by 2002:a05:6a00:c83:b0:666:617a:c3b5 with SMTP id a3-20020a056a000c8300b00666617ac3b5mr3302448pfv.21.1686941629800; Fri, 16 Jun 2023 11:53:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686941629; cv=none; d=google.com; s=arc-20160816; b=DaMu8xq7/QiRJ9FU60O1c5/uhhlck/IOkv0FyDoKq+ifg7Dq30X39exhWeleVNSK9n 2gX/VhZBC+mbqWZo2FT+Ck4dyd2ECPmhqLfNEKmY0wGy/9BZB3LYAmTjiPTjoaPSS/ZN o0ma+jM18kKCdZ/d9jYSPPEtEnIIajyud5k1ipRMnCAo8y6YtE+JFpXFKsKetXCrfN8e rk3TRH2lTgYt0PQb4F7XcgUa0NZc15ozl3aL+57xp702+GZZn+vdkufHEABhoMBqWU6T Kp1TZxRWlqNK3o8kY9/bNTVK6RcAt16g353zoDmanLXYIkpF3pE93yJJJb6szfHkBI1Y j8Ew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=OG/JAzqL2sGMrKnZ6iqLk4RXz0oKggFIMFoNTyNbnCY=; b=qWL++NVrEqQlJV2ZvXgf8RxQuQ8KjpP/HSidheR5+S8/U/omTFFtzq+VzoG+tDt4gY +zu1+V1RZ92RbBjG5VcsjG6r+21gT4PUcC7sFQWrtpqiFBgetilzQWSs1l5T2PyrSkLb eGeGWGtJumuUBV2348GeRTPx9i3CC30B9FwfpiIYoGGHWv7YXcToXgbCPquB6CC+TWaL dwe7772Xr05l44McPBnkBkn9W0S05UjKVccg5cW2+1KZ+8LqbI8gBUyM1h00MLwjYssg j3rmBaWBnDaAKpaldaanbflPnd48KJXyFDK2ECJ8lPPUUA3XSzovR7bKchbChCEPiq3R HiMQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=KPRWe+s1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u21-20020a627915000000b00666ee06402dsi240875pfc.84.2023.06.16.11.53.33; Fri, 16 Jun 2023 11:53:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=KPRWe+s1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344838AbjFPS2H (ORCPT + 99 others); Fri, 16 Jun 2023 14:28:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52144 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245204AbjFPS14 (ORCPT ); Fri, 16 Jun 2023 14:27:56 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E8963A89; Fri, 16 Jun 2023 11:27:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686940074; x=1718476074; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zEVnmX99I2MJ6rQrCRB0AQ3FXUujATL09To6bcMprnE=; b=KPRWe+s1M2plJGfan4TfB+8NKiCGPZsQs7DcZNaF34asM+vmlgzRxi1d mPf3iPoaJM9LTV1Hd2O0NZI+jGmdCbNqztjX7ESKOLHMCC+kMxTJMFK0A aW89UdiXJQJ4l2L/4RCx45bR+hJlHSMaaxCVIKV8VImekD5+ZzOeTK9Dx parPL1H68yFlMVin9NvVBXrQe8CmWTqI+eHlvaCgYkQGImP+lvz8mEtY+ eiPzjXWfe2r9+PEDzNARtmIEA/YQSI42dqj3YBEdBD4t3H3xCJpJBCKob v3Wm2YxBFJf5RqkhruIXdToa8QRzWvj07J5FoYWZKM78fYz6huo9ijFvn Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10743"; a="361815203" X-IronPort-AV: E=Sophos;i="6.00,248,1681196400"; d="scan'208";a="361815203" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jun 2023 11:27:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10743"; a="783018178" X-IronPort-AV: E=Sophos;i="6.00,248,1681196400"; d="scan'208";a="783018178" Received: from agluck-desk3.sc.intel.com ([172.25.222.74]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jun 2023 11:27:51 -0700 From: Tony Luck To: Borislav Petkov Cc: Yazen Ghannam , Smita.KoralahalliChannabasappa@amd.com, dave.hansen@linux.intel.com, x86@kernel.org, linux-edac@vger.kernel.org, linux-kernel@vger.kernel.org, patches@lists.linux.dev, Tony Luck Subject: [PATCH v6 4/4] x86/mce: Handle Intel threshold interrupt storms Date: Fri, 16 Jun 2023 11:27:44 -0700 Message-Id: <20230616182744.17632-5-tony.luck@intel.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230616182744.17632-1-tony.luck@intel.com> References: <20230411173841.70491-1-tony.luck@intel.com> <20230616182744.17632-1-tony.luck@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768886506178437009?= X-GMAIL-MSGID: =?utf-8?q?1768886506178437009?= Add an Intel specific hook into machine_check_poll() to keep track of per-CPU, per-bank corrected error logs (with a stub for the CONFIG_MCE_INTEL=n case). When a storm is observed the Rate of interrupts is reduced by setting a large threshold value for this bank in IA32_MCi_CTL2. This bank is added to the bitmap of banks for this CPU to poll. The polling rate is increased to once per second. When a storm ends reset the threshold in IA32_MCi_CTL2 back to 1, removes the bank from the bitmap for polling, and changes the polling rate back to the default. If a CPU with banks in storm mode is taken offline, the new CPU that inherits ownership of those banks takes over management of storm(s) in the inherited bank(s). The cmci_discover() function was already very large. These changes pushed it well over the top. Refactor with three helper functions to braing it back under control. Signed-off-by: Tony Luck --- arch/x86/kernel/cpu/mce/internal.h | 2 + arch/x86/kernel/cpu/mce/core.c | 3 + arch/x86/kernel/cpu/mce/intel.c | 202 +++++++++++++++++++++-------- 3 files changed, 156 insertions(+), 51 deletions(-) diff --git a/arch/x86/kernel/cpu/mce/internal.h b/arch/x86/kernel/cpu/mce/internal.h index 22899d28138f..1cd112f94028 100644 --- a/arch/x86/kernel/cpu/mce/internal.h +++ b/arch/x86/kernel/cpu/mce/internal.h @@ -43,12 +43,14 @@ extern mce_banks_t mce_banks_ce_disabled; void track_cmci_storm(int bank, u64 status); #ifdef CONFIG_X86_MCE_INTEL +void mce_intel_handle_storm(int bank, bool on); void cmci_disable_bank(int bank); void intel_init_cmci(void); void intel_init_lmce(void); void intel_clear_lmce(void); bool intel_filter_mce(struct mce *m); #else +static inline void mce_intel_handle_storm(int bank, bool on) { } static inline void cmci_disable_bank(int bank) { } static inline void intel_init_cmci(void) { } static inline void intel_init_lmce(void) { } diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c index d4c9dc194d56..bc5137c0c47b 100644 --- a/arch/x86/kernel/cpu/mce/core.c +++ b/arch/x86/kernel/cpu/mce/core.c @@ -2055,6 +2055,9 @@ static void mce_zhaoxin_feature_clear(struct cpuinfo_x86 *c) void mce_handle_storm(int bank, bool on) { switch (boot_cpu_data.x86_vendor) { + case X86_VENDOR_INTEL: + mce_intel_handle_storm(bank, on); + break; case X86_VENDOR_AMD: mce_amd_handle_storm(bank, on); break; diff --git a/arch/x86/kernel/cpu/mce/intel.c b/arch/x86/kernel/cpu/mce/intel.c index 052bf2708391..83bb800e408d 100644 --- a/arch/x86/kernel/cpu/mce/intel.c +++ b/arch/x86/kernel/cpu/mce/intel.c @@ -47,8 +47,27 @@ static DEFINE_PER_CPU(mce_banks_t, mce_banks_owned); */ static DEFINE_RAW_SPINLOCK(cmci_discover_lock); +/* Linux non-storm CMCI threshold (may be overridden by BIOS) */ #define CMCI_THRESHOLD 1 +/* + * MCi_CTL2 threshold for each bank when there is no storm. + * Default value for each bank may have been set by BIOS. + */ +static int cmci_threshold[MAX_NR_BANKS]; + +/* + * High threshold to limit CMCI rate during storms. Max supported is + * 0x7FFF. Use this slightly smaller value so it has a distinctive + * signature when some asks "Why am I not seeing all corrected errors?" + * A high threshold is used instead of just disabling CMCI for a + * bank because both corrected and uncorrected errors may be logged + * in the same bank and signalled with CMCI. The threshold only applies + * to corrected errors, so keeping CMCI enabled means that uncorrected + * errors will still be processed in a timely fashion. + */ +#define CMCI_STORM_THRESHOLD 32749 + static int cmci_supported(int *banks) { u64 cap; @@ -103,6 +122,31 @@ static bool lmce_supported(void) return tmp & FEAT_CTL_LMCE_ENABLED; } +/* + * Set a new CMCI threshold value. Preserve the state of the + * MCI_CTL2_CMCI_EN bit in case this happens during a + * cmci_rediscover() operation. + */ +static void cmci_set_threshold(int bank, int thresh) +{ + unsigned long flags; + u64 val; + + raw_spin_lock_irqsave(&cmci_discover_lock, flags); + rdmsrl(MSR_IA32_MCx_CTL2(bank), val); + val &= ~MCI_CTL2_CMCI_THRESHOLD_MASK; + wrmsrl(MSR_IA32_MCx_CTL2(bank), val | thresh); + raw_spin_unlock_irqrestore(&cmci_discover_lock, flags); +} + +void mce_intel_handle_storm(int bank, bool on) +{ + if (on) + cmci_set_threshold(bank, CMCI_STORM_THRESHOLD); + else + cmci_set_threshold(bank, cmci_threshold[bank]); +} + /* * The interrupt handler. This is called on every event. * Just call the poller directly to log any events. @@ -114,72 +158,126 @@ static void intel_threshold_interrupt(void) machine_check_poll(MCP_TIMESTAMP, this_cpu_ptr(&mce_banks_owned)); } +/* + * Check all the reasons why current CPU cannot claim + * ownership of a bank. + * 1: CPU already owns this bank + * 2: BIOS owns this bank + * 3: Some other CPU owns this bank + */ +static bool cmci_skip_bank(int bank, u64 *val) +{ + unsigned long *owned = (void *)this_cpu_ptr(&mce_banks_owned); + + if (test_bit(bank, owned)) + return true; + + /* Skip banks in firmware first mode */ + if (test_bit(bank, mce_banks_ce_disabled)) + return true; + + rdmsrl(MSR_IA32_MCx_CTL2(bank), *val); + + /* Already owned by someone else? */ + if (*val & MCI_CTL2_CMCI_EN) { + clear_bit(bank, owned); + __clear_bit(bank, this_cpu_ptr(mce_poll_banks)); + return true; + } + + return false; +} + +/* + * Decide which CMCI interrupt threshold to use: + * 1: If this bank is in storm mode from whichever CPU was + * the previous owner, stay in storm mode. + * 2: If ignoring any threshold set by BIOS, set Linux default + * 3: Try to honor BIOS threshold (unless buggy BIOS set it at zero). + */ +static u64 cmci_pick_threshold(u64 val, int *bios_zero_thresh) +{ + if ((val & MCI_CTL2_CMCI_THRESHOLD_MASK) == CMCI_STORM_THRESHOLD) + return val; + + if (!mca_cfg.bios_cmci_threshold) { + val &= ~MCI_CTL2_CMCI_THRESHOLD_MASK; + val |= CMCI_THRESHOLD; + } else if (!(val & MCI_CTL2_CMCI_THRESHOLD_MASK)) { + /* + * If bios_cmci_threshold boot option was specified + * but the threshold is zero, we'll try to initialize + * it to 1. + */ + *bios_zero_thresh = 1; + val |= CMCI_THRESHOLD; + } + + return val; +} + +/* + * Try to claim ownership of a bank. + */ +static void cmci_claim_bank(int bank, u64 val, int bios_zero_thresh, int *bios_wrong_thresh) +{ + struct mca_storm_desc *storm = this_cpu_ptr(&storm_desc); + + val |= MCI_CTL2_CMCI_EN; + wrmsrl(MSR_IA32_MCx_CTL2(bank), val); + rdmsrl(MSR_IA32_MCx_CTL2(bank), val); + + /* Did the enable bit stick? -- the bank supports CMCI */ + if (val & MCI_CTL2_CMCI_EN) { + set_bit(bank, (void *)this_cpu_ptr(&mce_banks_owned)); + if ((val & MCI_CTL2_CMCI_THRESHOLD_MASK) == CMCI_STORM_THRESHOLD) { + pr_notice("CPU%d BANK%d CMCI inherited storm\n", smp_processor_id(), bank); + storm->bank_history[bank] = ~0ull; + storm->bank_time_stamp[bank] = jiffies; + cmci_storm_begin(bank); + } else { + __clear_bit(bank, this_cpu_ptr(mce_poll_banks)); + } + /* + * We are able to set thresholds for some banks that + * had a threshold of 0. This means the BIOS has not + * set the thresholds properly or does not work with + * this boot option. Note down now and report later. + */ + if (mca_cfg.bios_cmci_threshold && bios_zero_thresh && + (val & MCI_CTL2_CMCI_THRESHOLD_MASK)) + *bios_wrong_thresh = 1; + + /* Save default threshold for each bank */ + if (cmci_threshold[bank] == 0) + cmci_threshold[bank] = val & MCI_CTL2_CMCI_THRESHOLD_MASK; + } else { + WARN_ON(!test_bit(bank, this_cpu_ptr(mce_poll_banks))); + } +} + /* * Enable CMCI (Corrected Machine Check Interrupt) for available MCE banks * on this CPU. Use the algorithm recommended in the SDM to discover shared - * banks. + * banks. Called during initial bootstrap, and also for hotplug CPU operations + * to rediscover/reassign machine check banks. */ static void cmci_discover(int banks) { - unsigned long *owned = (void *)this_cpu_ptr(&mce_banks_owned); - unsigned long flags; - int i; int bios_wrong_thresh = 0; + unsigned long flags; + int i; raw_spin_lock_irqsave(&cmci_discover_lock, flags); for (i = 0; i < banks; i++) { u64 val; int bios_zero_thresh = 0; - if (test_bit(i, owned)) + if (cmci_skip_bank(i, &val)) continue; - /* Skip banks in firmware first mode */ - if (test_bit(i, mce_banks_ce_disabled)) - continue; - - rdmsrl(MSR_IA32_MCx_CTL2(i), val); - - /* Already owned by someone else? */ - if (val & MCI_CTL2_CMCI_EN) { - clear_bit(i, owned); - __clear_bit(i, this_cpu_ptr(mce_poll_banks)); - continue; - } - - if (!mca_cfg.bios_cmci_threshold) { - val &= ~MCI_CTL2_CMCI_THRESHOLD_MASK; - val |= CMCI_THRESHOLD; - } else if (!(val & MCI_CTL2_CMCI_THRESHOLD_MASK)) { - /* - * If bios_cmci_threshold boot option was specified - * but the threshold is zero, we'll try to initialize - * it to 1. - */ - bios_zero_thresh = 1; - val |= CMCI_THRESHOLD; - } - - val |= MCI_CTL2_CMCI_EN; - wrmsrl(MSR_IA32_MCx_CTL2(i), val); - rdmsrl(MSR_IA32_MCx_CTL2(i), val); - - /* Did the enable bit stick? -- the bank supports CMCI */ - if (val & MCI_CTL2_CMCI_EN) { - set_bit(i, owned); - __clear_bit(i, this_cpu_ptr(mce_poll_banks)); - /* - * We are able to set thresholds for some banks that - * had a threshold of 0. This means the BIOS has not - * set the thresholds properly or does not work with - * this boot option. Note down now and report later. - */ - if (mca_cfg.bios_cmci_threshold && bios_zero_thresh && - (val & MCI_CTL2_CMCI_THRESHOLD_MASK)) - bios_wrong_thresh = 1; - } else { - WARN_ON(!test_bit(i, this_cpu_ptr(mce_poll_banks))); - } + val = cmci_pick_threshold(val, &bios_zero_thresh); + cmci_claim_bank(i, val, bios_zero_thresh, &bios_wrong_thresh); } raw_spin_unlock_irqrestore(&cmci_discover_lock, flags); if (mca_cfg.bios_cmci_threshold && bios_wrong_thresh) { @@ -218,6 +316,8 @@ static void __cmci_disable_bank(int bank) val &= ~MCI_CTL2_CMCI_EN; wrmsrl(MSR_IA32_MCx_CTL2(bank), val); __clear_bit(bank, this_cpu_ptr(mce_banks_owned)); + if ((val & MCI_CTL2_CMCI_THRESHOLD_MASK) == CMCI_STORM_THRESHOLD) + cmci_storm_end(bank); } /*