From patchwork Thu Aug 10 18:37:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 134206 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp650226vqi; Thu, 10 Aug 2023 12:56:14 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE1Y433kkbG3wyySayN2oANOVaeZMhnOBn4+kCW/5SDiir1aWOM4+oTygl1Cz34vLIK6fge X-Received: by 2002:a2e:878b:0:b0:2b6:d5af:1160 with SMTP id n11-20020a2e878b000000b002b6d5af1160mr13052lji.28.1691697373741; Thu, 10 Aug 2023 12:56:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691697373; cv=none; d=google.com; s=arc-20160816; b=kT5KpdhM+eq9fhmPlLLx1TFPma/CyPh3JkcPa6f+XFxyLB6s4Z6owVJARZOufajA+Z 4HIxtIIjj8lP7dDVrlw+DLWEvw8wyljuc6MXQHAZsgR+pLBIOpnu2GiNn1hZS6y5FPiU 88Q7nlRh/wemquRkmF/58rBACq9Fqg7mdzmlejMRnzD46BcJECsYTNP6BvRu1o8d3OUY djvpg0ZTUnehhSzTCrpIAEa6vcZIpqCclZfNhSw/yYTDgAqw5jOjPCsKbTayvy68CGgC PYqYuVyGV7wdT8Vv7edJqGacZJJUfc5Pgt4DJict04RfqcJjvg2DA9rgcRvoBfz5Bc1/ Di6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=Rab/GTu2nZ+oA1azvFmHvo6TFMlBo033l6ZZymTgpxs=; fh=Vp72a/EEov1VOHpGPkCjAnDQEmhhYrQPa+PSfT8H2jQ=; b=tkAFZF0tnGwpbuIUTwTR3SfjRtR4xwDtdZcs6SX6SDHH0IkYxg8RZjguP/BD9FDs9B 9bKPKxB4kRzpoiMLyuoF4Jqe0YPjfkSd6fSldrm/NsgPPtMul8YGLcrx7R3QNkcnJuoj v5yUQ7eRgMWTebpJlYEk9+TUzPxqUTdu7MzCv2MNIYoqEAmnmgiyzMmUZWr13aqc2/p0 /s9H8zRgbfPJHoXVF24nRb3HuiX5LNnrCZwe/eabBcoBzmA8Hxc9fNtgXOO22GFcHdSx 0Kl/Mu3g89imA4iZCIz0vupWR+24VWOiJo8PrkI5TgAur+D37dSVLSkD92RS6ObOn0UJ 5Heg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=I1tvp3sQ; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id qo21-20020a170907213500b0098cf3eaee47si2093277ejb.689.2023.08.10.12.55.49; Thu, 10 Aug 2023 12:56:13 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=I1tvp3sQ; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234557AbjHJSis (ORCPT + 99 others); Thu, 10 Aug 2023 14:38:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33346 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235854AbjHJSiO (ORCPT ); Thu, 10 Aug 2023 14:38:14 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B51EF35AD for ; Thu, 10 Aug 2023 11:37:58 -0700 (PDT) Message-ID: <20230810160805.990936778@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1691692673; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=Rab/GTu2nZ+oA1azvFmHvo6TFMlBo033l6ZZymTgpxs=; b=I1tvp3sQgBxkVj1XMQUfPcR79u9hZ/wT2F3k3w6fILe0jcuIPUUb1mMp1BiefG/PKBhXj9 LqXQSTXWKzLsS6FjeiQxa32Z0UrkHGKbG3tvJr78EFjzPKqbV5Ek4xJxHzgrdyqCh3I9t6 NsGMSjTK93pJy1car6PA7Lla2bBDnWD/51xfhGBBMSecEyL8C1GOXSchnl8b0GkHxKJA1l LDoNyGid1MFsrpB0XR7Uol06xc4PVyAOAhm7a1b2c0dSO0aEqFI3pFuh7ilzF9QCSIPAnU Ea+e3B5fnUiDrDmh/Y8YXvf6V4W68WzZQwvEL55PotCunuHYsokDp/Z26h4UyQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1691692673; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=Rab/GTu2nZ+oA1azvFmHvo6TFMlBo033l6ZZymTgpxs=; b=CKWtb0eoHJ4lTsVc7w/GvCKfFUNq6P8Ta+r+Sj9zC8hDbDyMh5B8drInP8IWAQwqRMeEYN 9VdZztY7xfAy6rCg== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Borislav Petkov , Ashok Raj , Arjan van de Ven Subject: [patch 18/30] x86/microcode: Handle "nosmt" correctly References: <20230810153317.850017756@linutronix.de> MIME-Version: 1.0 Date: Thu, 10 Aug 2023 20:37:52 +0200 (CEST) X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773873265533489311 X-GMAIL-MSGID: 1773873265533489311 From: Thomas Gleixner On CPUs where microcode loading is not NMI safe the SMT sibling which is parked in one of the play_dead() variants, these parked CPUs still react on NMIs. So if a NMI hits while the primary thread updates the microcode the resulting behaviour is undefined. The default play_dead() implementation on modern CPUs is using MWAIT, which is not guaranteed to be safe against an microcode update which affects MWAIT. Take the cpus_booted_once_mask into account to detect this case and refuse to load late if the vendor specific driver does not advertise that late loading is NMI safe. AMD stated that this is safe, so mark the AMD driver accordingly. This requirement will be partially lifted in later changes. Signed-off-by: Thomas Gleixner --- arch/x86/Kconfig | 2 - arch/x86/kernel/cpu/microcode/amd.c | 9 +++-- arch/x86/kernel/cpu/microcode/core.c | 51 +++++++++++++++++++------------ arch/x86/kernel/cpu/microcode/internal.h | 13 +++---- 4 files changed, 44 insertions(+), 31 deletions(-) --- --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -1314,7 +1314,7 @@ config MICROCODE config MICROCODE_LATE_LOADING bool "Late microcode loading (DANGEROUS)" default n - depends on MICROCODE + depends on MICROCODE && SMP help Loading microcode late, when the system is up and executing instructions is a tricky business and should be avoided if possible. Just the sequence --- a/arch/x86/kernel/cpu/microcode/amd.c +++ b/arch/x86/kernel/cpu/microcode/amd.c @@ -909,10 +909,11 @@ static void microcode_fini_cpu_amd(int c } static struct microcode_ops microcode_amd_ops = { - .request_microcode_fw = request_microcode_amd, - .collect_cpu_info = collect_cpu_info_amd, - .apply_microcode = apply_microcode_amd, - .microcode_fini_cpu = microcode_fini_cpu_amd, + .request_microcode_fw = request_microcode_amd, + .collect_cpu_info = collect_cpu_info_amd, + .apply_microcode = apply_microcode_amd, + .microcode_fini_cpu = microcode_fini_cpu_amd, + .nmi_safe = true, }; struct microcode_ops * __init init_amd_microcode(void) --- a/arch/x86/kernel/cpu/microcode/core.c +++ b/arch/x86/kernel/cpu/microcode/core.c @@ -326,23 +326,6 @@ static struct platform_device *microcode */ #define SPINUNIT 100 /* 100 nsec */ -static int check_online_cpus(void) -{ - unsigned int cpu; - - /* - * Make sure all CPUs are online. It's fine for SMT to be disabled if - * all the primary threads are still online. - */ - for_each_present_cpu(cpu) { - if (topology_is_primary_thread(cpu) && !cpu_online(cpu)) { - pr_err("Not all CPUs online, aborting microcode update.\n"); - return -EINVAL; - } - } - - return 0; -} static atomic_t late_cpus_in; static atomic_t late_cpus_out; @@ -459,6 +442,35 @@ static int microcode_reload_late(void) return ret; } +/* + * Ensure that all required CPUs which are present and have been booted + * once are online. + * + * To pass this check, all primary threads must be online. + * + * If the microcode load is not safe against NMI then all SMT threads + * must be online as well because they still react on NMI when they are + * soft-offlined and parked in one of the play_dead() variants. So if a + * NMI hits while the primary thread updates the microcode the resulting + * behaviour is undefined. The default play_dead() implementation on + * modern CPUs is using MWAIT, which is also not guaranteed to be safe + * against a microcode update which affects MWAIT. + */ +static bool ensure_cpus_are_online(void) +{ + unsigned int cpu; + + for_each_cpu_and(cpu, cpu_present_mask, &cpus_booted_once_mask) { + if (!cpu_online(cpu)) { + if (topology_is_primary_thread(cpu) || !microcode_ops->nmi_safe) { + pr_err("CPU %u not online\n", cpu); + return false; + } + } + } + return true; +} + static ssize_t reload_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t size) @@ -474,9 +486,10 @@ static ssize_t reload_store(struct devic cpus_read_lock(); - ret = check_online_cpus(); - if (ret) + if (!ensure_cpus_are_online()) { + ret = -EBUSY; goto put; + } tmp_ret = microcode_ops->request_microcode_fw(bsp, µcode_pdev->dev); if (tmp_ret != UCODE_NEW) --- a/arch/x86/kernel/cpu/microcode/internal.h +++ b/arch/x86/kernel/cpu/microcode/internal.h @@ -20,18 +20,17 @@ enum ucode_state { struct microcode_ops { enum ucode_state (*request_microcode_fw)(int cpu, struct device *dev); - void (*microcode_fini_cpu)(int cpu); /* - * The generic 'microcode_core' part guarantees that - * the callbacks below run on a target cpu when they - * are being called. + * The generic 'microcode_core' part guarantees that the callbacks + * below run on a target cpu when they are being called. * See also the "Synchronization" section in microcode_core.c. */ - enum ucode_state (*apply_microcode)(int cpu); - int (*collect_cpu_info)(int cpu, struct cpu_signature *csig); - void (*finalize_late_load)(int result); + enum ucode_state (*apply_microcode)(int cpu); + int (*collect_cpu_info)(int cpu, struct cpu_signature *csig); + void (*finalize_late_load)(int result); + unsigned int nmi_safe : 1; }; extern struct ucode_cpu_info ucode_cpu_info[];