From patchwork Tue Jun 13 12:17:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 10707 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp507074vqr; Tue, 13 Jun 2023 05:32:08 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ70vVW2qIlM4LDbdC4RA0PRzxvQhe/BO3NUBE6E63ysYjvw45jNwPzPtAFaYVZX06bLFdK8 X-Received: by 2002:aa7:d6d5:0:b0:516:2dcf:d027 with SMTP id x21-20020aa7d6d5000000b005162dcfd027mr8799415edr.10.1686659528607; Tue, 13 Jun 2023 05:32:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686659528; cv=none; d=google.com; s=arc-20160816; b=UMIFKCbCGYCSlN2EAJ+bcKC88RFljzgW2QE3TaGDs9OW7K+mSQoDl8nG7XMkWJda7v HuQ2sCVDHpwHJ60XFYmP+YYdNbUF8k9eqqtIe/5Xipsg8fbYX9svGEfPgf+9tt9yBRbw bco8/3UnfR/hvB1hUrDkok81Stl0OwfhWxyI49a+8HJ7JSxbWgUdvK+pGy3kpXm3qiha 9XOYIOKRkibVBewkEzsI+RLrvDoVRshj2HalDi2+wvEk4P9n14brjep/go6+uvNSh9pf UaQrTdQOWMqQbpPgsSfAsKgvLnuvgbeBP+rQwHdGxXA2j8wTRaFQsHbi59l8KRKkcQkm SfSw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:subject:cc:to:from:dkim-signature :dkim-signature:message-id; bh=+xkDu2ezsZFUK0UriVeN880YzwjasDfhm6+0tLf5Wqg=; b=sN413wPH+zLWrAJUK1GdmZUBSX6jHdRSLmGLphkZ5BgYvGcnlDw/FDcZ43H5rTfCQS 0L6jv9bcONm5QCdZvNthRvrpR4pfeJ+VDvgOI+ri79vMvDoZk9Ik4sK+QX5E3aJelpkr gVg4Hj+NA2BJs2y5lsowDNMeueUBYYDbSF9hATdZfVkf8HSLawWigBGzGAfhKumqbDtw N6m+X6WQ/IWL+2je3DGtAckiyYrSq/FjTnixIMnQJXbIlhCfjHUUC8oazaHmPKzhXL1w ZZVZdnoczGbitsN9L8zaQ89tjais9SYFGjnfTJn7pyDg2nr5yslfu//hDUlW5vHjOn2W kSHA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="bK72HX/U"; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=kC7etQnY; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e9-20020a056402148900b00514511f3ac9si3705597edv.201.2023.06.13.05.31.40; Tue, 13 Jun 2023 05:32:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b="bK72HX/U"; dkim=neutral (no key) header.i=@linutronix.de header.s=2020e header.b=kC7etQnY; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242232AbjFMMSA (ORCPT + 99 others); Tue, 13 Jun 2023 08:18:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57100 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232194AbjFMMR7 (ORCPT ); Tue, 13 Jun 2023 08:17:59 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1D98D10CC for ; Tue, 13 Jun 2023 05:17:57 -0700 (PDT) Message-ID: <20230613115353.599087484@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1686658674; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc; bh=+xkDu2ezsZFUK0UriVeN880YzwjasDfhm6+0tLf5Wqg=; b=bK72HX/UGPRhfy+hIwUqzr9tro3gQA9S+uXuH7t3vqXAK/DoM4H56/ieXpSfSTGvw/H52E avIKN0L1g0YeNRd7jg5mBW2R2LWcg+FCTURLtw1md52pCYdL5xhdUP7j22gId0hcfzPlPp Hw6gJSVPn/9SwKWYHH34KeA3hDLPfNLcwweNDt49d79lGuSKDMz2XPTx5wxLzJQ5IZmRrL mFC+I41WheWghH3/xQT125SGpc0cuSY8CzmIAqxjeENYrKGu3/x2HbAjvmlom5vziz0Dxv zLdBKOWw/meyb7uOiOWhb/70jE/irvid9i7YnM2ilUqdyqCLxbbOwh9dlhSPrg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1686658674; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc; bh=+xkDu2ezsZFUK0UriVeN880YzwjasDfhm6+0tLf5Wqg=; b=kC7etQnYqwYKWNpRF3IIDMVI4RH3gfQAJKS4YSGnHY+2ULnNgl+g0rlN6iFKdGdos6aePg /z3d2SFcd5BOgeDg== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Mario Limonciello , Tom Lendacky , Tony Battersby , Ashok Raj , Tony Luck , Arjan van de Veen , Eric Biederman Subject: [patch V2 0/8] x86/smp: Cure stop_other_cpus() and kexec() troubles Date: Tue, 13 Jun 2023 14:17:54 +0200 (CEST) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768590701812339749?= X-GMAIL-MSGID: =?utf-8?q?1768590701812339749?= This is the second version of the kexec() vs. mwait_play_dead() series. Version 1 can be found here: https://lore.kernel.org/r/20230603193439.502645149@linutronix.de Aside of picking up the correction of the original patch 5 this also integrates a fix for intermittend reboot hangs reported by Tony: https://lore.kernel.org/r/3817d810-e0f1-8ef8-0bbd-663b919ca49b@cybernetics.com which touches the same area. While halfways independent I added them here as these changes conflict nicely. So the two issues are: 1) stop_other_cpus() continues after observing num_online_cpus() == 1. This is problematic because the to be stopped CPUs clear their online bit first and then invoke eventually WBINVD, which can take a long time. There seems to be an interaction between the WBINVD and the reboot mechanics as this intermittendly results in hangs. 2) kexec() kernel can overwrite the memory locations which "offline" CPUs are monitoring. This write brings them out of MWAIT and they resume execution on overwritten text, page tables, data and stacks resulting in triple faults. Cure them by: #1 Synchronizing stop_other_cpus() with an atomic variable which is decremented in stop_this_cpu() _after_ WBINVD completes. #2 Bringing offline CPUs out of MWAIT and move them into HLT before starting the kexec() kernel. Optionaly send them an INIT IPI so they go back into wait for startup state. The series is also available from git: git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git x86/kexec Thanks, tglx --- include/asm/cpu.h | 2 include/asm/smp.h | 4 + kernel/process.c | 16 +++++ kernel/smp.c | 79 ++++++++++++++++++---------- kernel/smpboot.c | 149 ++++++++++++++++++++++++++++++++++++++++-------------- 5 files changed, 183 insertions(+), 67 deletions(-)