From patchwork Sat Jun 3 20:07:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 102905 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp1840891vqr; Sat, 3 Jun 2023 13:55:42 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ62mPcnxoE7EJVIVXyJBNH0RBndwgwC5BBLm35CSiQuCLF0fJVuUq+9qvdVqb+ffGcHRASS X-Received: by 2002:a17:90a:c211:b0:256:2efc:270e with SMTP id e17-20020a17090ac21100b002562efc270emr3943962pjt.5.1685825741712; Sat, 03 Jun 2023 13:55:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685825741; cv=none; d=google.com; s=arc-20160816; b=r8ETLWe31yeUTId9NHr9E/+p2huiOZVExoQDrsSHPoNEt4Q91kXlN6/r5fzIwHIM6c LTDEEpmr382wI7l+VkUD7RLrfjhepoeqWImwDF98Roms2TSAdyIVCyRKQjmHsAnSFpPG LapAyunio3M8peMUObme5FoRvUM/oAJenD9m7QfDpfPvveZvCkIeEr8PNzSj1NNrf7wA pdfDQyFyqs+IxVRRjJ75OhmZ2yPRUIddZ70vlA0Jnw82Xhqvh4qhkjt+R9En12vZM3DS uHxbRCgh69A0nUPOsmUHAV2JLcX2gUpOsCRatlUBQPu4CG9N6u0uuUknbCfAE6hTR4gf gsVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:date:mime-version:references:subject:cc:to:from :dkim-signature:dkim-signature:message-id; bh=fh2uTBEHG28Fh31y7f2tmR7yGLyMLoAyt+FF1TixMCg=; b=I3dCP/0nbR1uKHdzsLncf6JtVL5rUQGG+vsFSE6CMLt6b6orbNQQWyNzKAlg82Xs1x liCXUlKgr0CfWtW8Ze4Qvesngk8R2hAis956vX5JOBsAjI5LKOqMtNldI6UEke6ap5ks E0/Y4BMMaReMYqGauhJNXLO6Rdxj/WvBZ6euY0Vhu/Hez/qVFPUxYWR7p9LdsQef7e2i kQFvv04tH30icTaeE4ttlrpXGeZWQIEg9ToIbMA3fori892RIgEYvkQF7ae2M9UT780r 3Qz9NKTLBku4AgCbFeeArnsGWYdL3DOHGsvJTsWkYPy6kJ4jtHNdW9mMieNasuqvGKOj NwCg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=1nVnHoky; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id qi2-20020a17090b274200b00256a6aab74esi4916607pjb.111.2023.06.03.13.55.29; Sat, 03 Jun 2023 13:55:41 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=1nVnHoky; dkim=neutral (no key) header.i=@linutronix.de; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230492AbjFCUJB (ORCPT + 99 others); Sat, 3 Jun 2023 16:09:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231628AbjFCUIp (ORCPT ); Sat, 3 Jun 2023 16:08:45 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BCACFE60 for ; Sat, 3 Jun 2023 13:07:53 -0700 (PDT) Message-ID: <20230603200459.775471968@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1685822820; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=fh2uTBEHG28Fh31y7f2tmR7yGLyMLoAyt+FF1TixMCg=; b=1nVnHokyojIXwrcsLwEvNrRA4MA/Gp4A/mBpI6uAEV1v/jCxYGSnkpIOK13V1qC9G1cZ68 E0Wt+8KqnQK18DnJRPKdRxwf91S4QA5Y42Tq1WmTqsOF/yqIlLllKO5pQ4bFhhY3wTqQgr 4Px0WvA8jO3bRnsrseb5mKoj67Vo4TPrNsdaa7Dwj1VLeq0KvQPdBOCvei4ulO0CvRwoue 5bdcxZywog18NE2kVcZJv5vK/CVo4mGvrgvPcQdw786B/16F0MRU9jxZ0PFOlLKF711UiT vLQs43veq6Pkof2vLWl6tpJjJYljK/tnGppsn6lnxiP2WWUF4zrYmoP5GdpuUQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1685822820; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=fh2uTBEHG28Fh31y7f2tmR7yGLyMLoAyt+FF1TixMCg=; b=ZYmnES9mtLf+uuLZaMIpIFu5yZDJIJKyATWf0PD4/i7pu+IKi4nksO4gNyQ6S9IsqkhH1t YllL+o1xr/zXinCg== From: Thomas Gleixner To: LKML Cc: x86@kernel.org, Ashok Raj , Dave Hansen , Tony Luck , Arjan van de Veen , Peter Zijlstra , Eric Biederman Subject: [patch 3/6] x86/smp: Use dedicated cache-line for mwait_play_dead() References: <20230603193439.502645149@linutronix.de> MIME-Version: 1.0 Date: Sat, 3 Jun 2023 22:07:00 +0200 (CEST) X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767716412817304908?= X-GMAIL-MSGID: =?utf-8?q?1767716412817304908?= Monitoring idletask::thread_info::flags in mwait_play_dead() has been an obvious choice as all what is needed is a cache line which is not written by other CPUs. But there is a use case where a "dead" CPU needs to be brought out of that mwait(): kexec(). The CPU needs to be brought out of mwait before kexec() as kexec() can overwrite text, pagetables, stacks and the monitored cacheline of the original kernel. The latter causes mwait to resume execution which obviously causes havoc on the kexec kernel which results usually in triple faults. Use a dedicated per CPU storage to prepare for that. Signed-off-by: Thomas Gleixner --- arch/x86/kernel/smpboot.c | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -101,6 +101,17 @@ EXPORT_PER_CPU_SYMBOL(cpu_die_map); DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info); EXPORT_PER_CPU_SYMBOL(cpu_info); +struct mwait_cpu_dead { + unsigned int control; + unsigned int status; +}; + +/* + * Cache line aligned data for mwait_play_dead(). Separate on purpose so + * that it's unlikely to be touched by other CPUs. + */ +static DEFINE_PER_CPU_ALIGNED(struct mwait_cpu_dead, mwait_cpu_dead); + /* Logical package management. We might want to allocate that dynamically */ unsigned int __max_logical_packages __read_mostly; EXPORT_SYMBOL(__max_logical_packages); @@ -1758,10 +1769,10 @@ EXPORT_SYMBOL_GPL(cond_wakeup_cpu0); */ static inline void mwait_play_dead(void) { + struct mwait_cpu_dead *md = this_cpu_ptr(&mwait_cpu_dead); unsigned int eax, ebx, ecx, edx; unsigned int highest_cstate = 0; unsigned int highest_subcstate = 0; - void *mwait_ptr; int i; if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD || @@ -1796,13 +1807,6 @@ static inline void mwait_play_dead(void) (highest_subcstate - 1); } - /* - * This should be a memory location in a cache line which is - * unlikely to be touched by other processors. The actual - * content is immaterial as it is not actually modified in any way. - */ - mwait_ptr = ¤t_thread_info()->flags; - wbinvd(); while (1) { @@ -1814,9 +1818,9 @@ static inline void mwait_play_dead(void) * case where we return around the loop. */ mb(); - clflush(mwait_ptr); + clflush(md); mb(); - __monitor(mwait_ptr, 0, 0); + __monitor(md, 0, 0); mb(); __mwait(eax, 0);