From patchwork Thu Dec 1 23:26:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 28591 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp540401wrr; Thu, 1 Dec 2022 15:28:13 -0800 (PST) X-Google-Smtp-Source: AA0mqf78GVXphJvvvncpzMpy040l9iS9fouvGOhHbvnIJW1Xrd/ZZTRsGVP6IP8L2v1D5i3hXUCj X-Received: by 2002:a17:906:9f02:b0:7b5:f5c9:b450 with SMTP id fy2-20020a1709069f0200b007b5f5c9b450mr47538963ejc.65.1669937293796; Thu, 01 Dec 2022 15:28:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669937293; cv=none; d=google.com; s=arc-20160816; b=z+oIilqBmUQ5Y/SHPi3RdWCcvKhQwlELqpNSoCgVyHfnM7vPrZuTMyiogI4K9oFjzU gKijC7tvF+LZXWOufAW52kb2eEBM7n+T6VgJXxdM48R4j28zXCr/I7EJwXI/zyama0U0 zEKLCn0z/xBszMggtUu1zaZrcmpWj4BSUYEv/j3g2/Kkh4tvfybwznG4IYJNJCG01xot h3FeScRyclHyyuAIkzPybZdP9mQfeuxLcSZmntXHkpCFjny8EGtKaof7/sbLeBM5vskm AChaTj8Aqf2rxarYD/DVF4yZKwf+sqFrtN/Tyi0uXjitYUlmNbQVVzK0ChTOG9XxpZZf SSxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:dkim-signature; bh=DA91i2oZnY42qV1reMA6fFQYAyKGGbDT6HRfvlU2Axc=; b=WjawUmKpKOcgOE7FyhpWoiA+lFiWMmKvq80Za2hXqe1h6jWLbhaHLyBhCZiAbAdHVO NCotZoDuw8eVOTDYMHYn7h0mehtfHZlgXvJS9S3x4NzSjdUc5pzip32MHH3AcXwsYRev Nfsnf7Fqh+oHuM9JAvn7qevjFLyN0jNIkKXrzeCpg9Dkof62H+0+sNzKra0GogZu9hnN 5nAxnH1qosMREbyL7KSaAvLaK2YmcNvgvPjKCgHF6fSdpdQltlCL3DUGMjkD5NTAEC/U vOJ7UnrDJ13x30Y0IoKsDCnezgLb3EQiwl2F44wzO5UsCRyYZEuAqDf+Q2wuNJJFxPbI Kusg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=dknQcxep; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q18-20020a056402033200b004534c6c4bd7si4894591edw.433.2022.12.01.15.27.49; Thu, 01 Dec 2022 15:28:13 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=dknQcxep; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231799AbiLAX1K (ORCPT + 99 others); Thu, 1 Dec 2022 18:27:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55678 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231560AbiLAX1F (ORCPT ); Thu, 1 Dec 2022 18:27:05 -0500 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AFE1A578F1 for ; Thu, 1 Dec 2022 15:27:04 -0800 (PST) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-352e29ff8c2so31913837b3.21 for ; Thu, 01 Dec 2022 15:27:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=DA91i2oZnY42qV1reMA6fFQYAyKGGbDT6HRfvlU2Axc=; b=dknQcxepnbfWDiba9noWUVCaonlLdg8B/11fnai9Q3F7mag317asvQdsf+FqFF9fd4 YU6VtAXT0iwrRI/BP8oIfrzZ36BYaLecoa0Y+JQWbdR6ND1/pSbs9yG/uIb3zLfB99HR 15iYofue1KKJehKnekWfgNyetv0tN219KR31yZ3OcGv4/JAeBpNwsSBwFIV0ivD6Ehu1 E7lV6r966hKxFtahrqXwdoN7YkcvS/eenu2dh/wg16EfPdX0lqpSxYgUfidTR97UrPzn SEerIUXx6vUxrXpJCoAJqGccyLjb49C80+wqNcmXB0tDuKET6CkPYqMRRrTRc4OvePrg LKAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=DA91i2oZnY42qV1reMA6fFQYAyKGGbDT6HRfvlU2Axc=; b=Iq3gQmzWOzc9vWMzr3mnjI6zcsDHcsQbREKjFQscbUtmYfCtKjVjXQd078Xwi4rBIa sw7M0L+e21fWLZDsvVTNty/BooxhT+J/Vo/2IcxbXaQUb3MLRQRAQpOYnakI4SJF36y1 VBvIrQllIugYq9Dq4i/xkIE2exmNO0mZE8EOTvarXXprS0KYRb2LCu0PWqJ4FvQ2wlDq b70RPEp4wnqLU0YLXwT+57ojDuichPHfWkHKYMTo71yWfMy9lCmU/1NZ5qovx7tyoUa3 zUGscaTuxVcnENS/Wrta4fVx03bY7G7f1/Jd4sw7JdjAZM+ZefsbvmICsdR/Eu+f8SDQ AcYg== X-Gm-Message-State: ANoB5plQZ9P6dR//0KfLxX0SF2rNbaNU+SKaCGrmlbr6k0b/MR+6biPN +8bBXQKg63pH8ZMQeetZ0J8n0VmccvA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:6602:0:b0:6f9:890c:6468 with SMTP id a2-20020a256602000000b006f9890c6468mr12986187ybc.610.1669937223951; Thu, 01 Dec 2022 15:27:03 -0800 (PST) Reply-To: Sean Christopherson Date: Thu, 1 Dec 2022 23:26:40 +0000 In-Reply-To: <20221201232655.290720-1-seanjc@google.com> Mime-Version: 1.0 References: <20221201232655.290720-1-seanjc@google.com> X-Mailer: git-send-email 2.39.0.rc0.267.gcb52ba06e7-goog Message-ID: <20221201232655.290720-2-seanjc@google.com> Subject: [PATCH 01/16] x86/reboot: VMCLEAR active VMCSes before emergency reboot From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: "H. Peter Anvin" , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751056167284258330?= X-GMAIL-MSGID: =?utf-8?q?1751056167284258330?= VMCLEAR active VMCSes before any emergency reboot, not just if the kernel may kexec into a new kernel after a crash. Per Intel's SDM, the VMX architecture doesn't require the CPU to flush the VMCS cache on INIT. If an emergency reboot doesn't RESET CPUs, cached VMCSes could theoretically be kept and only be written back to memory after the new kernel is booted, i.e. could effectively corrupt memory after reboot. Opportunistically remove the setting of the global pointer to NULL to make checkpatch happy. Cc: Andrew Cooper Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kexec.h | 2 -- arch/x86/include/asm/reboot.h | 2 ++ arch/x86/kernel/crash.c | 31 ------------------------------- arch/x86/kernel/reboot.c | 22 ++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 10 +++------- 5 files changed, 27 insertions(+), 40 deletions(-) diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h index a3760ca796aa..256eee99afc8 100644 --- a/arch/x86/include/asm/kexec.h +++ b/arch/x86/include/asm/kexec.h @@ -208,8 +208,6 @@ int arch_kimage_file_post_load_cleanup(struct kimage *image); #endif #endif -typedef void crash_vmclear_fn(void); -extern crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss; extern void kdump_nmi_shootdown_cpus(void); #endif /* __ASSEMBLY__ */ diff --git a/arch/x86/include/asm/reboot.h b/arch/x86/include/asm/reboot.h index bc5b4d788c08..2551baec927d 100644 --- a/arch/x86/include/asm/reboot.h +++ b/arch/x86/include/asm/reboot.h @@ -25,6 +25,8 @@ void __noreturn machine_real_restart(unsigned int type); #define MRR_BIOS 0 #define MRR_APM 1 +typedef void crash_vmclear_fn(void); +extern crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss; void cpu_emergency_disable_virtualization(void); typedef void (*nmi_shootdown_cb)(int, struct pt_regs*); diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c index 6b8a6aae02e3..7611dd5f90a4 100644 --- a/arch/x86/kernel/crash.c +++ b/arch/x86/kernel/crash.c @@ -48,38 +48,12 @@ struct crash_memmap_data { unsigned int type; }; -/* - * This is used to VMCLEAR all VMCSs loaded on the - * processor. And when loading kvm_intel module, the - * callback function pointer will be assigned. - * - * protected by rcu. - */ -crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss = NULL; -EXPORT_SYMBOL_GPL(crash_vmclear_loaded_vmcss); - -static inline void cpu_crash_vmclear_loaded_vmcss(void) -{ - crash_vmclear_fn *do_vmclear_operation = NULL; - - rcu_read_lock(); - do_vmclear_operation = rcu_dereference(crash_vmclear_loaded_vmcss); - if (do_vmclear_operation) - do_vmclear_operation(); - rcu_read_unlock(); -} - #if defined(CONFIG_SMP) && defined(CONFIG_X86_LOCAL_APIC) static void kdump_nmi_callback(int cpu, struct pt_regs *regs) { crash_save_cpu(regs, cpu); - /* - * VMCLEAR VMCSs loaded on all cpus if needed. - */ - cpu_crash_vmclear_loaded_vmcss(); - /* * Disable Intel PT to stop its logging */ @@ -133,11 +107,6 @@ void native_machine_crash_shutdown(struct pt_regs *regs) crash_smp_send_stop(); - /* - * VMCLEAR VMCSs loaded on this cpu if needed. - */ - cpu_crash_vmclear_loaded_vmcss(); - cpu_emergency_disable_virtualization(); /* diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c index d03c551defcc..299b970e5f82 100644 --- a/arch/x86/kernel/reboot.c +++ b/arch/x86/kernel/reboot.c @@ -787,6 +787,26 @@ void machine_crash_shutdown(struct pt_regs *regs) } #endif +/* + * This is used to VMCLEAR all VMCSs loaded on the + * processor. And when loading kvm_intel module, the + * callback function pointer will be assigned. + * + * protected by rcu. + */ +crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss; +EXPORT_SYMBOL_GPL(crash_vmclear_loaded_vmcss); + +static inline void cpu_crash_vmclear_loaded_vmcss(void) +{ + crash_vmclear_fn *do_vmclear_operation = NULL; + + rcu_read_lock(); + do_vmclear_operation = rcu_dereference(crash_vmclear_loaded_vmcss); + if (do_vmclear_operation) + do_vmclear_operation(); + rcu_read_unlock(); +} /* This is the CPU performing the emergency shutdown work. */ int crashing_cpu = -1; @@ -798,6 +818,8 @@ int crashing_cpu = -1; */ void cpu_emergency_disable_virtualization(void) { + cpu_crash_vmclear_loaded_vmcss(); + cpu_emergency_vmxoff(); cpu_emergency_svm_disable(); } diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 39dd3082fcd8..7ea020456dd0 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -41,7 +41,7 @@ #include #include #include -#include +#include #include #include #include @@ -743,7 +743,6 @@ static int vmx_set_guest_uret_msr(struct vcpu_vmx *vmx, return ret; } -#ifdef CONFIG_KEXEC_CORE static void crash_vmclear_local_loaded_vmcss(void) { int cpu = raw_smp_processor_id(); @@ -753,7 +752,6 @@ static void crash_vmclear_local_loaded_vmcss(void) loaded_vmcss_on_cpu_link) vmcs_clear(v->vmcs); } -#endif /* CONFIG_KEXEC_CORE */ static void __loaded_vmcs_clear(void *arg) { @@ -8528,10 +8526,9 @@ static void __vmx_exit(void) { allow_smaller_maxphyaddr = false; -#ifdef CONFIG_KEXEC_CORE RCU_INIT_POINTER(crash_vmclear_loaded_vmcss, NULL); synchronize_rcu(); -#endif + vmx_cleanup_l1d_flush(); } @@ -8580,10 +8577,9 @@ static int __init vmx_init(void) pi_init_cpu(cpu); } -#ifdef CONFIG_KEXEC_CORE rcu_assign_pointer(crash_vmclear_loaded_vmcss, crash_vmclear_local_loaded_vmcss); -#endif + vmx_check_vmcs12_offsets(); /*