From patchwork Fri Jul 21 20:18:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 124093 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9010:0:b0:3e4:2afc:c1 with SMTP id l16csp467133vqg; Fri, 21 Jul 2023 14:12:32 -0700 (PDT) X-Google-Smtp-Source: APBJJlEEp4SmQRMsi+rkpqitKKe7BaqtQTFY2CT5JnPPxidF0HLmKXcBt6iSVxRHhjNkicayoZ5G X-Received: by 2002:a17:906:20de:b0:99b:574a:42b3 with SMTP id c30-20020a17090620de00b0099b574a42b3mr2878839ejc.43.1689973951776; Fri, 21 Jul 2023 14:12:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689973951; cv=none; d=google.com; s=arc-20160816; b=oXV4E3GI9jz9G00GFRnOjHaeWxX/8AdFgKxOCQh/fgraBfN8rguxyhZY7sVzJalcIV ooroqxYGoWoJtzBj1b/UfUnKyp/EtzNBFu6UdLTWNSRoZDW/R+GvymwPTLWxjnlFp327 TacvcDFxCqtIhe3Pr0s3C8PnffWzfi2GtuhX4ACtf/WtYRised+KNsRkqYlfsCBhr1FM niqRRsu+X5f9/z54Q9Q2S8qOXl+2dCO19O8BxctUqDYhOyKPCR5S7IPWKh34z271KCTP kgPc5nZoNXyojtVwhSDnuMtir5SxDR9vsn2a4K2HZVG8iEVec6ANJvXQtw36Mis9l2DI D7Cw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:dkim-signature; bh=xY+3Pnlxin9cJMvMOS3842OXvOZxwo3V1XbzLrWDw/E=; fh=GitwJflddDxrBBVhnTGEzIZYOBTU7RKSqL+OyCDv/0M=; b=Q/SbBddHhXe8oaI7dBPxxhWHhI/R3QnsHVbO6Ko6B/SFIVhlg9mBODNMRJHxIpAS0r 27eUt4+LV3f/+KIKsfJ1gPrIh5Wo8ft3/f3L3Gpl/IaE3v3NdqiQejmg6rFYhKHbxCEq 87w7+YUUbdni4ljRnB3spajmFxz3SSOzzeikCP1orlGARUOKfksbCWZn8NfN6gYp+kzP LHVjKZIv5DRSg82/hyvE6nG5n84ckMJuPbQpGHUmo//CL6Yr61ImP8hUfdHRb6TVAqnK B7QlBwy91sEf75H7r8U1OcYKF8TgvLN+f/dNggHIcbRLeEaV9uk9z5N3+NIsbo+Runix vvKg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=Lqb3RehZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id pj25-20020a170906d79900b0099367350a8asi2767865ejb.899.2023.07.21.14.12.07; Fri, 21 Jul 2023 14:12:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=Lqb3RehZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230503AbjGUUTZ (ORCPT + 99 others); Fri, 21 Jul 2023 16:19:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37980 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230044AbjGUUTR (ORCPT ); Fri, 21 Jul 2023 16:19:17 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 81C79359F for ; Fri, 21 Jul 2023 13:19:06 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-ccabb20111dso2289578276.3 for ; Fri, 21 Jul 2023 13:19:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689970746; x=1690575546; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=xY+3Pnlxin9cJMvMOS3842OXvOZxwo3V1XbzLrWDw/E=; b=Lqb3RehZeO7ntuc5ijbgqhf+wouAzLMMAFKmweKe4G70L9uc4Bw2Jl5XydLCvynLja MxugEfg74h0RFAs1g/mPDWIA6CQEQjP/1tCMXpzbUBwwHi1wUVOx7FGGl6cj3EKgt4DI 2serW/L/1xq/mdpALeeGTUFcMs8dGf7LsjjD0F7Qj7rNn0+spOMJVHUn7lKygW1pXfr6 V4l0GVwRDsM4qqERhw/rUjtXUDV9O3+sw+gB5y+qOA/uO7E4/u1wlbkArtHA17Pz8aqj lI6iYWVldo4NzF+NtwvTmo0m32Ap85XnNI96DTf2N43IXGAmmG5+ay/zpvN3HuCBAR7U b77Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689970746; x=1690575546; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=xY+3Pnlxin9cJMvMOS3842OXvOZxwo3V1XbzLrWDw/E=; b=bGuMXEiHTk4sUIkQOoQ6QsbddYs6SydKyUGk6tDW0mUzaXqOD4qeUkaXhIMiMmy3LV V0IWQ1YDtkBNHDdFRqtCQLybUxoPWG+M/bGzEDY+H6uIAq+TXWGUP9q6vggOQK+I43Q+ c7+RQ2Y3CS3VMwBR6VjtlyzRCefJfp2Jn9XQBF2LCE+wCEbEuksI27HI3cs3se5EmzuG hHm7iELHwn4pxWJ4+TnDYxdQacnB+R/yWr4WGaG22eBvspf4dncZ1CW8vH8j/gJgd9p5 LOGLKuQZnlesTkgMGICjRn40ddoOuzcpH5D2OT2phvUfuoHnCi7uiQSunaGgA9AMKECZ Vc6g== X-Gm-Message-State: ABy/qLaNQs13xtxAs8IxmpOdtb5f6b3kJ9SJ/S1Fq+QoXkWQbPz5WS6/ WfX4RKYj+AbDFyV20y1ZuPTqk67WpKw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:428d:0:b0:c65:7352:4b5 with SMTP id p135-20020a25428d000000b00c65735204b5mr21381yba.0.1689970745731; Fri, 21 Jul 2023 13:19:05 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 21 Jul 2023 13:18:41 -0700 In-Reply-To: <20230721201859.2307736-1-seanjc@google.com> Mime-Version: 1.0 References: <20230721201859.2307736-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.487.g6d72f3e995-goog Message-ID: <20230721201859.2307736-2-seanjc@google.com> Subject: [PATCH v4 01/19] x86/reboot: VMCLEAR active VMCSes before emergency reboot From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772066126355738693 X-GMAIL-MSGID: 1772066126355738693 VMCLEAR active VMCSes before any emergency reboot, not just if the kernel may kexec into a new kernel after a crash. Per Intel's SDM, the VMX architecture doesn't require the CPU to flush the VMCS cache on INIT. If an emergency reboot doesn't RESET CPUs, cached VMCSes could theoretically be kept and only be written back to memory after the new kernel is booted, i.e. could effectively corrupt memory after reboot. Opportunistically remove the setting of the global pointer to NULL to make checkpatch happy. Cc: Andrew Cooper Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kexec.h | 2 -- arch/x86/include/asm/reboot.h | 2 ++ arch/x86/kernel/crash.c | 31 ------------------------------- arch/x86/kernel/reboot.c | 22 ++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 10 +++------- 5 files changed, 27 insertions(+), 40 deletions(-) diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h index 5b77bbc28f96..819046974b99 100644 --- a/arch/x86/include/asm/kexec.h +++ b/arch/x86/include/asm/kexec.h @@ -205,8 +205,6 @@ int arch_kimage_file_post_load_cleanup(struct kimage *image); #endif #endif -typedef void crash_vmclear_fn(void); -extern crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss; extern void kdump_nmi_shootdown_cpus(void); #endif /* __ASSEMBLY__ */ diff --git a/arch/x86/include/asm/reboot.h b/arch/x86/include/asm/reboot.h index 9177b4354c3f..dc201724a643 100644 --- a/arch/x86/include/asm/reboot.h +++ b/arch/x86/include/asm/reboot.h @@ -25,6 +25,8 @@ void __noreturn machine_real_restart(unsigned int type); #define MRR_BIOS 0 #define MRR_APM 1 +typedef void crash_vmclear_fn(void); +extern crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss; void cpu_emergency_disable_virtualization(void); typedef void (*nmi_shootdown_cb)(int, struct pt_regs*); diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c index cdd92ab43cda..54cd959cb316 100644 --- a/arch/x86/kernel/crash.c +++ b/arch/x86/kernel/crash.c @@ -48,38 +48,12 @@ struct crash_memmap_data { unsigned int type; }; -/* - * This is used to VMCLEAR all VMCSs loaded on the - * processor. And when loading kvm_intel module, the - * callback function pointer will be assigned. - * - * protected by rcu. - */ -crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss = NULL; -EXPORT_SYMBOL_GPL(crash_vmclear_loaded_vmcss); - -static inline void cpu_crash_vmclear_loaded_vmcss(void) -{ - crash_vmclear_fn *do_vmclear_operation = NULL; - - rcu_read_lock(); - do_vmclear_operation = rcu_dereference(crash_vmclear_loaded_vmcss); - if (do_vmclear_operation) - do_vmclear_operation(); - rcu_read_unlock(); -} - #if defined(CONFIG_SMP) && defined(CONFIG_X86_LOCAL_APIC) static void kdump_nmi_callback(int cpu, struct pt_regs *regs) { crash_save_cpu(regs, cpu); - /* - * VMCLEAR VMCSs loaded on all cpus if needed. - */ - cpu_crash_vmclear_loaded_vmcss(); - /* * Disable Intel PT to stop its logging */ @@ -133,11 +107,6 @@ void native_machine_crash_shutdown(struct pt_regs *regs) crash_smp_send_stop(); - /* - * VMCLEAR VMCSs loaded on this cpu if needed. - */ - cpu_crash_vmclear_loaded_vmcss(); - cpu_emergency_disable_virtualization(); /* diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c index 3adbe97015c1..3fa4c6717a1d 100644 --- a/arch/x86/kernel/reboot.c +++ b/arch/x86/kernel/reboot.c @@ -787,6 +787,26 @@ void machine_crash_shutdown(struct pt_regs *regs) } #endif +/* + * This is used to VMCLEAR all VMCSs loaded on the + * processor. And when loading kvm_intel module, the + * callback function pointer will be assigned. + * + * protected by rcu. + */ +crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss; +EXPORT_SYMBOL_GPL(crash_vmclear_loaded_vmcss); + +static inline void cpu_crash_vmclear_loaded_vmcss(void) +{ + crash_vmclear_fn *do_vmclear_operation = NULL; + + rcu_read_lock(); + do_vmclear_operation = rcu_dereference(crash_vmclear_loaded_vmcss); + if (do_vmclear_operation) + do_vmclear_operation(); + rcu_read_unlock(); +} /* This is the CPU performing the emergency shutdown work. */ int crashing_cpu = -1; @@ -798,6 +818,8 @@ int crashing_cpu = -1; */ void cpu_emergency_disable_virtualization(void) { + cpu_crash_vmclear_loaded_vmcss(); + cpu_emergency_vmxoff(); cpu_emergency_svm_disable(); } diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 0ecf4be2c6af..7f692d97a821 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -41,7 +41,7 @@ #include #include #include -#include +#include #include #include #include @@ -744,7 +744,6 @@ static int vmx_set_guest_uret_msr(struct vcpu_vmx *vmx, return ret; } -#ifdef CONFIG_KEXEC_CORE static void crash_vmclear_local_loaded_vmcss(void) { int cpu = raw_smp_processor_id(); @@ -754,7 +753,6 @@ static void crash_vmclear_local_loaded_vmcss(void) loaded_vmcss_on_cpu_link) vmcs_clear(v->vmcs); } -#endif /* CONFIG_KEXEC_CORE */ static void __loaded_vmcs_clear(void *arg) { @@ -8592,10 +8590,9 @@ static void __vmx_exit(void) { allow_smaller_maxphyaddr = false; -#ifdef CONFIG_KEXEC_CORE RCU_INIT_POINTER(crash_vmclear_loaded_vmcss, NULL); synchronize_rcu(); -#endif + vmx_cleanup_l1d_flush(); } @@ -8644,10 +8641,9 @@ static int __init vmx_init(void) pi_init_cpu(cpu); } -#ifdef CONFIG_KEXEC_CORE rcu_assign_pointer(crash_vmclear_loaded_vmcss, crash_vmclear_local_loaded_vmcss); -#endif + vmx_check_vmcs12_offsets(); /*