From patchwork Fri Mar 10 21:42:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 67727 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1113907wrd; Fri, 10 Mar 2023 13:45:49 -0800 (PST) X-Google-Smtp-Source: AK7set+5lHKokEIh/33kmJm1Yh+Kil/sXvbc7UfEDML5IqJjHT1OFclmDHb5/FXY7ibbLE1l3E2l X-Received: by 2002:a17:902:e5ce:b0:199:25d1:e559 with SMTP id u14-20020a170902e5ce00b0019925d1e559mr32762549plf.0.1678484749676; Fri, 10 Mar 2023 13:45:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1678484749; cv=none; d=google.com; s=arc-20160816; b=Qkl3+Ax0gwHUQ3Mz/Vf+R3Pu4PWIh3KQpxorLXmxXfV99XhKr3IBQOZOQv82mV8zfX jp19fNFZojFLqf1j5w/+pD8z1eS98iYHvVaIWEDRUd4BNV2gH8qWtjC0V9DFGtx3RsxF SM84undhv3mqraw1U9IA4FG4et+k9KETrLJWJWoNTA4nRLzapotNv2nzEGY/bypoySXJ y5EHnlULPS2NbQ/g9RVzHarT5IX8lWWR6FIae/amAI0yqq9zWypuLCetrgxKn6htf2tN GLLEyR9ThGXGb1YBhhJYOyxBjYV58ZyYMll8DzJO972WcI45E/PtftopPW1Wd42KcHBu Qcfg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:dkim-signature; bh=ef7PP0SQ7R8DCF78GLBtkjVAZkkgPEB1VXa9M7tBWJI=; b=hzVV39+MUbQ1nwC0vBnVr8he2Uuge7Oz+uP9+RHpU54UIF6qQzilX06lP5eNCaH8+o 8M2dfmGK1cgKTgqRty4CC2JOiZU0ik/5Stcd3LGdTyPARFjdYHSkFv83fdkAkJsmf5sf I+tPzAWvlpIlWbTGyVm49+UR9gyg11En1PidEHD2cqxGWGaPW8rElfrW7Yj6q+33aTEI u8n2Fm/LxR3uLvwPx3cghu57zlMwPLSdsA7G7BBdo19mAnsewwV6X1WmU0V20hAIArvW a8BJ5l/H5Z//0hF74S4BdwrdMqrcYmRjH+vK0IUWHQ9Ug9hIjvS/4Lhrg9PxvJaPc/PL Z7ZA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=Un8MBYCm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u6-20020a17090341c600b0019e2c25d670si866272ple.572.2023.03.10.13.45.32; Fri, 10 Mar 2023 13:45:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=Un8MBYCm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231878AbjCJVmn (ORCPT + 99 others); Fri, 10 Mar 2023 16:42:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57772 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231590AbjCJVmk (ORCPT ); Fri, 10 Mar 2023 16:42:40 -0500 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5A9D91308F8 for ; Fri, 10 Mar 2023 13:42:39 -0800 (PST) Received: by mail-pj1-x104a.google.com with SMTP id pl10-20020a17090b268a00b00239ed042afcso4912938pjb.4 for ; Fri, 10 Mar 2023 13:42:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1678484559; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=ef7PP0SQ7R8DCF78GLBtkjVAZkkgPEB1VXa9M7tBWJI=; b=Un8MBYCmNnLoukZjJV+nUJt8RrbN1Wuz/29ZYWJZGTk2dPZ5X9qMkdfmDLXpK6nWai hCnYZQbiJGX8XRqFa1KD6lVkYvkWoAbqL62YxFG/yJKKljUoX2BnFUaKgeVDc5u5LGxN PyLIxDhvz7dvMSK8rLvrMj/Db8H5ad5uFl7o2jROvRPRjCfyiaV4RReQzCegQGzv5VAK wCYubxPB2Q6XJlzhUIPpzUhA5azOQTTyIE/8R+SkPGSbhqr8wQgcwBYVLgtEl8IPWuOh ozerfx7inWzUZLxL/dlwiAilvkE4hChx0tvu45zH/d1mD9WrKJsnHVXyBa/ZXeLW1gpj tq4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678484559; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ef7PP0SQ7R8DCF78GLBtkjVAZkkgPEB1VXa9M7tBWJI=; b=pKTl05APapL3TmebQIOyBY+q8/xwa4/DcTIhPEDjSPYBRYoP8ZZqT5jT+8uIZSCPQL wnF7qUq6xCBxwBJ82ZttI8rzCvIwL39gQdvcuLPDz/b7EV4rm2kvLq/195g4moDYncqD Yy9BlZduFe05xenY6hZEUJLe0OPRx77zGgi1QMyJeSpJOeblNxTE7p4NRBlgEDzv/HLk P27VE8F+MPkcvk4dsI80OQv6+jAj+pFiZpUoR5AO1EQhJylY0STQENyzk5imO/opHU3W w/yrGpG9FidNaXWVdCycigxx2lpiBHBZWuX/HYV6iUFXZ4F+oChz5YbGt5oSbc0YsI0a MNkQ== X-Gm-Message-State: AO0yUKWMX9T2jhwIbW82NVWplbwTr6hpBCh81Jt2PPSZYKOTu/J9jMMQ cu5ImZwtNlh4FNgLbTVqzyrKLPcSzVM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:a38e:b0:19b:42d:79f8 with SMTP id x14-20020a170902a38e00b0019b042d79f8mr10655656pla.5.1678484558868; Fri, 10 Mar 2023 13:42:38 -0800 (PST) Reply-To: Sean Christopherson Date: Fri, 10 Mar 2023 13:42:15 -0800 In-Reply-To: <20230310214232.806108-1-seanjc@google.com> Mime-Version: 1.0 References: <20230310214232.806108-1-seanjc@google.com> X-Mailer: git-send-email 2.40.0.rc1.284.g88254d51c5-goog Message-ID: <20230310214232.806108-2-seanjc@google.com> Subject: [PATCH v2 01/18] x86/reboot: VMCLEAR active VMCSes before emergency reboot From: Sean Christopherson To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Sean Christopherson , Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Andrew Cooper , Kai Huang , Chao Gao X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760018824609467548?= X-GMAIL-MSGID: =?utf-8?q?1760018824609467548?= VMCLEAR active VMCSes before any emergency reboot, not just if the kernel may kexec into a new kernel after a crash. Per Intel's SDM, the VMX architecture doesn't require the CPU to flush the VMCS cache on INIT. If an emergency reboot doesn't RESET CPUs, cached VMCSes could theoretically be kept and only be written back to memory after the new kernel is booted, i.e. could effectively corrupt memory after reboot. Opportunistically remove the setting of the global pointer to NULL to make checkpatch happy. Cc: Andrew Cooper Signed-off-by: Sean Christopherson --- arch/x86/include/asm/kexec.h | 2 -- arch/x86/include/asm/reboot.h | 2 ++ arch/x86/kernel/crash.c | 31 ------------------------------- arch/x86/kernel/reboot.c | 22 ++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 10 +++------- 5 files changed, 27 insertions(+), 40 deletions(-) diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h index a3760ca796aa..256eee99afc8 100644 --- a/arch/x86/include/asm/kexec.h +++ b/arch/x86/include/asm/kexec.h @@ -208,8 +208,6 @@ int arch_kimage_file_post_load_cleanup(struct kimage *image); #endif #endif -typedef void crash_vmclear_fn(void); -extern crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss; extern void kdump_nmi_shootdown_cpus(void); #endif /* __ASSEMBLY__ */ diff --git a/arch/x86/include/asm/reboot.h b/arch/x86/include/asm/reboot.h index bc5b4d788c08..2551baec927d 100644 --- a/arch/x86/include/asm/reboot.h +++ b/arch/x86/include/asm/reboot.h @@ -25,6 +25,8 @@ void __noreturn machine_real_restart(unsigned int type); #define MRR_BIOS 0 #define MRR_APM 1 +typedef void crash_vmclear_fn(void); +extern crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss; void cpu_emergency_disable_virtualization(void); typedef void (*nmi_shootdown_cb)(int, struct pt_regs*); diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c index cdd92ab43cda..54cd959cb316 100644 --- a/arch/x86/kernel/crash.c +++ b/arch/x86/kernel/crash.c @@ -48,38 +48,12 @@ struct crash_memmap_data { unsigned int type; }; -/* - * This is used to VMCLEAR all VMCSs loaded on the - * processor. And when loading kvm_intel module, the - * callback function pointer will be assigned. - * - * protected by rcu. - */ -crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss = NULL; -EXPORT_SYMBOL_GPL(crash_vmclear_loaded_vmcss); - -static inline void cpu_crash_vmclear_loaded_vmcss(void) -{ - crash_vmclear_fn *do_vmclear_operation = NULL; - - rcu_read_lock(); - do_vmclear_operation = rcu_dereference(crash_vmclear_loaded_vmcss); - if (do_vmclear_operation) - do_vmclear_operation(); - rcu_read_unlock(); -} - #if defined(CONFIG_SMP) && defined(CONFIG_X86_LOCAL_APIC) static void kdump_nmi_callback(int cpu, struct pt_regs *regs) { crash_save_cpu(regs, cpu); - /* - * VMCLEAR VMCSs loaded on all cpus if needed. - */ - cpu_crash_vmclear_loaded_vmcss(); - /* * Disable Intel PT to stop its logging */ @@ -133,11 +107,6 @@ void native_machine_crash_shutdown(struct pt_regs *regs) crash_smp_send_stop(); - /* - * VMCLEAR VMCSs loaded on this cpu if needed. - */ - cpu_crash_vmclear_loaded_vmcss(); - cpu_emergency_disable_virtualization(); /* diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c index d03c551defcc..299b970e5f82 100644 --- a/arch/x86/kernel/reboot.c +++ b/arch/x86/kernel/reboot.c @@ -787,6 +787,26 @@ void machine_crash_shutdown(struct pt_regs *regs) } #endif +/* + * This is used to VMCLEAR all VMCSs loaded on the + * processor. And when loading kvm_intel module, the + * callback function pointer will be assigned. + * + * protected by rcu. + */ +crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss; +EXPORT_SYMBOL_GPL(crash_vmclear_loaded_vmcss); + +static inline void cpu_crash_vmclear_loaded_vmcss(void) +{ + crash_vmclear_fn *do_vmclear_operation = NULL; + + rcu_read_lock(); + do_vmclear_operation = rcu_dereference(crash_vmclear_loaded_vmcss); + if (do_vmclear_operation) + do_vmclear_operation(); + rcu_read_unlock(); +} /* This is the CPU performing the emergency shutdown work. */ int crashing_cpu = -1; @@ -798,6 +818,8 @@ int crashing_cpu = -1; */ void cpu_emergency_disable_virtualization(void) { + cpu_crash_vmclear_loaded_vmcss(); + cpu_emergency_vmxoff(); cpu_emergency_svm_disable(); } diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index bcac3efcde41..302086255be6 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -41,7 +41,7 @@ #include #include #include -#include +#include #include #include #include @@ -743,7 +743,6 @@ static int vmx_set_guest_uret_msr(struct vcpu_vmx *vmx, return ret; } -#ifdef CONFIG_KEXEC_CORE static void crash_vmclear_local_loaded_vmcss(void) { int cpu = raw_smp_processor_id(); @@ -753,7 +752,6 @@ static void crash_vmclear_local_loaded_vmcss(void) loaded_vmcss_on_cpu_link) vmcs_clear(v->vmcs); } -#endif /* CONFIG_KEXEC_CORE */ static void __loaded_vmcs_clear(void *arg) { @@ -8553,10 +8551,9 @@ static void __vmx_exit(void) { allow_smaller_maxphyaddr = false; -#ifdef CONFIG_KEXEC_CORE RCU_INIT_POINTER(crash_vmclear_loaded_vmcss, NULL); synchronize_rcu(); -#endif + vmx_cleanup_l1d_flush(); } @@ -8605,10 +8602,9 @@ static int __init vmx_init(void) pi_init_cpu(cpu); } -#ifdef CONFIG_KEXEC_CORE rcu_assign_pointer(crash_vmclear_loaded_vmcss, crash_vmclear_local_loaded_vmcss); -#endif + vmx_check_vmcs12_offsets(); /*