From patchwork Sun Oct 30 06:23:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 12918 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp1667124wru; Sat, 29 Oct 2022 23:32:45 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7wjAHbTyWu2V+IceJFkBD4qYYz7PvIs4Nsub5ahkcqE3pdvEDp2426TMJBoxOBHBeKgoVy X-Received: by 2002:a17:90b:50e:b0:213:d7cb:83e3 with SMTP id r14-20020a17090b050e00b00213d7cb83e3mr278988pjz.232.1667111565319; Sat, 29 Oct 2022 23:32:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667111565; cv=none; d=google.com; s=arc-20160816; b=CWGdzzw7LLE683UE0Ox6pyvHoe23zRQTYrVvSBNNT/eKPQ0i9ulpjXBrmjS5Dx244f A6PiEsM20OV5MvyaJ7Ik2sWsKfVF9PTeA+RvAafy3bO3PvvYN0T2qakcNXMJX1qs/4p/ qm9OuBbnBGeA58Z+/KgDGRNzaAazSn32dPBlNfz+fuulUciZna0MKyU9rSOEDS5hB0qC 2lTv51XuQn2RFDP0mSIkveYOg1ME3rqm98sv7OXgHKY1Laag876OU4YTLspW8SbMI3rG wne/sp6q7gzbUq8wo2uKqIrSuxE1OF7l/IqAIhOh64FUpwR2XlPdMdGk72L/+QVlQV5T F5/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=PuLixola/pmuUouMVgTh0Pmxg328WE63T8fLiplCd9Q=; b=og4sul5Dw2Qmr4ojTOUmV1URH6S0o7liBiVnBgVIlPg+lQLA+ocP0mpiWslUqJT08x WYEMCHYAQblHEO9aAVsyLJHgLYTwZ68KKkQ215NA2jGERY3UQC/7dWo53QFw+WsvNNIO DwH8Go/MjfGjnqCUYUlzLOLqkP9urkoTkEYnzdvyDWp4kwF+6IhZoPmqRvf0imzV1Kb8 GEDJ8OEgJYhlql4wrKK0AsB+w4VZfUt06t1Db3gwjxUzkl5BX28HW/uqSQCHkeOwCdLf 06TRSMN3hQEYw0yefN2tBblUZ74CU7wjZR8qO2RkBYDNBVNYnDlMrvX3vmc/KrGwRf66 Y1Gg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=daJ4Om+s; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l14-20020a17090b078e00b0020d2d54066csi3782127pjz.171.2022.10.29.23.32.32; Sat, 29 Oct 2022 23:32:45 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=daJ4Om+s; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231434AbiJ3Gbk (ORCPT + 99 others); Sun, 30 Oct 2022 02:31:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58236 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231285AbiJ3GaI (ORCPT ); Sun, 30 Oct 2022 02:30:08 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4F4BD3A1; Sat, 29 Oct 2022 23:24:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1667111090; x=1698647090; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=U1/poVuEMcIZBzQv/9oQmn7JVsou64bidpPaDtiRnlQ=; b=daJ4Om+seASM5P0V4wQAXce6GQO7uWG340Xb8NgPN/+WMz+Ppk1GTdaV 23cHGe1bW2w6TntkznvGV1BJXqnLj5+wWjFGcbG3MxnTXI1X7lFFimGvN 4VQ7wAUby7QHLAeJ02ufbESJXpVEZ6xuhJs5i7UZ+RiarXW2oZEHvuYHE OkN965uaI1k2jCWi79Ehhoa3Yl5hgkVNnjtMD+KhhMsYk704FWpwfYy+r /ZIeqTPiE50FELqNhU5U5pAWgE7KnU3OxSoHtBvCfOvIEtDjoNt36iZ9c vBv1L3h5S/QPaQcoPbT/9CBJWXMs8DEQFNIMzCf/TkwK3oCQPgYgsSma4 A==; X-IronPort-AV: E=McAfee;i="6500,9779,10515"; a="395037198" X-IronPort-AV: E=Sophos;i="5.95,225,1661842800"; d="scan'208";a="395037198" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Oct 2022 23:24:11 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10515"; a="878393116" X-IronPort-AV: E=Sophos;i="5.95,225,1661842800"; d="scan'208";a="878393116" Received: from ls.sc.intel.com (HELO localhost) ([143.183.96.54]) by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Oct 2022 23:24:11 -0700 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , David Matlack , Sean Christopherson Subject: [PATCH v10 085/108] KVM: VMX: Move NMI/exception handler to common helper Date: Sat, 29 Oct 2022 23:23:26 -0700 Message-Id: <2954b502dd4477a439e1ff1ffc299997ba91c5f7.1667110240.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748093176225200052?= X-GMAIL-MSGID: =?utf-8?q?1748093176225200052?= From: Sean Christopherson TDX mostly handles NMI/exception exit mostly the same to VMX case. The difference is how to retrieve exit qualification. To share the code with TDX, move NMI/exception to a common header, common.h. Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/common.h | 70 ++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 79 ++++----------------------------------- 2 files changed, 78 insertions(+), 71 deletions(-) diff --git a/arch/x86/kvm/vmx/common.h b/arch/x86/kvm/vmx/common.h index 747f993cf7de..04836d88baaa 100644 --- a/arch/x86/kvm/vmx/common.h +++ b/arch/x86/kvm/vmx/common.h @@ -4,8 +4,78 @@ #include +#include + #include "posted_intr.h" #include "mmu.h" +#include "vmcs.h" +#include "x86.h" + +extern unsigned long vmx_host_idt_base; +void vmx_do_interrupt_nmi_irqoff(unsigned long entry); + +static inline void vmx_handle_interrupt_nmi_irqoff(struct kvm_vcpu *vcpu, + unsigned long entry) +{ + bool is_nmi = entry == (unsigned long)asm_exc_nmi_noist; + + kvm_before_interrupt(vcpu, is_nmi ? KVM_HANDLING_NMI : KVM_HANDLING_IRQ); + vmx_do_interrupt_nmi_irqoff(entry); + kvm_after_interrupt(vcpu); +} + +static inline void vmx_handle_nm_fault_irqoff(struct kvm_vcpu *vcpu) +{ + /* + * Save xfd_err to guest_fpu before interrupt is enabled, so the + * MSR value is not clobbered by the host activity before the guest + * has chance to consume it. + * + * Do not blindly read xfd_err here, since this exception might + * be caused by L1 interception on a platform which doesn't + * support xfd at all. + * + * Do it conditionally upon guest_fpu::xfd. xfd_err matters + * only when xfd contains a non-zero value. + * + * Queuing exception is done in vmx_handle_exit. See comment there. + */ + if (vcpu->arch.guest_fpu.fpstate->xfd) + rdmsrl(MSR_IA32_XFD_ERR, vcpu->arch.guest_fpu.xfd_err); +} + +static inline void vmx_handle_exception_nmi_irqoff(struct kvm_vcpu *vcpu, + u32 intr_info) +{ + const unsigned long nmi_entry = (unsigned long)asm_exc_nmi_noist; + + /* if exit due to PF check for async PF */ + if (is_page_fault(intr_info)) + vcpu->arch.apf.host_apf_flags = kvm_read_and_reset_apf_flags(); + /* if exit due to NM, handle before interrupts are enabled */ + else if (is_nm_fault(intr_info)) + vmx_handle_nm_fault_irqoff(vcpu); + /* Handle machine checks before interrupts are enabled */ + else if (is_machine_check(intr_info)) + kvm_machine_check(); + /* We need to handle NMIs before interrupts are enabled */ + else if (is_nmi(intr_info)) + vmx_handle_interrupt_nmi_irqoff(vcpu, nmi_entry); +} + +static inline void vmx_handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu, + u32 intr_info) +{ + unsigned int vector = intr_info & INTR_INFO_VECTOR_MASK; + gate_desc *desc = (gate_desc *)vmx_host_idt_base + vector; + + if (KVM_BUG(!is_external_intr(intr_info), vcpu->kvm, + "KVM: unexpected VM-Exit interrupt info: 0x%x", intr_info)) + return; + + vmx_handle_interrupt_nmi_irqoff(vcpu, gate_offset(desc)); + vcpu->arch.at_instruction_boundary = true; +} static inline int __vmx_handle_ept_violation(struct kvm_vcpu *vcpu, gpa_t gpa, unsigned long exit_qualification) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 79d8d6a89516..ee2705707266 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -525,7 +525,7 @@ static inline void vmx_segment_cache_clear(struct vcpu_vmx *vmx) vmx->segment_cache.bitmask = 0; } -static unsigned long host_idt_base; +unsigned long vmx_host_idt_base; #if IS_ENABLED(CONFIG_HYPERV) static bool __read_mostly enlightened_vmcs = true; @@ -4236,7 +4236,7 @@ void vmx_set_constant_host_state(struct vcpu_vmx *vmx) vmcs_write16(HOST_SS_SELECTOR, __KERNEL_DS); /* 22.2.4 */ vmcs_write16(HOST_TR_SELECTOR, GDT_ENTRY_TSS*8); /* 22.2.4 */ - vmcs_writel(HOST_IDTR_BASE, host_idt_base); /* 22.2.4 */ + vmcs_writel(HOST_IDTR_BASE, vmx_host_idt_base); /* 22.2.4 */ vmcs_writel(HOST_RIP, (unsigned long)vmx_vmexit); /* 22.2.5 */ @@ -5120,10 +5120,10 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu) intr_info = vmx_get_intr_info(vcpu); if (is_machine_check(intr_info) || is_nmi(intr_info)) - return 1; /* handled by handle_exception_nmi_irqoff() */ + return 1; /* handled by vmx_handle_exception_nmi_irqoff() */ /* - * Queue the exception here instead of in handle_nm_fault_irqoff(). + * Queue the exception here instead of in vmx_handle_nm_fault_irqoff(). * This ensures the nested_vmx check is not skipped so vmexit can * be reflected to L1 (when it intercepts #NM) before reaching this * point. @@ -6798,70 +6798,6 @@ void vmx_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap) vmcs_write64(EOI_EXIT_BITMAP3, eoi_exit_bitmap[3]); } -void vmx_do_interrupt_nmi_irqoff(unsigned long entry); - -static void handle_interrupt_nmi_irqoff(struct kvm_vcpu *vcpu, - unsigned long entry) -{ - bool is_nmi = entry == (unsigned long)asm_exc_nmi_noist; - - kvm_before_interrupt(vcpu, is_nmi ? KVM_HANDLING_NMI : KVM_HANDLING_IRQ); - vmx_do_interrupt_nmi_irqoff(entry); - kvm_after_interrupt(vcpu); -} - -static void handle_nm_fault_irqoff(struct kvm_vcpu *vcpu) -{ - /* - * Save xfd_err to guest_fpu before interrupt is enabled, so the - * MSR value is not clobbered by the host activity before the guest - * has chance to consume it. - * - * Do not blindly read xfd_err here, since this exception might - * be caused by L1 interception on a platform which doesn't - * support xfd at all. - * - * Do it conditionally upon guest_fpu::xfd. xfd_err matters - * only when xfd contains a non-zero value. - * - * Queuing exception is done in vmx_handle_exit. See comment there. - */ - if (vcpu->arch.guest_fpu.fpstate->xfd) - rdmsrl(MSR_IA32_XFD_ERR, vcpu->arch.guest_fpu.xfd_err); -} - -static void handle_exception_nmi_irqoff(struct kvm_vcpu *vcpu, u32 intr_info) -{ - const unsigned long nmi_entry = (unsigned long)asm_exc_nmi_noist; - - /* if exit due to PF check for async PF */ - if (is_page_fault(intr_info)) - vcpu->arch.apf.host_apf_flags = kvm_read_and_reset_apf_flags(); - /* if exit due to NM, handle before interrupts are enabled */ - else if (is_nm_fault(intr_info)) - handle_nm_fault_irqoff(vcpu); - /* Handle machine checks before interrupts are enabled */ - else if (is_machine_check(intr_info)) - kvm_machine_check(); - /* We need to handle NMIs before interrupts are enabled */ - else if (is_nmi(intr_info)) - handle_interrupt_nmi_irqoff(vcpu, nmi_entry); -} - -static void handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu, - u32 intr_info) -{ - unsigned int vector = intr_info & INTR_INFO_VECTOR_MASK; - gate_desc *desc = (gate_desc *)host_idt_base + vector; - - if (KVM_BUG(!is_external_intr(intr_info), vcpu->kvm, - "KVM: unexpected VM-Exit interrupt info: 0x%x", intr_info)) - return; - - handle_interrupt_nmi_irqoff(vcpu, gate_offset(desc)); - vcpu->arch.at_instruction_boundary = true; -} - void vmx_handle_exit_irqoff(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); @@ -6870,9 +6806,10 @@ void vmx_handle_exit_irqoff(struct kvm_vcpu *vcpu) return; if (vmx->exit_reason.basic == EXIT_REASON_EXTERNAL_INTERRUPT) - handle_external_interrupt_irqoff(vcpu, vmx_get_intr_info(vcpu)); + vmx_handle_external_interrupt_irqoff(vcpu, + vmx_get_intr_info(vcpu)); else if (vmx->exit_reason.basic == EXIT_REASON_EXCEPTION_NMI) - handle_exception_nmi_irqoff(vcpu, vmx_get_intr_info(vcpu)); + vmx_handle_exception_nmi_irqoff(vcpu, vmx_get_intr_info(vcpu)); } /* @@ -8120,7 +8057,7 @@ __init int vmx_hardware_setup(void) int r; store_idt(&dt); - host_idt_base = dt.address; + vmx_host_idt_base = dt.address; vmx_setup_user_return_msrs();