From patchwork Sat Feb 11 00:35:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 55661 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp1256936wrn; Fri, 10 Feb 2023 16:41:04 -0800 (PST) X-Google-Smtp-Source: AK7set+Z3Wim+XJQwaH6CrhwWxN0GDN7AbWS7eiRNfKxFxJLIJLz/rf3rRtz9CtrUfEs+9ytM+gh X-Received: by 2002:a17:906:b5a:b0:89f:aa50:22e with SMTP id v26-20020a1709060b5a00b0089faa50022emr15995132ejg.64.1676076063848; Fri, 10 Feb 2023 16:41:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676076063; cv=none; d=google.com; s=arc-20160816; b=eHwwmPlzSPebdmjEIpddiGS6YvzVB7PQyLCiTAl2x6PyAQzL1NgNn7jR7GBSV4m/D/ MCLY8hpkixU/pkvgBPn2zgBYwlTguN1qUOrFXWf9O1dCiYXVwCxlTSiSCN+GFCLC8hyn nWXXl62vFtZ1/viTGlZC9EFJ/wB25FAJPSKF8/i+BawJiyLMz0b+ti2mY736YK0bCvYY VPYvcER508MRStpcQc6btmCqIEzsTFLb/W23VQFAhGmbO+bowwMF5npJwkSV22I68Kde +gY2GVFYctjm7S8Qs7bM2ACy8/kN1C+QCwpEYmfCXQ2p1xpSiTgyGmF1fUwVSxJWJijZ bpuA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:dkim-signature; bh=lmsg+5V35acSdfbJb/dgFwZCkcY3fbnY+ZUgRxDIMyc=; b=bFHBjOqIiGmPWKTerGYinFcE3ByzEnnEui9qHHHFHA9vp09Po2a6ZZ18poGMaODrTa fqy9A1KabJj06jFHs1poIWjQATZTFywoLb9GpmI71GVWCEbhLR5SHL1EZ59qkqfmfSmR Y1xJmYhLcLORvB5hV6f61RE5Y52tDRNOKQi0UzGa1vIs/ovD3X/NxeKXJQ32PKSwmKVs 9+ZMNVLeBSs51sKvxnFB+pB7deEDbUhwUrINrrxvrQ8OWdWLTbfxrEdAz1Dh4lcAbLhd jeSw3p4pmJSQCOZpCFkEAB0IxRd+WllqLrIy188U7h7kYxqvaGInrIZx339ANSkO9hLG zD2g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=BAOAgMai; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bp8-20020a170907918800b007c0c7c42636si6672141ejb.683.2023.02.10.16.40.38; Fri, 10 Feb 2023 16:41:03 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=BAOAgMai; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229712AbjBKAfu (ORCPT + 99 others); Fri, 10 Feb 2023 19:35:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55178 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229657AbjBKAfp (ORCPT ); Fri, 10 Feb 2023 19:35:45 -0500 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [IPv6:2607:f8b0:4864:20::1049]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C553F5A930 for ; Fri, 10 Feb 2023 16:35:41 -0800 (PST) Received: by mail-pj1-x1049.google.com with SMTP id dt6-20020a17090afa4600b00230183006c3so3022515pjb.7 for ; Fri, 10 Feb 2023 16:35:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=lmsg+5V35acSdfbJb/dgFwZCkcY3fbnY+ZUgRxDIMyc=; b=BAOAgMaiw1HfVC6cI2qDycwby87Gn/+VElcBlW7UjLY/VI8ABpo3kU+saozZqz6HA8 mObqLrElTKxPoMdDCXblOecR4tM389vyuVazvjCoMNTjVq5XKVlAosWEud/zlxbApoDo 3MBNOtyYyJCuZsxAy2+qAX08Fze2quXl3/gWedcP1UxsWd7oAc4cbf8zCps2hMRGlApE Bl3ntMnbjAqLXPLuxxbkuYTsmVpi118SYA/4nCfncGDn8i3x9/6qtfiGt+1epC9b4vnc ZvtGAIZ1uz8by2AJllatxH0jpQIlTJrFhyFE0uYX1yizkhP6SLuufky/qNWPBBdxSPBD +jyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=lmsg+5V35acSdfbJb/dgFwZCkcY3fbnY+ZUgRxDIMyc=; b=qmjYZd/zqMCQcyfGYzGK6jrTWBQ2iyCxWph5M9BK7fxyAY1jeb2nJY6J5QPxXJASHK /46tOdScaIsuncJHvSsZitZbVL2pdou82qpU7UL4EbZCpsb/+O/Domuj3F6Moe62rSWI hKBjNF/1s5RKwt+cuIVahISdXpOdYbKDlOvWgi9nIymfhldkhPED9jSGMWcCuTRklvfz LOSNDS78Yoj5ye8r1p/IP9M+cseAdeRPEzFW1VIz16vrMYv2JQSmtnVr6dK21CQWXYso MPNSuLvOj5PMIe9tFgc53EKXPVDC1aCvtO/pzQGH/YbZCmrope8h2+1XIOmiaL2pwMaL NeGg== X-Gm-Message-State: AO0yUKXc2s6mkEUwwPks2g99hQ4/yqr7scHpjFUmMwf51XpaFo19Bdpe yX3YstcDf9fOWsOgQLB5WLTDGhwm8+Y= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a62:61c7:0:b0:593:c940:f0c7 with SMTP id v190-20020a6261c7000000b00593c940f0c7mr3886470pfb.16.1676075740854; Fri, 10 Feb 2023 16:35:40 -0800 (PST) Reply-To: Sean Christopherson Date: Sat, 11 Feb 2023 00:35:33 +0000 In-Reply-To: <20230211003534.564198-1-seanjc@google.com> Mime-Version: 1.0 References: <20230211003534.564198-1-seanjc@google.com> X-Mailer: git-send-email 2.39.1.581.gbfd45094c4-goog Message-ID: <20230211003534.564198-3-seanjc@google.com> Subject: [PATCH v2 2/3] KVM: VMX: Stub out enable_evmcs static key for CONFIG_HYPERV=n From: Sean Christopherson To: Vitaly Kuznetsov , Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757493135146383103?= X-GMAIL-MSGID: =?utf-8?q?1757493135146383103?= Wrap enable_evmcs in a helper and stub it out when CONFIG_HYPERV=n in order to eliminate the static branch nop placeholders. clang-14 is clever enough to elide the nop, but gcc-12 is not. Stubbing out the key reduces the size of kvm-intel.ko by ~7.5% (200KiB) when compiled with gcc-12 (there are a _lot_ of VMCS accesses throughout KVM). Reviewed-by: Vitaly Kuznetsov Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/hyperv.c | 4 ++-- arch/x86/kvm/vmx/hyperv.h | 10 ++++++++-- arch/x86/kvm/vmx/vmx.c | 15 +++++++-------- arch/x86/kvm/vmx/vmx_ops.h | 22 +++++++++++----------- 4 files changed, 28 insertions(+), 23 deletions(-) diff --git a/arch/x86/kvm/vmx/hyperv.c b/arch/x86/kvm/vmx/hyperv.c index b6748055c586..274fbd38c64e 100644 --- a/arch/x86/kvm/vmx/hyperv.c +++ b/arch/x86/kvm/vmx/hyperv.c @@ -118,8 +118,6 @@ #define EVMCS1_SUPPORTED_VMFUNC (0) -DEFINE_STATIC_KEY_FALSE(enable_evmcs); - #define EVMCS1_OFFSET(x) offsetof(struct hv_enlightened_vmcs, x) #define EVMCS1_FIELD(number, name, clean_field)[ROL16(number, 6)] = \ {EVMCS1_OFFSET(name), clean_field} @@ -611,6 +609,8 @@ int nested_evmcs_check_controls(struct vmcs12 *vmcs12) } #if IS_ENABLED(CONFIG_HYPERV) +DEFINE_STATIC_KEY_FALSE(enable_evmcs); + /* * KVM on Hyper-V always uses the latest known eVMCSv1 revision, the assumption * is: in case a feature has corresponding fields in eVMCS described and it was diff --git a/arch/x86/kvm/vmx/hyperv.h b/arch/x86/kvm/vmx/hyperv.h index 1299143d00df..a54a2fdf0a5b 100644 --- a/arch/x86/kvm/vmx/hyperv.h +++ b/arch/x86/kvm/vmx/hyperv.h @@ -16,8 +16,6 @@ struct vmcs_config; -DECLARE_STATIC_KEY_FALSE(enable_evmcs); - #define current_evmcs ((struct hv_enlightened_vmcs *)this_cpu_read(current_vmcs)) #define KVM_EVMCS_VERSION 1 @@ -69,6 +67,13 @@ static inline u64 evmcs_read_any(struct hv_enlightened_vmcs *evmcs, #if IS_ENABLED(CONFIG_HYPERV) +DECLARE_STATIC_KEY_FALSE(enable_evmcs); + +static __always_inline bool kvm_is_using_evmcs(void) +{ + return static_branch_unlikely(&enable_evmcs); +} + static __always_inline int get_evmcs_offset(unsigned long field, u16 *clean_field) { @@ -158,6 +163,7 @@ static inline void evmcs_load(u64 phys_addr) void evmcs_sanitize_exec_ctrls(struct vmcs_config *vmcs_conf); #else /* !IS_ENABLED(CONFIG_HYPERV) */ +static __always_inline bool kvm_is_using_evmcs(void) { return false; } static __always_inline void evmcs_write64(unsigned long field, u64 value) {} static __always_inline void evmcs_write32(unsigned long field, u32 value) {} static __always_inline void evmcs_write16(unsigned long field, u16 value) {} diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 33614ee2cd67..d80cbe01b5d7 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -595,7 +595,7 @@ static void hv_reset_evmcs(void) { struct hv_vp_assist_page *vp_ap; - if (!static_branch_unlikely(&enable_evmcs)) + if (!kvm_is_using_evmcs()) return; /* @@ -2818,8 +2818,7 @@ static int vmx_hardware_enable(void) * This can happen if we hot-added a CPU but failed to allocate * VP assist page for it. */ - if (static_branch_unlikely(&enable_evmcs) && - !hv_get_vp_assist_page(cpu)) + if (kvm_is_using_evmcs() && !hv_get_vp_assist_page(cpu)) return -EFAULT; intel_pt_handle_vmx(1); @@ -2871,7 +2870,7 @@ struct vmcs *alloc_vmcs_cpu(bool shadow, int cpu, gfp_t flags) memset(vmcs, 0, vmcs_config.size); /* KVM supports Enlightened VMCS v1 only */ - if (static_branch_unlikely(&enable_evmcs)) + if (kvm_is_using_evmcs()) vmcs->hdr.revision_id = KVM_EVMCS_VERSION; else vmcs->hdr.revision_id = vmcs_config.revision_id; @@ -2966,7 +2965,7 @@ static __init int alloc_kvm_area(void) * still be marked with revision_id reported by * physical CPU. */ - if (static_branch_unlikely(&enable_evmcs)) + if (kvm_is_using_evmcs()) vmcs->hdr.revision_id = vmcs_config.revision_id; per_cpu(vmxarea, cpu) = vmcs; @@ -3936,7 +3935,7 @@ static void vmx_msr_bitmap_l01_changed(struct vcpu_vmx *vmx) * 'Enlightened MSR Bitmap' feature L0 needs to know that MSR * bitmap has changed. */ - if (IS_ENABLED(CONFIG_HYPERV) && static_branch_unlikely(&enable_evmcs)) { + if (kvm_is_using_evmcs()) { struct hv_enlightened_vmcs *evmcs = (void *)vmx->vmcs01.vmcs; if (evmcs->hv_enlightenments_control.msr_bitmap) @@ -7313,7 +7312,7 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu) vmx_vcpu_enter_exit(vcpu, __vmx_vcpu_run_flags(vmx)); /* All fields are clean at this point */ - if (static_branch_unlikely(&enable_evmcs)) { + if (kvm_is_using_evmcs()) { current_evmcs->hv_clean_fields |= HV_VMX_ENLIGHTENED_CLEAN_FIELD_ALL; @@ -7443,7 +7442,7 @@ static int vmx_vcpu_create(struct kvm_vcpu *vcpu) * feature only for vmcs01, KVM currently isn't equipped to realize any * performance benefits from enabling it for vmcs02. */ - if (IS_ENABLED(CONFIG_HYPERV) && static_branch_unlikely(&enable_evmcs) && + if (kvm_is_using_evmcs() && (ms_hyperv.nested_features & HV_X64_NESTED_MSR_BITMAP)) { struct hv_enlightened_vmcs *evmcs = (void *)vmx->vmcs01.vmcs; diff --git a/arch/x86/kvm/vmx/vmx_ops.h b/arch/x86/kvm/vmx/vmx_ops.h index db95bde52998..ce47dc265f89 100644 --- a/arch/x86/kvm/vmx/vmx_ops.h +++ b/arch/x86/kvm/vmx/vmx_ops.h @@ -147,7 +147,7 @@ static __always_inline unsigned long __vmcs_readl(unsigned long field) static __always_inline u16 vmcs_read16(unsigned long field) { vmcs_check16(field); - if (static_branch_unlikely(&enable_evmcs)) + if (kvm_is_using_evmcs()) return evmcs_read16(field); return __vmcs_readl(field); } @@ -155,7 +155,7 @@ static __always_inline u16 vmcs_read16(unsigned long field) static __always_inline u32 vmcs_read32(unsigned long field) { vmcs_check32(field); - if (static_branch_unlikely(&enable_evmcs)) + if (kvm_is_using_evmcs()) return evmcs_read32(field); return __vmcs_readl(field); } @@ -163,7 +163,7 @@ static __always_inline u32 vmcs_read32(unsigned long field) static __always_inline u64 vmcs_read64(unsigned long field) { vmcs_check64(field); - if (static_branch_unlikely(&enable_evmcs)) + if (kvm_is_using_evmcs()) return evmcs_read64(field); #ifdef CONFIG_X86_64 return __vmcs_readl(field); @@ -175,7 +175,7 @@ static __always_inline u64 vmcs_read64(unsigned long field) static __always_inline unsigned long vmcs_readl(unsigned long field) { vmcs_checkl(field); - if (static_branch_unlikely(&enable_evmcs)) + if (kvm_is_using_evmcs()) return evmcs_read64(field); return __vmcs_readl(field); } @@ -222,7 +222,7 @@ static __always_inline void __vmcs_writel(unsigned long field, unsigned long val static __always_inline void vmcs_write16(unsigned long field, u16 value) { vmcs_check16(field); - if (static_branch_unlikely(&enable_evmcs)) + if (kvm_is_using_evmcs()) return evmcs_write16(field, value); __vmcs_writel(field, value); @@ -231,7 +231,7 @@ static __always_inline void vmcs_write16(unsigned long field, u16 value) static __always_inline void vmcs_write32(unsigned long field, u32 value) { vmcs_check32(field); - if (static_branch_unlikely(&enable_evmcs)) + if (kvm_is_using_evmcs()) return evmcs_write32(field, value); __vmcs_writel(field, value); @@ -240,7 +240,7 @@ static __always_inline void vmcs_write32(unsigned long field, u32 value) static __always_inline void vmcs_write64(unsigned long field, u64 value) { vmcs_check64(field); - if (static_branch_unlikely(&enable_evmcs)) + if (kvm_is_using_evmcs()) return evmcs_write64(field, value); __vmcs_writel(field, value); @@ -252,7 +252,7 @@ static __always_inline void vmcs_write64(unsigned long field, u64 value) static __always_inline void vmcs_writel(unsigned long field, unsigned long value) { vmcs_checkl(field); - if (static_branch_unlikely(&enable_evmcs)) + if (kvm_is_using_evmcs()) return evmcs_write64(field, value); __vmcs_writel(field, value); @@ -262,7 +262,7 @@ static __always_inline void vmcs_clear_bits(unsigned long field, u32 mask) { BUILD_BUG_ON_MSG(__builtin_constant_p(field) && ((field) & 0x6000) == 0x2000, "vmcs_clear_bits does not support 64-bit fields"); - if (static_branch_unlikely(&enable_evmcs)) + if (kvm_is_using_evmcs()) return evmcs_write32(field, evmcs_read32(field) & ~mask); __vmcs_writel(field, __vmcs_readl(field) & ~mask); @@ -272,7 +272,7 @@ static __always_inline void vmcs_set_bits(unsigned long field, u32 mask) { BUILD_BUG_ON_MSG(__builtin_constant_p(field) && ((field) & 0x6000) == 0x2000, "vmcs_set_bits does not support 64-bit fields"); - if (static_branch_unlikely(&enable_evmcs)) + if (kvm_is_using_evmcs()) return evmcs_write32(field, evmcs_read32(field) | mask); __vmcs_writel(field, __vmcs_readl(field) | mask); @@ -289,7 +289,7 @@ static inline void vmcs_load(struct vmcs *vmcs) { u64 phys_addr = __pa(vmcs); - if (static_branch_unlikely(&enable_evmcs)) + if (kvm_is_using_evmcs()) return evmcs_load(phys_addr); vmx_asm1(vmptrld, "m"(phys_addr), vmcs, phys_addr);