From patchwork Sat Feb 3 09:12:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Liu X-Patchwork-Id: 196311 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:9bc1:b0:106:209c:c626 with SMTP id op1csp1044884dyc; Sat, 3 Feb 2024 06:30:19 -0800 (PST) X-Google-Smtp-Source: AGHT+IEXW2csnIqLTNFMi9Y8yBi1v9HuG0iU2FYYoywnaQ+YPWmyEijFe/qW3OtmQ5XsGx5U3nKo X-Received: by 2002:a05:620a:2881:b0:783:ae14:9ee5 with SMTP id j1-20020a05620a288100b00783ae149ee5mr5429160qkp.5.1706970619440; Sat, 03 Feb 2024 06:30:19 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706970619; cv=pass; d=google.com; s=arc-20160816; b=VFBjZWTU/xuHfg9y+eKlXPpyubsO+3Q1MzdqI8jgCYyZCZFrdvi5y120SW5yNqG78s lHxnljTTk7yqwUp3l56XOkofXNryCjQymGXd0dmP4ZO5NCXw+iaHrEy+ubez2Q/C14kS /DkGyoRqLBD7neWeelENkypfFlREePVShwseZa1dAIVwWqnmmj8jb5SfxK/tqR6oca9H DB812VcQ2QguHtyk34V2OMGZVaicRVT6Zzc/RfIpuUzjvQMKKfQaTRf7vB1ysKoxQKIJ 8NqUFMbq9yOkve2jU9AA1mIGbg/th0GV+abhnRAqskPA+mmooOMcmud0UYfCvWmP0lK8 CxSg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=moOa5RvG7KEfIPxkO7/GK64XRlrEiZStM6+96txr6XQ=; fh=l5yKkavAu0SN/IylzOw+z8RiGv+vS77qIuEEPoVBrTY=; b=b0WfiHabCE7yJGJSudWoXPfC8CRDtlYg8uoducUjWn6ZGWda1wyBA/H0bWc546Cb0Q C5SMyI8eUynL29Kzfg0+lnvMNgqVUdIWg4ycKskmJQ0U5tSUjB4Ys6Vz0yMpbSWPKSiX /VnZTHhyWifNiMTe+k01huJqtfwA3OezDiPy+weHxbADXVkAhfuYRB4hz9+mNrTLjLVF v2Pnx6wDF4F7wygtlnHVgka3WnpidoQZpn8Y57EIzS/Oe/tHSlihHE4XLJc/TTLlXVJ1 nP2WfQ/PvsRudFCqwDxUT+NaLndfD/NlEU+/gOW3w1fftSKTQRe8alfIUIZOiS7fLe5Q xtoQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=WUvoFJal; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-50992-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-50992-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Forwarded-Encrypted: i=1; AJvYcCWBRqQtVB/g3q1GH+AXqn0PnLCm65dkEcA7vvb4Q9MV93KXk9l7ETyam/rSVXVDijHgsY9cGsCGuq+zIVvf34RzOrfM3Q== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id k13-20020a05620a07ed00b0078407afa8d4si4329617qkk.254.2024.02.03.06.30.19 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 03 Feb 2024 06:30:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-50992-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=WUvoFJal; arc=pass (i=1 dkim=pass dkdomain=intel.com dmarc=pass fromdomain=linux.intel.com); spf=pass (google.com: domain of linux-kernel+bounces-50992-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-50992-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 7165D1C287D1 for ; Sat, 3 Feb 2024 09:05:12 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 57A1D6024E; Sat, 3 Feb 2024 09:01:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="WUvoFJal" Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9B7325FEF8; Sat, 3 Feb 2024 09:01:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.14 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706950868; cv=none; b=uqTvmk30TqcNMYNXiY2DDSVk4NyEZwa6C+JymrqDwGkLcUVk9NIgfXa5N6nAVTvQmkZG1ejlgWirgsHFZadqQ0MyYBxJKQJw0rSQm/S1ATf7bviHX2uUfwdpPBcg+udqLOI8tGjFgIanm1v3OFJN4xmYHdGXDqyM3XIrUki6a94= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706950868; c=relaxed/simple; bh=jaSpkN47fYUzN6obTA+Fv5SVXL8pgNQFR+mSAgkAyoY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=g1+bRW+MUyYl0Bf8xOvOTxbkWj/bm/qosPckRRpIL7zUqs5qiuwQ8ntutdEBbo+bdkjNEkfexsiz7CkTkWe+/4i1Vz9T8Eqr3vflqqkULYh1mOtbUER9ae810DlFqqccK52Fac9Qd8x+FphyQzg7cC7e7MPhqt/TwT6iuZkqG6k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=WUvoFJal; arc=none smtp.client-ip=198.175.65.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706950867; x=1738486867; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=jaSpkN47fYUzN6obTA+Fv5SVXL8pgNQFR+mSAgkAyoY=; b=WUvoFJal5haVfkyljBUyamGtlIUZ1MjP3iQsB8q/hgYQqgp8U9zg9sRY VnpoaGiT4v0tOzA2joZnFdBocnopNyXILz1plYCCx8AA/wygz+WrleDeF cK1j/8vwYsA9x/8hvw+YhACGGSreULeRyZbA+1SpXrhw8BWiGWqbsEm0e dRreTo35TxaG0h78FReEKOWO5zhfgZ1/LMjUXI0aITik3bFIDIN504c/Y hhR9xqnnFX3/2YTPKIWVU/zVgnhmMDPvR7xOA6v5gaCpZMpCwdkWVszBh J2F6dlfoNrl4EaRMLxrOs9T5oDMItXfRxPK1UQUMGvkwqlfeAX6yZner3 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="4132038" X-IronPort-AV: E=Sophos;i="6.05,240,1701158400"; d="scan'208";a="4132038" Received: from fmviesa009.fm.intel.com ([10.60.135.149]) by orvoesa106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Feb 2024 01:01:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,240,1701158400"; d="scan'208";a="291420" Received: from liuzhao-optiplex-7080.sh.intel.com ([10.239.160.36]) by fmviesa009.fm.intel.com with ESMTP; 03 Feb 2024 01:01:00 -0800 From: Zhao Liu To: Paolo Bonzini , Sean Christopherson , "Rafael J . Wysocki" , Daniel Lezcano , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H . Peter Anvin" , kvm@vger.kernel.org, linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org Cc: Ricardo Neri , Len Brown , Zhang Rui , Zhenyu Wang , Zhuocheng Ding , Dapeng Mi , Yanting Jiang , Yongwei Ma , Vineeth Pillai , Suleiman Souhlal , Masami Hiramatsu , David Dai , Saravana Kannan , Zhao Liu Subject: [RFC 14/26] KVM: x86: Introduce the HFI dynamic update request and kvm_x86_ops Date: Sat, 3 Feb 2024 17:12:02 +0800 Message-Id: <20240203091214.411862-15-zhao1.liu@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240203091214.411862-1-zhao1.liu@linux.intel.com> References: <20240203091214.411862-1-zhao1.liu@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789888424512248817 X-GMAIL-MSGID: 1789888424512248817 From: Zhao Liu There're 2 cases that we need to update Guest HFI table dynamically: 1. When Host's HFI table has update, we need to sync the change to Guest. 2. When vCPU thread migrates to another pCPU, we need rebuild the new Guest HFI table based the HFI data of the pCPU that the vCPU is running on. So add the updating mechanism with a new request and a new op to prepare for the above 2 cases: - New KVM request type to perform HFI updating at vcpu_enter_guest(). Updating the VM's HFI table will result in writing to the VM's memory. This requires vCPU context, so we pend HFI updates via kvm request until vCPU is running. Here we only make request for one vCPU per VM because all vCPUs of the same VM share the same HFI table. This allows one vCPU to update the HFI table for the entire VM. - New kvm_x86_op (optional for x86). When KVM processes KVM_REQ_HFI_UPDATE, this ops is called to update the corresponding HFI table raw for the specified vCPU. Tested-by: Yanting Jiang Signed-off-by: Zhao Liu --- arch/x86/include/asm/kvm-x86-ops.h | 3 ++- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/vmx/vmx.c | 30 ++++++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 2 ++ 4 files changed, 36 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index 378ed944b849..1b16de7a03eb 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -136,8 +136,9 @@ KVM_X86_OP_OPTIONAL(migrate_timers) KVM_X86_OP(msr_filter_changed) KVM_X86_OP(complete_emulated_msr) KVM_X86_OP(vcpu_deliver_sipi_vector) -KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons); +KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons) KVM_X86_OP_OPTIONAL(get_untagged_addr) +KVM_X86_OP_OPTIONAL(update_hfi) #undef KVM_X86_OP #undef KVM_X86_OP_OPTIONAL diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index b5b2d0fde579..e476a86b0766 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -121,6 +121,7 @@ KVM_ARCH_REQ_FLAGS(31, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) #define KVM_REQ_HV_TLB_FLUSH \ KVM_ARCH_REQ_FLAGS(32, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) +#define KVM_REQ_HFI_UPDATE KVM_ARCH_REQ(33) #define CR0_RESERVED_BITS \ (~(unsigned long)(X86_CR0_PE | X86_CR0_MP | X86_CR0_EM | X86_CR0_TS \ @@ -1794,6 +1795,7 @@ struct kvm_x86_ops { unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu); gva_t (*get_untagged_addr)(struct kvm_vcpu *vcpu, gva_t gva, unsigned int flags); + void (*update_hfi)(struct kvm_vcpu *vcpu); }; struct kvm_x86_nested_ops { diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 7881f6b51daa..93c47ba0817b 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -1651,6 +1651,35 @@ static void vmx_update_hfi_table(struct kvm *kvm) vmx_inject_therm_interrupt(kvm_get_vcpu(kvm, 0)); } +static void vmx_dynamic_update_hfi_table(struct kvm_vcpu *vcpu) +{ + struct kvm_vmx *kvm_vmx = to_kvm_vmx(vcpu->kvm); + struct hfi_desc *kvm_vmx_hfi = &kvm_vmx->pkg_therm.hfi_desc; + + if (!intel_hfi_enabled()) + return; + + mutex_lock(&kvm_vmx->pkg_therm.pkg_therm_lock); + + /* + * If Guest hasn't handled the previous update, just mark a pending + * flag to indicate that Host has more updates that KVM needs to sync. + */ + if (kvm_vmx_hfi->hfi_update_status) { + kvm_vmx_hfi->hfi_update_pending = true; + mutex_unlock(&kvm_vmx->pkg_therm.pkg_therm_lock); + return; + } + + /* + * The virtual HFI table is maintained at VM level so that vCPUs + * of the same VM are sharing the one HFI table. Therefore, one + * vCPU can update the HFI table for the whole VM. + */ + vmx_update_hfi_table(vcpu->kvm); + mutex_unlock(&kvm_vmx->pkg_therm.pkg_therm_lock); +} + /* * Switches to specified vcpu, until a matching vcpu_put(), but assumes * vcpu mutex is already taken. @@ -8703,6 +8732,7 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector, .get_untagged_addr = vmx_get_untagged_addr, + .update_hfi = vmx_dynamic_update_hfi_table, }; static unsigned int vmx_handle_intel_pt_intr(void) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 7d787ced513f..bea3def6a4b1 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10850,6 +10850,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) if (kvm_check_request(KVM_REQ_UPDATE_CPU_DIRTY_LOGGING, vcpu)) static_call(kvm_x86_update_cpu_dirty_logging)(vcpu); + if (kvm_check_request(KVM_REQ_HFI_UPDATE, vcpu)) + static_call(kvm_x86_update_hfi)(vcpu); } if (kvm_check_request(KVM_REQ_EVENT, vcpu) || req_int_win ||