From patchwork Mon Oct 2 11:57:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Levitsky X-Patchwork-Id: 147506 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2a8e:b0:403:3b70:6f57 with SMTP id in14csp1716248vqb; Mon, 2 Oct 2023 15:05:00 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEZwEikqHLDnNyOhN9/jNm9XX8NtMnZVsNtLg6a7gKkMGoCmOOXQDEsZgsqw7cexy8k41t4 X-Received: by 2002:a05:6358:7e07:b0:142:fba8:dcfc with SMTP id o7-20020a0563587e0700b00142fba8dcfcmr15441417rwm.16.1696284299751; Mon, 02 Oct 2023 15:04:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696284299; cv=none; d=google.com; s=arc-20160816; b=QpV/8AWphsqthied4/FGxqLvq35zeUbLNzo7m86VngT4ooq/eCgDGtoHfpBAyCTck+ 1TIgxpuMwkS5WS39Y1lOGRYO/c8vLdqaSB5XZqSApgcvDmwg4bdVMDcpgtK4X21rYVRr nup6gbsJeCqVz1uGuR9RJ9QgY4u8DKd5gFdC/qed/jlirN98ltSOKOvVgUojuP6EP0W2 YF1ZdSINd2QiZbLLD94Q5fZAIIs17AX9Ibi9vEcHGACNgIzoYKjjHwkAyWV8/rgSHSLg ZAGq6/DwncDaapPLF0/QP7oPS5MpXewxebElyRPpL6eo3ZJFHW5Zs0TLF3tocwwGVMVx HMVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=KOVqc9nE1wzSloZSIZglCt8Fd/e7Bb/HSk7V8DxF+zY=; fh=OogiIvQ2zUR209vET+yuT7qDV8D5we5HX6IGkxNqqHY=; b=EeX3DA6/3MMrJlxbqJkje5y3E6Oc+7FvE/o/Y0v9J8etq0yG1lwMs7CJHqRFDfki7w /u39VZA/i6FJgMFdr3Eaudzw3yurGL4XawO6hedxnElkDqgQiUDTNjDRQp/m1bqRrnAE E6G1bGUREHD2QbnzWpscSANWvz1KZp2mxGbFJ7PxCjILWGCS/Tv1nGc+oKYp6uMdN1Rk RScNHOAXOTRSMzJTjS0SfO1oaFGGLl7eKKdlxaNui5km9wuZ7pF9U6LFl9HC6kNBzzBz OsSVEGUMGxQ8VKiezVAA9JoRaP33aKO2DhXcxl1de4qSZs2F8DGRBUlA+0zFDbX6LV9L NPJg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=gyRNX6aj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id f7-20020a056a001ac700b0068fdfe29f0asi29553744pfv.88.2023.10.02.15.04.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Oct 2023 15:04:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=gyRNX6aj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 8D64E806A60A; Mon, 2 Oct 2023 04:59:02 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236843AbjJBL6n (ORCPT + 18 others); Mon, 2 Oct 2023 07:58:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55636 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236855AbjJBL6e (ORCPT ); Mon, 2 Oct 2023 07:58:34 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D1FBCE1 for ; Mon, 2 Oct 2023 04:57:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696247862; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KOVqc9nE1wzSloZSIZglCt8Fd/e7Bb/HSk7V8DxF+zY=; b=gyRNX6aj63lvovshkCfRcf+x1UYYyia77hDy5WQIgr7+2rl393YrQgBU3D0WbqyFEnck9/ m2yF7hpN28k98JIlJMxL/Al0LxkMOMkexy4PfuUYN997JDlXj9j+5FZhzEpUi5+qUmjqRO 4Zb8ST392UOWRCiBaE79dtQT5pKVajo= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-217-AAsvoonTPgymJCVqpBS66A-1; Mon, 02 Oct 2023 07:57:33 -0400 X-MC-Unique: AAsvoonTPgymJCVqpBS66A-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5225F101B042; Mon, 2 Oct 2023 11:57:32 +0000 (UTC) Received: from localhost.localdomain (unknown [10.45.224.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id 12BAE140E953; Mon, 2 Oct 2023 11:57:28 +0000 (UTC) From: Maxim Levitsky To: kvm@vger.kernel.org Cc: Will Deacon , linux-kernel@vger.kernel.org, Borislav Petkov , Dave Hansen , x86@kernel.org, Ingo Molnar , "H. Peter Anvin" , Thomas Gleixner , Joerg Roedel , Suravee Suthikulpanit , Sean Christopherson , Maxim Levitsky , Robin Murphy , iommu@lists.linux.dev, Paolo Bonzini Subject: [PATCH v3 1/4] KVM: Add per vCPU flag specifying that a vCPU is loaded Date: Mon, 2 Oct 2023 14:57:20 +0300 Message-Id: <20231002115723.175344-2-mlevitsk@redhat.com> In-Reply-To: <20231002115723.175344-1-mlevitsk@redhat.com> References: <20231002115723.175344-1-mlevitsk@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Mon, 02 Oct 2023 04:59:02 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778683005806985278 X-GMAIL-MSGID: 1778683005806985278 Add vcpu->loaded boolean flag specifying that a vCPU is loaded. Such flag can be useful in a vendor code (e.g AVIC) to make decisions based on it. Signed-off-by: Maxim Levitsky --- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 10 ++++++++++ 2 files changed, 11 insertions(+) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index fb6c6109fdcad69..331432d86e44d51 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -379,6 +379,7 @@ struct kvm_vcpu { #endif bool preempted; bool ready; + bool loaded; struct kvm_vcpu_arch arch; struct kvm_vcpu_stat stat; char stats_id[KVM_STATS_NAME_SIZE]; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 486800a7024b373..615f2a02b7cb97f 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -214,6 +214,10 @@ void vcpu_load(struct kvm_vcpu *vcpu) __this_cpu_write(kvm_running_vcpu, vcpu); preempt_notifier_register(&vcpu->preempt_notifier); kvm_arch_vcpu_load(vcpu, cpu); + + /* Ensure that vcpu->cpu is visible before vcpu->loaded is set to true */ + smp_wmb(); + WRITE_ONCE(vcpu->loaded, true); put_cpu(); } EXPORT_SYMBOL_GPL(vcpu_load); @@ -221,6 +225,12 @@ EXPORT_SYMBOL_GPL(vcpu_load); void vcpu_put(struct kvm_vcpu *vcpu) { preempt_disable(); + WRITE_ONCE(vcpu->loaded, false); + /* + * Ensure that vcpu->loaded is set and visible, + * before KVM actually unloads the vCPU. + */ + smp_wmb(); kvm_arch_vcpu_put(vcpu); preempt_notifier_unregister(&vcpu->preempt_notifier); __this_cpu_write(kvm_running_vcpu, NULL); From patchwork Mon Oct 2 11:57:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Levitsky X-Patchwork-Id: 147471 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2a8e:b0:403:3b70:6f57 with SMTP id in14csp1691903vqb; Mon, 2 Oct 2023 14:08:40 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEYmNlFz2guHT02znoPDevU5J/MNj0gVigRcX9r8pLBpof3aZeCtkVHCUFs/u0lERsmJHD1 X-Received: by 2002:a05:6a20:748d:b0:158:7fdf:66df with SMTP id p13-20020a056a20748d00b001587fdf66dfmr15256135pzd.18.1696280920444; Mon, 02 Oct 2023 14:08:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696280920; cv=none; d=google.com; s=arc-20160816; b=CAFc57TVQfvbsd5Z71aH7pYMa57P2e3tZJMBqlpjE5NLe3zRSPdiih6hMfebnbkulF qa9RfTa5cVoQhFiY39l+QBT8kOJ9ssKZNYbnb6NRON6XVJP+CGkdxr6wU0bxYWeNGa4h MYwcmBfjbQsD4bJmM0CPBn26lmapkDa+giaslOrbrKGDqw6EH2HemRtSbC/o90xsnSjG BUfZeXhlN7z8TkdQKP+BRsOpSUJRMA+w3Vwx+7pYi+5ajJxY15WU2GW2PfpW4Co7Srlf Nw07pXlbHHhOfKpIPp7YH0f8L5ynOBUfy9pkFRqmPrNSMGLPDzNT9WStHvvGtzH+sSjc WgbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=k2JA8Z38D8eUi/xmH+rBm9nb9Bwfoc30nugRqNPb0OY=; fh=OogiIvQ2zUR209vET+yuT7qDV8D5we5HX6IGkxNqqHY=; b=Sk00o7O0DKrLo++/0tku9usi4Hq18dtAfsSxh1whcIpAqmxdY9BQWgH0sfXmaHGAv/ tZ3unY9j/shhQKmDG9FSQ1OJG+Pwdi/JMzu/enBTm8OiNBSs0jJV6F+QOckrZ7m4K3zU Nha1hE9lilTX1diMNkWQYlL9i3nLKrFNxjU3KrVc5x3h/k+r6WEecXLOgdJHv0de+l1P nX62k2F6xfFC/Hla0vAs3rydNlex+fwx7w4azb/FwkU73eDFfGVT60qunh4G3oFPQyzC wiBQxJYS17uhoO2LSraWmE973LrrV09y25x6UxATSeTkzLSmiaYJ2XRHlv2c3pvBNNnG xSAg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=d4fMeZt5; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from groat.vger.email (groat.vger.email. [2620:137:e000::3:5]) by mx.google.com with ESMTPS id w14-20020a056a0014ce00b0068a6eb3b548si30347952pfu.401.2023.10.02.14.08.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Oct 2023 14:08:40 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=d4fMeZt5; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 6225B80965BD; Mon, 2 Oct 2023 04:59:00 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236835AbjJBL62 (ORCPT + 18 others); Mon, 2 Oct 2023 07:58:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55614 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236604AbjJBL6Z (ORCPT ); Mon, 2 Oct 2023 07:58:25 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 65F0ED8 for ; Mon, 2 Oct 2023 04:57:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696247860; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=k2JA8Z38D8eUi/xmH+rBm9nb9Bwfoc30nugRqNPb0OY=; b=d4fMeZt522x/Yg7vVF3sug+c1n8kyw1YcJ95IBZ7hLG+B4pyPnmqfc1sR8a2P7NJyv1M+v 9v6/RfRAtWakLpBOLwDBdPup3CUajvyw6hMUBmbh2uoN16tSmVqQG62UAFnh4rL0l+szfM FvTUtG/jcn+QPdYXRUdOS/tVbco2Llk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-249-CWvtSVzZOEiXnzY8rUs16w-1; Mon, 02 Oct 2023 07:57:37 -0400 X-MC-Unique: CWvtSVzZOEiXnzY8rUs16w-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E4A5F8007A4; Mon, 2 Oct 2023 11:57:35 +0000 (UTC) Received: from localhost.localdomain (unknown [10.45.224.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id A5CB6140E950; Mon, 2 Oct 2023 11:57:32 +0000 (UTC) From: Maxim Levitsky To: kvm@vger.kernel.org Cc: Will Deacon , linux-kernel@vger.kernel.org, Borislav Petkov , Dave Hansen , x86@kernel.org, Ingo Molnar , "H. Peter Anvin" , Thomas Gleixner , Joerg Roedel , Suravee Suthikulpanit , Sean Christopherson , Maxim Levitsky , Robin Murphy , iommu@lists.linux.dev, Paolo Bonzini Subject: [PATCH v3 2/4] x86: KVM: AVIC: stop using 'is_running' bit in avic_vcpu_put() Date: Mon, 2 Oct 2023 14:57:21 +0300 Message-Id: <20231002115723.175344-3-mlevitsk@redhat.com> In-Reply-To: <20231002115723.175344-1-mlevitsk@redhat.com> References: <20231002115723.175344-1-mlevitsk@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Mon, 02 Oct 2023 04:59:00 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778679462188419358 X-GMAIL-MSGID: 1778679462188419358 An optimization was added to avic_vcpu_load() in commit 782f64558de7 ("KVM: SVM: Skip AVIC and IRTE updates when loading blocking vCPU") to avoid re-enabling AVIC if the vCPU is about to block. Such situation arises when a vCPU thread is preempted in between the call to kvm_arch_vcpu_blocking() and before the matching call to kvm_arch_vcpu_unblocking() which in case of AVIC disables/enables the AVIC on this vCPU. The same optimization was done in avic_vcpu_put() however the code was based on physical id table's 'is_running' bit, building upon assumption that if avic_vcpu_load() didn't set it, then kvm doesn't need to disable avic (since it wasn't really enabled). However, once AVIC's IPI virtualization is made optional, this bit might be always false regardless if a vCPU is running or not. To fix this, instead of checking this bit, check the same 'kvm_vcpu_is_blocking()' condition. Also, as a bonus, re-enable the warning for already set 'is_running' bit, if it was found set, during avic_vcpu_put() execution and the vCPU was not blocking a condition which indicates a bug. Signed-off-by: Maxim Levitsky --- arch/x86/kvm/svm/avic.c | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/svm/avic.c b/arch/x86/kvm/svm/avic.c index 2092db892d7d052..4c75ca15999fcd4 100644 --- a/arch/x86/kvm/svm/avic.c +++ b/arch/x86/kvm/svm/avic.c @@ -1075,16 +1075,10 @@ void avic_vcpu_put(struct kvm_vcpu *vcpu) lockdep_assert_preemption_disabled(); /* - * Note, reading the Physical ID entry outside of ir_list_lock is safe - * as only the pCPU that has loaded (or is loading) the vCPU is allowed - * to modify the entry, and preemption is disabled. I.e. the vCPU - * can't be scheduled out and thus avic_vcpu_{put,load}() can't run - * recursively. + * If the vcpu is blocking, there is no need to do anything. + * See the comment in avic_vcpu_load(). */ - entry = READ_ONCE(*(svm->avic_physical_id_cache)); - - /* Nothing to do if IsRunning == '0' due to vCPU blocking. */ - if (!(entry & AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK)) + if (kvm_vcpu_is_blocking(vcpu)) return; /* @@ -1099,6 +1093,9 @@ void avic_vcpu_put(struct kvm_vcpu *vcpu) avic_update_iommu_vcpu_affinity(vcpu, -1, 0); + entry = READ_ONCE(*(svm->avic_physical_id_cache)); + WARN_ON_ONCE(!(entry & AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK)); + entry &= ~AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK; WRITE_ONCE(*(svm->avic_physical_id_cache), entry); From patchwork Mon Oct 2 11:57:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Levitsky X-Patchwork-Id: 147501 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2a8e:b0:403:3b70:6f57 with SMTP id in14csp1713949vqb; Mon, 2 Oct 2023 15:01:13 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGcKY/CAS3NXNgBCGx+tzhnzS3e5S1/5+EyULfRttEfSCnKOjyQ1woYKgc3DHdKh5y7OE/B X-Received: by 2002:a05:6358:2619:b0:145:6e16:fa86 with SMTP id l25-20020a056358261900b001456e16fa86mr16609816rwc.22.1696284073424; Mon, 02 Oct 2023 15:01:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696284073; cv=none; d=google.com; s=arc-20160816; b=uszd5AKY8JO/nq6d9UYE93FTzgcfh4yCT0U6jNfwLwdykiaZxr1iOT0C5Is2yUyYly LYhbZxrky1NbN1n1cD/NjgN6Kc5iBSqjS2GPnS/AGztlacFeZKJyfo3OZ47FcYiJSOxl SWH6IzG5xTUypMp5ynU3ENv4HIqOcJ+leJOR+7oa+szEjY0aT4UCe/wx6YOojxEck64l ivqiYPtHtI5FT6/AHbsR2fM1x2Z/89jSaZQ9ynv5xw87KaWT7Fa8KH7+VFw7JgO27VIA YILM4KZG0rivudnXz9UYlfxhtPiIQVdG4ACQM66UWWOtE2RoeZJZVWpx9/1nZyJPQnHB Z6OA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ar0urwZ0bmfOZ0eV2FeowpE2tYaVs6PwEG08j6g7CC4=; fh=OogiIvQ2zUR209vET+yuT7qDV8D5we5HX6IGkxNqqHY=; b=s/EP3iSxu+h0mZQza+H80T8xJ0BCMr27PAJp2k74Q4rS5sBdUoTTAsdLsXrUx3bjgH oLuGr5/de4YHq/bicNZRtruaBYqe/FQ8fmPNayGuGnW8qJTw0LDrxypVx7uIxLTntKOU RAtYx+P9xnJ0V/pg3Y+w5rdDgWuwAcFvXGPoMWMF7ZJIZaKyGswE8VcKw+DUn3oXHAVx 7ObNwW/eOlFTP7lme730ue5i1pULAI8p9S8vmF90eICZYn1dJY8Oo6avgCrAU0QsJ7pF LeShtwsNjqy4ARAE7i5nxb93yeGkeEbTK0WOTWcLEoJuvpyvERQNVITW8A/371blNcJY uwmw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=ZN8sBt91; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id cp5-20020a056a00348500b006910a45a234si28440710pfb.202.2023.10.02.15.01.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Oct 2023 15:01:13 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=ZN8sBt91; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id A1934806E15C; Mon, 2 Oct 2023 04:58:46 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236883AbjJBL6h (ORCPT + 18 others); Mon, 2 Oct 2023 07:58:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55626 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236836AbjJBL63 (ORCPT ); Mon, 2 Oct 2023 07:58:29 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 77812C6 for ; Mon, 2 Oct 2023 04:57:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696247865; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ar0urwZ0bmfOZ0eV2FeowpE2tYaVs6PwEG08j6g7CC4=; b=ZN8sBt91ZghtDyophpNQfQoAK9mIROIoJ3oHypC7vn5dq1S1i3Tie33ALOtCIUwTP1ZSHM sNtEtRmwUheMnZY83MHnK8WhTjx5XtExvaKLQZX5/sfH6ft6lTAsnxyDDt9Efra+7iYeXr jCuv5OO4MnIuNyq/qt5jl82Jw9iaDZA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-652-eA8zi65nNnCoX1cK0Oq2uA-1; Mon, 02 Oct 2023 07:57:40 -0400 X-MC-Unique: eA8zi65nNnCoX1cK0Oq2uA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F3B4285A5BA; Mon, 2 Oct 2023 11:57:39 +0000 (UTC) Received: from localhost.localdomain (unknown [10.45.224.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4471C140E950; Mon, 2 Oct 2023 11:57:36 +0000 (UTC) From: Maxim Levitsky To: kvm@vger.kernel.org Cc: Will Deacon , linux-kernel@vger.kernel.org, Borislav Petkov , Dave Hansen , x86@kernel.org, Ingo Molnar , "H. Peter Anvin" , Thomas Gleixner , Joerg Roedel , Suravee Suthikulpanit , Sean Christopherson , Maxim Levitsky , Robin Murphy , iommu@lists.linux.dev, Paolo Bonzini Subject: [PATCH v3 3/4] x86: KVM: don't read physical ID table entry in avic_pi_update_irte() Date: Mon, 2 Oct 2023 14:57:22 +0300 Message-Id: <20231002115723.175344-4-mlevitsk@redhat.com> In-Reply-To: <20231002115723.175344-1-mlevitsk@redhat.com> References: <20231002115723.175344-1-mlevitsk@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Mon, 02 Oct 2023 04:58:46 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778682768248735162 X-GMAIL-MSGID: 1778682768248735162 Change AVIC's code to use vcpu->loaded and vcpu->cpu instead of reading back the cpu and 'is_running' bit from the avic's physical id entry. Once AVIC's IPI virtualization is made optional, the 'is_running' bit might always be false regardless if a vCPU is running or not. Signed-off-by: Maxim Levitsky --- arch/x86/kvm/svm/avic.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/svm/avic.c b/arch/x86/kvm/svm/avic.c index 4c75ca15999fcd4..bdab28005ad3405 100644 --- a/arch/x86/kvm/svm/avic.c +++ b/arch/x86/kvm/svm/avic.c @@ -791,7 +791,6 @@ static int svm_ir_list_add(struct vcpu_svm *svm, struct amd_iommu_pi_data *pi) int ret = 0; unsigned long flags; struct amd_svm_iommu_ir *ir; - u64 entry; /** * In some cases, the existing irte is updated and re-set, @@ -832,10 +831,11 @@ static int svm_ir_list_add(struct vcpu_svm *svm, struct amd_iommu_pi_data *pi) * will update the pCPU info when the vCPU awkened and/or scheduled in. * See also avic_vcpu_load(). */ - entry = READ_ONCE(*(svm->avic_physical_id_cache)); - if (entry & AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK) - amd_iommu_update_ga(entry & AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK, - true, pi->ir_data); + if (READ_ONCE(svm->vcpu.loaded)) { + /* Ensure that the vcpu->loaded is read before the vcpu->cpu */ + smp_rmb(); + amd_iommu_update_ga(READ_ONCE(svm->vcpu.cpu), true, pi->ir_data); + } list_add(&ir->node, &svm->ir_list); spin_unlock_irqrestore(&svm->ir_list_lock, flags); From patchwork Mon Oct 2 11:57:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Levitsky X-Patchwork-Id: 147360 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2a8e:b0:403:3b70:6f57 with SMTP id in14csp1535582vqb; Mon, 2 Oct 2023 09:17:19 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEwWazwZg9/xpKGhN1Z8YL2N7MqG64yzVTn7jAFMbF0QbQoECNWvY/FLwLfaVQZXH5tPOIk X-Received: by 2002:a05:6a20:432a:b0:159:f5fb:bf74 with SMTP id h42-20020a056a20432a00b00159f5fbbf74mr13654889pzk.3.1696263438681; Mon, 02 Oct 2023 09:17:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696263438; cv=none; d=google.com; s=arc-20160816; b=QfR2I6hS5KGinCQ+zfQ8dyXgxCpfeFZ2RgKGdd+lbqFvzoiD4JrN1iq05pZqflb7Xt MLjJSf9oDW5ePVYzvYbRzZ0VzEzdtjZq2RzMbspAI2wVEvqa8iegbSVTlqccjutg6gGo 9Yogw9w5123YYG7SbekRdswkoil0ETE3hEzJgFd/SI153x0lgdTxCE+V0opyfTx1XofB DdSRNfCOfXyzmzl2zl2Kwgx7km9FM0uzQ5X7T6jZZh00sd3brRKIGcrhOJI2gEaw5YXt cxJYycIKvNvevN8zy8iz84r+LjcO6sXjmAlo4gIEA4fQmI1JgZkzpSb9ohW/q0KkK/qs fVAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=KRNcUMO/3qZE3ezJDf3zkvX+ebI+kEjoeR2cSJmrJs8=; fh=OogiIvQ2zUR209vET+yuT7qDV8D5we5HX6IGkxNqqHY=; b=eXgUgr7vzr5vsLOfwyi8j1E6cF/u9KaiW4qbEP+2keo9kvcZzAe2a14FqVBP/uH4YU vUCc/VI6xshUhtRD2BaTXkVVm7Ddl1+J4veryUt1Njm4MGP5qEVHj3jbmZKhwlDE/+wl h84ofajNc0sjfHVB32u+hGb2qoEihTn2/XR0Dc4Xq2ggHiBmQe4wUZ0gg9SMdBWy5IFH DkhJbf3BGgbxI+GrsZjo4MyQsEmLOP8b7s7OM7NyocJetc4u6rh7a93OE0L8/NQkmWAM ZrlFCGufykY62NXYwX0Y7ePanc9eK1c9b4PRwbmtircrpRMaUhGFf6OB0NYlUAVq9B08 G2aw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=SEA4rwaw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id y4-20020a636404000000b00578c8ce14edsi28976549pgb.252.2023.10.02.09.17.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Oct 2023 09:17:18 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=SEA4rwaw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 4E706808A91D; Mon, 2 Oct 2023 05:04:17 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236875AbjJBMD6 (ORCPT + 18 others); Mon, 2 Oct 2023 08:03:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47138 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236843AbjJBMD4 (ORCPT ); Mon, 2 Oct 2023 08:03:56 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4FE6110D0 for ; Mon, 2 Oct 2023 05:02:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696248169; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KRNcUMO/3qZE3ezJDf3zkvX+ebI+kEjoeR2cSJmrJs8=; b=SEA4rwawn7EncTuRa72DuAvAyeid85OZjsOV7wMfSlbka1qGJDkKvnb/5oVfIY0sRH/dHn QgGQTIO5jfB818DB+CssgY4VJjs+LCXRMC/DHS93M7ePO3WPOgmO+fT/mDF9Vz89Um1mGl zHAnevh2BO3TEYsnq0DrhNRENEEjs0E= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-499-gLzkORjIPI-dG0TowIN7Kw-1; Mon, 02 Oct 2023 08:02:45 -0400 X-MC-Unique: gLzkORjIPI-dG0TowIN7Kw-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 958F8811E7B; Mon, 2 Oct 2023 11:57:43 +0000 (UTC) Received: from localhost.localdomain (unknown [10.45.224.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5446C140E953; Mon, 2 Oct 2023 11:57:40 +0000 (UTC) From: Maxim Levitsky To: kvm@vger.kernel.org Cc: Will Deacon , linux-kernel@vger.kernel.org, Borislav Petkov , Dave Hansen , x86@kernel.org, Ingo Molnar , "H. Peter Anvin" , Thomas Gleixner , Joerg Roedel , Suravee Suthikulpanit , Sean Christopherson , Maxim Levitsky , Robin Murphy , iommu@lists.linux.dev, Paolo Bonzini Subject: [PATCH v3 4/4] x86: KVM: SVM: allow optionally to disable AVIC's IPI virtualization Date: Mon, 2 Oct 2023 14:57:23 +0300 Message-Id: <20231002115723.175344-5-mlevitsk@redhat.com> In-Reply-To: <20231002115723.175344-1-mlevitsk@redhat.com> References: <20231002115723.175344-1-mlevitsk@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Mon, 02 Oct 2023 05:04:17 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778661131185494960 X-GMAIL-MSGID: 1778661131185494960 On Zen2 (and likely on Zen1 as well), AVIC doesn't reliably detect a change in the 'is_running' bit during ICR write emulation and might skip a VM exit, if that bit was recently cleared. The absence of the VM exit, leads to the KVM not waking up / triggering nested vm exit on the target(s) of the IPI, which can, in some cases, lead to unbounded delays in the guest execution. As I recently discovered, a reasonable workaround exists: make the KVM never set the is_running bit, which in essence disables the IPI virtualization portion of AVIC making it equal to APICv without IPI virtualization. This workaround ensures that (*) all ICR writes always cause a VM exit and therefore correctly emulated, in expense of never enjoying VM exit-less ICR write emulation. To let the user control the workaround, a new kvm_amd module parameter was added: 'enable_ipiv', using the same name as IPI virtualization of VMX. However unlike VMX, this parameter is tri-state: 0, 1, -1. -1 is the default value which instructs KVM to choose the default based on the CPU model. (*) More correctly all ICR writes except when the 'Self' shorthand is used: In this case AVIC skips reading physid table and just sets bits in IRR of local APIC. Thankfully in this case, the errata is not possible, therefore an extra workaround is not needed. Signed-off-by: Maxim Levitsky --- arch/x86/kvm/svm/avic.c | 51 +++++++++++++++++++++++++++++++---------- 1 file changed, 39 insertions(+), 12 deletions(-) diff --git a/arch/x86/kvm/svm/avic.c b/arch/x86/kvm/svm/avic.c index bdab28005ad3405..b3ec693083cc883 100644 --- a/arch/x86/kvm/svm/avic.c +++ b/arch/x86/kvm/svm/avic.c @@ -62,6 +62,9 @@ static_assert(__AVIC_GATAG(AVIC_VM_ID_MASK, AVIC_VCPU_ID_MASK) == -1u); static bool force_avic; module_param_unsafe(force_avic, bool, 0444); +static int enable_ipiv = -1; +module_param(enable_ipiv, int, 0444); + /* Note: * This hash table is used to map VM_ID to a struct kvm_svm, * when handling AMD IOMMU GALOG notification to schedule in @@ -1024,7 +1027,6 @@ avic_update_iommu_vcpu_affinity(struct kvm_vcpu *vcpu, int cpu, bool r) void avic_vcpu_load(struct kvm_vcpu *vcpu, int cpu) { - u64 entry; int h_physical_id = kvm_cpu_get_apicid(cpu); struct vcpu_svm *svm = to_svm(vcpu); unsigned long flags; @@ -1053,14 +1055,22 @@ void avic_vcpu_load(struct kvm_vcpu *vcpu, int cpu) */ spin_lock_irqsave(&svm->ir_list_lock, flags); - entry = READ_ONCE(*(svm->avic_physical_id_cache)); - WARN_ON_ONCE(entry & AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK); + /* + * Do not update the actual physical id table entry, if the IPI + * virtualization portion of AVIC is not enabled. + * In this case all ICR writes except Self IPIs will be intercepted. + */ + + if (enable_ipiv) { + u64 entry = READ_ONCE(*svm->avic_physical_id_cache); - entry &= ~AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK; - entry |= (h_physical_id & AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK); - entry |= AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK; + WARN_ON_ONCE(entry & AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK); + entry &= ~AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK; + entry |= (h_physical_id & AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK); + entry |= AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK; + WRITE_ONCE(*(svm->avic_physical_id_cache), entry); + } - WRITE_ONCE(*(svm->avic_physical_id_cache), entry); avic_update_iommu_vcpu_affinity(vcpu, h_physical_id, true); spin_unlock_irqrestore(&svm->ir_list_lock, flags); @@ -1068,7 +1078,6 @@ void avic_vcpu_load(struct kvm_vcpu *vcpu, int cpu) void avic_vcpu_put(struct kvm_vcpu *vcpu) { - u64 entry; struct vcpu_svm *svm = to_svm(vcpu); unsigned long flags; @@ -1093,11 +1102,17 @@ void avic_vcpu_put(struct kvm_vcpu *vcpu) avic_update_iommu_vcpu_affinity(vcpu, -1, 0); - entry = READ_ONCE(*(svm->avic_physical_id_cache)); - WARN_ON_ONCE(!(entry & AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK)); + /* + * Do not update the actual physical id table entry if the IPI + * virtualization is disabled. See explanation in avic_vcpu_load(). + */ + if (enable_ipiv) { + u64 entry = READ_ONCE(*svm->avic_physical_id_cache); - entry &= ~AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK; - WRITE_ONCE(*(svm->avic_physical_id_cache), entry); + WARN_ON_ONCE(!(entry & AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK)); + entry &= ~AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK; + WRITE_ONCE(*(svm->avic_physical_id_cache), entry); + } spin_unlock_irqrestore(&svm->ir_list_lock, flags); @@ -1211,5 +1226,17 @@ bool avic_hardware_setup(void) amd_iommu_register_ga_log_notifier(&avic_ga_log_notifier); + if (enable_ipiv == -1) { + enable_ipiv = 1; + /* Assume that Zen1 and Zen2 have errata #1235 */ + if (boot_cpu_data.x86 == 0x17) { + pr_info("AVIC's IPI virtualization disabled due to errata #1235\n"); + enable_ipiv = 0; + } + } + + if (enable_ipiv) + pr_info("AVIC's IPI virtualization enabled\n"); + return true; }