From patchwork Fri Jun 2 01:15:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 102256 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp724614vqr; Thu, 1 Jun 2023 18:25:59 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4Xtei2KjOdHbMro/kIzW9TPMwyFeebhNBdsExFWi8u9fqMdMfMeb6frvXyGvDR+ST7eVOS X-Received: by 2002:a92:d907:0:b0:33a:9ccb:2259 with SMTP id s7-20020a92d907000000b0033a9ccb2259mr7615857iln.3.1685669158798; Thu, 01 Jun 2023 18:25:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685669158; cv=none; d=google.com; s=arc-20160816; b=cffhe7n7gnPKr+ZG1YH4pPwnbee+56WCm+Aj+aRdUKP9c4OvKBKelEJiPJa6hURrqR MPvqTeL+tKqKQdHe5BqzuGy4dvue5Gz506JqGeeL/WMM4GwkYKs9Qrt/wkFxl3WL3DvQ 8cLCRF4EtT7Lj3EsvVTrO7j+L32AWU8zwraBRABf6yqBzvRP1sPF5LpvornS3x6Bb6Ve XKW3J/psqPfp9IhMTMvWf83LdpKWt/SFyz9+b/Ju499LQ1xEvBvBFbMTXLkZPeZl5U+p 50B1UO0QLuNo7Beu2xlOHpTwmIk4SvctHY1PfdbFOWQqgU+vjAYDhbsL0uEu+pzbseWF csZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:dkim-signature; bh=tmG5Gj4Mwq4ntwod8eDhcTKnVqwdZ3BWjaq2d7Ob80k=; b=B+G3JBReFAm5Mh+17QxhksA+WeA1+tUSW47NtWyx+UaA9wUY+K/lKnxn2hMOfMOWvz 2celWAnlDX+GPQtkkYrz/c7aZO+wWMxWaxfQHAIbae8SDK4AEYm6TUHJ+kOeM5kNGf3m ayZgBHg+13jswkNIIxRun960hB5MFte08BxODekCQaDofwYF98ZPAhPUmj1DGeKzw1b/ MEaPlLnjcEbK//YDbl5BtR5R6xeQQbz+3Vhte1yTicWlPTB/ourHofkND/TDjK+wteet KacY1PCNhW3BQbH2Bhw/7d/h9SF+Pcp2r+oY4FAMFEpX44jTapmnl3BK+XigPwdztGqA eC2w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=uC2XLUjv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k16-20020a637b50000000b005347aba7376si124305pgn.297.2023.06.01.18.25.43; Thu, 01 Jun 2023 18:25:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=uC2XLUjv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233267AbjFBBP2 (ORCPT + 99 others); Thu, 1 Jun 2023 21:15:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41366 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233251AbjFBBP0 (ORCPT ); Thu, 1 Jun 2023 21:15:26 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9AE941A7 for ; Thu, 1 Jun 2023 18:15:23 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-565d1b86a64so20936757b3.3 for ; Thu, 01 Jun 2023 18:15:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1685668523; x=1688260523; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=tmG5Gj4Mwq4ntwod8eDhcTKnVqwdZ3BWjaq2d7Ob80k=; b=uC2XLUjv1JyRrK7ep7CVM4ZlT2APqmo2uQ6DEC+UkTT7G4T5nodeNoub/YBwvFRguI PslfYWKsTdTj4HRUhOMESTHjjITLrF++g2d95YBnetmlqjkh4Lv7sHnfD84BU4IrrLr1 Y5btKdmMLEjN0rli82gVKnmJetTXW7TIlzAhgKrhCFh2EB93DEqW98F0N4CKWyb3NKFe /4auYE/4xbSdPJJGl4joVAFDBd7DD7YCEKNCJGP22ATCcQktaQFTJ4boUisttl+4im18 LgdVZ1pi3R27V3/VTdX4GGBBI6KUSrPwDxMKSZHt1pIbFB+sgE3lARNsbPLX286Nkozl rqlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685668523; x=1688260523; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=tmG5Gj4Mwq4ntwod8eDhcTKnVqwdZ3BWjaq2d7Ob80k=; b=dXKi5bamr/fkz0UHkJCdERE+Ivsgkni+wQUDfLUoL4QBl9O8Z3jfxotVcHEngWH58B ArSbddOSpoXVrVYC8lhjCPhjSI/L+zOBvuVUcSyFcOQrMxluXb90I1VEaBRVlvnXEMBS CFZCyPvunR2rxSfUQrMPZtbgQ6bIeaED9dWu9YRRvTZfLYsr+EYsQ+XMntBsqF+CqZNA QNNHB7YXe6KTDwitDB4DZfmVtmnwrau7x/P9kxtBAC4bV4pCSNjhqUlWcu13G2iUe4jo XyvKYTNuCvC8LveUVSkyQ1S2NixwDwPNdUKCkud5Lp9dggOZFj/OqoHwT/vE4fXQ9Bkb TD7A== X-Gm-Message-State: AC+VfDwdrKojG7oB+xENuL1LWEy9h59B7MalvYjtKL1yjj//lIOYIQ81 Vd3IqAg/tMrKKXTqhDBkjVifNtPJkeo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a81:c509:0:b0:552:b607:634b with SMTP id k9-20020a81c509000000b00552b607634bmr6259856ywi.4.1685668522848; Thu, 01 Jun 2023 18:15:22 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 1 Jun 2023 18:15:16 -0700 In-Reply-To: <20230602011518.787006-1-seanjc@google.com> Mime-Version: 1.0 References: <20230602011518.787006-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.rc2.161.g9c6817b8e7-goog Message-ID: <20230602011518.787006-2-seanjc@google.com> Subject: [PATCH 1/3] KVM: VMX: Retry APIC-access page reload if invalidation is in-progress From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Jason Gunthorpe , Alistair Popple , Robin Murphy X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767552223467087276?= X-GMAIL-MSGID: =?utf-8?q?1767552223467087276?= Re-request an APIC-access page reload if there is a relevant mmu_notifier invalidation in-progress when KVM retrieves the backing pfn, i.e. stall vCPUs until the backing pfn for the APIC-access page is "officially" stable. Relying on the primary MMU to not make changes after invoking ->invalidate_range() works, e.g. any additional changes to a PRESENT PTE would also trigger an ->invalidate_range(), but using ->invalidate_range() to fudge around KVM not honoring past and in-progress invalidations is a bit hacky. Honoring invalidations will allow using KVM's standard mmu_notifier hooks to detect APIC-access page reloads, which will in turn allow removing KVM's implementation of ->invalidate_range() (the APIC-access page case is a true one-off). Opportunistically add a comment to explain why doing nothing if a memslot isn't found is functionally correct. Suggested-by: Jason Gunthorpe Cc: Alistair Popple Cc: Robin Murphy Signed-off-by: Sean Christopherson Reviewed-by: Alistair Popple --- arch/x86/kvm/vmx/vmx.c | 50 +++++++++++++++++++++++++++++++++++++----- 1 file changed, 45 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 44fb619803b8..59195f0dc7a5 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6708,7 +6708,12 @@ void vmx_set_virtual_apic_mode(struct kvm_vcpu *vcpu) static void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu) { - struct page *page; + const gfn_t gfn = APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT; + struct kvm *kvm = vcpu->kvm; + struct kvm_memslots *slots = kvm_memslots(kvm); + struct kvm_memory_slot *slot; + unsigned long mmu_seq; + kvm_pfn_t pfn; /* Defer reload until vmcs01 is the current VMCS. */ if (is_guest_mode(vcpu)) { @@ -6720,18 +6725,53 @@ static void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu) SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES)) return; - page = gfn_to_page(vcpu->kvm, APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT); - if (is_error_page(page)) + /* + * Grab the memslot so that the hva lookup for the mmu_notifier retry + * is guaranteed to use the same memslot as the pfn lookup, i.e. rely + * on the pfn lookup's validation of the memslot to ensure a valid hva + * is used for the retry check. + */ + slot = id_to_memslot(slots, APIC_ACCESS_PAGE_PRIVATE_MEMSLOT); + if (!slot || slot->flags & KVM_MEMSLOT_INVALID) return; - vmcs_write64(APIC_ACCESS_ADDR, page_to_phys(page)); + /* + * Ensure that the mmu_notifier sequence count is read before KVM + * retrieves the pfn from the primary MMU. Note, the memslot is + * protected by SRCU, not the mmu_notifier. Pairs with the smp_wmb() + * in kvm_mmu_invalidate_end(). + */ + mmu_seq = kvm->mmu_invalidate_seq; + smp_rmb(); + + /* + * No need to retry if the memslot does not exist or is invalid. KVM + * controls the APIC-access page memslot, and only deletes the memslot + * if APICv is permanently inhibited, i.e. the memslot won't reappear. + */ + pfn = gfn_to_pfn_memslot(slot, gfn); + if (is_error_noslot_pfn(pfn)) + return; + + read_lock(&vcpu->kvm->mmu_lock); + if (mmu_invalidate_retry_hva(kvm, mmu_seq, + gfn_to_hva_memslot(slot, gfn))) { + kvm_make_request(KVM_REQ_APIC_PAGE_RELOAD, vcpu); + read_unlock(&vcpu->kvm->mmu_lock); + goto out; + } + + vmcs_write64(APIC_ACCESS_ADDR, pfn_to_hpa(pfn)); + read_unlock(&vcpu->kvm->mmu_lock); + vmx_flush_tlb_current(vcpu); +out: /* * Do not pin apic access page in memory, the MMU notifier * will call us again if it is migrated or swapped out. */ - put_page(page); + kvm_release_pfn_clean(pfn); } static void vmx_hwapic_isr_update(int max_isr) From patchwork Fri Jun 2 01:15:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 102257 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp724668vqr; Thu, 1 Jun 2023 18:26:09 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7RZaWR/eziNvx6A8xArbUIOQpdW8cJaukF7wSThirM92IkfnG9RXYNM5dJ9M8qhoGvPKG2 X-Received: by 2002:a92:d1c4:0:b0:32b:75bc:cda2 with SMTP id u4-20020a92d1c4000000b0032b75bccda2mr6590818ilg.22.1685669169026; Thu, 01 Jun 2023 18:26:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685669168; cv=none; d=google.com; s=arc-20160816; b=M6d6YD+9ubTLdBZ4ZjHUS5YPTvuIgbzM3NtronSCHa5CyfUwXZMnrshTMox/gklhKG zZoEYgS4nQ6FEd+BttWHKUCvF3yctDVdmMr4FmiOP7r3rDQjLBbPABu+mo9R1I7+cXnI WbZbTs/6VVQ9yLr7i3MGIEUKdvuRpjKz8iaGps9up1Zpngo51dL0b29qo6ouQPkZkt4r kpiIayuo9zfLBqWSnkXGf+defeSZwR+HiDico0bbGD+Ecspto3ShwmUefkL7f4TD5saF cMhP+DAD44G7LAc1OXZMABlGMNYsCdzFGcEIIx7GVwItm2ibImtMWdVycJxh9Ee5Y1+8 v/ig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:dkim-signature; bh=QZyedvEO4VqsOzfsB28/8dG+6mqt6s6B82uH8fSmwq8=; b=WR6H8kCaewQTit6Jn0urDgWV85sL/ZeH6o0U83YsU2PGSCmv6NqwoJxUW4bqxPoeCB 6/XuOWtRE9HLIP3XIlhHuCcV/R0TM5CPKnW0mLdyF0+KE0ay7twupvTeUqib16jVJz2v G+UhN5v1Teca6iR6Rv9RRxebnmd0ua93fNwt2hdPwEQQz1s863k40kKuwMIjKbuDtZsk NAm1BAK58jNXprSxlqGMVFsbOW/xkhZjjEQfhSgan6DsTBAXcyYr+WN/AZsnRf8Y7gei HbKyKjGt3f5jDTnxLm1hDyCOcZjrq/L0dA0VtRJyxj893qDlr8U6q6rlK5CfmseoFEJY GxAw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=j6CUWl3a; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 200-20020a6301d1000000b0053f2803af48si124546pgb.581.2023.06.01.18.25.55; Thu, 01 Jun 2023 18:26:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=j6CUWl3a; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233274AbjFBBPe (ORCPT + 99 others); Thu, 1 Jun 2023 21:15:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41464 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233277AbjFBBP2 (ORCPT ); Thu, 1 Jun 2023 21:15:28 -0400 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 26C6A180 for ; Thu, 1 Jun 2023 18:15:25 -0700 (PDT) Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-1b02cd4b829so10253765ad.1 for ; Thu, 01 Jun 2023 18:15:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1685668524; x=1688260524; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=QZyedvEO4VqsOzfsB28/8dG+6mqt6s6B82uH8fSmwq8=; b=j6CUWl3a3FC0QTS2njOofjKDlfQDmtDJaHSEouWHPCamUcCop7S0W1f56KyCl2MYCz DstIw0CLzrHL+bqnN/88nI8F9zPPVgGPmafeNxh7uQExuP98qW5YpkuGO7fz/JN016ot ONsEzzryAmfo9bNAIsMV34WY7aScQG1sHNQQqUac+jFmNo0cLvSgRWIGQnE1Figc4LgJ TMq/TnB4/PGyJaxLGEgGG93zTw8uRjt4FOxjcHttzl6sUqQzSHIqeSP3ip5fqROjFUlK 1n4cxYxyeLfHcXrK9mKx7vMZUAHJl1sv0YgnAkvwpveJ12L1s6NLxOR3dH5ccmW4qY0f iT9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685668524; x=1688260524; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=QZyedvEO4VqsOzfsB28/8dG+6mqt6s6B82uH8fSmwq8=; b=S0DehrrQbOSvNmKArdefLawL6BSJsG/1eOwwHcpVxDZjdAYHKJPnVDSvHtEbcDUxtV eDUPmQqTfU1eezmFceYfZemqtf7Kw6btM0yJxQqYHVXGbiggp7VnLBxbVXUhLQjZQkfy 3ageIqouRDHXtFXwbOhE9VlvjpGMMjdIOoWN4pWxviM3/5/tUIG8VI2mM97wnviD0XCQ 5cEELg/I6Nl8ML+yRfDTvpNcaPzxJiv7OuXU+EiqNCtnERZiKSH58UX3RqOqYY0OlJV+ wfIZPTYNGfo7VRqLs5ElWCpRn98b8djzzVgUERujyfhFpvD5GIPfUm1ReTIVWJhPKnr+ tLQg== X-Gm-Message-State: AC+VfDxlc1wuUDIZpI5WwGM4HLVR1OsG89U4eS43hGNrf0U27/doM0ee 1/4BU2MUnnGfxt5gjeC1IhvBZqvm0Ok= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:fb85:b0:1a6:8a07:960a with SMTP id lg5-20020a170902fb8500b001a68a07960amr166701plb.0.1685668524538; Thu, 01 Jun 2023 18:15:24 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 1 Jun 2023 18:15:17 -0700 In-Reply-To: <20230602011518.787006-1-seanjc@google.com> Mime-Version: 1.0 References: <20230602011518.787006-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.rc2.161.g9c6817b8e7-goog Message-ID: <20230602011518.787006-3-seanjc@google.com> Subject: [PATCH 2/3] KVM: x86: Use standard mmu_notifier invalidate hooks for APIC access page From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Jason Gunthorpe , Alistair Popple , Robin Murphy X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767552234345168685?= X-GMAIL-MSGID: =?utf-8?q?1767552234345168685?= Now that KVM honors past and in-progress mmu_notifier invalidations when reloading the APIC-access page, use KVM's "standard" invalidation hooks to trigger a reload and delete the one-off usage of invalidate_range(). Aside from eliminating one-off code in KVM, dropping KVM's use of invalidate_range() will allow common mmu_notifier to redefine the API to be more strictly focused on invalidating secondary TLBs that share the primary MMU's page tables. Suggested-by: Jason Gunthorpe Cc: Alistair Popple Cc: Robin Murphy Signed-off-by: Sean Christopherson Reviewed-by: Alistair Popple --- arch/x86/kvm/mmu/mmu.c | 3 +++ arch/x86/kvm/x86.c | 14 -------------- include/linux/kvm_host.h | 3 --- virt/kvm/kvm_main.c | 18 ------------------ 4 files changed, 3 insertions(+), 35 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index c8961f45e3b1..01a11ce68e57 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -1600,6 +1600,9 @@ bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range) if (tdp_mmu_enabled) flush = kvm_tdp_mmu_unmap_gfn_range(kvm, range, flush); + if (range->slot->id == APIC_ACCESS_PAGE_PRIVATE_MEMSLOT) + kvm_make_all_cpus_request(kvm, KVM_REQ_APIC_PAGE_RELOAD); + return flush; } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index ceb7c5e9cf9e..141a62e59d2b 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10431,20 +10431,6 @@ static void vcpu_load_eoi_exitmap(struct kvm_vcpu *vcpu) vcpu, (u64 *)vcpu->arch.ioapic_handled_vectors); } -void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, - unsigned long start, unsigned long end) -{ - unsigned long apic_address; - - /* - * The physical address of apic access page is stored in the VMCS. - * Update it when it becomes invalid. - */ - apic_address = gfn_to_hva(kvm, APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT); - if (start <= apic_address && apic_address < end) - kvm_make_all_cpus_request(kvm, KVM_REQ_APIC_PAGE_RELOAD); -} - void kvm_arch_guest_memory_reclaimed(struct kvm *kvm) { static_call_cond(kvm_x86_guest_memory_reclaimed)(kvm); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 0e571e973bc2..cb66f4100be7 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2237,9 +2237,6 @@ static inline long kvm_arch_vcpu_async_ioctl(struct file *filp, } #endif /* CONFIG_HAVE_KVM_VCPU_ASYNC_IOCTL */ -void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, - unsigned long start, unsigned long end); - void kvm_arch_guest_memory_reclaimed(struct kvm *kvm); #ifdef CONFIG_HAVE_KVM_VCPU_RUN_PID_CHANGE diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index cb5c13eee193..844ff6b0b21d 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -154,11 +154,6 @@ static unsigned long long kvm_active_vms; static DEFINE_PER_CPU(cpumask_var_t, cpu_kick_mask); -__weak void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, - unsigned long start, unsigned long end) -{ -} - __weak void kvm_arch_guest_memory_reclaimed(struct kvm *kvm) { } @@ -521,18 +516,6 @@ static inline struct kvm *mmu_notifier_to_kvm(struct mmu_notifier *mn) return container_of(mn, struct kvm, mmu_notifier); } -static void kvm_mmu_notifier_invalidate_range(struct mmu_notifier *mn, - struct mm_struct *mm, - unsigned long start, unsigned long end) -{ - struct kvm *kvm = mmu_notifier_to_kvm(mn); - int idx; - - idx = srcu_read_lock(&kvm->srcu); - kvm_arch_mmu_notifier_invalidate_range(kvm, start, end); - srcu_read_unlock(&kvm->srcu, idx); -} - typedef bool (*hva_handler_t)(struct kvm *kvm, struct kvm_gfn_range *range); typedef void (*on_lock_fn_t)(struct kvm *kvm, unsigned long start, @@ -892,7 +875,6 @@ static void kvm_mmu_notifier_release(struct mmu_notifier *mn, } static const struct mmu_notifier_ops kvm_mmu_notifier_ops = { - .invalidate_range = kvm_mmu_notifier_invalidate_range, .invalidate_range_start = kvm_mmu_notifier_invalidate_range_start, .invalidate_range_end = kvm_mmu_notifier_invalidate_range_end, .clear_flush_young = kvm_mmu_notifier_clear_flush_young, From patchwork Fri Jun 2 01:15:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 102254 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp722357vqr; Thu, 1 Jun 2023 18:19:24 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ41r3y68ShhSlk80/UL/7B7JJFs6DbX1t8SbhMIAWBBoXqdypIsVACbFQ47uH9Fw6738O98 X-Received: by 2002:a05:6a20:2447:b0:10f:708b:bb28 with SMTP id t7-20020a056a20244700b0010f708bbb28mr14059782pzc.7.1685668764513; Thu, 01 Jun 2023 18:19:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685668764; cv=none; d=google.com; s=arc-20160816; b=aaI5D1rlroxMIHcN6CZW6lRxEyYqxWf+g5oOcZn3EjpRj6PqU5rfz2v05HfhCrNf75 7fOjwv1sY4w4IxAXfm0ejNtUSKGeqR0qWGt0xM4v9MgN8INFd3yA+jYSmF/1omY779r2 B0O9rWiu6iJRxHyyEjj1xJ88XW2eJxgvCztdkbtzZsymIsCdjgfbu82lKfBAW5il55oz LMFoQWE8r8FnCoQ7WaPf705m3JK7Z6T1FzSrLcpnozC184DCY+FFV7f0PHdycmUTTDDe k04SyDLEjdCFgXQpXbYuJIBFC5tteR1ulGcpVSJ2ODVhZvhxF6rfsf6MNIjDUjSPySys 6Plw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:dkim-signature; bh=XNDrDuYa3iWJuXJKyD6gXhX5XFme5N3YEsdm0o31DEg=; b=I6nCN60J4/5FAUlT2ZSfYBABF1yxiQ4SuEhXNWtABOIaXv6hbmJxFlKchON9wqWXt7 8bSNImcQDuBcEvAs8ToarHP5dMiJdGxIKhVjAKAmKLX+Lm7mInZBCTE3y/zwV85jV7gr v8+yldbk6v3/TR0sXOE3qj3pj1jayhIqSlJzFb/tALFMTo7Lv26S+FU67l7tOZXbq0PH XJKpHdodguKdiQ18Po1/piTulOlfPkwUzyzD5zSYDNrLd8aOG1r4sC7X1vBo3jlGrKgq mdi7guBzuZv3m5arp/YjwTKarwyXrs71CCdrOhR2KD1RHO9mVlbwULiDmbdKzSBMscuE ZeLA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=Po+suhv+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n26-20020a63721a000000b00540107fc033si118337pgc.282.2023.06.01.18.19.10; Thu, 01 Jun 2023 18:19:24 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=Po+suhv+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233330AbjFBBPi (ORCPT + 99 others); Thu, 1 Jun 2023 21:15:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41508 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233251AbjFBBPb (ORCPT ); Thu, 1 Jun 2023 21:15:31 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EB85E199 for ; Thu, 1 Jun 2023 18:15:26 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-bacfa4eefd2so2089016276.0 for ; Thu, 01 Jun 2023 18:15:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1685668526; x=1688260526; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=XNDrDuYa3iWJuXJKyD6gXhX5XFme5N3YEsdm0o31DEg=; b=Po+suhv+8tQfEJPqRxrACOHdjZJr6yLdW6VxkO02vb+MHxsu4VMYOZCpTcDebrWjnK 5738//eSrPOHNp0DY8SragBWqRMzBLxVXuVgfE4ougaCogMIJw+nETG6kjdb36Z2Htzq lNYjNs/6RjAfL2Jqnx6VBZgmnJRAAZJvBw4rqJCA+LsNxX8JDl/scqSO3oqh9fopdqQI t9LFrhwbk8UNBGby9BfGJsomRdX8VCDme789sKFtZDc0jjmTUsnKOmec0hPIYSPkMfYW nrMt+UVVLwW/9wqxKSlQupZAiVTTRmQIP84xeBBr8ThQH5X5pnhXfNofP/yD4wyFV22W JLQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685668526; x=1688260526; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=XNDrDuYa3iWJuXJKyD6gXhX5XFme5N3YEsdm0o31DEg=; b=NfaccLyLt7Em4dRDcqo2obuINgsRz/wAgk4VdUBzt4MFzhB+0e5+QFE4EFN4z7Ymet 5xY1TMzzWZyD4AZ76X6r+nBe9tV8k0YWGbKv+viIl58QkC3NmDPznRixXEvQ74ESYtMZ 3eQzPWEBW2AONkXIg1E7x8vKQCq/BBxMlpR5BliA7AEPfdUEXzoiNL1eVtfLaqhHxpml 30C1Nak5NeBbyCWuac35+MZxrU3PEGywtuIvEXTlyQb11bMbvOPk2gpL+QjpIakl7QAw NxwgwgJwyhdaqAKqHQHzcpfalS8jPz8H7E5p862WHhxw1rcGwioZE9ll6X+xa09tZM0Y f6zQ== X-Gm-Message-State: AC+VfDyVZQ+c3leO44koY60hw6FsYWVin9IOvHnI1b/aiO6YEdxfSTmo JViv8H7WHgzEOl4o0g/zRLcEiswRYas= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:adc6:0:b0:b8f:6b84:33cb with SMTP id d6-20020a25adc6000000b00b8f6b8433cbmr714516ybe.11.1685668526648; Thu, 01 Jun 2023 18:15:26 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 1 Jun 2023 18:15:18 -0700 In-Reply-To: <20230602011518.787006-1-seanjc@google.com> Mime-Version: 1.0 References: <20230602011518.787006-1-seanjc@google.com> X-Mailer: git-send-email 2.41.0.rc2.161.g9c6817b8e7-goog Message-ID: <20230602011518.787006-4-seanjc@google.com> Subject: [PATCH 3/3] KVM: x86/mmu: Trigger APIC-access page reload iff vendor code cares From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Jason Gunthorpe , Alistair Popple , Robin Murphy X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767551810344224836?= X-GMAIL-MSGID: =?utf-8?q?1767551810344224836?= Request an APIC-access page reload when the backing page is migrated (or unmapped) if and only if vendor code actually plugs the backing pfn into structures that reside outside of KVM's MMU. This avoids kicking all vCPUs in the (hopefully infrequent) scenario where the backing page is migrated/invalidated. Unlike VMX's APICv, SVM's AVIC doesn't plug the backing pfn directly into the VMCB and so doesn't need a hook to invalidate an out-of-MMU "mapping". Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 01a11ce68e57..beb507d82adf 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -1600,7 +1600,8 @@ bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range) if (tdp_mmu_enabled) flush = kvm_tdp_mmu_unmap_gfn_range(kvm, range, flush); - if (range->slot->id == APIC_ACCESS_PAGE_PRIVATE_MEMSLOT) + if (kvm_x86_ops.set_apic_access_page_addr && + range->slot->id == APIC_ACCESS_PAGE_PRIVATE_MEMSLOT) kvm_make_all_cpus_request(kvm, KVM_REQ_APIC_PAGE_RELOAD); return flush;