From patchwork Thu Oct 13 21:12:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 2374 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp483837wrs; Thu, 13 Oct 2022 14:14:31 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6DbLwr0rGYmAjE8U0Q4E4MZa7DmhsSOsZFgfaXv2aLifngjUCmzOKMPTRt15IjlUl2lPLv X-Received: by 2002:a17:907:2c5b:b0:78d:3f8a:19d0 with SMTP id hf27-20020a1709072c5b00b0078d3f8a19d0mr1196178ejc.369.1665695671566; Thu, 13 Oct 2022 14:14:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1665695671; cv=none; d=google.com; s=arc-20160816; b=SRAv2YwaFOtKI+s2Rqzps1Mi4JOiWj7oeT9/zmwLVR1wfuoBvsxIXarhZ0T7BZFbOU G6Thwyl4HfpNAuc2F2pBCADtnfXwQwUVaPn91MomgMRlFQp85FU5PAjyh7zSAbBPvqme h2VYg7DF5O5ezeMz4nrvuDOhGI+sXmW+ElkMzospZZQ44WhbHaGQAoxY6I2qrXSMBO3G 9qhjf1VbTFE8Sxdvgl8pe5fdlMNr4rgAWh5mSBG1M1XcX1PxPDY62OJtMbLvVqjY2PQ1 YEeFtEuXGYKHdvWAyi3UdBKlmODQasjciL7bS0QMRZwVTOTvZ2fXpw+2nidV3nrduzD3 QVwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:cc:to:from:subject :message-id:references:mime-version:in-reply-to:date:reply-to :dkim-signature; bh=BR8P37dIvaTzFUy4lljy43WBv5S0DCgP55/AwwfIul8=; b=mcPJoFUP8tG0Jn6m/Y2mhrSutqWMobUOP77SV3qf03nkG0waKHoc1obqPqGkJd0C9z vFwa9MUdWdhL/YsN7r0rdxMi3oyBy+VNPeSrl7gjLNWiRDFRiHTtQgSMOVYA0EkobOI7 wHaGCeVWF7GH8l0xRpNUwHGgPDk90DPDcUrcHBqUZcid5gzOoADp2eZt1Khc52GxKk5q 66YP3vsFbOXA6NiMO43V/InxRYdU8S0uzVN0jVoye5zS+lKpeIZnqcOLvDZTBPFvQ31o FMQ/K5M75rCRTSsQInohzVFB8SlMdYuukxyMRO4x5Dk+NrNyaz/ZmFHWihbf4YQnZhI+ 9HUg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=CCO6wUna; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id js19-20020a17090797d300b0077fda35e1adsi460171ejc.618.2022.10.13.14.14.05; Thu, 13 Oct 2022 14:14:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=CCO6wUna; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229737AbiJMVNP (ORCPT + 99 others); Thu, 13 Oct 2022 17:13:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52748 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229940AbiJMVNL (ORCPT ); Thu, 13 Oct 2022 17:13:11 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F2337192D97 for ; Thu, 13 Oct 2022 14:12:56 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id b14-20020a056902030e00b006a827d81fd8so2562650ybs.17 for ; Thu, 13 Oct 2022 14:12:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=BR8P37dIvaTzFUy4lljy43WBv5S0DCgP55/AwwfIul8=; b=CCO6wUnaRGcTstghOOGYPgobIQnpiUGsYX/hCr5+qDry/5v3Aund/1oMz7QmAiJ9Rc qHtK5CFnvyCSLEQwMgXznejUx9u+4wkGV8aoLCND0rOJeaKdeTe+kjeevWvOX3W3ZDap UtKRvvUyA6hYauHJYsNI8BHgtt6mO7R/Hr28mdLgdPebppsjoR5A+R8QYNujc3nOIPvm ZreplDyFfEtuvnJvSnH1L0+SYE42Lygeuk0bChDkmHAKdIUNA8mNcCi2rJSjF9uHjyRX s1BTh4U1dK1fd+RGvcHWbK/w6QqxqeLgE7Bzr1z0J+GXsMUXg/E2UGmmcphr0UDIkrmR HDVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=BR8P37dIvaTzFUy4lljy43WBv5S0DCgP55/AwwfIul8=; b=HeFk2O3W3TGfgpLlkZ+eQkXjSPAJjpNKf5AH+EsbnlqGyIYTeXdjGBMpJie0Y28lnv wXggspDgTkL1koanmsRT4Q4+H+TzPutQM7dCSBOURadz7aL1fDNAZA7a+llRBeLo8LdH p5hj+L3LuLHZyHRZ+I+aCO7fanLHdfY0VNMeMROyow0n2kyte62xapSX4wPCsm2O6zMb W5JHNF5dCQCZ5sX9e6razJnRe8EPDc2oIPR5oJ+sMw6iNiXRffog5KfeZc8IomFBIEnh yLsFzzcQd7J1WoE//w8u2BWAfhqL/1IqHgTmVh+jxHxInl9X4V1jk2SVoV1NT+/XdsbW BIrg== X-Gm-Message-State: ACrzQf3e+SQhdqeTWHKeiIxIxSkHnE5FEwimy/RQzgOyH/4TmqrJhcpK xyqauljdYdCREAhKsZEELuCQpncgzJU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:344e:0:b0:6c1:40e4:5a3b with SMTP id b75-20020a25344e000000b006c140e45a3bmr1712215yba.423.1665695562532; Thu, 13 Oct 2022 14:12:42 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 13 Oct 2022 21:12:20 +0000 In-Reply-To: <20221013211234.1318131-1-seanjc@google.com> Mime-Version: 1.0 References: <20221013211234.1318131-1-seanjc@google.com> X-Mailer: git-send-email 2.38.0.413.g74048e4d9e-goog Message-ID: <20221013211234.1318131-3-seanjc@google.com> Subject: [PATCH v2 02/16] KVM: Reject attempts to consume or refresh inactive gfn_to_pfn_cache From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Michal Luczaj , David Woodhouse X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1746608504447838011?= X-GMAIL-MSGID: =?utf-8?q?1746608504447838011?= Reject kvm_gpc_check() and kvm_gpc_refresh() if the cache is inactive. Not checking the active flag during refresh is particularly egregious, as KVM can end up with a valid, inactive cache, which can lead to a variety of use-after-free bugs, e.g. consuming a NULL kernel pointer or missing an mmu_notifier invalidation due to the cache not being on the list of gfns to invalidate. Note, "active" needs to be set if and only if the cache is on the list of caches, i.e. is reachable via mmu_notifier events. If a relevant mmu_notifier event occurs while the cache is "active" but not on the list, KVM will not acquire the cache's lock and so will not serailize the mmu_notifier event with active users and/or kvm_gpc_refresh(). A race between KVM_XEN_ATTR_TYPE_SHARED_INFO and KVM_XEN_HVM_EVTCHN_SEND can be exploited to trigger the bug. 1. Deactivate shinfo cache: kvm_xen_hvm_set_attr case KVM_XEN_ATTR_TYPE_SHARED_INFO kvm_gpc_deactivate kvm_gpc_unmap gpc->valid = false gpc->khva = NULL gpc->active = false Result: active = false, valid = false 2. Cause cache refresh: kvm_arch_vm_ioctl case KVM_XEN_HVM_EVTCHN_SEND kvm_xen_hvm_evtchn_send kvm_xen_set_evtchn kvm_xen_set_evtchn_fast kvm_gpc_check return -EWOULDBLOCK because !gpc->valid kvm_xen_set_evtchn_fast return -EWOULDBLOCK kvm_gpc_refresh hva_to_pfn_retry gpc->valid = true gpc->khva = not NULL Result: active = false, valid = true 3. Race ioctl KVM_XEN_HVM_EVTCHN_SEND against ioctl KVM_XEN_ATTR_TYPE_SHARED_INFO: kvm_arch_vm_ioctl case KVM_XEN_HVM_EVTCHN_SEND kvm_xen_hvm_evtchn_send kvm_xen_set_evtchn kvm_xen_set_evtchn_fast read_lock gpc->lock kvm_xen_hvm_set_attr case KVM_XEN_ATTR_TYPE_SHARED_INFO mutex_lock kvm->lock kvm_xen_shared_info_init kvm_gpc_activate gpc->khva = NULL kvm_gpc_check [ Check passes because gpc->valid is still true, even though gpc->khva is already NULL. ] shinfo = gpc->khva pending_bits = shinfo->evtchn_pending CRASH: test_and_set_bit(..., pending_bits) Fixes: 982ed0de4753 ("KVM: Reinstate gfn_to_pfn_cache with invalidation support") Cc: stable@vger.kernel.org Reported-by: : Michal Luczaj Signed-off-by: Sean Christopherson --- virt/kvm/pfncache.c | 41 ++++++++++++++++++++++++++++++++++------- 1 file changed, 34 insertions(+), 7 deletions(-) diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index 08f97cf97264..346e47f15572 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -81,6 +81,9 @@ bool kvm_gfn_to_pfn_cache_check(struct kvm *kvm, struct gfn_to_pfn_cache *gpc, { struct kvm_memslots *slots = kvm_memslots(kvm); + if (!gpc->active) + return false; + if ((gpa & ~PAGE_MASK) + len > PAGE_SIZE) return false; @@ -240,10 +243,11 @@ int kvm_gfn_to_pfn_cache_refresh(struct kvm *kvm, struct gfn_to_pfn_cache *gpc, { struct kvm_memslots *slots = kvm_memslots(kvm); unsigned long page_offset = gpa & ~PAGE_MASK; - kvm_pfn_t old_pfn, new_pfn; + bool unmap_old = false; unsigned long old_uhva; + kvm_pfn_t old_pfn; void *old_khva; - int ret = 0; + int ret; /* * If must fit within a single page. The 'len' argument is @@ -261,6 +265,11 @@ int kvm_gfn_to_pfn_cache_refresh(struct kvm *kvm, struct gfn_to_pfn_cache *gpc, write_lock_irq(&gpc->lock); + if (!gpc->active) { + ret = -EINVAL; + goto out_unlock; + } + old_pfn = gpc->pfn; old_khva = gpc->khva - offset_in_page(gpc->khva); old_uhva = gpc->uhva; @@ -291,6 +300,7 @@ int kvm_gfn_to_pfn_cache_refresh(struct kvm *kvm, struct gfn_to_pfn_cache *gpc, /* If the HVA→PFN mapping was already valid, don't unmap it. */ old_pfn = KVM_PFN_ERR_FAULT; old_khva = NULL; + ret = 0; } out: @@ -305,14 +315,15 @@ int kvm_gfn_to_pfn_cache_refresh(struct kvm *kvm, struct gfn_to_pfn_cache *gpc, gpc->khva = NULL; } - /* Snapshot the new pfn before dropping the lock! */ - new_pfn = gpc->pfn; + /* Detect a pfn change before dropping the lock! */ + unmap_old = (old_pfn != gpc->pfn); +out_unlock: write_unlock_irq(&gpc->lock); mutex_unlock(&gpc->refresh_lock); - if (old_pfn != new_pfn) + if (unmap_old) gpc_unmap_khva(kvm, old_pfn, old_khva); return ret; @@ -366,11 +377,19 @@ int kvm_gpc_activate(struct kvm *kvm, struct gfn_to_pfn_cache *gpc, gpc->vcpu = vcpu; gpc->usage = usage; gpc->valid = false; - gpc->active = true; spin_lock(&kvm->gpc_lock); list_add(&gpc->list, &kvm->gpc_list); spin_unlock(&kvm->gpc_lock); + + /* + * Activate the cache after adding it to the list, a concurrent + * refresh must not establish a mapping until the cache is + * reachable by mmu_notifier events. + */ + write_lock_irq(&gpc->lock); + gpc->active = true; + write_unlock_irq(&gpc->lock); } return kvm_gfn_to_pfn_cache_refresh(kvm, gpc, gpa, len); } @@ -379,12 +398,20 @@ EXPORT_SYMBOL_GPL(kvm_gpc_activate); void kvm_gpc_deactivate(struct kvm *kvm, struct gfn_to_pfn_cache *gpc) { if (gpc->active) { + /* + * Deactivate the cache before removing it from the list, KVM + * must stall mmu_notifier events until all users go away, i.e. + * until gpc->lock is dropped and refresh is guaranteed to fail. + */ + write_lock_irq(&gpc->lock); + gpc->active = false; + write_unlock_irq(&gpc->lock); + spin_lock(&kvm->gpc_lock); list_del(&gpc->list); spin_unlock(&kvm->gpc_lock); kvm_gfn_to_pfn_cache_unmap(kvm, gpc); - gpc->active = false; } } EXPORT_SYMBOL_GPL(kvm_gpc_deactivate);