From patchwork Fri Jan 20 15:05:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Liu X-Patchwork-Id: 46407 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp257530wrn; Fri, 20 Jan 2023 07:19:19 -0800 (PST) X-Google-Smtp-Source: AMrXdXuFMVHvMp+d5JIvSIuDiaVmfzr7uA1rxx1/xV5Ocm8dMmGk5kLy2i9wB7Gi+DrOC94oJtcf X-Received: by 2002:a05:6a20:3ca8:b0:b8:a56f:7612 with SMTP id b40-20020a056a203ca800b000b8a56f7612mr19720494pzj.55.1674227959494; Fri, 20 Jan 2023 07:19:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674227959; cv=none; d=google.com; s=arc-20160816; b=0dwoSIbJSgZV0EL+mgyt+nfjYhfySUf1cw16cRCiyAe43ipoV3lAeUJJHIrnqOH9KI h7aAr5l193IiCklzDUbrEGr/zqz45COtnQpNf48ZHzpqpjZpIPc/wprWkinkuvtQdloC x4WL/9FdOZ6EX+/gzahUFMILSmGOAXjnOPZNEig+egiESZth1dalyUJHbvBfLikqoRxO PoLANQrGMDoJ69siQsqNtQBR4bnnxNNfcYY4CZPa5pZlzZrVfDtQNgfom48MpFSWceSD mzWKq8ZqozTWyvreCSIbEJluS6KvDkfBe0v83f3zQw2aE3jtxDhArCnrQ+ekXFudAmFU ub3w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=rZf/PsF88cC4IiK9Tf38Hstprk7/2S5N3R+8Bskw0NM=; b=lLacj6s+NJAyW4XwQsBJnrOdOMJ5QGY6K8iYU4QPiGxlmPqbc6n8OQ8mT8CvvXJ9tI Qj5iTe4Gymr0XJzfuuA3Qe/SerDgLEJiJ8toI0CjE50PveMbZG8VuGnl6OHQdYA3NRo6 Zht9GlLq2hVaclQdAI5NasvAj7fAHGK21Ts4ZBdivr5tsep28j77Ha+9d1cyMhn/xeLJ qGo1tpzlMLqN+q9IMU8wZz+lvbmt/0tV8dXjzdOyV7uVBpCWT/JDpTXwXfCDAPFvez2q 0MGt2xHrdmaKxdTkYRuFXqz3NjX92SiGkrCxUc4rLU/LjfBbdnxIvWFJoJl54bxMadSQ 6oEQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="F/0vIVpV"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m18-20020a639412000000b00497e31916dfsi41638635pge.273.2023.01.20.07.19.06; Fri, 20 Jan 2023 07:19:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="F/0vIVpV"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230209AbjATPFe (ORCPT + 99 others); Fri, 20 Jan 2023 10:05:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40818 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229907AbjATPFc (ORCPT ); Fri, 20 Jan 2023 10:05:32 -0500 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C888B7CCC1; Fri, 20 Jan 2023 07:05:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1674227131; x=1705763131; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=Zn183ElO1bl64Y4pVPfF7h7rDHD+/IuJAJiKiAAXWvc=; b=F/0vIVpVwojTbSAlnf2veNQ3B7J09FMvOPCmAYS/wa+rX8SN9SZwu//F g27XxVYnlUsdx8FXerODDMbpDy6DaRZaL3aC3GBItoxhEjMNzDFtHNiMt gvgmlJVnNEP3sGPBrIwWceEyFKJkIBKb8AVshNFSBOJ73fvqW5FZIHANP iVzvG5cOsXYXT/hrVZKdA7hfxloRxb3FbQ66Vi9qQSaP6b5AJQrI2+/j4 0FepJEMCN/xG818aDRDF7mWx8VZKknMtjjRbh1vquc0Zww9VUHaUasmQP 5mBp8p7uwepP1pp8+fqrWjgLBj8/WSx/YTOj53x+s05xlsCUlb6HjWlXj A==; X-IronPort-AV: E=McAfee;i="6500,9779,10595"; a="309169480" X-IronPort-AV: E=Sophos;i="5.97,232,1669104000"; d="scan'208";a="309169480" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Jan 2023 07:05:31 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10595"; a="784532388" X-IronPort-AV: E=Sophos;i="5.97,232,1669104000"; d="scan'208";a="784532388" Received: from 984fee00a4c6.jf.intel.com ([10.165.58.231]) by orsmga004.jf.intel.com with ESMTP; 20 Jan 2023 07:05:30 -0800 From: Yi Liu To: alex.williamson@redhat.com, pbonzini@redhat.com, mjrosato@linux.ibm.com Cc: yi.l.liu@intel.com, jgg@nvidia.com, kevin.tian@intel.com, cohuck@redhat.com, farman@linux.ibm.com, pmorel@linux.ibm.com, borntraeger@linux.ibm.com, frankja@linux.ibm.com, imbrenda@linux.ibm.com, david@redhat.com, akrowiak@linux.ibm.com, jjherne@linux.ibm.com, pasic@linux.ibm.com, zhenyuw@linux.intel.com, zhi.a.wang@intel.com, seanjc@google.com, linux-s390@vger.kernel.org, kvm@vger.kernel.org, intel-gvt-dev@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org Subject: [PATCH] kvm/vfio: Fix potential deadlock on vfio group_lock Date: Fri, 20 Jan 2023 07:05:28 -0800 Message-Id: <20230120150528.471752-1-yi.l.liu@intel.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755555256994818070?= X-GMAIL-MSGID: =?utf-8?q?1755555256994818070?= Currently it is possible that the final put of a KVM reference comes from vfio during its device close operation. This occurs while the vfio group lock is held; however, if the vfio device is still in the kvm device list, then the following call chain could result in a deadlock: VFIO holds group->group_lock/group_rwsem -> kvm_put_kvm -> kvm_destroy_vm -> kvm_destroy_devices -> kvm_vfio_destroy -> kvm_vfio_file_set_kvm -> vfio_file_set_kvm -> try to hold group->group_lock/group_rwsem The key function is the kvm_destroy_devices() which triggers destroy cb of kvm_device_ops. It calls back to vfio and try to hold group_lock. So if this path doesn't call back to vfio, this dead lock would be fixed. Actually, there is a way for it. KVM provides another point to free the kvm-vfio device which is the point when the device file descriptor is closed. This can be achieved by providing the release cb instead of the destroy cb. Also rename kvm_vfio_destroy() to be kvm_vfio_release(). /* * Destroy is responsible for freeing dev. * * Destroy may be called before or after destructors are called * on emulated I/O regions, depending on whether a reference is * held by a vcpu or other kvm component that gets destroyed * after the emulated I/O. */ void (*destroy)(struct kvm_device *dev); /* * Release is an alternative method to free the device. It is * called when the device file descriptor is closed. Once * release is called, the destroy method will not be called * anymore as the device is removed from the device list of * the VM. kvm->lock is held. */ void (*release)(struct kvm_device *dev); Fixes: 421cfe6596f6 ("vfio: remove VFIO_GROUP_NOTIFY_SET_KVM") Reported-by: Alex Williamson Suggested-by: Kevin Tian Reviewed-by: Jason Gunthorpe Signed-off-by: Yi Liu Reviewed-by: Matthew Rosato --- virt/kvm/vfio.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/virt/kvm/vfio.c b/virt/kvm/vfio.c index 495ceabffe88..e94f3ea718e5 100644 --- a/virt/kvm/vfio.c +++ b/virt/kvm/vfio.c @@ -336,7 +336,7 @@ static int kvm_vfio_has_attr(struct kvm_device *dev, return -ENXIO; } -static void kvm_vfio_destroy(struct kvm_device *dev) +static void kvm_vfio_release(struct kvm_device *dev) { struct kvm_vfio *kv = dev->private; struct kvm_vfio_group *kvg, *tmp; @@ -363,7 +363,7 @@ static int kvm_vfio_create(struct kvm_device *dev, u32 type); static struct kvm_device_ops kvm_vfio_ops = { .name = "kvm-vfio", .create = kvm_vfio_create, - .destroy = kvm_vfio_destroy, + .release = kvm_vfio_release, .set_attr = kvm_vfio_set_attr, .has_attr = kvm_vfio_has_attr, };