From patchwork Wed Nov 1 23:31:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Danilo Krummrich X-Patchwork-Id: 160723 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:8f47:0:b0:403:3b70:6f57 with SMTP id j7csp21597vqu; Wed, 1 Nov 2023 16:32:50 -0700 (PDT) X-Google-Smtp-Source: AGHT+IElNG6y2ifzUOfHkdR1bqD9Z3mD3eeD1+XvdUYAOjbdVxpCeBue0A3FmTUGpw/Pia257RlS X-Received: by 2002:a17:90a:7563:b0:27d:853:9109 with SMTP id q90-20020a17090a756300b0027d08539109mr15813490pjk.20.1698881570085; Wed, 01 Nov 2023 16:32:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698881570; cv=none; d=google.com; s=arc-20160816; b=n10bFMfJVKwE8BuLIVG6sFas5mW5WV4dB2HwEFqbLRUWfgfC8hiTKk5gLEoLJ13kIG UE/D9GgKtYiJd3RrMiM5BPITnbQFzUX8476JhxEXsKrjujk7FRLzABGHMAt7oYHkfJyQ wAUlYs/fiXZCpVzNau0ragnV3ggyaIyGozU88Q+NRCrutuIlvsSbkvJ/72PpMPMML0N9 h3PDLhDVljxj/Mb4fH35BkF/v3eh2l25OkYCtShjRZ2o7CznAeYKCvuv/Ff9P6Q0n9IU r6a86YJzGljq4QJOusCxH9PWA0GDNPA7+J+EyKFy/gv+t7GXeXn7YCiDAU0xjZzk9a1w istw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=juQeCGfPWswDlP+Z0KdUDVDRgyhQp0+hsdRA02TinWo=; fh=8dtrvlz6eC1As92HJPwYWdp8qGhbMzdOhqcu6vG4iUM=; b=Z6Z+VJk/NYxs2+QcvW5VajAVFARpD1SfH70LC8LENcZONvlJ1nFxXQILmF9CtZ6/ht qS63K3HpDKfFTpxG1zYmzLc0cz/P3dkKJtGh5uVqAl8+LeUa/X6hwDWFF2Tdd+/qNkz6 MtRfFTVwUbf+JlZa5yvOoE2hg0ht4eSiT+TyFu50jK4wXhrngKZq6FqU9yWsoOsuWxfR 4r/6wXnXmuKUYy5gszC3VCm4pvZ5emBNUeAGizhhQwONGzaYOFU+O9ZTZQZRsSYQliP6 FrWPJNelBHG8n3tlCTV4lTd+SLPp17LNAlfkgZxXbZiy1MsLD8A46m0iUyM4BnfunLdx +ixw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=QcJT2oWo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id d14-20020a17090ad3ce00b002802da109fasi1744016pjw.165.2023.11.01.16.32.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Nov 2023 16:32:50 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=QcJT2oWo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 6314081A43E8; Wed, 1 Nov 2023 16:32:49 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347868AbjKAXco (ORCPT + 35 others); Wed, 1 Nov 2023 19:32:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54030 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347524AbjKAXcf (ORCPT ); Wed, 1 Nov 2023 19:32:35 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 64E1B101 for ; Wed, 1 Nov 2023 16:31:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698881515; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=juQeCGfPWswDlP+Z0KdUDVDRgyhQp0+hsdRA02TinWo=; b=QcJT2oWo/+Hv0/QlLS23Gv4jxHPG4kCSnBcPZr+GistP3SgkOZCF+CIHdMLUOxtgRuBT16 8n+r7WReL1cfOrfcg77MM7nrRBJGG2ofuOyiO3SUu6iWWF1bqqdl/FLLBQRCpwLr+Sft71 n0fU+oeJ400oz5v9qkNYtmOoGt4xoC0= Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-568-bYEKZXIIOyO99yl7ysFa0A-1; Wed, 01 Nov 2023 19:31:54 -0400 X-MC-Unique: bYEKZXIIOyO99yl7ysFa0A-1 Received: by mail-ej1-f72.google.com with SMTP id a640c23a62f3a-9d25d0788b8so18675966b.1 for ; Wed, 01 Nov 2023 16:31:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698881513; x=1699486313; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=juQeCGfPWswDlP+Z0KdUDVDRgyhQp0+hsdRA02TinWo=; b=RQ+qRE1lfR8rhJPxf5vDXN8mVOJOqwZlMutCPYUcREiu5TLe6aULdIKcbHIXqQ5D/Y Ewer7CCyKhCJDMJ2hPlXQcwf2sMGcbWIsiW9RLg8RhQTndlv5QVSnwjJqiMrwd1ebtR+ vkHyn0wnCrX05tBZJ8vOQSAGrb8vGywZNObWonZmrwQ1RAcB7yUIj/S+aoyvbeYPp/zC O5guX87QC1oGTzgIYdLQp9LhdL8Eo9Fdel73i96UFLHjbIuLazMjIYJ+7r4woi/gTjU7 b3zjzYS7aMTR8ITbQjBLJPd8brAHAPRxktnMgb9pXGaiDUFrRNdbeFZ/sxh+j2fo2pKT g4QQ== X-Gm-Message-State: AOJu0YzMvUcpk86XZ+He8vpsQXahwdGiUElUFzz2AHNP13FcrngTiUJ3 U+TuvzvlNJy5SiIX5oC3Xi91KOTryDo/KhrfzqMpXc7OyzCFiJwm43aKma8Q9hhuGzW6VQ3//4l TI461SM/tkV2Pm+fQ2DvQy5V/ X-Received: by 2002:a17:907:7da6:b0:9a9:ef41:e5c7 with SMTP id oz38-20020a1709077da600b009a9ef41e5c7mr3737995ejc.8.1698881513204; Wed, 01 Nov 2023 16:31:53 -0700 (PDT) X-Received: by 2002:a17:907:7da6:b0:9a9:ef41:e5c7 with SMTP id oz38-20020a1709077da600b009a9ef41e5c7mr3737989ejc.8.1698881512969; Wed, 01 Nov 2023 16:31:52 -0700 (PDT) Received: from cassiopeiae.. ([2a02:810d:4b3f:de9c:642:1aff:fe31:a19f]) by smtp.gmail.com with ESMTPSA id i17-20020a170906a29100b00992e265495csm456222ejz.212.2023.11.01.16.31.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Nov 2023 16:31:52 -0700 (PDT) From: Danilo Krummrich To: airlied@gmail.com, daniel@ffwll.ch, matthew.brost@intel.com, thomas.hellstrom@linux.intel.com, sarah.walker@imgtec.com, donald.robson@imgtec.com, boris.brezillon@collabora.com, christian.koenig@amd.com, faith@gfxstrand.net Cc: dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-kernel@vger.kernel.org, Danilo Krummrich Subject: [PATCH drm-misc-next v8 09/12] drm/gpuvm: reference count drm_gpuvm structures Date: Thu, 2 Nov 2023 00:31:01 +0100 Message-ID: <20231101233113.8059-10-dakr@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231101233113.8059-1-dakr@redhat.com> References: <20231101233113.8059-1-dakr@redhat.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_FILL_THIS_FORM_SHORT,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Wed, 01 Nov 2023 16:32:49 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781406441025482159 X-GMAIL-MSGID: 1781406441025482159 Implement reference counting for struct drm_gpuvm. Signed-off-by: Danilo Krummrich --- drivers/gpu/drm/drm_gpuvm.c | 44 +++++++++++++++++++------- drivers/gpu/drm/nouveau/nouveau_uvmm.c | 20 +++++++++--- include/drm/drm_gpuvm.h | 31 +++++++++++++++++- 3 files changed, 78 insertions(+), 17 deletions(-) diff --git a/drivers/gpu/drm/drm_gpuvm.c b/drivers/gpu/drm/drm_gpuvm.c index 53e2c406fb04..6a88eafc5229 100644 --- a/drivers/gpu/drm/drm_gpuvm.c +++ b/drivers/gpu/drm/drm_gpuvm.c @@ -746,6 +746,8 @@ drm_gpuvm_init(struct drm_gpuvm *gpuvm, const char *name, gpuvm->rb.tree = RB_ROOT_CACHED; INIT_LIST_HEAD(&gpuvm->rb.list); + kref_init(&gpuvm->kref); + gpuvm->name = name ? name : "unknown"; gpuvm->flags = flags; gpuvm->ops = ops; @@ -770,15 +772,8 @@ drm_gpuvm_init(struct drm_gpuvm *gpuvm, const char *name, } EXPORT_SYMBOL_GPL(drm_gpuvm_init); -/** - * drm_gpuvm_destroy() - cleanup a &drm_gpuvm - * @gpuvm: pointer to the &drm_gpuvm to clean up - * - * Note that it is a bug to call this function on a manager that still - * holds GPU VA mappings. - */ -void -drm_gpuvm_destroy(struct drm_gpuvm *gpuvm) +static void +drm_gpuvm_fini(struct drm_gpuvm *gpuvm) { gpuvm->name = NULL; @@ -790,7 +785,33 @@ drm_gpuvm_destroy(struct drm_gpuvm *gpuvm) drm_gem_object_put(gpuvm->r_obj); } -EXPORT_SYMBOL_GPL(drm_gpuvm_destroy); + +static void +drm_gpuvm_free(struct kref *kref) +{ + struct drm_gpuvm *gpuvm = container_of(kref, struct drm_gpuvm, kref); + + if (drm_WARN_ON(gpuvm->drm, !gpuvm->ops->vm_free)) + return; + + drm_gpuvm_fini(gpuvm); + + gpuvm->ops->vm_free(gpuvm); +} + +/** + * drm_gpuvm_bo_put() - drop a struct drm_gpuvm reference + * @gpuvm: the &drm_gpuvm to release the reference of + * + * This releases a reference to @gpuvm. + */ +void +drm_gpuvm_put(struct drm_gpuvm *gpuvm) +{ + if (gpuvm) + kref_put(&gpuvm->kref, drm_gpuvm_free); +} +EXPORT_SYMBOL_GPL(drm_gpuvm_put); static int __drm_gpuva_insert(struct drm_gpuvm *gpuvm, @@ -843,7 +864,7 @@ drm_gpuva_insert(struct drm_gpuvm *gpuvm, if (unlikely(!drm_gpuvm_range_valid(gpuvm, addr, range))) return -EINVAL; - return __drm_gpuva_insert(gpuvm, va); + return __drm_gpuva_insert(drm_gpuvm_get(gpuvm), va); } EXPORT_SYMBOL_GPL(drm_gpuva_insert); @@ -876,6 +897,7 @@ drm_gpuva_remove(struct drm_gpuva *va) } __drm_gpuva_remove(va); + drm_gpuvm_put(va->vm); } EXPORT_SYMBOL_GPL(drm_gpuva_remove); diff --git a/drivers/gpu/drm/nouveau/nouveau_uvmm.c b/drivers/gpu/drm/nouveau/nouveau_uvmm.c index 54be12c1272f..cb2f06565c46 100644 --- a/drivers/gpu/drm/nouveau/nouveau_uvmm.c +++ b/drivers/gpu/drm/nouveau/nouveau_uvmm.c @@ -1780,6 +1780,18 @@ nouveau_uvmm_bo_unmap_all(struct nouveau_bo *nvbo) } } +static void +nouveau_uvmm_free(struct drm_gpuvm *gpuvm) +{ + struct nouveau_uvmm *uvmm = uvmm_from_gpuvm(gpuvm); + + kfree(uvmm); +} + +static const struct drm_gpuvm_ops gpuvm_ops = { + .vm_free = nouveau_uvmm_free, +}; + int nouveau_uvmm_ioctl_vm_init(struct drm_device *dev, void *data, @@ -1830,7 +1842,7 @@ nouveau_uvmm_ioctl_vm_init(struct drm_device *dev, NOUVEAU_VA_SPACE_END, init->kernel_managed_addr, init->kernel_managed_size, - NULL); + &gpuvm_ops); /* GPUVM takes care from here on. */ drm_gem_object_put(r_obj); @@ -1849,8 +1861,7 @@ nouveau_uvmm_ioctl_vm_init(struct drm_device *dev, return 0; out_gpuvm_fini: - drm_gpuvm_destroy(&uvmm->base); - kfree(uvmm); + drm_gpuvm_put(&uvmm->base); out_unlock: mutex_unlock(&cli->mutex); return ret; @@ -1902,7 +1913,6 @@ nouveau_uvmm_fini(struct nouveau_uvmm *uvmm) mutex_lock(&cli->mutex); nouveau_vmm_fini(&uvmm->vmm); - drm_gpuvm_destroy(&uvmm->base); - kfree(uvmm); + drm_gpuvm_put(&uvmm->base); mutex_unlock(&cli->mutex); } diff --git a/include/drm/drm_gpuvm.h b/include/drm/drm_gpuvm.h index 0c2e24155a93..4e6e1fd3485a 100644 --- a/include/drm/drm_gpuvm.h +++ b/include/drm/drm_gpuvm.h @@ -247,6 +247,11 @@ struct drm_gpuvm { struct list_head list; } rb; + /** + * @kref: reference count of this object + */ + struct kref kref; + /** * @kernel_alloc_node: * @@ -273,7 +278,23 @@ void drm_gpuvm_init(struct drm_gpuvm *gpuvm, const char *name, u64 start_offset, u64 range, u64 reserve_offset, u64 reserve_range, const struct drm_gpuvm_ops *ops); -void drm_gpuvm_destroy(struct drm_gpuvm *gpuvm); + +/** + * drm_gpuvm_get() - acquire a struct drm_gpuvm reference + * @gpuvm: the &drm_gpuvm to acquire the reference of + * + * This function acquires an additional reference to @gpuvm. It is illegal to + * call this without already holding a reference. No locks required. + */ +static inline struct drm_gpuvm * +drm_gpuvm_get(struct drm_gpuvm *gpuvm) +{ + kref_get(&gpuvm->kref); + + return gpuvm; +} + +void drm_gpuvm_put(struct drm_gpuvm *gpuvm); bool drm_gpuvm_range_valid(struct drm_gpuvm *gpuvm, u64 addr, u64 range); bool drm_gpuvm_interval_empty(struct drm_gpuvm *gpuvm, u64 addr, u64 range); @@ -673,6 +694,14 @@ static inline void drm_gpuva_init_from_op(struct drm_gpuva *va, * operations to drivers. */ struct drm_gpuvm_ops { + /** + * @vm_free: called when the last reference of a struct drm_gpuvm is + * dropped + * + * This callback is mandatory. + */ + void (*vm_free)(struct drm_gpuvm *gpuvm); + /** * @op_alloc: called when the &drm_gpuvm allocates * a struct drm_gpuva_op