From patchwork Mon Mar 6 16:31:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fenghua Yu X-Patchwork-Id: 64952 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1951166wrd; Mon, 6 Mar 2023 08:56:11 -0800 (PST) X-Google-Smtp-Source: AK7set/2ugZb8v5z/p0uSI3bhFBLWTY/fongmlEUR1Y8DAYLU47wEuhRLpR+ija/+crz5NXT9pP/ X-Received: by 2002:a17:902:6a81:b0:19d:20f7:daf5 with SMTP id n1-20020a1709026a8100b0019d20f7daf5mr10031388plk.28.1678121771602; Mon, 06 Mar 2023 08:56:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1678121771; cv=none; d=google.com; s=arc-20160816; b=sN0UG0s7K6vJVUrBz066bi/Kz6PMj8PRY/4jGlSRx5RfISUZ2jedKUEsKm+TkCi7kx bTKkh2U/P/EFaYI8gzogTZw726nFK8lcL2UCd8hHFKUiyDl52kO6ov7q5+9kv1lM8ose Etj+sIvuaGiRFHYi9GQA0uWJSBabWRSilCm/8C1p6ECbgqERbmaL53i9mNdT2tdTYyxV kg3hvY7BPtplSCWeNIdqVhih9xw4uiqNu9iDVJ7M+p1Jevbr+lKONK1xVMJLC/0rt/9E RoeBj+IXYB2bYXUWCYG1O5q7X826KDvfQEYIc5a9ZXpvnzZ2qV7cprKvRhNlUlGbhjLU 2f2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=SL80H8ECzhi7b4y5IEi1MDrFbdVwp0DRgp+tQwlNbkY=; b=Sly7CrgFRYS5aVb0PRMHxZbm16QQGe3wxpIfFQtYvUnn0A5mXn66ItXrv8gs2/gV1C 6j5YEJKnOGTD/l1CI4o25vnHh6b6KMpGjhZOHjNNOL84u3KPW6Kvrd7U6wcNy5DExhKV dKpgc0SdZso3LQ96qHnVjIrDSIDbT80Cd7UFGif6uzTJvAeSBuP39OfLaNooNxhqFAQF mEAtmqJd/EB99T7doeFMUIb0YA22n74cSShMOGk1SSN7QwTSsxeP3D34/qeGKZj+YJO0 vngyCOnO7HQFdjc4gVle0V9IkU4YQnkxSCKxXQ3GmKBASom0c9/EMPbwQvLsKPmXaXqs xIJQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="QD/JCTT+"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 84-20020a630557000000b00502e4676ac9si9102054pgf.584.2023.03.06.08.55.58; Mon, 06 Mar 2023 08:56:11 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="QD/JCTT+"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230143AbjCFQq0 (ORCPT + 99 others); Mon, 6 Mar 2023 11:46:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34886 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229801AbjCFQol (ORCPT ); Mon, 6 Mar 2023 11:44:41 -0500 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 109FABC; Mon, 6 Mar 2023 08:44:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678121044; x=1709657044; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=o0tzkoJWUdCWbcbRIlF9BFXfJp21Z11MftALfWwp9uM=; b=QD/JCTT+SGbbivKqusSkyKpQ2ZVfC2db//w9PeAsvV5abKAtnLbmdjcy /qwaMPp5QRidHeQ7atvKhlHmhJIAfj9mmgvUWovh+YglGzdcYzkbdjhBd 3LWpraIZBfnudi7NxgZyj+TGiVnndoN7tFfCFgGAoB2R89hHIQf9bWsZy qQTfWVJCO8ApNiuhaLem3SoM66Yj+yQsPiTYT9LOfc5lPZ8zBj2BXoBFz m+/vaGM3Q8Ys89mH6P/gHw+2wkXzvc4D0bgrM2y9bVI3RI3ymOl1+kr88 ZNnpAHPu91Na/+55SoYy+FIWnOXuwCphE8wwdfzLB8Jj77m+Szaa5s1sK A==; X-IronPort-AV: E=McAfee;i="6500,9779,10641"; a="398181182" X-IronPort-AV: E=Sophos;i="5.98,238,1673942400"; d="scan'208";a="398181182" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Mar 2023 08:31:59 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10641"; a="669504553" X-IronPort-AV: E=Sophos;i="5.98,238,1673942400"; d="scan'208";a="669504553" Received: from fyu1.sc.intel.com ([172.25.103.126]) by orsmga007.jf.intel.com with ESMTP; 06 Mar 2023 08:31:59 -0800 From: Fenghua Yu To: "Vinod Koul" , "Dave Jiang" Cc: dmaengine@vger.kernel.org, "linux-kernel" , Tony Zhu , Fenghua Yu Subject: [PATCH v2 13/16] dmaengine: idxd: add a device to represent the file opened Date: Mon, 6 Mar 2023 08:31:35 -0800 Message-Id: <20230306163138.587484-14-fenghua.yu@intel.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230306163138.587484-1-fenghua.yu@intel.com> References: <20230306163138.587484-1-fenghua.yu@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1759638215100212249?= X-GMAIL-MSGID: =?utf-8?q?1759638215100212249?= From: Dave Jiang Embed a struct device for the user file context in order to export sysfs attributes related with the opened file. Tie the lifetime of the file context to the device. The sysfs entry will be added under the char device. Tested-by: Tony Zhu Signed-off-by: Dave Jiang Co-developed-by: Fenghua Yu Signed-off-by: Fenghua Yu --- drivers/dma/idxd/cdev.c | 122 +++++++++++++++++++++++++++++++--------- drivers/dma/idxd/idxd.h | 2 + 2 files changed, 97 insertions(+), 27 deletions(-) diff --git a/drivers/dma/idxd/cdev.c b/drivers/dma/idxd/cdev.c index 3ce134afa867..2ff9a680ea5e 100644 --- a/drivers/dma/idxd/cdev.c +++ b/drivers/dma/idxd/cdev.c @@ -22,6 +22,13 @@ struct idxd_cdev_context { struct ida minor_ida; }; +/* + * Since user file names are global in DSA devices, define their ida's as + * global to avoid conflict file names. + */ +static DEFINE_IDA(file_ida); +static DEFINE_MUTEX(ida_lock); + /* * ictx is an array based off of accelerator types. enum idxd_type * is used as index @@ -37,7 +44,60 @@ struct idxd_user_context { unsigned int pasid; unsigned int flags; struct iommu_sva *sva; + struct idxd_dev idxd_dev; u64 counters[COUNTER_MAX]; + int id; +}; + +static void idxd_cdev_evl_drain_pasid(struct idxd_wq *wq, u32 pasid); +static void idxd_xa_pasid_remove(struct idxd_user_context *ctx); + +static inline struct idxd_user_context *dev_to_uctx(struct device *dev) +{ + struct idxd_dev *idxd_dev = confdev_to_idxd_dev(dev); + + return container_of(idxd_dev, struct idxd_user_context, idxd_dev); +} + +static void idxd_file_dev_release(struct device *dev) +{ + struct idxd_user_context *ctx = dev_to_uctx(dev); + struct idxd_wq *wq = ctx->wq; + struct idxd_device *idxd = wq->idxd; + int rc; + + mutex_lock(&ida_lock); + ida_free(&file_ida, ctx->id); + mutex_unlock(&ida_lock); + + /* Wait for in-flight operations to complete. */ + if (wq_shared(wq)) { + idxd_device_drain_pasid(idxd, ctx->pasid); + } else { + if (device_user_pasid_enabled(idxd)) { + /* The wq disable in the disable pasid function will drain the wq */ + rc = idxd_wq_disable_pasid(wq); + if (rc < 0) + dev_err(dev, "wq disable pasid failed.\n"); + } else { + idxd_wq_drain(wq); + } + } + + if (ctx->sva) { + idxd_cdev_evl_drain_pasid(wq, ctx->pasid); + iommu_sva_unbind_device(ctx->sva); + idxd_xa_pasid_remove(ctx); + } + kfree(ctx); + mutex_lock(&wq->wq_lock); + idxd_wq_put(wq); + mutex_unlock(&wq->wq_lock); +} + +static struct device_type idxd_cdev_file_type = { + .name = "idxd_file", + .release = idxd_file_dev_release, }; static void idxd_cdev_dev_release(struct device *dev) @@ -105,10 +165,11 @@ static int idxd_cdev_open(struct inode *inode, struct file *filp) struct idxd_user_context *ctx; struct idxd_device *idxd; struct idxd_wq *wq; - struct device *dev; + struct device *dev, *fdev; int rc = 0; struct iommu_sva *sva; unsigned int pasid; + struct idxd_cdev *idxd_cdev; wq = inode_wq(inode); idxd = wq->idxd; @@ -163,10 +224,41 @@ static int idxd_cdev_open(struct inode *inode, struct file *filp) } } + idxd_cdev = wq->idxd_cdev; + mutex_lock(&ida_lock); + ctx->id = ida_alloc(&file_ida, GFP_KERNEL); + mutex_unlock(&ida_lock); + if (ctx->id < 0) { + dev_warn(dev, "ida alloc failure\n"); + goto failed_ida; + } + ctx->idxd_dev.type = IDXD_DEV_CDEV_FILE; + fdev = user_ctx_dev(ctx); + device_initialize(fdev); + fdev->parent = cdev_dev(idxd_cdev); + fdev->bus = &dsa_bus_type; + fdev->type = &idxd_cdev_file_type; + + rc = dev_set_name(fdev, "file%d", ctx->id); + if (rc < 0) { + dev_warn(dev, "set name failure\n"); + goto failed_dev_name; + } + + rc = device_add(fdev); + if (rc < 0) { + dev_warn(dev, "file device add failure\n"); + goto failed_dev_add; + } + idxd_wq_get(wq); mutex_unlock(&wq->wq_lock); return 0; + failed_dev_add: + failed_dev_name: + put_device(fdev); + failed_ida: failed_set_pasid: if (device_user_pasid_enabled(idxd)) idxd_xa_pasid_remove(ctx); @@ -214,35 +306,10 @@ static int idxd_cdev_release(struct inode *node, struct file *filep) struct idxd_wq *wq = ctx->wq; struct idxd_device *idxd = wq->idxd; struct device *dev = &idxd->pdev->dev; - int rc; dev_dbg(dev, "%s called\n", __func__); filep->private_data = NULL; - - /* Wait for in-flight operations to complete. */ - if (wq_shared(wq)) { - idxd_device_drain_pasid(idxd, ctx->pasid); - } else { - if (device_user_pasid_enabled(idxd)) { - /* The wq disable in the disable pasid function will drain the wq */ - rc = idxd_wq_disable_pasid(wq); - if (rc < 0) - dev_err(dev, "wq disable pasid failed.\n"); - } else { - idxd_wq_drain(wq); - } - } - - if (ctx->sva) { - idxd_cdev_evl_drain_pasid(wq, ctx->pasid); - iommu_sva_unbind_device(ctx->sva); - idxd_xa_pasid_remove(ctx); - } - - kfree(ctx); - mutex_lock(&wq->wq_lock); - idxd_wq_put(wq); - mutex_unlock(&wq->wq_lock); + device_unregister(user_ctx_dev(ctx)); return 0; } @@ -373,6 +440,7 @@ void idxd_wq_del_cdev(struct idxd_wq *wq) struct idxd_cdev *idxd_cdev; idxd_cdev = wq->idxd_cdev; + ida_destroy(&file_ida); wq->idxd_cdev = NULL; cdev_device_del(&idxd_cdev->cdev, cdev_dev(idxd_cdev)); put_device(cdev_dev(idxd_cdev)); diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h index f92db20015fb..b5d7ef611bae 100644 --- a/drivers/dma/idxd/idxd.h +++ b/drivers/dma/idxd/idxd.h @@ -32,6 +32,7 @@ enum idxd_dev_type { IDXD_DEV_GROUP, IDXD_DEV_ENGINE, IDXD_DEV_CDEV, + IDXD_DEV_CDEV_FILE, IDXD_DEV_MAX_TYPE, }; @@ -405,6 +406,7 @@ enum idxd_completion_status { #define engine_confdev(engine) &engine->idxd_dev.conf_dev #define group_confdev(group) &group->idxd_dev.conf_dev #define cdev_dev(cdev) &cdev->idxd_dev.conf_dev +#define user_ctx_dev(ctx) (&(ctx)->idxd_dev.conf_dev) #define confdev_to_idxd_dev(dev) container_of(dev, struct idxd_dev, conf_dev) #define idxd_dev_to_idxd(idxd_dev) container_of(idxd_dev, struct idxd_device, idxd_dev)