From patchwork Mon Jan 29 17:01:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Cercueil X-Patchwork-Id: 193608 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp714499dyb; Mon, 29 Jan 2024 09:22:44 -0800 (PST) X-Google-Smtp-Source: AGHT+IEgkCYrzB6b2Bt3KEDKWC3nSLm2HPWK2TxthSthQRTyw1yFAUVG1Ge/OhslKlCtk3R6dLDJ X-Received: by 2002:a05:6402:51ca:b0:55e:e829:1461 with SMTP id r10-20020a05640251ca00b0055ee8291461mr3079986edd.42.1706548964627; Mon, 29 Jan 2024 09:22:44 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706548964; cv=pass; d=google.com; s=arc-20160816; b=yQa2/qx+IKpsX5Cn3d4bhTQ+rMl0ckTvxPtWCfUk94P6s0L016FQAz7zj7TXawV2wN U6XXQJuOBPKYqI72kbZFypQgtfjB6bzmCW6uookx4eAfnEMlsJkufUQ/UlJjwGhFt7/w KLOk23lbUj6H5e57n8nT/qpJxQjaB7VBagmPS6LfT001yufSBsatQccPrzDp3ntsK/A2 7rMOwwrXOyU45DSl6Vmg2pIufdCoiVr9viiOwZKJGO343nbCdhjTeN4b52/dPyw+lnx3 IzwZgx2XKmW5TsRCuDGeSy67i0VhjLVQxjuOYyHqM+qn22qzPP4UeCALxuBS0zsa6Ndc SiAA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=ciNLk5XaUXAioykenKcf7E1KkXSWDnIcRp3Oxutuxu0=; fh=UUvZraMiTlolibKjuaTYSz8c8N6g0y2n/C3Uf8DFNMA=; b=wdtxIDyYcfdR8/yDDmJ9tWwfU0O4daey+hmZrwB8PM4KPoHuSUt58giVtgLfhTZCRw yQtI4euRLmI2ReG0FsWgh8tVgKCh06dYfEPgJh2Fhaw+YYg6L+Hk4iDj0nDjRBKmFXlE GRYw8IttZ+OVnhyPAZ3lFTF4enrEyVPpL6SLpz9UCyRLw43rPyhE7CSZk6ZbJ7VXFWmA FMzEfPFOaCUPzzQAXrTvMQm8DWldPVy2R1wqMXN/IkI/OJSJMyQFyV3bKl3blYQhXMz9 jBWUyHvdR0zBAXuxG3hPiKx6we0wwiC7fmwEFBVl34E43xeoayfIaLofOQG7wHQ0CcDR r/lg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b="ROERA/vs"; arc=pass (i=1 spf=pass spfdomain=crapouillou.net dkim=pass dkdomain=crapouillou.net dmarc=pass fromdomain=crapouillou.net); spf=pass (google.com: domain of linux-kernel+bounces-43174-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43174-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id f10-20020a056402194a00b0055e7e8a977fsi2888093edz.380.2024.01.29.09.22.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 09:22:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43174-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b="ROERA/vs"; arc=pass (i=1 spf=pass spfdomain=crapouillou.net dkim=pass dkdomain=crapouillou.net dmarc=pass fromdomain=crapouillou.net); spf=pass (google.com: domain of linux-kernel+bounces-43174-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43174-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 5429A1F29A3C for ; Mon, 29 Jan 2024 17:03:13 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6954F15956B; Mon, 29 Jan 2024 17:02:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=crapouillou.net header.i=@crapouillou.net header.b="ROERA/vs" Received: from aposti.net (aposti.net [89.234.176.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 07121157026; Mon, 29 Jan 2024 17:02:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=89.234.176.197 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706547747; cv=none; b=Yo64Khv9v3fo9N/8z+nIKV0vQdOhEk31KEJzaRLAuD/L6niC3h4kcYuKV6xF7L9e1EuunUrP1cGfrivanFB53z/keOWsSmByO6zx0Wiqjm4v0mNemMj/HG2H1dR8Oz0fUJXnOtl3maECLndJgE0p42g2tOq9HTwSZFiMeQW0oQg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706547747; c=relaxed/simple; bh=Zq5Xbj+rWU8akfSZxarVMv3U6m+GKxNoE54+0tQg5Fc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JpbcLAZHQWGM5j/lBJcC5Jo0nkrO4SkH1PKf/VI+qUBfRDwF7jo/nPTNFeTQ7kXsTpmvY9yfnIvds9hi0pka/q9hoOQSj/KURAjYo6PAyZOm0VvKbuVqsvBOPJu6T4zNsVqDiC2/Rifgddk9AmBSd+9UAglLjaSlhbJMXMgag4k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=crapouillou.net; spf=pass smtp.mailfrom=crapouillou.net; dkim=pass (1024-bit key) header.d=crapouillou.net header.i=@crapouillou.net header.b=ROERA/vs; arc=none smtp.client-ip=89.234.176.197 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=crapouillou.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=crapouillou.net DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=crapouillou.net; s=mail; t=1706547736; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ciNLk5XaUXAioykenKcf7E1KkXSWDnIcRp3Oxutuxu0=; b=ROERA/vshX8isvB0THFWyZBHPRQ3mRtms+JQHpKmfHNwLf3v8hy8SK0RzrSrf8PqWpYJ5A HqSDGa+GfTVW/uhOVp3EZ+mT8TwlYxnIAcYjcq2rDUG2msXKy5MinDfeIRRvlZcNglPKsf Ygr0mAttNSk7DNo6Qb7ZuiqXDTBbHls= From: Paul Cercueil To: Jonathan Cameron , Jonathan Corbet , Lars-Peter Clausen , Vinod Koul , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: Daniel Vetter , Nuno Sa , Michael Hennerich , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, dmaengine@vger.kernel.org, linux-iio@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, Paul Cercueil Subject: [PATCH v6 1/6] dmaengine: Add API function dmaengine_prep_slave_dma_vec() Date: Mon, 29 Jan 2024 18:01:56 +0100 Message-ID: <20240129170201.133785-2-paul@crapouillou.net> In-Reply-To: <20240129170201.133785-1-paul@crapouillou.net> References: <20240129170201.133785-1-paul@crapouillou.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789446287245020420 X-GMAIL-MSGID: 1789446287245020420 This function can be used to initiate a scatter-gather DMA transfer, where the address and size of each segment is located in one entry of the dma_vec array. The major difference with dmaengine_prep_slave_sg() is that it supports specifying the lengths of each DMA transfer; as trying to override the length of the transfer with dmaengine_prep_slave_sg() is a very tedious process. The introduction of a new API function is also justified by the fact that scatterlists are on their way out. Note that dmaengine_prep_interleaved_dma() is not helpful either in that case, as it assumes that the address of each segment will be higher than the one of the previous segment, which we just cannot guarantee in case of a scatter-gather transfer. Signed-off-by: Paul Cercueil --- v3: New patch v5: Replace with function dmaengine_prep_slave_dma_vec(), and struct 'dma_vec'. Note that at some point we will need to support cyclic transfers using dmaengine_prep_slave_dma_vec(). Maybe with a new "flags" parameter to the function? --- include/linux/dmaengine.h | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/include/linux/dmaengine.h b/include/linux/dmaengine.h index 3df70d6131c8..ee5931ddb42f 100644 --- a/include/linux/dmaengine.h +++ b/include/linux/dmaengine.h @@ -160,6 +160,16 @@ struct dma_interleaved_template { struct data_chunk sgl[]; }; +/** + * struct dma_vec - DMA vector + * @addr: Bus address of the start of the vector + * @len: Length in bytes of the DMA vector + */ +struct dma_vec { + dma_addr_t addr; + size_t len; +}; + /** * enum dma_ctrl_flags - DMA flags to augment operation preparation, * control completion, and communicate status. @@ -910,6 +920,10 @@ struct dma_device { struct dma_async_tx_descriptor *(*device_prep_dma_interrupt)( struct dma_chan *chan, unsigned long flags); + struct dma_async_tx_descriptor *(*device_prep_slave_dma_vec)( + struct dma_chan *chan, const struct dma_vec *vecs, + size_t nents, enum dma_transfer_direction direction, + unsigned long flags); struct dma_async_tx_descriptor *(*device_prep_slave_sg)( struct dma_chan *chan, struct scatterlist *sgl, unsigned int sg_len, enum dma_transfer_direction direction, @@ -972,6 +986,17 @@ static inline struct dma_async_tx_descriptor *dmaengine_prep_slave_single( dir, flags, NULL); } +static inline struct dma_async_tx_descriptor *dmaengine_prep_slave_dma_vec( + struct dma_chan *chan, const struct dma_vec *vecs, size_t nents, + enum dma_transfer_direction dir, unsigned long flags) +{ + if (!chan || !chan->device || !chan->device->device_prep_slave_dma_vec) + return NULL; + + return chan->device->device_prep_slave_dma_vec(chan, vecs, nents, + dir, flags); +} + static inline struct dma_async_tx_descriptor *dmaengine_prep_slave_sg( struct dma_chan *chan, struct scatterlist *sgl, unsigned int sg_len, enum dma_transfer_direction dir, unsigned long flags) From patchwork Mon Jan 29 17:01:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Cercueil X-Patchwork-Id: 193607 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp714206dyb; Mon, 29 Jan 2024 09:22:13 -0800 (PST) X-Google-Smtp-Source: AGHT+IHKEwrHGJ7tPPy+tS6mbFbRmWa05EWFa9/KPvxnz/6Rt9oNeiRpfk9hNioNBhecVzqPvVjQ X-Received: by 2002:a17:906:6848:b0:a35:deb0:cd19 with SMTP id a8-20020a170906684800b00a35deb0cd19mr1631754ejs.41.1706548933145; Mon, 29 Jan 2024 09:22:13 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706548933; cv=pass; d=google.com; s=arc-20160816; b=YcjDTiHNwuEkX0UYnTUoRF28vN/MglJbnO9oTkCg+OOAMAVRgCPrXu7t1yWWmGY2wH 3MCS6zRIlgMm/FoRA3+h3nbmwZebfXKoB9zkYZBSm3BuvDCegqPcK7YfsM2ew1f04x4O IKerLu6G7+KvMA/VRUOQv4nYyha2gtxtua19B9oX/AkJI9nEaIAy8iiavYd9vOaONdN7 RTX4umiEi+BpRIa2PL3bX9xcBibOqysX+oNYgm8lx+Dk/z7WAeSHRvDvCf7vRX0nNAfg LmK/3OhrJAZ2Xe4MizwpouRHDvKcWBh+48TA6cPjTNcZkwOC41v8X87HH1K7e0S2FaMr X6Wg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=AurlEA6DoJCzg+71bkiCFnfLG7gMwOBZhYf30UJ9HR8=; fh=UUvZraMiTlolibKjuaTYSz8c8N6g0y2n/C3Uf8DFNMA=; b=x3BVBi5x1JnSczkto/rDJ92ylwAj/einimOjNJhB/92hspdLAGImJLB8VXeB0OxMdz HfkkWxx39N9DTyBQ3cohpScnVYVxl3bz9yaq+w6EnKgZZVQPY6oop+e+o1vL88peM9vU 5Z7wcaMw4WLybin+ApF3vg7nGZfMNgz9lK4kwLw5ExFa6u4H/ZNIZR8eHE2fwxuQkzNV qrWkhYq7YgxsdornaD4Zux2XmZ7pJKgLywLjlhYr18EsqIpsxwfhtlN6L59XO/nAlesc lUOU8qGHznNFqP0i9eU5hxHBRAr/gPlglmDs05aAudFcXKw2I1wuVDUARP5fFSIzCH6b SkBA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b=reCqO7KH; arc=pass (i=1 spf=pass spfdomain=crapouillou.net dkim=pass dkdomain=crapouillou.net dmarc=pass fromdomain=crapouillou.net); spf=pass (google.com: domain of linux-kernel+bounces-43175-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43175-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id le6-20020a170906ae0600b00a342c1f050dsi3566873ejb.551.2024.01.29.09.22.12 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 09:22:13 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43175-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b=reCqO7KH; arc=pass (i=1 spf=pass spfdomain=crapouillou.net dkim=pass dkdomain=crapouillou.net dmarc=pass fromdomain=crapouillou.net); spf=pass (google.com: domain of linux-kernel+bounces-43175-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43175-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 47D4F1F29E9C for ; Mon, 29 Jan 2024 17:03:36 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id F1916157E62; Mon, 29 Jan 2024 17:02:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=crapouillou.net header.i=@crapouillou.net header.b="reCqO7KH" Received: from aposti.net (aposti.net [89.234.176.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C878C159578; Mon, 29 Jan 2024 17:02:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=89.234.176.197 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706547755; cv=none; b=q+Kdn7zDbA41J+nceGGZKT03e+gRW0tC4QziDIrORH1Qd970LBaXdTSnvpeHKYDlAI7Vw3QSgS6BmPQ3mKH3Na9aFep77RPQMqDMTC91sTsULBFhYLWeHYYq1faqfsrSjr+5PXR3Kk02jfML5A6+Skm69G8JBdKA8u9QCWbyyzQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706547755; c=relaxed/simple; bh=0y8kLw+aA7K36Gjm6NOLYCOcdvQW78uBQ7ekLR5rBGo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=m9fDZv8vXm0ZHoFzS1a/w9sGVnTULykWWflG/tCT26QO/RxVLB85f9kwl4HIYKoKNypIX7PSeBw6sZHx4Dq3EeJKtlOi6STBJ0r3jT7iZMlzORxgAzhZn+09gfmzhiP8USZbL9aeD8lnnrMp2NoFPcU+JRRnXxu2PQ1rgDeUVfM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=crapouillou.net; spf=pass smtp.mailfrom=crapouillou.net; dkim=pass (1024-bit key) header.d=crapouillou.net header.i=@crapouillou.net header.b=reCqO7KH; arc=none smtp.client-ip=89.234.176.197 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=crapouillou.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=crapouillou.net DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=crapouillou.net; s=mail; t=1706547737; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AurlEA6DoJCzg+71bkiCFnfLG7gMwOBZhYf30UJ9HR8=; b=reCqO7KHV08j9c8YnlrUDobYtPOoXhcSBM5TvFFi9A2IoHtmdScjc6h1aDrHx00MQwOnpV JB9OaENhyi9FKNLFet5DZB5rXl2JCxyTs091MFUPhbNLuU6ijt2AAqctWNg2D1/jL3YLMD soUqE3clcatavU8p5WDb6lYM21HG648= From: Paul Cercueil To: Jonathan Cameron , Jonathan Corbet , Lars-Peter Clausen , Vinod Koul , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: Daniel Vetter , Nuno Sa , Michael Hennerich , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, dmaengine@vger.kernel.org, linux-iio@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, Paul Cercueil Subject: [PATCH v6 2/6] dmaengine: dma-axi-dmac: Implement device_prep_slave_dma_vec Date: Mon, 29 Jan 2024 18:01:57 +0100 Message-ID: <20240129170201.133785-3-paul@crapouillou.net> In-Reply-To: <20240129170201.133785-1-paul@crapouillou.net> References: <20240129170201.133785-1-paul@crapouillou.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789446254329082885 X-GMAIL-MSGID: 1789446254329082885 Add implementation of the .device_prep_slave_dma_vec() callback. Signed-off-by: Paul Cercueil --- v3: New patch v5: Implement .device_prep_slave_dma_vec() instead of v3's .device_prep_slave_dma_array(). v6: Use new prototype for axi_dmac_alloc_desc() as it changed upstream. --- drivers/dma/dma-axi-dmac.c | 40 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/drivers/dma/dma-axi-dmac.c b/drivers/dma/dma-axi-dmac.c index 4e339c04fc1e..276856a1742d 100644 --- a/drivers/dma/dma-axi-dmac.c +++ b/drivers/dma/dma-axi-dmac.c @@ -620,6 +620,45 @@ static struct axi_dmac_sg *axi_dmac_fill_linear_sg(struct axi_dmac_chan *chan, return sg; } +static struct dma_async_tx_descriptor * +axi_dmac_prep_slave_dma_vec(struct dma_chan *c, const struct dma_vec *vecs, + size_t nb, enum dma_transfer_direction direction, + unsigned long flags) +{ + struct axi_dmac_chan *chan = to_axi_dmac_chan(c); + struct axi_dmac_desc *desc; + unsigned int num_sgs = 0; + struct axi_dmac_sg *dsg; + size_t i; + + if (direction != chan->direction) + return NULL; + + for (i = 0; i < nb; i++) + num_sgs += DIV_ROUND_UP(vecs[i].len, chan->max_length); + + desc = axi_dmac_alloc_desc(chan, num_sgs); + if (!desc) + return NULL; + + dsg = desc->sg; + + for (i = 0; i < nb; i++) { + if (!axi_dmac_check_addr(chan, vecs[i].addr) || + !axi_dmac_check_len(chan, vecs[i].len)) { + kfree(desc); + return NULL; + } + + dsg = axi_dmac_fill_linear_sg(chan, direction, vecs[i].addr, 1, + vecs[i].len, dsg); + } + + desc->cyclic = false; + + return vchan_tx_prep(&chan->vchan, &desc->vdesc, flags); +} + static struct dma_async_tx_descriptor *axi_dmac_prep_slave_sg( struct dma_chan *c, struct scatterlist *sgl, unsigned int sg_len, enum dma_transfer_direction direction, @@ -1055,6 +1094,7 @@ static int axi_dmac_probe(struct platform_device *pdev) dma_dev->device_tx_status = dma_cookie_status; dma_dev->device_issue_pending = axi_dmac_issue_pending; dma_dev->device_prep_slave_sg = axi_dmac_prep_slave_sg; + dma_dev->device_prep_slave_dma_vec = axi_dmac_prep_slave_dma_vec; dma_dev->device_prep_dma_cyclic = axi_dmac_prep_dma_cyclic; dma_dev->device_prep_interleaved_dma = axi_dmac_prep_interleaved; dma_dev->device_terminate_all = axi_dmac_terminate_all; From patchwork Mon Jan 29 17:01:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Cercueil X-Patchwork-Id: 193599 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp701802dyb; Mon, 29 Jan 2024 09:04:03 -0800 (PST) X-Google-Smtp-Source: AGHT+IFMM8yZleCH/++BjvOFV0BfyClrSj0NjL9uZyXKzKmoVRUCXZGrTkLOWjMvZzys0nvGxpXq X-Received: by 2002:a05:6214:1d23:b0:686:aa78:df4d with SMTP id f3-20020a0562141d2300b00686aa78df4dmr8195889qvd.53.1706547843581; Mon, 29 Jan 2024 09:04:03 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706547843; cv=pass; d=google.com; s=arc-20160816; b=IYvxGjw0Zq8NKLk5rKYOdXlvQcKrq1cI9KXd9kQAghjDQhTVXAXkR0pSkhUEdg6BUj +0Nud9CYcGRFUlAMhG9YooQMRrSFV/g+VXdYdNTdpR0TO5VorqqmNuvdIJk53CC/Tai+ 9M7gQ96PqvVq3hLFQQC5aXfQeZpdYkUx6fv5eSmc9ExlwJEy0zmQLLxeguTzNZA0rBoU fo4Z8Vu3BJY+q3Ak7gbmEGpIrh/bGLWuUb7z2ELu8Ngtid3d4fIyFWRED31MqbEE37iP qjFZlm99SIGe08FvjIxMurVwaGAtArLQ016u94C60CLGPU0YXmClXf+Ak8hvOf+MSdsf ffpQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=AgdsXnVTon/FRLXp7TQxqQVcJiHh41Rxyt8QgmpirfA=; fh=UUvZraMiTlolibKjuaTYSz8c8N6g0y2n/C3Uf8DFNMA=; b=m2BDDWO3LLKt5ADYSyhsR0F0Y+m4dlArPYmQBOI9D/HPFk66C/vibZXaFIs9s8KcXe UHyrQM80LdQVwTNXNTrhqnVvo5ovbNH9KltMjXsGXO2jdI2RNfTyqx3MBF1hH+v1rXlm gGcU34VZjkpXNn3aA6MxT3ewpV7nCzEEQJGxTIoAxMBFZasUcHuFRK1ZKevLq+dY+ACq GTOvEI9F9oKEmArKGHsu46TZfokRVqX6pnm8uD0RCdG9oI65Hvz/7yBtlgSbFB2F1kha 0zuSmo+rGhohIFGu1MbCwK/vm22lkFholuAyxk0j1Sut6VQbb8fUOMQzV7QJMIBQlJPx Xrfg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b=Az9rWxhf; arc=pass (i=1 spf=pass spfdomain=crapouillou.net dkim=pass dkdomain=crapouillou.net dmarc=pass fromdomain=crapouillou.net); spf=pass (google.com: domain of linux-kernel+bounces-43176-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43176-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id i15-20020a05620a248f00b00783fa16e04fsi3372548qkn.737.2024.01.29.09.04.03 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 09:04:03 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43176-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b=Az9rWxhf; arc=pass (i=1 spf=pass spfdomain=crapouillou.net dkim=pass dkdomain=crapouillou.net dmarc=pass fromdomain=crapouillou.net); spf=pass (google.com: domain of linux-kernel+bounces-43176-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43176-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 66C251C22ECA for ; Mon, 29 Jan 2024 17:03:54 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7DFAB15A48C; Mon, 29 Jan 2024 17:02:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=crapouillou.net header.i=@crapouillou.net header.b="Az9rWxhf" Received: from aposti.net (aposti.net [89.234.176.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EBBB6157E68; Mon, 29 Jan 2024 17:02:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=89.234.176.197 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706547764; cv=none; b=HKyy9GUsjlKxxYLHgES0xSFjVhOFpLGhc27lr2lA6GAXQ1+t3Zz+14NX2DXiWL/JANAliAbPqeTlkE59y7c/YFb1EK11O8yINKCB2R++d4IncSkpWDU0xtMCtjHiO5WlANADaT9X8qtqiR2ney96SVTYEygAn/5ztK82hI4zl38= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706547764; c=relaxed/simple; bh=I2wiwsBViu2/r7p1CJYm4ak8ATxaZ3OcqUuVknnl9Vg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fLxueAK70VDx1byPpggwWmD0r22sScQnaMyOieLjPk6wr6m7yjPfEnzZPubVr1qS1A9Z10yJ1D/qohQcwoYSHW2ZLYjCBWeXL+5AfmHgM2tCC5NHTdmlMFo3eQkMSPH3wG9lJviyGCvBJ+Gpds8ep7XH2cBisQ4VNgSTZuIFjl0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=crapouillou.net; spf=pass smtp.mailfrom=crapouillou.net; dkim=pass (1024-bit key) header.d=crapouillou.net header.i=@crapouillou.net header.b=Az9rWxhf; arc=none smtp.client-ip=89.234.176.197 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=crapouillou.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=crapouillou.net DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=crapouillou.net; s=mail; t=1706547738; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AgdsXnVTon/FRLXp7TQxqQVcJiHh41Rxyt8QgmpirfA=; b=Az9rWxhfq+cGRnNe0zl1dKyjOjTwjtdON2b4Ar2V5hY2ahT7DN3lxGjbd6nw8qKiJfjmfA yOqK1HDmIXDNC6nY8BCJZB6eLYwU25ITkdfkgI5Jz6VpbcBtLrK6/u5OlpWsgh+gHTsGlv FNiyCqwTT7SKJHe9U2fnuOGpkW3FWKw= From: Paul Cercueil To: Jonathan Cameron , Jonathan Corbet , Lars-Peter Clausen , Vinod Koul , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: Daniel Vetter , Nuno Sa , Michael Hennerich , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, dmaengine@vger.kernel.org, linux-iio@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, Paul Cercueil Subject: [PATCH v6 3/6] iio: core: Add new DMABUF interface infrastructure Date: Mon, 29 Jan 2024 18:01:58 +0100 Message-ID: <20240129170201.133785-4-paul@crapouillou.net> In-Reply-To: <20240129170201.133785-1-paul@crapouillou.net> References: <20240129170201.133785-1-paul@crapouillou.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789445111216904860 X-GMAIL-MSGID: 1789445111216904860 Add the necessary infrastructure to the IIO core to support a new optional DMABUF based interface. With this new interface, DMABUF objects (externally created) can be attached to a IIO buffer, and subsequently used for data transfer. A userspace application can then use this interface to share DMABUF objects between several interfaces, allowing it to transfer data in a zero-copy fashion, for instance between IIO and the USB stack. The userspace application can also memory-map the DMABUF objects, and access the sample data directly. The advantage of doing this vs. the read() interface is that it avoids an extra copy of the data between the kernel and userspace. This is particularly userful for high-speed devices which produce several megabytes or even gigabytes of data per second. As part of the interface, 3 new IOCTLs have been added: IIO_BUFFER_DMABUF_ATTACH_IOCTL(int fd): Attach the DMABUF object identified by the given file descriptor to the buffer. IIO_BUFFER_DMABUF_DETACH_IOCTL(int fd): Detach the DMABUF object identified by the given file descriptor from the buffer. Note that closing the IIO buffer's file descriptor will automatically detach all previously attached DMABUF objects. IIO_BUFFER_DMABUF_ENQUEUE_IOCTL(struct iio_dmabuf *): Request a data transfer to/from the given DMABUF object. Its file descriptor, as well as the transfer size and flags are provided in the "iio_dmabuf" structure. These three IOCTLs have to be performed on the IIO buffer's file descriptor, obtained using the IIO_BUFFER_GET_FD_IOCTL() ioctl. Signed-off-by: Paul Cercueil --- v2: Only allow the new IOCTLs on the buffer FD created with IIO_BUFFER_GET_FD_IOCTL(). v3: - Get rid of the old IOCTLs. The IIO subsystem does not create or manage DMABUFs anymore, and only attaches/detaches externally created DMABUFs. - Add IIO_BUFFER_DMABUF_CYCLIC to the supported flags. v5: - Use dev_err() instead of pr_err() - Inline to_iio_dma_fence() - Add comment to explain why we unref twice when detaching dmabuf - Remove TODO comment. It is actually safe to free the file's private data even when transfers are still pending because it won't be accessed. - Fix documentation of new fields in struct iio_buffer_access_funcs - iio_dma_resv_lock() does not need to be exported, make it static v6: - Remove dead code in iio_dma_resv_lock() - Fix non-block actually blocking - Cache dma_buf_attachment instead of mapping/unmapping it for every transfer - Return -EINVAL instead of IIO_IOCTL_UNHANDLED for unknown ioctl - Make .block_enqueue() callback take a dma_fence pointer, which will be passed to iio_buffer_signal_dmabuf_done() instead of the dma_buf_attachment; and remove the backpointer from the priv structure to the dma_fence. - Use dma_fence_begin/end_signalling in the dma_fence critical sections - Unref dma_fence and dma_buf_attachment in worker, because they might try to lock the dma_resv, which would deadlock. - Add buffer ops to lock/unlock the queue. This is motivated by the fact that once the dma_fence has been installed, we cannot lock anything anymore - so the queue must be locked before the dma_fence is installed. - Use 'long retl' variable to handle the return value of dma_resv_wait_timeout() - Protect dmabufs list access with a mutex - Rework iio_buffer_find_attachment() to use the internal dmabufs list, instead of messing with dmabufs private data. - Add an atomically-increasing sequence number for fences --- drivers/iio/industrialio-buffer.c | 462 ++++++++++++++++++++++++++++++ include/linux/iio/buffer_impl.h | 33 +++ include/uapi/linux/iio/buffer.h | 22 ++ 3 files changed, 517 insertions(+) diff --git a/drivers/iio/industrialio-buffer.c b/drivers/iio/industrialio-buffer.c index b581a7e80566..0e63a09fa90a 100644 --- a/drivers/iio/industrialio-buffer.c +++ b/drivers/iio/industrialio-buffer.c @@ -9,14 +9,19 @@ * - Better memory allocation techniques? * - Alternative access techniques? */ +#include #include #include #include #include +#include +#include +#include #include #include #include #include +#include #include #include @@ -28,6 +33,32 @@ #include #include +#define DMABUF_ENQUEUE_TIMEOUT_MS 5000 + +struct iio_dma_fence; + +struct iio_dmabuf_priv { + struct list_head entry; + struct kref ref; + + struct iio_buffer *buffer; + struct iio_dma_buffer_block *block; + + u64 context; + spinlock_t lock; + + struct dma_buf_attachment *attach; + struct sg_table *sgt; + enum dma_data_direction dir; + atomic_t seqno; +}; + +struct iio_dma_fence { + struct dma_fence base; + struct iio_dmabuf_priv *priv; + struct work_struct work; +}; + static const char * const iio_endian_prefix[] = { [IIO_BE] = "be", [IIO_LE] = "le", @@ -332,6 +363,8 @@ void iio_buffer_init(struct iio_buffer *buffer) { INIT_LIST_HEAD(&buffer->demux_list); INIT_LIST_HEAD(&buffer->buffer_list); + INIT_LIST_HEAD(&buffer->dmabufs); + mutex_init(&buffer->dmabufs_mutex); init_waitqueue_head(&buffer->pollq); kref_init(&buffer->ref); if (!buffer->watermark) @@ -1519,14 +1552,62 @@ static void iio_buffer_unregister_legacy_sysfs_groups(struct iio_dev *indio_dev) kfree(iio_dev_opaque->legacy_scan_el_group.attrs); } +static void iio_buffer_dmabuf_release(struct kref *ref) +{ + struct iio_dmabuf_priv *priv = container_of(ref, struct iio_dmabuf_priv, ref); + struct dma_buf_attachment *attach = priv->attach; + struct iio_buffer *buffer = priv->buffer; + struct dma_buf *dmabuf = attach->dmabuf; + + dma_resv_lock(dmabuf->resv, NULL); + dma_buf_unmap_attachment(attach, priv->sgt, priv->dir); + dma_resv_unlock(dmabuf->resv); + + buffer->access->detach_dmabuf(buffer, priv->block); + + dma_buf_detach(attach->dmabuf, attach); + dma_buf_put(dmabuf); + kfree(priv); +} + +void iio_buffer_dmabuf_get(struct dma_buf_attachment *attach) +{ + struct iio_dmabuf_priv *priv = attach->importer_priv; + + kref_get(&priv->ref); +} +EXPORT_SYMBOL_GPL(iio_buffer_dmabuf_get); + +void iio_buffer_dmabuf_put(struct dma_buf_attachment *attach) +{ + struct iio_dmabuf_priv *priv = attach->importer_priv; + + kref_put(&priv->ref, iio_buffer_dmabuf_release); +} +EXPORT_SYMBOL_GPL(iio_buffer_dmabuf_put); + static int iio_buffer_chrdev_release(struct inode *inode, struct file *filep) { struct iio_dev_buffer_pair *ib = filep->private_data; struct iio_dev *indio_dev = ib->indio_dev; struct iio_buffer *buffer = ib->buffer; + struct iio_dmabuf_priv *priv, *tmp; wake_up(&buffer->pollq); + mutex_lock(&buffer->dmabufs_mutex); + + /* Close all attached DMABUFs */ + list_for_each_entry_safe(priv, tmp, &buffer->dmabufs, entry) { + list_del_init(&priv->entry); + iio_buffer_dmabuf_put(priv->attach); + } + + if (!list_empty(&buffer->dmabufs)) + dev_warn(&indio_dev->dev, "Buffer FD closed with active transfers\n"); + + mutex_unlock(&buffer->dmabufs_mutex); + kfree(ib); clear_bit(IIO_BUSY_BIT_POS, &buffer->flags); iio_device_put(indio_dev); @@ -1534,11 +1615,391 @@ static int iio_buffer_chrdev_release(struct inode *inode, struct file *filep) return 0; } +static int iio_dma_resv_lock(struct dma_buf *dmabuf, bool nonblock) +{ + if (!nonblock) + return dma_resv_lock_interruptible(dmabuf->resv, NULL); + + if (!dma_resv_trylock(dmabuf->resv)) + return -EBUSY; + + return 0; +} + +static struct dma_buf_attachment * +iio_buffer_find_attachment(struct iio_dev_buffer_pair *ib, + struct dma_buf *dmabuf, bool nonblock) +{ + struct device *dev = ib->indio_dev->dev.parent; + struct iio_buffer *buffer = ib->buffer; + struct dma_buf_attachment *attach = NULL; + struct iio_dmabuf_priv *priv; + + mutex_lock(&buffer->dmabufs_mutex); + + list_for_each_entry(priv, &buffer->dmabufs, entry) { + if (priv->attach->dev == dev + && priv->attach->dmabuf == dmabuf) { + attach = priv->attach; + break; + } + } + + if (attach) + iio_buffer_dmabuf_get(attach); + + mutex_unlock(&buffer->dmabufs_mutex); + + return attach ?: ERR_PTR(-EPERM); +} + +static int iio_buffer_attach_dmabuf(struct iio_dev_buffer_pair *ib, + int __user *user_fd, bool nonblock) +{ + struct iio_dev *indio_dev = ib->indio_dev; + struct iio_buffer *buffer = ib->buffer; + struct dma_buf_attachment *attach; + struct iio_dmabuf_priv *priv; + struct dma_buf *dmabuf; + int err, fd; + + if (!buffer->access->attach_dmabuf + || !buffer->access->detach_dmabuf + || !buffer->access->enqueue_dmabuf) + return -EPERM; + + if (copy_from_user(&fd, user_fd, sizeof(fd))) + return -EFAULT; + + priv = kzalloc(sizeof(*priv), GFP_KERNEL); + if (!priv) + return -ENOMEM; + + spin_lock_init(&priv->lock); + priv->context = dma_fence_context_alloc(1); + + dmabuf = dma_buf_get(fd); + if (IS_ERR(dmabuf)) { + err = PTR_ERR(dmabuf); + goto err_free_priv; + } + + attach = dma_buf_attach(dmabuf, indio_dev->dev.parent); + if (IS_ERR(attach)) { + err = PTR_ERR(attach); + goto err_dmabuf_put; + } + + err = iio_dma_resv_lock(dmabuf, nonblock); + if (err) + goto err_dmabuf_detach; + + priv->dir = buffer->direction == IIO_BUFFER_DIRECTION_IN + ? DMA_FROM_DEVICE : DMA_TO_DEVICE; + + priv->sgt = dma_buf_map_attachment(attach, priv->dir); + if (IS_ERR(priv->sgt)) { + err = PTR_ERR(priv->sgt); + dev_err(&indio_dev->dev, "Unable to map attachment: %d\n", err); + goto err_resv_unlock; + } + + kref_init(&priv->ref); + priv->buffer = buffer; + priv->attach = attach; + attach->importer_priv = priv; + + priv->block = buffer->access->attach_dmabuf(buffer, attach); + if (IS_ERR(priv->block)) { + err = PTR_ERR(priv->block); + goto err_dmabuf_unmap_attachment; + } + + dma_resv_unlock(dmabuf->resv); + + mutex_lock(&buffer->dmabufs_mutex); + list_add(&priv->entry, &buffer->dmabufs); + mutex_unlock(&buffer->dmabufs_mutex); + + return 0; + +err_dmabuf_unmap_attachment: + dma_buf_unmap_attachment(attach, priv->sgt, priv->dir); +err_resv_unlock: + dma_resv_unlock(dmabuf->resv); +err_dmabuf_detach: + dma_buf_detach(dmabuf, attach); +err_dmabuf_put: + dma_buf_put(dmabuf); +err_free_priv: + kfree(priv); + + return err; +} + +static int iio_buffer_detach_dmabuf(struct iio_dev_buffer_pair *ib, + int *user_req, bool nonblock) +{ + struct iio_buffer *buffer = ib->buffer; + struct iio_dev *indio_dev = ib->indio_dev; + struct iio_dmabuf_priv *priv; + struct dma_buf *dmabuf; + int dmabuf_fd, ret = -EPERM; + + if (copy_from_user(&dmabuf_fd, user_req, sizeof(dmabuf_fd))) + return -EFAULT; + + dmabuf = dma_buf_get(dmabuf_fd); + if (IS_ERR(dmabuf)) + return PTR_ERR(dmabuf); + + mutex_lock(&buffer->dmabufs_mutex); + + list_for_each_entry(priv, &buffer->dmabufs, entry) { + if (priv->attach->dev == indio_dev->dev.parent + && priv->attach->dmabuf == dmabuf) { + list_del(&priv->entry); + + /* Unref the reference from iio_buffer_attach_dmabuf() */ + iio_buffer_dmabuf_put(priv->attach); + ret = 0; + break; + } + } + + mutex_unlock(&buffer->dmabufs_mutex); + dma_buf_put(dmabuf); + + return ret; +} + +static const char * +iio_buffer_dma_fence_get_driver_name(struct dma_fence *fence) +{ + return "iio"; +} + +static void iio_buffer_dma_fence_release(struct dma_fence *fence) +{ + struct iio_dma_fence *iio_fence = + container_of(fence, struct iio_dma_fence, base); + + kfree(iio_fence); +} + +static const struct dma_fence_ops iio_buffer_dma_fence_ops = { + .get_driver_name = iio_buffer_dma_fence_get_driver_name, + .get_timeline_name = iio_buffer_dma_fence_get_driver_name, + .release = iio_buffer_dma_fence_release, +}; + +static int iio_buffer_enqueue_dmabuf(struct iio_dev_buffer_pair *ib, + struct iio_dmabuf __user *iio_dmabuf_req, + bool nonblock) +{ + struct iio_buffer *buffer = ib->buffer; + struct iio_dmabuf iio_dmabuf; + struct dma_buf_attachment *attach; + struct iio_dmabuf_priv *priv; + struct iio_dma_fence *fence; + struct dma_buf *dmabuf; + unsigned long timeout; + bool cookie, cyclic, dma_to_ram; + long retl; + u32 seqno; + int ret; + + if (copy_from_user(&iio_dmabuf, iio_dmabuf_req, sizeof(iio_dmabuf))) + return -EFAULT; + + if (iio_dmabuf.flags & ~IIO_BUFFER_DMABUF_SUPPORTED_FLAGS) + return -EINVAL; + + cyclic = iio_dmabuf.flags & IIO_BUFFER_DMABUF_CYCLIC; + + /* Cyclic flag is only supported on output buffers */ + if (cyclic && buffer->direction != IIO_BUFFER_DIRECTION_OUT) + return -EINVAL; + + dmabuf = dma_buf_get(iio_dmabuf.fd); + if (IS_ERR(dmabuf)) + return PTR_ERR(dmabuf); + + if (!iio_dmabuf.bytes_used || iio_dmabuf.bytes_used > dmabuf->size) { + ret = -EINVAL; + goto err_dmabuf_put; + } + + attach = iio_buffer_find_attachment(ib, dmabuf, nonblock); + if (IS_ERR(attach)) { + ret = PTR_ERR(attach); + goto err_dmabuf_put; + } + + priv = attach->importer_priv; + + fence = kmalloc(sizeof(*fence), GFP_KERNEL); + if (!fence) { + ret = -ENOMEM; + goto err_attachment_put; + } + + fence->priv = priv; + + seqno = atomic_add_return(1, &priv->seqno); + + /* + * The transfers are guaranteed to be processed in the order they are + * enqueued, so we can use a simple incrementing sequence number for + * the dma_fence. + */ + dma_fence_init(&fence->base, &iio_buffer_dma_fence_ops, + &priv->lock, priv->context, seqno); + + ret = iio_dma_resv_lock(dmabuf, nonblock); + if (ret) + goto err_fence_put; + + timeout = nonblock ? 0 : msecs_to_jiffies(DMABUF_ENQUEUE_TIMEOUT_MS); + + /* Make sure we don't have writers */ + retl = dma_resv_wait_timeout(dmabuf->resv, DMA_RESV_USAGE_WRITE, + true, timeout); + if (retl == 0) + retl = -EBUSY; + if (retl < 0) { + ret = (int)retl; + goto err_resv_unlock; + } + + dma_to_ram = buffer->direction == IIO_BUFFER_DIRECTION_IN; + + if (dma_to_ram) { + /* + * If we're writing to the DMABUF, make sure we don't have + * readers + */ + retl = dma_resv_wait_timeout(dmabuf->resv, + DMA_RESV_USAGE_READ, true, + timeout); + if (retl == 0) + retl = -EBUSY; + if (retl < 0) { + ret = (int)retl; + goto err_resv_unlock; + } + } + + if (buffer->access->lock_queue) + buffer->access->lock_queue(buffer); + + ret = dma_resv_reserve_fences(dmabuf->resv, 1); + if (ret) + goto err_queue_unlock; + + dma_resv_add_fence(dmabuf->resv, &fence->base, + dma_resv_usage_rw(dma_to_ram)); + dma_resv_unlock(dmabuf->resv); + + cookie = dma_fence_begin_signalling(); + + ret = buffer->access->enqueue_dmabuf(buffer, priv->block, &fence->base, + priv->sgt, iio_dmabuf.bytes_used, + cyclic); + if (ret) { + /* + * DMABUF enqueue failed, but we already added the fence. + * Signal the error through the fence completion mechanism. + */ + iio_buffer_signal_dmabuf_done(&fence->base, ret); + } + + if (buffer->access->unlock_queue) + buffer->access->unlock_queue(buffer); + + dma_fence_end_signalling(cookie); + dma_buf_put(dmabuf); + + return ret; + +err_queue_unlock: + if (buffer->access->unlock_queue) + buffer->access->unlock_queue(buffer); +err_resv_unlock: + dma_resv_unlock(dmabuf->resv); +err_fence_put: + dma_fence_put(&fence->base); +err_attachment_put: + iio_buffer_dmabuf_put(attach); +err_dmabuf_put: + dma_buf_put(dmabuf); + + return ret; +} + +static void iio_buffer_cleanup(struct work_struct *work) +{ + struct iio_dma_fence *fence = + container_of(work, struct iio_dma_fence, work); + struct iio_dmabuf_priv *priv = fence->priv; + struct dma_buf_attachment *attach = priv->attach; + + dma_fence_put(&fence->base); + iio_buffer_dmabuf_put(attach); +} + +void iio_buffer_signal_dmabuf_done(struct dma_fence *fence, int ret) +{ + struct iio_dma_fence *iio_fence = + container_of(fence, struct iio_dma_fence, base); + bool cookie = dma_fence_begin_signalling(); + + /* + * Get a reference to the fence, so that it's not freed as soon as + * it's signaled. + */ + dma_fence_get(fence); + + fence->error = ret; + dma_fence_signal(fence); + dma_fence_end_signalling(cookie); + + /* + * The fence will be unref'd in iio_buffer_cleanup. + * It can't be done here, as the unref functions might try to lock the + * resv object, which can deadlock. + */ + INIT_WORK(&iio_fence->work, iio_buffer_cleanup); + schedule_work(&iio_fence->work); +} +EXPORT_SYMBOL_GPL(iio_buffer_signal_dmabuf_done); + +static long iio_buffer_chrdev_ioctl(struct file *filp, + unsigned int cmd, unsigned long arg) +{ + struct iio_dev_buffer_pair *ib = filp->private_data; + void __user *_arg = (void __user *)arg; + bool nonblock = filp->f_flags & O_NONBLOCK; + + switch (cmd) { + case IIO_BUFFER_DMABUF_ATTACH_IOCTL: + return iio_buffer_attach_dmabuf(ib, _arg, nonblock); + case IIO_BUFFER_DMABUF_DETACH_IOCTL: + return iio_buffer_detach_dmabuf(ib, _arg, nonblock); + case IIO_BUFFER_DMABUF_ENQUEUE_IOCTL: + return iio_buffer_enqueue_dmabuf(ib, _arg, nonblock); + default: + return -EINVAL; + } +} + static const struct file_operations iio_buffer_chrdev_fileops = { .owner = THIS_MODULE, .llseek = noop_llseek, .read = iio_buffer_read, .write = iio_buffer_write, + .unlocked_ioctl = iio_buffer_chrdev_ioctl, + .compat_ioctl = compat_ptr_ioctl, .poll = iio_buffer_poll, .release = iio_buffer_chrdev_release, }; @@ -1953,6 +2414,7 @@ static void iio_buffer_release(struct kref *ref) { struct iio_buffer *buffer = container_of(ref, struct iio_buffer, ref); + mutex_destroy(&buffer->dmabufs_mutex); buffer->access->release(buffer); } diff --git a/include/linux/iio/buffer_impl.h b/include/linux/iio/buffer_impl.h index 89c3fd7c29ca..f4b1147291e5 100644 --- a/include/linux/iio/buffer_impl.h +++ b/include/linux/iio/buffer_impl.h @@ -9,8 +9,12 @@ #include #include +struct dma_buf_attachment; +struct dma_fence; struct iio_dev; +struct iio_dma_buffer_block; struct iio_buffer; +struct sg_table; /** * INDIO_BUFFER_FLAG_FIXED_WATERMARK - Watermark level of the buffer can not be @@ -39,6 +43,13 @@ struct iio_buffer; * device stops sampling. Calles are balanced with @enable. * @release: called when the last reference to the buffer is dropped, * should free all resources allocated by the buffer. + * @attach_dmabuf: called from userspace via ioctl to attach one external + * DMABUF. + * @detach_dmabuf: called from userspace via ioctl to detach one previously + * attached DMABUF. + * @enqueue_dmabuf: called from userspace via ioctl to queue this DMABUF + * object to this buffer. Requires a valid DMABUF fd, that + * was previouly attached to this buffer. * @modes: Supported operating modes by this buffer type * @flags: A bitmask combination of INDIO_BUFFER_FLAG_* * @@ -68,6 +79,17 @@ struct iio_buffer_access_funcs { void (*release)(struct iio_buffer *buffer); + struct iio_dma_buffer_block * (*attach_dmabuf)(struct iio_buffer *buffer, + struct dma_buf_attachment *attach); + void (*detach_dmabuf)(struct iio_buffer *buffer, + struct iio_dma_buffer_block *block); + int (*enqueue_dmabuf)(struct iio_buffer *buffer, + struct iio_dma_buffer_block *block, + struct dma_fence *fence, struct sg_table *sgt, + size_t size, bool cyclic); + void (*lock_queue)(struct iio_buffer *buffer); + void (*unlock_queue)(struct iio_buffer *buffer); + unsigned int modes; unsigned int flags; }; @@ -136,6 +158,12 @@ struct iio_buffer { /* @ref: Reference count of the buffer. */ struct kref ref; + + /* @dmabufs: List of DMABUF attachments */ + struct list_head dmabufs; /* P: dmabufs_mutex */ + + /* @dmabufs_mutex: Protects dmabufs */ + struct mutex dmabufs_mutex; }; /** @@ -156,9 +184,14 @@ int iio_update_buffers(struct iio_dev *indio_dev, **/ void iio_buffer_init(struct iio_buffer *buffer); +void iio_buffer_dmabuf_get(struct dma_buf_attachment *attach); +void iio_buffer_dmabuf_put(struct dma_buf_attachment *attach); + struct iio_buffer *iio_buffer_get(struct iio_buffer *buffer); void iio_buffer_put(struct iio_buffer *buffer); +void iio_buffer_signal_dmabuf_done(struct dma_fence *fence, int ret); + #else /* CONFIG_IIO_BUFFER */ static inline void iio_buffer_get(struct iio_buffer *buffer) {} diff --git a/include/uapi/linux/iio/buffer.h b/include/uapi/linux/iio/buffer.h index 13939032b3f6..c666aa95e532 100644 --- a/include/uapi/linux/iio/buffer.h +++ b/include/uapi/linux/iio/buffer.h @@ -5,6 +5,28 @@ #ifndef _UAPI_IIO_BUFFER_H_ #define _UAPI_IIO_BUFFER_H_ +#include + +/* Flags for iio_dmabuf.flags */ +#define IIO_BUFFER_DMABUF_CYCLIC (1 << 0) +#define IIO_BUFFER_DMABUF_SUPPORTED_FLAGS 0x00000001 + +/** + * struct iio_dmabuf - Descriptor for a single IIO DMABUF object + * @fd: file descriptor of the DMABUF object + * @flags: one or more IIO_BUFFER_DMABUF_* flags + * @bytes_used: number of bytes used in this DMABUF for the data transfer. + * Should generally be set to the DMABUF's size. + */ +struct iio_dmabuf { + __u32 fd; + __u32 flags; + __u64 bytes_used; +}; + #define IIO_BUFFER_GET_FD_IOCTL _IOWR('i', 0x91, int) +#define IIO_BUFFER_DMABUF_ATTACH_IOCTL _IOW('i', 0x92, int) +#define IIO_BUFFER_DMABUF_DETACH_IOCTL _IOW('i', 0x93, int) +#define IIO_BUFFER_DMABUF_ENQUEUE_IOCTL _IOW('i', 0x94, struct iio_dmabuf) #endif /* _UAPI_IIO_BUFFER_H_ */ From patchwork Mon Jan 29 17:01:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Cercueil X-Patchwork-Id: 193602 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp704874dyb; Mon, 29 Jan 2024 09:07:53 -0800 (PST) X-Google-Smtp-Source: AGHT+IGCtI9ufKOKWNBFldCl7O9dI/Hkq86JZYnj5K13S+A7LfGAaZ0I3CQDucwFnFLF/j4Z9Rbl X-Received: by 2002:a05:6a00:3d15:b0:6db:dd2d:b4e2 with SMTP id lo21-20020a056a003d1500b006dbdd2db4e2mr2832931pfb.8.1706548072770; Mon, 29 Jan 2024 09:07:52 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706548072; cv=pass; d=google.com; s=arc-20160816; b=oOtC7KWI9bjyOVd1OfOIS4OXdAPbFDYWRbxHz6kolwJ4p5gDgpo/fvQQcgI498Q7lj QOQtjOS41BK4Eq9miJUChOEuh1J2PSeU/ao5b/7koPjOXqmp5uHG6hp6cTTNJXdxRxSB 9b29TB4T/4enWzgi5FF8Z5M8QGxBPetmBu9nzvrz1e/2udtwRFiq8fkdo7MQkh298Qew gltlVESMd6sTGFHMB+9o9mS4KBrF6QGNwPnrfJEtRodposVNfNV91EunoVBLfoJUL9Pr ZvERfiW8rETvONZYiGL7Fzct4ZQNgY1WUUdg8YQYMvF1V5RNGd3pHGtsHOltwxAyiOk7 yy7w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=U+LY8goyLQKkbUw8Rbl+fN90hSYYz2kVkllDTHptXMg=; fh=UUvZraMiTlolibKjuaTYSz8c8N6g0y2n/C3Uf8DFNMA=; b=gfL1LT689ikGSiSWb6LZSTMKoyxyq48BW8z3KeqwWPdF6cds+SMYeKYxIQ89ixB1zq S8zJvtRGmkDB5U3qq/pkqE5zhAKvMnSUkFHOXy4Bkkihk+RRKu6F9F9ca+W9T19ZOhz6 yd6UCg3zILkPt2yA7L5m8HRs677JR9cNGlxfVKOhaHQJwMy1w63UMvTPLw9bZfVBpFmr nU4H4nRyZv7vQT89rKiyszbsxiUj8VlJN5g7cvQ6x9SuutCLcbnliYuHkW7TOi07u88o StL2qkuXCg1dbtwYDhj0G9vV3qziipxvaW2Uq/xazccRBZAAFnDhH9ZUBUxGGBqvs6rT mvAQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b="ta/1CnnS"; arc=pass (i=1 spf=pass spfdomain=crapouillou.net dkim=pass dkdomain=crapouillou.net dmarc=pass fromdomain=crapouillou.net); spf=pass (google.com: domain of linux-kernel+bounces-43177-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43177-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id t71-20020a63814a000000b005cee03a1bf9si5848510pgd.448.2024.01.29.09.07.52 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 09:07:52 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43177-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b="ta/1CnnS"; arc=pass (i=1 spf=pass spfdomain=crapouillou.net dkim=pass dkdomain=crapouillou.net dmarc=pass fromdomain=crapouillou.net); spf=pass (google.com: domain of linux-kernel+bounces-43177-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43177-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id E3BC228C848 for ; Mon, 29 Jan 2024 17:04:12 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 68D0415A4A9; Mon, 29 Jan 2024 17:02:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=crapouillou.net header.i=@crapouillou.net header.b="ta/1CnnS" Received: from aposti.net (aposti.net [89.234.176.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A107E155308; Mon, 29 Jan 2024 17:02:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=89.234.176.197 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706547772; cv=none; b=PvMKHiNrhEJeAlTZ8b1oPSKzYe8cSzgTTLo5STD3EN5xu7wQCRFt7/VjV8KKiG/r4gk3raNfkz5tpyULSNm27HxxtjpDszpkp0zQ02nHyvcaJ8EHeRVoXw+7gOH7w4Rax+DEFGM13hTGNmXm4mxeZgroo/e2ISXO0eIM00+ke5g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706547772; c=relaxed/simple; bh=trjw935VDwEIeOU0qg2uhxE1OiIa43EkLjDtCqLmxaE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SMQWV1lPkUuT3FWK5GqgFmApaIhaTPY+o1WqjyXCh3M7cmgo2SKfSLkoa6UenRRASR8NXpK4Mgfeo+FHob6OtwwClsjTnlL+tRDa0OopkunhMUojQEg4DY6JS3zTNZM7Xeth/T6k3H/dRDmBFZlC5PfSRVKPQsNM+FhW7FCY2cE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=crapouillou.net; spf=pass smtp.mailfrom=crapouillou.net; dkim=pass (1024-bit key) header.d=crapouillou.net header.i=@crapouillou.net header.b=ta/1CnnS; arc=none smtp.client-ip=89.234.176.197 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=crapouillou.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=crapouillou.net DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=crapouillou.net; s=mail; t=1706547739; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=U+LY8goyLQKkbUw8Rbl+fN90hSYYz2kVkllDTHptXMg=; b=ta/1CnnSm2Wl77kAN7mLMFTKRNmZOuFZ18vdn+mO82mqHDPQ4CAxL+juWPQMLQX1xosbcp u+ExyzCorCWhSLOKV0gX2ireza//M8JwLo4Zr3cOc+L6uAqoKWl3ceRGrxumsRWzxWfFAF Edc7OVslLSF4hxc9/GSaMPAsNp4rUio= From: Paul Cercueil To: Jonathan Cameron , Jonathan Corbet , Lars-Peter Clausen , Vinod Koul , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: Daniel Vetter , Nuno Sa , Michael Hennerich , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, dmaengine@vger.kernel.org, linux-iio@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, Paul Cercueil Subject: [PATCH v6 4/6] iio: buffer-dma: Enable support for DMABUFs Date: Mon, 29 Jan 2024 18:01:59 +0100 Message-ID: <20240129170201.133785-5-paul@crapouillou.net> In-Reply-To: <20240129170201.133785-1-paul@crapouillou.net> References: <20240129170201.133785-1-paul@crapouillou.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789445352092662723 X-GMAIL-MSGID: 1789445352092662723 Implement iio_dma_buffer_attach_dmabuf(), iio_dma_buffer_detach_dmabuf() and iio_dma_buffer_transfer_dmabuf(), which can then be used by the IIO DMA buffer implementations. Signed-off-by: Paul Cercueil --- v3: Update code to provide the functions that will be used as callbacks for the new IOCTLs. v6: - Update iio_dma_buffer_enqueue_dmabuf() to take a dma_fence pointer - Pass that dma_fence pointer along to iio_buffer_signal_dmabuf_done() - Add iio_dma_buffer_lock_queue() / iio_dma_buffer_unlock_queue() - Do not lock the queue in iio_dma_buffer_enqueue_dmabuf(). The caller will ensure that it has been locked already. - Replace "int += bool;" by "if (bool) int++;" - Use dma_fence_begin/end_signalling in the dma_fence critical sections - Use one "num_dmabufs" fields instead of one "num_blocks" and one "num_fileio_blocks". Make it an atomic_t, which makes it possible to decrement it atomically in iio_buffer_block_release() without having to lock the queue mutex; and in turn, it means that we don't need to use iio_buffer_block_put_atomic() everywhere to avoid locking the queue mutex twice. - Use cleanup.h guard(mutex) when possible - Explicitely list all states in the switch in iio_dma_can_enqueue_block() - Rename iio_dma_buffer_fileio_mode() to iio_dma_buffer_can_use_fileio(), and add a comment explaining why it cannot race vs. DMABUF. --- drivers/iio/buffer/industrialio-buffer-dma.c | 181 +++++++++++++++++-- include/linux/iio/buffer-dma.h | 31 ++++ 2 files changed, 201 insertions(+), 11 deletions(-) diff --git a/drivers/iio/buffer/industrialio-buffer-dma.c b/drivers/iio/buffer/industrialio-buffer-dma.c index 5610ba67925e..c0f539af98f9 100644 --- a/drivers/iio/buffer/industrialio-buffer-dma.c +++ b/drivers/iio/buffer/industrialio-buffer-dma.c @@ -4,6 +4,8 @@ * Author: Lars-Peter Clausen */ +#include +#include #include #include #include @@ -14,6 +16,8 @@ #include #include #include +#include +#include #include #include @@ -94,13 +98,18 @@ static void iio_buffer_block_release(struct kref *kref) { struct iio_dma_buffer_block *block = container_of(kref, struct iio_dma_buffer_block, kref); + struct iio_dma_buffer_queue *queue = block->queue; - WARN_ON(block->state != IIO_BLOCK_STATE_DEAD); + WARN_ON(block->fileio && block->state != IIO_BLOCK_STATE_DEAD); - dma_free_coherent(block->queue->dev, PAGE_ALIGN(block->size), - block->vaddr, block->phys_addr); + if (block->fileio) { + dma_free_coherent(queue->dev, PAGE_ALIGN(block->size), + block->vaddr, block->phys_addr); + } else { + atomic_dec(&queue->num_dmabufs); + } - iio_buffer_put(&block->queue->buffer); + iio_buffer_put(&queue->buffer); kfree(block); } @@ -163,7 +172,7 @@ static struct iio_dma_buffer_queue *iio_buffer_to_queue(struct iio_buffer *buf) } static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block( - struct iio_dma_buffer_queue *queue, size_t size) + struct iio_dma_buffer_queue *queue, size_t size, bool fileio) { struct iio_dma_buffer_block *block; @@ -171,13 +180,16 @@ static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block( if (!block) return NULL; - block->vaddr = dma_alloc_coherent(queue->dev, PAGE_ALIGN(size), - &block->phys_addr, GFP_KERNEL); - if (!block->vaddr) { - kfree(block); - return NULL; + if (fileio) { + block->vaddr = dma_alloc_coherent(queue->dev, PAGE_ALIGN(size), + &block->phys_addr, GFP_KERNEL); + if (!block->vaddr) { + kfree(block); + return NULL; + } } + block->fileio = fileio; block->size = size; block->state = IIO_BLOCK_STATE_DONE; block->queue = queue; @@ -186,6 +198,9 @@ static struct iio_dma_buffer_block *iio_dma_buffer_alloc_block( iio_buffer_get(&queue->buffer); + if (!fileio) + atomic_inc(&queue->num_dmabufs); + return block; } @@ -206,13 +221,21 @@ void iio_dma_buffer_block_done(struct iio_dma_buffer_block *block) { struct iio_dma_buffer_queue *queue = block->queue; unsigned long flags; + bool cookie; + + cookie = dma_fence_begin_signalling(); spin_lock_irqsave(&queue->list_lock, flags); _iio_dma_buffer_block_done(block); spin_unlock_irqrestore(&queue->list_lock, flags); + if (!block->fileio) + iio_buffer_signal_dmabuf_done(block->fence, 0); + iio_buffer_block_put_atomic(block); wake_up_interruptible_poll(&queue->buffer.pollq, EPOLLIN | EPOLLRDNORM); + + dma_fence_end_signalling(cookie); } EXPORT_SYMBOL_GPL(iio_dma_buffer_block_done); @@ -231,17 +254,27 @@ void iio_dma_buffer_block_list_abort(struct iio_dma_buffer_queue *queue, { struct iio_dma_buffer_block *block, *_block; unsigned long flags; + bool cookie; + + cookie = dma_fence_begin_signalling(); spin_lock_irqsave(&queue->list_lock, flags); list_for_each_entry_safe(block, _block, list, head) { list_del(&block->head); block->bytes_used = 0; _iio_dma_buffer_block_done(block); + + if (!block->fileio) + iio_buffer_signal_dmabuf_done(block->fence, -EINTR); iio_buffer_block_put_atomic(block); } spin_unlock_irqrestore(&queue->list_lock, flags); + if (queue->fileio.enabled) + queue->fileio.enabled = false; + wake_up_interruptible_poll(&queue->buffer.pollq, EPOLLIN | EPOLLRDNORM); + dma_fence_end_signalling(cookie); } EXPORT_SYMBOL_GPL(iio_dma_buffer_block_list_abort); @@ -261,6 +294,16 @@ static bool iio_dma_block_reusable(struct iio_dma_buffer_block *block) } } +static bool iio_dma_buffer_can_use_fileio(struct iio_dma_buffer_queue *queue) +{ + /* + * Note that queue->num_dmabufs cannot increase while the queue is + * locked, it can only decrease, so it does not race against + * iio_dma_buffer_alloc_block(). + */ + return queue->fileio.enabled || !atomic_read(&queue->num_dmabufs); +} + /** * iio_dma_buffer_request_update() - DMA buffer request_update callback * @buffer: The buffer which to request an update @@ -287,6 +330,12 @@ int iio_dma_buffer_request_update(struct iio_buffer *buffer) mutex_lock(&queue->lock); + queue->fileio.enabled = iio_dma_buffer_can_use_fileio(queue); + + /* If DMABUFs were created, disable fileio interface */ + if (!queue->fileio.enabled) + goto out_unlock; + /* Allocations are page aligned */ if (PAGE_ALIGN(queue->fileio.block_size) == PAGE_ALIGN(size)) try_reuse = true; @@ -327,7 +376,7 @@ int iio_dma_buffer_request_update(struct iio_buffer *buffer) } if (!block) { - block = iio_dma_buffer_alloc_block(queue, size); + block = iio_dma_buffer_alloc_block(queue, size, true); if (!block) { ret = -ENOMEM; goto out_unlock; @@ -384,8 +433,12 @@ static void iio_dma_buffer_submit_block(struct iio_dma_buffer_queue *queue, block->state = IIO_BLOCK_STATE_ACTIVE; iio_buffer_block_get(block); + ret = queue->ops->submit(queue, block); if (ret) { + if (!block->fileio) + iio_buffer_signal_dmabuf_done(block->fence, ret); + /* * This is a bit of a problem and there is not much we can do * other then wait for the buffer to be disabled and re-enabled @@ -588,6 +641,112 @@ size_t iio_dma_buffer_data_available(struct iio_buffer *buf) } EXPORT_SYMBOL_GPL(iio_dma_buffer_data_available); +struct iio_dma_buffer_block * +iio_dma_buffer_attach_dmabuf(struct iio_buffer *buffer, + struct dma_buf_attachment *attach) +{ + struct iio_dma_buffer_queue *queue = iio_buffer_to_queue(buffer); + struct iio_dma_buffer_block *block; + + guard(mutex)(&queue->lock); + + /* + * If the buffer is enabled and in fileio mode new blocks can't be + * allocated. + */ + if (queue->fileio.enabled) + return ERR_PTR(-EBUSY); + + block = iio_dma_buffer_alloc_block(queue, attach->dmabuf->size, false); + if (!block) + return ERR_PTR(-ENOMEM); + + block->attach = attach; + + /* Free memory that might be in use for fileio mode */ + iio_dma_buffer_fileio_free(queue); + + return block; +} +EXPORT_SYMBOL_GPL(iio_dma_buffer_attach_dmabuf); + +void iio_dma_buffer_detach_dmabuf(struct iio_buffer *buffer, + struct iio_dma_buffer_block *block) +{ + block->state = IIO_BLOCK_STATE_DEAD; + iio_buffer_block_put_atomic(block); +} +EXPORT_SYMBOL_GPL(iio_dma_buffer_detach_dmabuf); + +static int iio_dma_can_enqueue_block(struct iio_dma_buffer_block *block) +{ + struct iio_dma_buffer_queue *queue = block->queue; + + /* If in fileio mode buffers can't be enqueued. */ + if (queue->fileio.enabled) + return -EBUSY; + + switch (block->state) { + case IIO_BLOCK_STATE_QUEUED: + return -EPERM; + case IIO_BLOCK_STATE_ACTIVE: + case IIO_BLOCK_STATE_DEAD: + return -EBUSY; + case IIO_BLOCK_STATE_DONE: + break; + } + + return 0; +} + +int iio_dma_buffer_enqueue_dmabuf(struct iio_buffer *buffer, + struct iio_dma_buffer_block *block, + struct dma_fence *fence, + struct sg_table *sgt, + size_t size, bool cyclic) +{ + struct iio_dma_buffer_queue *queue = iio_buffer_to_queue(buffer); + bool cookie; + int ret; + + WARN_ON(!mutex_is_locked(&queue->lock)); + + cookie = dma_fence_begin_signalling(); + + ret = iio_dma_can_enqueue_block(block); + if (ret < 0) + goto out_end_signalling; + + block->bytes_used = size; + block->cyclic = cyclic; + block->sg_table = sgt; + block->fence = fence; + + iio_dma_buffer_enqueue(queue, block); + +out_end_signalling: + dma_fence_end_signalling(cookie); + + return ret; +} +EXPORT_SYMBOL_GPL(iio_dma_buffer_enqueue_dmabuf); + +void iio_dma_buffer_lock_queue(struct iio_buffer *buffer) +{ + struct iio_dma_buffer_queue *queue = iio_buffer_to_queue(buffer); + + mutex_lock(&queue->lock); +} +EXPORT_SYMBOL_GPL(iio_dma_buffer_lock_queue); + +void iio_dma_buffer_unlock_queue(struct iio_buffer *buffer) +{ + struct iio_dma_buffer_queue *queue = iio_buffer_to_queue(buffer); + + mutex_unlock(&queue->lock); +} +EXPORT_SYMBOL_GPL(iio_dma_buffer_unlock_queue); + /** * iio_dma_buffer_set_bytes_per_datum() - DMA buffer set_bytes_per_datum callback * @buffer: Buffer to set the bytes-per-datum for diff --git a/include/linux/iio/buffer-dma.h b/include/linux/iio/buffer-dma.h index 18d3702fa95d..b62ff2a3f554 100644 --- a/include/linux/iio/buffer-dma.h +++ b/include/linux/iio/buffer-dma.h @@ -7,6 +7,7 @@ #ifndef __INDUSTRIALIO_DMA_BUFFER_H__ #define __INDUSTRIALIO_DMA_BUFFER_H__ +#include #include #include #include @@ -16,6 +17,9 @@ struct iio_dma_buffer_queue; struct iio_dma_buffer_ops; struct device; +struct dma_buf_attachment; +struct dma_fence; +struct sg_table; /** * enum iio_block_state - State of a struct iio_dma_buffer_block @@ -41,6 +45,8 @@ enum iio_block_state { * @queue: Parent DMA buffer queue * @kref: kref used to manage the lifetime of block * @state: Current state of the block + * @cyclic: True if this is a cyclic buffer + * @fileio: True if this buffer is used for fileio mode */ struct iio_dma_buffer_block { /* May only be accessed by the owner of the block */ @@ -63,6 +69,14 @@ struct iio_dma_buffer_block { * queue->list_lock if the block is not owned by the core. */ enum iio_block_state state; + + bool cyclic; + bool fileio; + + struct dma_buf_attachment *attach; + struct sg_table *sg_table; + + struct dma_fence *fence; }; /** @@ -72,6 +86,7 @@ struct iio_dma_buffer_block { * @pos: Read offset in the active block * @block_size: Size of each block * @next_dequeue: index of next block that will be dequeued + * @enabled: Whether the buffer is operating in fileio mode */ struct iio_dma_buffer_queue_fileio { struct iio_dma_buffer_block *blocks[2]; @@ -80,6 +95,7 @@ struct iio_dma_buffer_queue_fileio { size_t block_size; unsigned int next_dequeue; + bool enabled; }; /** @@ -95,6 +111,7 @@ struct iio_dma_buffer_queue_fileio { * the DMA controller * @incoming: List of buffers on the incoming queue * @active: Whether the buffer is currently active + * @num_dmabufs: Total number of DMABUFs attached to this queue * @fileio: FileIO state */ struct iio_dma_buffer_queue { @@ -107,6 +124,7 @@ struct iio_dma_buffer_queue { struct list_head incoming; bool active; + atomic_t num_dmabufs; struct iio_dma_buffer_queue_fileio fileio; }; @@ -142,4 +160,17 @@ int iio_dma_buffer_init(struct iio_dma_buffer_queue *queue, void iio_dma_buffer_exit(struct iio_dma_buffer_queue *queue); void iio_dma_buffer_release(struct iio_dma_buffer_queue *queue); +struct iio_dma_buffer_block * +iio_dma_buffer_attach_dmabuf(struct iio_buffer *buffer, + struct dma_buf_attachment *attach); +void iio_dma_buffer_detach_dmabuf(struct iio_buffer *buffer, + struct iio_dma_buffer_block *block); +int iio_dma_buffer_enqueue_dmabuf(struct iio_buffer *buffer, + struct iio_dma_buffer_block *block, + struct dma_fence *fence, + struct sg_table *sgt, + size_t size, bool cyclic); +void iio_dma_buffer_lock_queue(struct iio_buffer *buffer); +void iio_dma_buffer_unlock_queue(struct iio_buffer *buffer); + #endif From patchwork Mon Jan 29 17:02:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Cercueil X-Patchwork-Id: 193603 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp705080dyb; Mon, 29 Jan 2024 09:08:08 -0800 (PST) X-Google-Smtp-Source: AGHT+IHfEjszZSueLbWvzVlICABciwC4MMnpWvyuQExJOL5UB3nYEf5nHrP35NW5qZOxKsrM5V0Z X-Received: by 2002:a05:6a00:b90:b0:6da:e7c2:42ca with SMTP id g16-20020a056a000b9000b006dae7c242camr5073542pfj.21.1706548088715; Mon, 29 Jan 2024 09:08:08 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706548088; cv=pass; d=google.com; s=arc-20160816; b=R7n0GEFx/BfW+Ieo83SNjvpo/PScBHZesA+n9gUJHFIjGm0rzeJj/XjwT7KkqBI+EH CuwCditMx04eRtIIyw0dCeYroMrmT7/B+b5edOP2ZerPHhvX2X+1IlNdQK81Go6yb6si bNzg2OMc/FAHKKW+LGUBGVnmIoFWXEguZv8LepQ0btSXq2nAMCPj3f4L8YddLNgNi5/A GMsmxd4KZi/HrnsFgIQbpB9uhEaBaWuYDTfcaYRRgKlvxTaBDEJ5E6Tm38OxQuQVbPLX F+R20c+hrupDrUtX9K3NbDviVGwaXbiKKQJVyMfueLOGk1Cd6G4eaJonKCXxdVmeBKk+ BaXg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=/0YZL7D5wfQtXZEBgkttnzz1qHadNtN8B09YGKH9qPc=; fh=UUvZraMiTlolibKjuaTYSz8c8N6g0y2n/C3Uf8DFNMA=; b=cpixanix5Elj3OJpJw5Ug6o82XpX9/ccda9EFk/FjMh5+q8GwHxt3Q/QcRWG4I3aQC ujJAIY6sLo8KzQN3GSnpESCsWCA5CZpnk2wQBvor8g/hg1u0dSDPRaCjgRnuH1n1eFqN DzMDfQKEdXv9Y4ptFkR+rUNbAVJ9fL0yO0zgScXn4ksV7Xxm7QUYQbLkb8VXMlqKrNh1 8t7j7sFtsXiEcKZvpugtA7SkNxOpm9Hf93BB8zov6ZyiiyX0GjYpL7nE+rVBwbaSiK6L dc0Gri8UXJrCuvl+aCgOoWLyYAexswp6MwF1A/WqXfOl5RxN97UjD+jAVltQWdv6SWrz kzVQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b=4QQrUwGx; arc=pass (i=1 spf=pass spfdomain=crapouillou.net dkim=pass dkdomain=crapouillou.net dmarc=pass fromdomain=crapouillou.net); spf=pass (google.com: domain of linux-kernel+bounces-43178-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43178-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id t39-20020a056a0013a700b006dde33be7f6si6062195pfg.218.2024.01.29.09.08.08 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 09:08:08 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43178-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b=4QQrUwGx; arc=pass (i=1 spf=pass spfdomain=crapouillou.net dkim=pass dkdomain=crapouillou.net dmarc=pass fromdomain=crapouillou.net); spf=pass (google.com: domain of linux-kernel+bounces-43178-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43178-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id F257B28C7E4 for ; Mon, 29 Jan 2024 17:04:30 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 27D22157041; Mon, 29 Jan 2024 17:03:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=crapouillou.net header.i=@crapouillou.net header.b="4QQrUwGx" Received: from aposti.net (aposti.net [89.234.176.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C053915A4AC; Mon, 29 Jan 2024 17:02:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=89.234.176.197 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706547781; cv=none; b=OJLz/O9puoXGg7YYcbciu88XW/LagzY7WdsTdZxEDu3gVi1F2g30iSmAeFKpEOlVNxrstl0KWbZ4dbGSK43RcCBpz9h80mzuwEpLPXAucVId52Ewt2v/W7AEfw/UqX2gAIDSRnRJNIV+3WCUWzZSNYz+EF63WRF/GVVlHUmZNns= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706547781; c=relaxed/simple; bh=UmEplxjcutZ/aaDgkwmjqVZy3IpHxf4/5tz4vN9kxyA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GJx12hkzffpgXFYaPxoTJqYbwbQgeQ+0KLD1uv0Kxe0jyGD3lPIizNXlzEugmHR1YkjEzI5/9xO0V7dID0shpvdgFFFqt9Tk0q27JnDTK/4lUqi9gAIsPUHYqI21PLE0Oys3jDFVOBk7/ts29sQWrbHXfHtGqCvLk0NLcpDPNc8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=crapouillou.net; spf=pass smtp.mailfrom=crapouillou.net; dkim=pass (1024-bit key) header.d=crapouillou.net header.i=@crapouillou.net header.b=4QQrUwGx; arc=none smtp.client-ip=89.234.176.197 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=crapouillou.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=crapouillou.net DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=crapouillou.net; s=mail; t=1706547740; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/0YZL7D5wfQtXZEBgkttnzz1qHadNtN8B09YGKH9qPc=; b=4QQrUwGxB/krxFoltCT9bjAvaHymTQh0LMEK3wG0UcOru+QYEW0LnHIiMi5pBKBFnrPdWD QUXwFZX7/nCdx6A1J4S6+m5Cvl1SB41GLrhxMaULwjkcl0/1VTEDcc/Oa7jEhZRQLSRfqq LfKqEqppAygtDNmFgpsKFnIn/5Epu+I= From: Paul Cercueil To: Jonathan Cameron , Jonathan Corbet , Lars-Peter Clausen , Vinod Koul , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: Daniel Vetter , Nuno Sa , Michael Hennerich , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, dmaengine@vger.kernel.org, linux-iio@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, Paul Cercueil Subject: [PATCH v6 5/6] iio: buffer-dmaengine: Support new DMABUF based userspace API Date: Mon, 29 Jan 2024 18:02:00 +0100 Message-ID: <20240129170201.133785-6-paul@crapouillou.net> In-Reply-To: <20240129170201.133785-1-paul@crapouillou.net> References: <20240129170201.133785-1-paul@crapouillou.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789445368738619205 X-GMAIL-MSGID: 1789445368738619205 Use the functions provided by the buffer-dma core to implement the DMABUF userspace API in the buffer-dmaengine IIO buffer implementation. Since we want to be able to transfer an arbitrary number of bytes and not necesarily the full DMABUF, the associated scatterlist is converted to an array of DMA addresses + lengths, which is then passed to dmaengine_prep_slave_dma_array(). Signed-off-by: Paul Cercueil --- v3: Use the new dmaengine_prep_slave_dma_array(), and adapt the code to work with the new functions introduced in industrialio-buffer-dma.c. v5: - Use the new dmaengine_prep_slave_dma_vec(). - Restrict to input buffers, since output buffers are not yet supported by IIO buffers. v6: - Populate .lock_queue / .unlock_queue callbacks - Switch to atomic memory allocations in .submit_queue, because of the dma_fence critical section - Make sure that the size of the scatterlist is enough --- .../buffer/industrialio-buffer-dmaengine.c | 58 +++++++++++++++++-- 1 file changed, 52 insertions(+), 6 deletions(-) diff --git a/drivers/iio/buffer/industrialio-buffer-dmaengine.c b/drivers/iio/buffer/industrialio-buffer-dmaengine.c index 45fe7d0d42ee..c4cfdb0c1231 100644 --- a/drivers/iio/buffer/industrialio-buffer-dmaengine.c +++ b/drivers/iio/buffer/industrialio-buffer-dmaengine.c @@ -64,15 +64,54 @@ static int iio_dmaengine_buffer_submit_block(struct iio_dma_buffer_queue *queue, struct dmaengine_buffer *dmaengine_buffer = iio_buffer_to_dmaengine_buffer(&queue->buffer); struct dma_async_tx_descriptor *desc; + struct scatterlist *sgl; + struct dma_vec *vecs; dma_cookie_t cookie; + size_t len_total; + size_t max_size; + unsigned int i; + int nents; - block->bytes_used = min(block->size, dmaengine_buffer->max_size); - block->bytes_used = round_down(block->bytes_used, - dmaengine_buffer->align); + if (queue->buffer.direction != IIO_BUFFER_DIRECTION_IN) { + /* We do not yet support output buffers. */ + return -EINVAL; + } - desc = dmaengine_prep_slave_single(dmaengine_buffer->chan, - block->phys_addr, block->bytes_used, DMA_DEV_TO_MEM, - DMA_PREP_INTERRUPT); + if (block->sg_table) { + sgl = block->sg_table->sgl; + nents = sg_nents_for_len(sgl, block->bytes_used); + if (nents < 0) + return nents; + + vecs = kmalloc_array(nents, sizeof(*vecs), GFP_ATOMIC); + if (!vecs) + return -ENOMEM; + + len_total = block->bytes_used; + + for (i = 0; i < nents; i++) { + vecs[i].addr = sg_dma_address(sgl); + vecs[i].len = min(sg_dma_len(sgl), len_total); + len_total -= vecs[i].len; + + sgl = sg_next(sgl); + } + + desc = dmaengine_prep_slave_dma_vec(dmaengine_buffer->chan, + vecs, nents, DMA_DEV_TO_MEM, + DMA_PREP_INTERRUPT); + kfree(vecs); + } else { + max_size = min(block->size, dmaengine_buffer->max_size); + max_size = round_down(max_size, dmaengine_buffer->align); + block->bytes_used = max_size; + + desc = dmaengine_prep_slave_single(dmaengine_buffer->chan, + block->phys_addr, + block->bytes_used, + DMA_DEV_TO_MEM, + DMA_PREP_INTERRUPT); + } if (!desc) return -ENOMEM; @@ -120,6 +159,13 @@ static const struct iio_buffer_access_funcs iio_dmaengine_buffer_ops = { .data_available = iio_dma_buffer_data_available, .release = iio_dmaengine_buffer_release, + .enqueue_dmabuf = iio_dma_buffer_enqueue_dmabuf, + .attach_dmabuf = iio_dma_buffer_attach_dmabuf, + .detach_dmabuf = iio_dma_buffer_detach_dmabuf, + + .lock_queue = iio_dma_buffer_lock_queue, + .unlock_queue = iio_dma_buffer_unlock_queue, + .modes = INDIO_BUFFER_HARDWARE, .flags = INDIO_BUFFER_FLAG_FIXED_WATERMARK, }; From patchwork Mon Jan 29 17:02:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Cercueil X-Patchwork-Id: 193600 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp702527dyb; Mon, 29 Jan 2024 09:04:55 -0800 (PST) X-Google-Smtp-Source: AGHT+IH6aFNL9N+Em8sWrFdmVba3Jc8wrovTspqviq1JTUGW/3JVuw7KS0UsDcONPg87MimJO27o X-Received: by 2002:a05:622a:24e:b0:42a:41f2:3ae4 with SMTP id c14-20020a05622a024e00b0042a41f23ae4mr6826175qtx.116.1706547895140; Mon, 29 Jan 2024 09:04:55 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706547895; cv=pass; d=google.com; s=arc-20160816; b=aF/9X4hDQAXop+Xq1BWxb1PnqXXeKx6kVtSNKCoUEE4dIySabuStz5ghNs6D97s11O 12PMKVmsWl6PVIfRh51+5C+M8hcfA4+3H1uy89nGAjVsYjclS76XjfI9ogSKANS6+b6R Uru8WwX30TPQJMv9GGu+hMqJtfadoKRpq95M67eeivzjPw8NStuk36xiyvdGzMAHu9ua RNdMoaTc4kRAyepZg8mDjD4803R0qnUZqBDP+eXMtzzGzV6pX/TSjrLWMGYilKFFQOyR 2FNAUwvMM1P84GoF1NHNLTqKKHonX4gzLADp+pZCPj/acwHCE28qZeIeRQRPJmZ2r/Bd ql4g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=6SosLHUfbvBusyQrjhEdH6qlujZTce4ql106/+fLWzw=; fh=UUvZraMiTlolibKjuaTYSz8c8N6g0y2n/C3Uf8DFNMA=; b=p9Do3Rmyl9wkpAa+9L6ihkXhICF9EuqDdFM8yv1qfOdODfsQ6ZUME70HUZRj5A68Fu FZNFLS1gzcwtr+mNzgc+6Z6HjWVCIcXp4/WoSp2n5SAKZk9kGyFru1oxZ5Id+WvN2YYm 9w6SkIvPx30lhJy6fVueFMyh+yl4dDUdkjlzpL3N7/cxZb1/M8KfGOw3Wj5VFVJCRCpW yZmb8qPxxwlJ0vDaZf+dM8D1FXU4Cgi0P+OzqCLbxMXFZX9pWQrM4Ymqnx6ta5aOblgd MKEmInKq8oZ804lNsxBFf7VZwv9yeAO6P0WbnCtL0Q7Y3cDD56O4d68wv+NG74L+A6vW Ah8w== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b=Hz5iSkzr; arc=pass (i=1 spf=pass spfdomain=crapouillou.net dkim=pass dkdomain=crapouillou.net dmarc=pass fromdomain=crapouillou.net); spf=pass (google.com: domain of linux-kernel+bounces-43179-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43179-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id w6-20020a05622a190600b0042a67eca0f8si8009114qtc.169.2024.01.29.09.04.55 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 09:04:55 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43179-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b=Hz5iSkzr; arc=pass (i=1 spf=pass spfdomain=crapouillou.net dkim=pass dkdomain=crapouillou.net dmarc=pass fromdomain=crapouillou.net); spf=pass (google.com: domain of linux-kernel+bounces-43179-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43179-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id D1CDA1C23782 for ; Mon, 29 Jan 2024 17:04:48 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5B9A315703F; Mon, 29 Jan 2024 17:03:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=crapouillou.net header.i=@crapouillou.net header.b="Hz5iSkzr" Received: from aposti.net (aposti.net [89.234.176.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE05E15AAAF; Mon, 29 Jan 2024 17:03:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=89.234.176.197 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706547789; cv=none; b=orjqPGhqEh/2lVXlx1AfGXGF2iklxejE25phoBnmskr9snv6i3rXvTJCwjNX5saVBK87Zpgale5/dKAlzge6GJO7xFcPzbBgd6s17gnnlKygceC9DEAjjJZQcyZrMY3/BZqoSkdWflXj4a4dvMAyHUT9VJSLW3DNqDNXWWQpCMI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706547789; c=relaxed/simple; bh=wHWg4LZRwCfhResqeH0XOYfyJt+0f2JWYpJlo3R3dtA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Rp2Onbvusjs830PS7Va0/UJ9zo+zfjboyQVxb49f5ebChjPapgTpAQ8zb8uSa8yhBTYfHVfkJRN9Ck5OD0GZljRq5k7NMXYor2+bxyC3Q/OqWPiNo/aqO7KJTQQEUtSFazTiJwThK7DOvA8FhwS/LKA0eY3WkqZ+LsI6NjhO30g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=crapouillou.net; spf=pass smtp.mailfrom=crapouillou.net; dkim=pass (1024-bit key) header.d=crapouillou.net header.i=@crapouillou.net header.b=Hz5iSkzr; arc=none smtp.client-ip=89.234.176.197 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=crapouillou.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=crapouillou.net DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=crapouillou.net; s=mail; t=1706547741; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6SosLHUfbvBusyQrjhEdH6qlujZTce4ql106/+fLWzw=; b=Hz5iSkzreG374dszhrBcWBdOw/8E9kBUgkLUyYVVOzM0dIA3qjeGkDueAhT1yesWaD3R/J CU0Dav7pa+LPws6gK3o25zLvdwVZOMMblQ+ZLMEMbNg+y0SSabHObT9U9BAeFIJSWcS+Ht 4Mel07g7k6sXDOVHXLU9KIMPE1H9a9U= From: Paul Cercueil To: Jonathan Cameron , Jonathan Corbet , Lars-Peter Clausen , Vinod Koul , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= Cc: Daniel Vetter , Nuno Sa , Michael Hennerich , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, dmaengine@vger.kernel.org, linux-iio@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, Paul Cercueil Subject: [PATCH v6 6/6] Documentation: iio: Document high-speed DMABUF based API Date: Mon, 29 Jan 2024 18:02:01 +0100 Message-ID: <20240129170201.133785-7-paul@crapouillou.net> In-Reply-To: <20240129170201.133785-1-paul@crapouillou.net> References: <20240129170201.133785-1-paul@crapouillou.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789445165484882190 X-GMAIL-MSGID: 1789445165484882190 Document the new DMABUF based API. Signed-off-by: Paul Cercueil --- v2: - Explicitly state that the new interface is optional and is not implemented by all drivers. - The IOCTLs can now only be called on the buffer FD returned by IIO_BUFFER_GET_FD_IOCTL. - Move the page up a bit in the index since it is core stuff and not driver-specific. v3: Update the documentation to reflect the new API. v5: Use description lists for the documentation of the three new IOCTLs instead of abusing subsections. --- Documentation/iio/dmabuf_api.rst | 54 ++++++++++++++++++++++++++++++++ Documentation/iio/index.rst | 2 ++ 2 files changed, 56 insertions(+) create mode 100644 Documentation/iio/dmabuf_api.rst diff --git a/Documentation/iio/dmabuf_api.rst b/Documentation/iio/dmabuf_api.rst new file mode 100644 index 000000000000..1cd6cd51a582 --- /dev/null +++ b/Documentation/iio/dmabuf_api.rst @@ -0,0 +1,54 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=================================== +High-speed DMABUF interface for IIO +=================================== + +1. Overview +=========== + +The Industrial I/O subsystem supports access to buffers through a +file-based interface, with read() and write() access calls through the +IIO device's dev node. + +It additionally supports a DMABUF based interface, where the userspace +can attach DMABUF objects (externally created) to a IIO buffer, and +subsequently use them for data transfers. + +A userspace application can then use this interface to share DMABUF +objects between several interfaces, allowing it to transfer data in a +zero-copy fashion, for instance between IIO and the USB stack. + +The userspace application can also memory-map the DMABUF objects, and +access the sample data directly. The advantage of doing this vs. the +read() interface is that it avoids an extra copy of the data between the +kernel and userspace. This is particularly useful for high-speed devices +which produce several megabytes or even gigabytes of data per second. +It does however increase the userspace-kernelspace synchronization +overhead, as the DMA_BUF_SYNC_START and DMA_BUF_SYNC_END IOCTLs have to +be used for data integrity. + +2. User API +=========== + +As part of this interface, three new IOCTLs have been added. These three +IOCTLs have to be performed on the IIO buffer's file descriptor, +obtained using the IIO_BUFFER_GET_FD_IOCTL() ioctl. + + ``IIO_BUFFER_DMABUF_ATTACH_IOCTL(int)`` + Attach the DMABUF object, identified by its file descriptor, to the + IIO buffer. Returns zero on success, and a negative errno value on + error. + + ``IIO_BUFFER_DMABUF_DETACH_IOCTL(int)`` + Detach the given DMABUF object, identified by its file descriptor, + from the IIO buffer. Returns zero on success, and a negative errno + value on error. + + Note that closing the IIO buffer's file descriptor will + automatically detach all previously attached DMABUF objects. + + ``IIO_BUFFER_DMABUF_ENQUEUE_IOCTL(struct iio_dmabuf *iio_dmabuf)`` + Enqueue a previously attached DMABUF object to the buffer queue. + Enqueued DMABUFs will be read from (if output buffer) or written to + (if input buffer) as long as the buffer is enabled. diff --git a/Documentation/iio/index.rst b/Documentation/iio/index.rst index 1b7292c58cd0..3eae8fcb1938 100644 --- a/Documentation/iio/index.rst +++ b/Documentation/iio/index.rst @@ -9,6 +9,8 @@ Industrial I/O iio_configfs + dmabuf_api + ep93xx_adc bno055