From patchwork Fri Dec 15 13:13:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Cercueil X-Patchwork-Id: 179266 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9263486dys; Fri, 15 Dec 2023 05:14:50 -0800 (PST) X-Google-Smtp-Source: AGHT+IGFqhNuWO7S6Cfk8y1yzSV3Cl1LhikzembbproSNodjvPRI1p7bCDYcHer5PTi8n/LbXjvH X-Received: by 2002:a17:902:f745:b0:1d0:6ffd:6122 with SMTP id g5-20020a170902f74500b001d06ffd6122mr14038386plw.68.1702646090580; Fri, 15 Dec 2023 05:14:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702646090; cv=none; d=google.com; s=arc-20160816; b=IGmILQq6/du5pdiPA7Vue1vK9YNbSveTMZosmz9hbi/XErN8WHfeOoUAKnQUfwGZ11 XZQ/FpWpTWKItHyNLFF5KkX2IgmQXvWsv6ToEe3aLrym3PMK3M/QrebnyleX3QK0MeUZ iOp1E0pmht1UapYveQqEth9qPtnPly/Miw9/3C73OPweDu3/UeAArh9cU4MlepIOgYGN 2BanflF8VbQr69KJM8nrRslec0SWW/wZR4HQRWKtxWLRbDL6DJqxJapnoGp8VH7jfsSd O9RIeeAiih1y5E95MLQznyHKN4EIUihg756wzcQCqvk7/ulznOSVzXxfzBOQsIr3dF8K Yk0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=21xrIJTonl2JSPX53jr10oHQ64LwYtc2R0VBvZ9/BIs=; fh=lq3uIejE9+UCb9GhFK9s8FDaMDYUiH23kGvWD8KW0Eg=; b=MiE13/PAXbGksrRMximWpzRXw4sIzgN30JboNiREtC7YEuP9VonVpWR5IebFerJVQb RQlP/T2B/kypzJVcj5xNsGCtMkCa1RHbYWmLNmqRJn7qbfKMu9X21K7xT1fG1XLC/0zP ar5o9jDioJ8l76/Gdy+2MohKwv5+5YkcsZrkgm32Bi8Q9iAa811ByAH+99nJd5ILgrjA pdQwgDCOSiJSER216MeCfWvGQGIWsKYs3q9yC/ZYSdyLCP5YBH5Fl2KrMQlj425x3I15 iIjIygVYzTOTSplXISGUKN49GhFd/prk8mS6T5NzyQz0+fzhUkx369d6EYmFH3U7rZA8 VO0g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b=Srd2SsiP; spf=pass (google.com: domain of linux-kernel+bounces-979-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-979-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id kh5-20020a170903064500b001d003f4a193si2103269plb.384.2023.12.15.05.14.50 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 05:14:50 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-979-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b=Srd2SsiP; spf=pass (google.com: domain of linux-kernel+bounces-979-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-979-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 93CEAB21355 for ; Fri, 15 Dec 2023 13:14:21 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 177E62DB88; Fri, 15 Dec 2023 13:14:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=crapouillou.net header.i=@crapouillou.net header.b="Srd2SsiP" X-Original-To: linux-kernel@vger.kernel.org Received: from aposti.net (aposti.net [89.234.176.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 96DFF2D791; Fri, 15 Dec 2023 13:14:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=crapouillou.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=crapouillou.net DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=crapouillou.net; s=mail; t=1702646029; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=21xrIJTonl2JSPX53jr10oHQ64LwYtc2R0VBvZ9/BIs=; b=Srd2SsiPVhfJfsyWju8h+Dm7x5+BPCtFQl+HxHQbQDmI+PE4bBDNFI8XEZ/F00eK+WVL0E 1WRaVjqdmk26n/7H3E5q1RbKKjB/g34lRMRLvxcKJPC2sqozzM8vmaANDeLiVdaMqTPdyM /RQRKKg/IfZopXUG219fOc6KrlKNbXY= From: Paul Cercueil To: Vinod Koul Cc: Lars-Peter Clausen , =?utf-8?q?Nuno_S=C3=A1?= , Michael Hennerich , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org, Paul Cercueil Subject: [PATCH v2 1/5] dmaengine: axi-dmac: Small code cleanup Date: Fri, 15 Dec 2023 14:13:09 +0100 Message-ID: <20231215131313.23840-2-paul@crapouillou.net> In-Reply-To: <20231215131313.23840-1-paul@crapouillou.net> References: <20231215131313.23840-1-paul@crapouillou.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam: Yes X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785353826858403549 X-GMAIL-MSGID: 1785353826858403549 Use a for() loop instead of a while() loop in axi_dmac_fill_linear_sg(). This makes the code leaner and cleaner overall, and does not introduce any functional change. Signed-off-by: Paul Cercueil --- v2: Improve commit message --- drivers/dma/dma-axi-dmac.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/drivers/dma/dma-axi-dmac.c b/drivers/dma/dma-axi-dmac.c index 2457a420c13d..760940b21eab 100644 --- a/drivers/dma/dma-axi-dmac.c +++ b/drivers/dma/dma-axi-dmac.c @@ -508,16 +508,13 @@ static struct axi_dmac_sg *axi_dmac_fill_linear_sg(struct axi_dmac_chan *chan, segment_size = ((segment_size - 1) | chan->length_align_mask) + 1; for (i = 0; i < num_periods; i++) { - len = period_len; - - while (len > segment_size) { + for (len = period_len; len > segment_size; sg++) { if (direction == DMA_DEV_TO_MEM) sg->dest_addr = addr; else sg->src_addr = addr; sg->x_len = segment_size; sg->y_len = 1; - sg++; addr += segment_size; len -= segment_size; } From patchwork Fri Dec 15 13:13:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Cercueil X-Patchwork-Id: 179268 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9263672dys; Fri, 15 Dec 2023 05:15:06 -0800 (PST) X-Google-Smtp-Source: AGHT+IFUX9A6GXFiX63yXsZvr43GJ3GrCFeSgRHuC6C1VXNptmBMxNrbxHm3cX53t5zBt8kqr5Q7 X-Received: by 2002:a05:6358:528c:b0:171:4aa4:51 with SMTP id g12-20020a056358528c00b001714aa40051mr5123683rwa.54.1702646105905; Fri, 15 Dec 2023 05:15:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702646105; cv=none; d=google.com; s=arc-20160816; b=z/CmeuXfJuA4a0QUixxZKd+y4uQkdSU7PZCaUQAKsXyZ7SzEBK+BIKCwRnKOhkNkML cNc2sZOipHJxzFvdzSAl4kBvQs/FBLG3hf5TYRIq67RaallNoXtjqJDdhb919rOkbcGG asCOIqvc/mVZ7UBbDLt2MUgprWdTdJv/DV5GcsJ4gQlNW7VKNzTFTzvvU7vl1wyFUNT+ Yc3+pww1tEdFp1grXr3SrpBKYue4bL8kbNxRdl0lxm8RXXP+3drQUa1GFaMYuMvDcg+a 3OBOaIwoEy49wlwgWXhJncjnGvy9pWQfF2PJwHXud9L9INdXZ013/PVbAy3HGr/3BmVt dQrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=dX1OOI1l+mZGv2ZWGYC1ua/+SRhxUb4vYorhXQG2m0s=; fh=lq3uIejE9+UCb9GhFK9s8FDaMDYUiH23kGvWD8KW0Eg=; b=CQHAK2aEhxggY681VIn7w+xqvNPXdB5jrp1WxlHogNxCZ7RSzU7kizPCB81Erl4AQU 8Y8KHarcIweCtQ4pEISjb1xVfFatDTlxgH32mxkPjkgy0PR2RaCYBGrFOnjUWVVdIzXI 8F/5IJQXhRNRdQ0LD2NOyqYhM+bYPRZQkhrX63ej8GoUjyHX433MIMvmjnhYOsRq6OO6 ttrRRJTHs7DhsiNCSbUnCXf9oucT9/a4tC5KG+Majqp4maRkAspk47ccbBK5kDsrmDwf JvjmJQzrKEGhefFLdX+1+b3JoEpzZB+LkExSs3vVTBTKWyTWlo6FB9uootYAhOTMiHEd embw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b=n73GF01n; spf=pass (google.com: domain of linux-kernel+bounces-980-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-980-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id 6-20020a630f46000000b00578b487825asi12901989pgp.208.2023.12.15.05.15.05 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 05:15:05 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-980-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b=n73GF01n; spf=pass (google.com: domain of linux-kernel+bounces-980-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-980-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id F0AFBB22831 for ; Fri, 15 Dec 2023 13:14:36 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 111092DB9B; Fri, 15 Dec 2023 13:14:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=crapouillou.net header.i=@crapouillou.net header.b="n73GF01n" X-Original-To: linux-kernel@vger.kernel.org Received: from aposti.net (aposti.net [89.234.176.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A43082DB78; Fri, 15 Dec 2023 13:14:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=crapouillou.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=crapouillou.net DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=crapouillou.net; s=mail; t=1702646030; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dX1OOI1l+mZGv2ZWGYC1ua/+SRhxUb4vYorhXQG2m0s=; b=n73GF01nlvHq339iRA74SaYN1veT8I1ymfb0jyl4RXfsslvWnxW8Sigt6UGYAq0KKmDwt/ l0mmWMMHKRX4jyKHL6DdB9ueF2czOEgP0AbgpUzqhX4I8+vBvKmer9DU0oUwCM/txc2hdU TmL1k3pWMFkaKPOw9u2qHVQN4x3utqk= From: Paul Cercueil To: Vinod Koul Cc: Lars-Peter Clausen , =?utf-8?q?Nuno_S=C3=A1?= , Michael Hennerich , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org, Paul Cercueil Subject: [PATCH v2 2/5] dmaengine: axi-dmac: Allocate hardware descriptors Date: Fri, 15 Dec 2023 14:13:10 +0100 Message-ID: <20231215131313.23840-3-paul@crapouillou.net> In-Reply-To: <20231215131313.23840-1-paul@crapouillou.net> References: <20231215131313.23840-1-paul@crapouillou.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam: Yes X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785353843055558794 X-GMAIL-MSGID: 1785353843055558794 Change where and how the DMA transfers meta-data is stored, to prepare for the upcoming introduction of scatter-gather support. Allocate hardware descriptors in the format that the HDL core will be expecting them when the scatter-gather feature is enabled, and use these fields to store the data that was previously stored in the axi_dmac_sg structure. Note that the 'x_len' and 'y_len' fields now contain the transfer length minus one, since that's what the hardware will expect in these fields. Signed-off-by: Paul Cercueil --- drivers/dma/dma-axi-dmac.c | 134 ++++++++++++++++++++++++------------- 1 file changed, 88 insertions(+), 46 deletions(-) diff --git a/drivers/dma/dma-axi-dmac.c b/drivers/dma/dma-axi-dmac.c index 760940b21eab..185230a769b9 100644 --- a/drivers/dma/dma-axi-dmac.c +++ b/drivers/dma/dma-axi-dmac.c @@ -97,20 +97,31 @@ /* The maximum ID allocated by the hardware is 31 */ #define AXI_DMAC_SG_UNUSED 32U +struct axi_dmac_hw_desc { + u32 flags; + u32 id; + u64 dest_addr; + u64 src_addr; + u64 __unused; + u32 y_len; + u32 x_len; + u32 src_stride; + u32 dst_stride; + u64 __pad[2]; +}; + struct axi_dmac_sg { - dma_addr_t src_addr; - dma_addr_t dest_addr; - unsigned int x_len; - unsigned int y_len; - unsigned int dest_stride; - unsigned int src_stride; - unsigned int id; unsigned int partial_len; bool schedule_when_free; + + struct axi_dmac_hw_desc *hw; + dma_addr_t hw_phys; }; struct axi_dmac_desc { struct virt_dma_desc vdesc; + struct axi_dmac_chan *chan; + bool cyclic; bool have_partial_xfer; @@ -229,7 +240,7 @@ static void axi_dmac_start_transfer(struct axi_dmac_chan *chan) sg = &desc->sg[desc->num_submitted]; /* Already queued in cyclic mode. Wait for it to finish */ - if (sg->id != AXI_DMAC_SG_UNUSED) { + if (sg->hw->id != AXI_DMAC_SG_UNUSED) { sg->schedule_when_free = true; return; } @@ -246,16 +257,16 @@ static void axi_dmac_start_transfer(struct axi_dmac_chan *chan) chan->next_desc = desc; } - sg->id = axi_dmac_read(dmac, AXI_DMAC_REG_TRANSFER_ID); + sg->hw->id = axi_dmac_read(dmac, AXI_DMAC_REG_TRANSFER_ID); if (axi_dmac_dest_is_mem(chan)) { - axi_dmac_write(dmac, AXI_DMAC_REG_DEST_ADDRESS, sg->dest_addr); - axi_dmac_write(dmac, AXI_DMAC_REG_DEST_STRIDE, sg->dest_stride); + axi_dmac_write(dmac, AXI_DMAC_REG_DEST_ADDRESS, sg->hw->dest_addr); + axi_dmac_write(dmac, AXI_DMAC_REG_DEST_STRIDE, sg->hw->dst_stride); } if (axi_dmac_src_is_mem(chan)) { - axi_dmac_write(dmac, AXI_DMAC_REG_SRC_ADDRESS, sg->src_addr); - axi_dmac_write(dmac, AXI_DMAC_REG_SRC_STRIDE, sg->src_stride); + axi_dmac_write(dmac, AXI_DMAC_REG_SRC_ADDRESS, sg->hw->src_addr); + axi_dmac_write(dmac, AXI_DMAC_REG_SRC_STRIDE, sg->hw->src_stride); } /* @@ -270,8 +281,8 @@ static void axi_dmac_start_transfer(struct axi_dmac_chan *chan) if (chan->hw_partial_xfer) flags |= AXI_DMAC_FLAG_PARTIAL_REPORT; - axi_dmac_write(dmac, AXI_DMAC_REG_X_LENGTH, sg->x_len - 1); - axi_dmac_write(dmac, AXI_DMAC_REG_Y_LENGTH, sg->y_len - 1); + axi_dmac_write(dmac, AXI_DMAC_REG_X_LENGTH, sg->hw->x_len); + axi_dmac_write(dmac, AXI_DMAC_REG_Y_LENGTH, sg->hw->y_len); axi_dmac_write(dmac, AXI_DMAC_REG_FLAGS, flags); axi_dmac_write(dmac, AXI_DMAC_REG_START_TRANSFER, 1); } @@ -286,9 +297,9 @@ static inline unsigned int axi_dmac_total_sg_bytes(struct axi_dmac_chan *chan, struct axi_dmac_sg *sg) { if (chan->hw_2d) - return sg->x_len * sg->y_len; + return (sg->hw->x_len + 1) * (sg->hw->y_len + 1); else - return sg->x_len; + return (sg->hw->x_len + 1); } static void axi_dmac_dequeue_partial_xfers(struct axi_dmac_chan *chan) @@ -307,9 +318,9 @@ static void axi_dmac_dequeue_partial_xfers(struct axi_dmac_chan *chan) list_for_each_entry(desc, &chan->active_descs, vdesc.node) { for (i = 0; i < desc->num_sgs; i++) { sg = &desc->sg[i]; - if (sg->id == AXI_DMAC_SG_UNUSED) + if (sg->hw->id == AXI_DMAC_SG_UNUSED) continue; - if (sg->id == id) { + if (sg->hw->id == id) { desc->have_partial_xfer = true; sg->partial_len = len; found_sg = true; @@ -376,12 +387,12 @@ static bool axi_dmac_transfer_done(struct axi_dmac_chan *chan, do { sg = &active->sg[active->num_completed]; - if (sg->id == AXI_DMAC_SG_UNUSED) /* Not yet submitted */ + if (sg->hw->id == AXI_DMAC_SG_UNUSED) /* Not yet submitted */ break; - if (!(BIT(sg->id) & completed_transfers)) + if (!(BIT(sg->hw->id) & completed_transfers)) break; active->num_completed++; - sg->id = AXI_DMAC_SG_UNUSED; + sg->hw->id = AXI_DMAC_SG_UNUSED; if (sg->schedule_when_free) { sg->schedule_when_free = false; start_next = true; @@ -476,22 +487,52 @@ static void axi_dmac_issue_pending(struct dma_chan *c) spin_unlock_irqrestore(&chan->vchan.lock, flags); } -static struct axi_dmac_desc *axi_dmac_alloc_desc(unsigned int num_sgs) +static struct axi_dmac_desc * +axi_dmac_alloc_desc(struct axi_dmac_chan *chan, unsigned int num_sgs) { + struct axi_dmac *dmac = chan_to_axi_dmac(chan); + struct device *dev = dmac->dma_dev.dev; + struct axi_dmac_hw_desc *hws; struct axi_dmac_desc *desc; + dma_addr_t hw_phys; unsigned int i; desc = kzalloc(struct_size(desc, sg, num_sgs), GFP_NOWAIT); if (!desc) return NULL; desc->num_sgs = num_sgs; + desc->chan = chan; - for (i = 0; i < num_sgs; i++) - desc->sg[i].id = AXI_DMAC_SG_UNUSED; + hws = dma_alloc_coherent(dev, PAGE_ALIGN(num_sgs * sizeof(*hws)), + &hw_phys, GFP_ATOMIC); + if (!hws) { + kfree(desc); + return NULL; + } + + for (i = 0; i < num_sgs; i++) { + desc->sg[i].hw = &hws[i]; + desc->sg[i].hw_phys = hw_phys + i * sizeof(*hws); + + hws[i].id = AXI_DMAC_SG_UNUSED; + hws[i].flags = 0; + } return desc; } +static void axi_dmac_free_desc(struct axi_dmac_desc *desc) +{ + struct axi_dmac *dmac = chan_to_axi_dmac(desc->chan); + struct device *dev = dmac->dma_dev.dev; + struct axi_dmac_hw_desc *hw = desc->sg[0].hw; + dma_addr_t hw_phys = desc->sg[0].hw_phys; + + dma_free_coherent(dev, PAGE_ALIGN(desc->num_sgs * sizeof(*hw)), + hw, hw_phys); + kfree(desc); +} + static struct axi_dmac_sg *axi_dmac_fill_linear_sg(struct axi_dmac_chan *chan, enum dma_transfer_direction direction, dma_addr_t addr, unsigned int num_periods, unsigned int period_len, @@ -510,21 +551,22 @@ static struct axi_dmac_sg *axi_dmac_fill_linear_sg(struct axi_dmac_chan *chan, for (i = 0; i < num_periods; i++) { for (len = period_len; len > segment_size; sg++) { if (direction == DMA_DEV_TO_MEM) - sg->dest_addr = addr; + sg->hw->dest_addr = addr; else - sg->src_addr = addr; - sg->x_len = segment_size; - sg->y_len = 1; + sg->hw->src_addr = addr; + sg->hw->x_len = segment_size - 1; + sg->hw->y_len = 0; + sg->hw->flags = 0; addr += segment_size; len -= segment_size; } if (direction == DMA_DEV_TO_MEM) - sg->dest_addr = addr; + sg->hw->dest_addr = addr; else - sg->src_addr = addr; - sg->x_len = len; - sg->y_len = 1; + sg->hw->src_addr = addr; + sg->hw->x_len = len - 1; + sg->hw->y_len = 0; sg++; addr += len; } @@ -551,7 +593,7 @@ static struct dma_async_tx_descriptor *axi_dmac_prep_slave_sg( for_each_sg(sgl, sg, sg_len, i) num_sgs += DIV_ROUND_UP(sg_dma_len(sg), chan->max_length); - desc = axi_dmac_alloc_desc(num_sgs); + desc = axi_dmac_alloc_desc(chan, num_sgs); if (!desc) return NULL; @@ -560,7 +602,7 @@ static struct dma_async_tx_descriptor *axi_dmac_prep_slave_sg( for_each_sg(sgl, sg, sg_len, i) { if (!axi_dmac_check_addr(chan, sg_dma_address(sg)) || !axi_dmac_check_len(chan, sg_dma_len(sg))) { - kfree(desc); + axi_dmac_free_desc(desc); return NULL; } @@ -595,7 +637,7 @@ static struct dma_async_tx_descriptor *axi_dmac_prep_dma_cyclic( num_periods = buf_len / period_len; num_segments = DIV_ROUND_UP(period_len, chan->max_length); - desc = axi_dmac_alloc_desc(num_periods * num_segments); + desc = axi_dmac_alloc_desc(chan, num_periods * num_segments); if (!desc) return NULL; @@ -650,26 +692,26 @@ static struct dma_async_tx_descriptor *axi_dmac_prep_interleaved( return NULL; } - desc = axi_dmac_alloc_desc(1); + desc = axi_dmac_alloc_desc(chan, 1); if (!desc) return NULL; if (axi_dmac_src_is_mem(chan)) { - desc->sg[0].src_addr = xt->src_start; - desc->sg[0].src_stride = xt->sgl[0].size + src_icg; + desc->sg[0].hw->src_addr = xt->src_start; + desc->sg[0].hw->src_stride = xt->sgl[0].size + src_icg; } if (axi_dmac_dest_is_mem(chan)) { - desc->sg[0].dest_addr = xt->dst_start; - desc->sg[0].dest_stride = xt->sgl[0].size + dst_icg; + desc->sg[0].hw->dest_addr = xt->dst_start; + desc->sg[0].hw->dst_stride = xt->sgl[0].size + dst_icg; } if (chan->hw_2d) { - desc->sg[0].x_len = xt->sgl[0].size; - desc->sg[0].y_len = xt->numf; + desc->sg[0].hw->x_len = xt->sgl[0].size - 1; + desc->sg[0].hw->y_len = xt->numf - 1; } else { - desc->sg[0].x_len = xt->sgl[0].size * xt->numf; - desc->sg[0].y_len = 1; + desc->sg[0].hw->x_len = xt->sgl[0].size * xt->numf - 1; + desc->sg[0].hw->y_len = 0; } if (flags & DMA_CYCLIC) @@ -685,7 +727,7 @@ static void axi_dmac_free_chan_resources(struct dma_chan *c) static void axi_dmac_desc_free(struct virt_dma_desc *vdesc) { - kfree(container_of(vdesc, struct axi_dmac_desc, vdesc)); + axi_dmac_free_desc(to_axi_dmac_desc(vdesc)); } static bool axi_dmac_regmap_rdwr(struct device *dev, unsigned int reg) From patchwork Fri Dec 15 13:13:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Cercueil X-Patchwork-Id: 179267 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9263518dys; Fri, 15 Dec 2023 05:14:54 -0800 (PST) X-Google-Smtp-Source: AGHT+IGzm7XlmTw6jqTaa2e0sjshnb3ldpLCNrTQu5EidPt3bUkMmsSzWMUyuuMe8DuPJA3LEa/f X-Received: by 2002:a05:651c:24b:b0:2cc:2087:3de2 with SMTP id x11-20020a05651c024b00b002cc20873de2mr5835685ljn.46.1702646093833; Fri, 15 Dec 2023 05:14:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702646093; cv=none; d=google.com; s=arc-20160816; b=y8k3oviqguevU8xXA2s/Awm5vTuyUEbpPW+X7ZlRU3AEhSAN8Wlsu0yyTqKTm7ET32 7u4kPQtX8r8AJp27wLhs1qsBik6872ROG3mpAzOyd1OO6f8eXem3PdsuTfkLDKX/6vcJ Rm8b7yGY39w9qxFqMJc7rrjc/dj+zattnvrQHOulDs0wvSMKonIwrVN39eFZIKMxjPXD 30McMgXQZDJoNuYwBP74RKa4Mo22JK75rZ+AaCh1ciylP8TB4eDWl0H1D34bmVRP+H/6 QSLmo89EiZ8dLk7ro1sW5zW7Q+xa3TwPfCxQ/jXYvC2rswFyhuri/9IwZzmN1wOfFX4y EwZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=DfTyNXzfHCBEsP75+j9tuzi+q9BjmgIsVWTMiU/h6PU=; fh=lq3uIejE9+UCb9GhFK9s8FDaMDYUiH23kGvWD8KW0Eg=; b=HfoxbFdC87/7lnm6lbxPjfAvTSAZ2E03hGwAJ4CJreXBgk2zlK0a//mbiZOHDInoXV gFDPvcWdxWU2nw2vSx2ra5iX0+7APvQoXE5NrJBpB865Uhcu3PYFEynQG4M9oJAd5v17 aKvORs4fTYf9hFCbf50gsjC+Z8x2ekYnDKOv2alF8SUHSXkODG1y/wV/yolwe0aBvVgi tNddAX1dTeTSx+LUom4qUDUgLQmVB10lIGImQJpvs5XGfzhFV4lKYOz3U7sUNPLMdefD dcAFK+fFYpR8YbnIhifcXEHaqm4H400aJyOs/MYnYQO8CGaCgcDbemJuLk7lJIa0/5VV dc+g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b=UIcXbG7C; spf=pass (google.com: domain of linux-kernel+bounces-981-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-981-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id i2-20020a05640242c200b0054ca2f95675si7415386edc.562.2023.12.15.05.14.53 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 05:14:53 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-981-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b=UIcXbG7C; spf=pass (google.com: domain of linux-kernel+bounces-981-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-981-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 46D3F1F249E7 for ; Fri, 15 Dec 2023 13:14:53 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E92382EB1A; Fri, 15 Dec 2023 13:14:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=crapouillou.net header.i=@crapouillou.net header.b="UIcXbG7C" X-Original-To: linux-kernel@vger.kernel.org Received: from aposti.net (aposti.net [89.234.176.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 896C62DF8F; Fri, 15 Dec 2023 13:14:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=crapouillou.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=crapouillou.net DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=crapouillou.net; s=mail; t=1702646030; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DfTyNXzfHCBEsP75+j9tuzi+q9BjmgIsVWTMiU/h6PU=; b=UIcXbG7CBMh88X7t8sZevtwrmaHlWv/g0RP3uQ6xH5EHBISmC30BrAsOhMogM5QvY/RQb9 sDnWahVnoDWEl0gC976GnESmk1OOQRn/iBQleCIREPLL1vGp1FCpwEzb4yq+K0It3rjtmn 7c4/ox+xPUj1aoI9v8eALHbOI5fi1bo= From: Paul Cercueil To: Vinod Koul Cc: Lars-Peter Clausen , =?utf-8?q?Nuno_S=C3=A1?= , Michael Hennerich , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org, Paul Cercueil Subject: [PATCH v2 3/5] dmaengine: axi-dmac: Add support for scatter-gather transfers Date: Fri, 15 Dec 2023 14:13:11 +0100 Message-ID: <20231215131313.23840-4-paul@crapouillou.net> In-Reply-To: <20231215131313.23840-1-paul@crapouillou.net> References: <20231215131313.23840-1-paul@crapouillou.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam: Yes X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785353830262319379 X-GMAIL-MSGID: 1785353830262319379 Implement support for scatter-gather transfers. Build a chain of hardware descriptors, each one corresponding to a segment of the transfer, and linked to the next one. The hardware will transfer the chain and only fire interrupts when the whole chain has been transferred. Support for scatter-gather is automatically enabled when the driver detects that the hardware supports it, by writing then reading the AXI_DMAC_REG_SG_ADDRESS register. If not available, the driver will fall back to standard DMA transfers. Signed-off-by: Paul Cercueil --- drivers/dma/dma-axi-dmac.c | 135 +++++++++++++++++++++++++------------ 1 file changed, 93 insertions(+), 42 deletions(-) diff --git a/drivers/dma/dma-axi-dmac.c b/drivers/dma/dma-axi-dmac.c index 185230a769b9..5109530b66de 100644 --- a/drivers/dma/dma-axi-dmac.c +++ b/drivers/dma/dma-axi-dmac.c @@ -81,9 +81,13 @@ #define AXI_DMAC_REG_CURRENT_DEST_ADDR 0x438 #define AXI_DMAC_REG_PARTIAL_XFER_LEN 0x44c #define AXI_DMAC_REG_PARTIAL_XFER_ID 0x450 +#define AXI_DMAC_REG_CURRENT_SG_ID 0x454 +#define AXI_DMAC_REG_SG_ADDRESS 0x47c +#define AXI_DMAC_REG_SG_ADDRESS_HIGH 0x4bc #define AXI_DMAC_CTRL_ENABLE BIT(0) #define AXI_DMAC_CTRL_PAUSE BIT(1) +#define AXI_DMAC_CTRL_ENABLE_SG BIT(2) #define AXI_DMAC_IRQ_SOT BIT(0) #define AXI_DMAC_IRQ_EOT BIT(1) @@ -97,12 +101,16 @@ /* The maximum ID allocated by the hardware is 31 */ #define AXI_DMAC_SG_UNUSED 32U +/* Flags for axi_dmac_hw_desc.flags */ +#define AXI_DMAC_HW_FLAG_LAST BIT(0) +#define AXI_DMAC_HW_FLAG_IRQ BIT(1) + struct axi_dmac_hw_desc { u32 flags; u32 id; u64 dest_addr; u64 src_addr; - u64 __unused; + u64 next_sg_addr; u32 y_len; u32 x_len; u32 src_stride; @@ -150,6 +158,7 @@ struct axi_dmac_chan { bool hw_partial_xfer; bool hw_cyclic; bool hw_2d; + bool hw_sg; }; struct axi_dmac { @@ -224,9 +233,11 @@ static void axi_dmac_start_transfer(struct axi_dmac_chan *chan) unsigned int flags = 0; unsigned int val; - val = axi_dmac_read(dmac, AXI_DMAC_REG_START_TRANSFER); - if (val) /* Queue is full, wait for the next SOT IRQ */ - return; + if (!chan->hw_sg) { + val = axi_dmac_read(dmac, AXI_DMAC_REG_START_TRANSFER); + if (val) /* Queue is full, wait for the next SOT IRQ */ + return; + } desc = chan->next_desc; @@ -245,9 +256,10 @@ static void axi_dmac_start_transfer(struct axi_dmac_chan *chan) return; } - desc->num_submitted++; - if (desc->num_submitted == desc->num_sgs || - desc->have_partial_xfer) { + if (chan->hw_sg) { + chan->next_desc = NULL; + } else if (++desc->num_submitted == desc->num_sgs || + desc->have_partial_xfer) { if (desc->cyclic) desc->num_submitted = 0; /* Start again */ else @@ -259,14 +271,16 @@ static void axi_dmac_start_transfer(struct axi_dmac_chan *chan) sg->hw->id = axi_dmac_read(dmac, AXI_DMAC_REG_TRANSFER_ID); - if (axi_dmac_dest_is_mem(chan)) { - axi_dmac_write(dmac, AXI_DMAC_REG_DEST_ADDRESS, sg->hw->dest_addr); - axi_dmac_write(dmac, AXI_DMAC_REG_DEST_STRIDE, sg->hw->dst_stride); - } + if (!chan->hw_sg) { + if (axi_dmac_dest_is_mem(chan)) { + axi_dmac_write(dmac, AXI_DMAC_REG_DEST_ADDRESS, sg->hw->dest_addr); + axi_dmac_write(dmac, AXI_DMAC_REG_DEST_STRIDE, sg->hw->dst_stride); + } - if (axi_dmac_src_is_mem(chan)) { - axi_dmac_write(dmac, AXI_DMAC_REG_SRC_ADDRESS, sg->hw->src_addr); - axi_dmac_write(dmac, AXI_DMAC_REG_SRC_STRIDE, sg->hw->src_stride); + if (axi_dmac_src_is_mem(chan)) { + axi_dmac_write(dmac, AXI_DMAC_REG_SRC_ADDRESS, sg->hw->src_addr); + axi_dmac_write(dmac, AXI_DMAC_REG_SRC_STRIDE, sg->hw->src_stride); + } } /* @@ -281,8 +295,14 @@ static void axi_dmac_start_transfer(struct axi_dmac_chan *chan) if (chan->hw_partial_xfer) flags |= AXI_DMAC_FLAG_PARTIAL_REPORT; - axi_dmac_write(dmac, AXI_DMAC_REG_X_LENGTH, sg->hw->x_len); - axi_dmac_write(dmac, AXI_DMAC_REG_Y_LENGTH, sg->hw->y_len); + if (chan->hw_sg) { + axi_dmac_write(dmac, AXI_DMAC_REG_SG_ADDRESS, (u32)sg->hw_phys); + axi_dmac_write(dmac, AXI_DMAC_REG_SG_ADDRESS_HIGH, + (u64)sg->hw_phys >> 32); + } else { + axi_dmac_write(dmac, AXI_DMAC_REG_X_LENGTH, sg->hw->x_len); + axi_dmac_write(dmac, AXI_DMAC_REG_Y_LENGTH, sg->hw->y_len); + } axi_dmac_write(dmac, AXI_DMAC_REG_FLAGS, flags); axi_dmac_write(dmac, AXI_DMAC_REG_START_TRANSFER, 1); } @@ -359,6 +379,9 @@ static void axi_dmac_compute_residue(struct axi_dmac_chan *chan, rslt->result = DMA_TRANS_NOERROR; rslt->residue = 0; + if (chan->hw_sg) + return; + /* * We get here if the last completed segment is partial, which * means we can compute the residue from that segment onwards @@ -385,36 +408,46 @@ static bool axi_dmac_transfer_done(struct axi_dmac_chan *chan, (completed_transfers & AXI_DMAC_FLAG_PARTIAL_XFER_DONE)) axi_dmac_dequeue_partial_xfers(chan); - do { - sg = &active->sg[active->num_completed]; - if (sg->hw->id == AXI_DMAC_SG_UNUSED) /* Not yet submitted */ - break; - if (!(BIT(sg->hw->id) & completed_transfers)) - break; - active->num_completed++; - sg->hw->id = AXI_DMAC_SG_UNUSED; - if (sg->schedule_when_free) { - sg->schedule_when_free = false; - start_next = true; + if (chan->hw_sg) { + if (active->cyclic) { + vchan_cyclic_callback(&active->vdesc); + } else { + list_del(&active->vdesc.node); + vchan_cookie_complete(&active->vdesc); + active = axi_dmac_active_desc(chan); } + } else { + do { + sg = &active->sg[active->num_completed]; + if (sg->hw->id == AXI_DMAC_SG_UNUSED) /* Not yet submitted */ + break; + if (!(BIT(sg->hw->id) & completed_transfers)) + break; + active->num_completed++; + sg->hw->id = AXI_DMAC_SG_UNUSED; + if (sg->schedule_when_free) { + sg->schedule_when_free = false; + start_next = true; + } - if (sg->partial_len) - axi_dmac_compute_residue(chan, active); + if (sg->partial_len) + axi_dmac_compute_residue(chan, active); - if (active->cyclic) - vchan_cyclic_callback(&active->vdesc); + if (active->cyclic) + vchan_cyclic_callback(&active->vdesc); - if (active->num_completed == active->num_sgs || - sg->partial_len) { - if (active->cyclic) { - active->num_completed = 0; /* wrap around */ - } else { - list_del(&active->vdesc.node); - vchan_cookie_complete(&active->vdesc); - active = axi_dmac_active_desc(chan); + if (active->num_completed == active->num_sgs || + sg->partial_len) { + if (active->cyclic) { + active->num_completed = 0; /* wrap around */ + } else { + list_del(&active->vdesc.node); + vchan_cookie_complete(&active->vdesc); + active = axi_dmac_active_desc(chan); + } } - } - } while (active); + } while (active); + } return start_next; } @@ -478,8 +511,12 @@ static void axi_dmac_issue_pending(struct dma_chan *c) struct axi_dmac_chan *chan = to_axi_dmac_chan(c); struct axi_dmac *dmac = chan_to_axi_dmac(chan); unsigned long flags; + u32 ctrl = AXI_DMAC_CTRL_ENABLE; + + if (chan->hw_sg) + ctrl |= AXI_DMAC_CTRL_ENABLE_SG; - axi_dmac_write(dmac, AXI_DMAC_REG_CTRL, AXI_DMAC_CTRL_ENABLE); + axi_dmac_write(dmac, AXI_DMAC_REG_CTRL, ctrl); spin_lock_irqsave(&chan->vchan.lock, flags); if (vchan_issue_pending(&chan->vchan)) @@ -516,8 +553,14 @@ axi_dmac_alloc_desc(struct axi_dmac_chan *chan, unsigned int num_sgs) hws[i].id = AXI_DMAC_SG_UNUSED; hws[i].flags = 0; + + /* Link hardware descriptors */ + hws[i].next_sg_addr = hw_phys + (i + 1) * sizeof(*hws); } + /* The last hardware descriptor will trigger an interrupt */ + desc->sg[num_sgs - 1].hw->flags = AXI_DMAC_HW_FLAG_LAST | AXI_DMAC_HW_FLAG_IRQ; + return desc; } @@ -753,6 +796,9 @@ static bool axi_dmac_regmap_rdwr(struct device *dev, unsigned int reg) case AXI_DMAC_REG_CURRENT_DEST_ADDR: case AXI_DMAC_REG_PARTIAL_XFER_LEN: case AXI_DMAC_REG_PARTIAL_XFER_ID: + case AXI_DMAC_REG_CURRENT_SG_ID: + case AXI_DMAC_REG_SG_ADDRESS: + case AXI_DMAC_REG_SG_ADDRESS_HIGH: return true; default: return false; @@ -905,6 +951,10 @@ static int axi_dmac_detect_caps(struct axi_dmac *dmac, unsigned int version) if (axi_dmac_read(dmac, AXI_DMAC_REG_FLAGS) == AXI_DMAC_FLAG_CYCLIC) chan->hw_cyclic = true; + axi_dmac_write(dmac, AXI_DMAC_REG_SG_ADDRESS, 0xffffffff); + if (axi_dmac_read(dmac, AXI_DMAC_REG_SG_ADDRESS)) + chan->hw_sg = true; + axi_dmac_write(dmac, AXI_DMAC_REG_Y_LENGTH, 1); if (axi_dmac_read(dmac, AXI_DMAC_REG_Y_LENGTH) == 1) chan->hw_2d = true; @@ -1005,6 +1055,7 @@ static int axi_dmac_probe(struct platform_device *pdev) dma_dev->dst_addr_widths = BIT(dmac->chan.dest_width); dma_dev->directions = BIT(dmac->chan.direction); dma_dev->residue_granularity = DMA_RESIDUE_GRANULARITY_DESCRIPTOR; + dma_dev->max_sg_burst = 31; /* 31 SGs maximum in one burst */ INIT_LIST_HEAD(&dma_dev->channels); dmac->chan.vchan.desc_free = axi_dmac_desc_free; From patchwork Fri Dec 15 13:13:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Cercueil X-Patchwork-Id: 179270 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9264130dys; Fri, 15 Dec 2023 05:15:39 -0800 (PST) X-Google-Smtp-Source: AGHT+IFKPHn/KLco1/h5Enp5+zMmXEfRbAT04nsctJ+JmYLmbeyJFeHSfTQsm0eY6uDKsT0tFgHj X-Received: by 2002:a17:902:e888:b0:1d0:6ffd:ceb9 with SMTP id w8-20020a170902e88800b001d06ffdceb9mr8087810plg.114.1702646139230; Fri, 15 Dec 2023 05:15:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702646139; cv=none; d=google.com; s=arc-20160816; b=OCszjTxPeC62/VS+ZOEJsiqlVO+Kt9kJEvaxmm1Y315RqfpC+oghDKpU/Ahyl97C6v YjPyDP2zKPCpdvgJ/hzeamnHQHLgEebDgvyDa60F9wdH3N4nc02ZwU3GbOJtJrHCeZcR 1lWrh5mGcB7seLPMICS9cqSI99Vhrj8WNH1BLQNRaMwc0JKOVLKnpKI7mGUJ/EvNDCwS FGNp0W/a+MAbhvXxwlB2XpPJM8lI6VY/7E5oEAHaUqDzVkxp9vkyIXftZfHWglCk1TIa q/+dW4rVf95H9Tas1rVh5uk8xJZmEAOlU9qerCpcfe2r+Kt6td9mA0uicegpqUcba4dJ wppw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=LZDpp4w6x3Qe5YwK/dl5k4LHlomX4XHjxnuuTbRgfVA=; fh=lq3uIejE9+UCb9GhFK9s8FDaMDYUiH23kGvWD8KW0Eg=; b=CnfFJckKGh1GM8AE3UbqbdsBi34Cn+2CWuVgIAT86nXSCBko5SEbte7YUHCk/bnsJ0 CRm6Fwv2V8XdzQcRWS4A02dmsGkHbgsxWPxePnljqCizEsVd1Raf4EVZjnUUZTW1fVCP Vn9m/vWiCM37sLGB4cbl3ZiNN+j09oI2CUBziQMzbGSMacRvJ5FsdiNnFvUGm+JIK4Eg EN+lcdlrS35z+LJhMnl0N9IlLzkfKucGjD2AkiOc5+n/VO1TUaVFxdamKHxlYc4giMNn /vDRI1tIMCsKXsv5RRgxRD2AGmbO4dJy1e3VVpGsGDG8Gu+01bpc9oyRo3CRnoSfWBOy MKlg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b=3r0VeLvc; spf=pass (google.com: domain of linux-kernel+bounces-982-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-982-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id kh5-20020a170903064500b001d003f4a193si2103269plb.384.2023.12.15.05.15.38 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 05:15:39 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-982-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b=3r0VeLvc; spf=pass (google.com: domain of linux-kernel+bounces-982-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-982-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 4FF59B21FED for ; Fri, 15 Dec 2023 13:15:08 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CA6DD30662; Fri, 15 Dec 2023 13:14:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=crapouillou.net header.i=@crapouillou.net header.b="3r0VeLvc" X-Original-To: linux-kernel@vger.kernel.org Received: from aposti.net (aposti.net [89.234.176.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 06ABE30333; Fri, 15 Dec 2023 13:14:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=crapouillou.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=crapouillou.net DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=crapouillou.net; s=mail; t=1702646031; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LZDpp4w6x3Qe5YwK/dl5k4LHlomX4XHjxnuuTbRgfVA=; b=3r0VeLvc+diNuJCQ549YKzeVipFNByVP6djLsD20mm7iuqlmv/dAO415SlHWgLomrGZejG Wu9jBwmb8Fxxu2gYB1V7gcWIoRPEYObosJdkYoLyZGKdMTX6057PnYXEs8zeEDx7zFR2ik M7KqINO1qBecKD/eGbLBfbc0X+OVMM8= From: Paul Cercueil To: Vinod Koul Cc: Lars-Peter Clausen , =?utf-8?q?Nuno_S=C3=A1?= , Michael Hennerich , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org, Paul Cercueil Subject: [PATCH v2 4/5] dmaengine: axi-dmac: Use only EOT interrupts when doing scatter-gather Date: Fri, 15 Dec 2023 14:13:12 +0100 Message-ID: <20231215131313.23840-5-paul@crapouillou.net> In-Reply-To: <20231215131313.23840-1-paul@crapouillou.net> References: <20231215131313.23840-1-paul@crapouillou.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam: Yes X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785353878169360650 X-GMAIL-MSGID: 1785353878169360650 Instead of notifying userspace in the end-of-transfer (EOT) interrupt and program the hardware in the start-of-transfer (SOT) interrupt, we can do both things in the EOT, allowing us to mask the SOT, and halve the number of interrupts sent by the HDL core. Signed-off-by: Paul Cercueil --- v2: Make sure cyclic buffers are restarted as well --- drivers/dma/dma-axi-dmac.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/drivers/dma/dma-axi-dmac.c b/drivers/dma/dma-axi-dmac.c index 5109530b66de..f63acae511fb 100644 --- a/drivers/dma/dma-axi-dmac.c +++ b/drivers/dma/dma-axi-dmac.c @@ -411,10 +411,12 @@ static bool axi_dmac_transfer_done(struct axi_dmac_chan *chan, if (chan->hw_sg) { if (active->cyclic) { vchan_cyclic_callback(&active->vdesc); + start_next = true; } else { list_del(&active->vdesc.node); vchan_cookie_complete(&active->vdesc); active = axi_dmac_active_desc(chan); + start_next = !!active; } } else { do { @@ -1000,6 +1002,7 @@ static int axi_dmac_probe(struct platform_device *pdev) struct axi_dmac *dmac; struct regmap *regmap; unsigned int version; + u32 irq_mask = 0; int ret; dmac = devm_kzalloc(&pdev->dev, sizeof(*dmac), GFP_KERNEL); @@ -1067,7 +1070,10 @@ static int axi_dmac_probe(struct platform_device *pdev) dma_dev->copy_align = (dmac->chan.address_align_mask + 1); - axi_dmac_write(dmac, AXI_DMAC_REG_IRQ_MASK, 0x00); + if (dmac->chan.hw_sg) + irq_mask |= AXI_DMAC_IRQ_SOT; + + axi_dmac_write(dmac, AXI_DMAC_REG_IRQ_MASK, irq_mask); if (of_dma_is_coherent(pdev->dev.of_node)) { ret = axi_dmac_read(dmac, AXI_DMAC_REG_COHERENCY_DESC); From patchwork Fri Dec 15 13:13:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Cercueil X-Patchwork-Id: 179269 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp9263920dys; Fri, 15 Dec 2023 05:15:23 -0800 (PST) X-Google-Smtp-Source: AGHT+IELmdOsxaqiPO91FDksVYcEyxs3LAjTatsFrSxDFez6i0bfSLt9jAeOeslK8ENnCM1ab1rz X-Received: by 2002:a67:ed50:0:b0:464:a39f:c58b with SMTP id m16-20020a67ed50000000b00464a39fc58bmr7204723vsp.62.1702646123060; Fri, 15 Dec 2023 05:15:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702646123; cv=none; d=google.com; s=arc-20160816; b=Xqe2/Seql+ZaRl0dzXzVjGvF/Fdq2woNbwbFwdh8oK+1YHbPi5mOvnPl7DPlkf1i0B aOBp0foeH8TPsyDFUMAIeppDhz5ap1E3C4CItXY4zBO636ATCO/yFXtduQ3sHR8YGn4G ewmZKTYbT8QafMNTieMH73ifDvR8C/I8zs5VlRDt+AhLhKLV5UTg3bMxZpHBPT4Et+VB 5LjTQ/zkpFKassp9eL/Mw/WGxavp7riOtF3pC2HPyMxqPJtZqfkOC9pFmBTutVuKvlYe 6wFk7XA4ehtglOQzerHE/ygSutms+VspMwTlMzQKTeOKpc1ZISKoKbt2sHvrHyUJLqDL dwNA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=kAeMuIy1XyR2uI2Lp4H2je64QseMIBGkzrSyKbKGd4Y=; fh=lq3uIejE9+UCb9GhFK9s8FDaMDYUiH23kGvWD8KW0Eg=; b=PIMn+co2tgqqiCOJMiJ+bbqVUkO9I6AVcoUszW6Kf4ulB8zvUz8rXtb1yXT5h7PaOy GDI4HDCB9iOI8/q99Z2A4LwM4ei+7lef4Zril2D4RuEd6ognT5I7OkoXgpKLEzddGAe/ RHrOkW9Ch0Y8BjTpRpvOydQZaYbj3l7IV3aqu8cC3QYOS/Unb/jdMR2VP4a9Q+3UHOMF tZ0eM6eGgwe0YosLtxNFrpGQwUl9HGq2w3D/WrtgTTXGnYTyydz3W/BFfQmrOMAYYrtW U0Fqxzrb/0TH7CbYI6CEHJouf22Rqr4BECCq2KMNOYyZc+vQM6GHt+Eo8xFQH4oPEt9N tn7A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b=DIiMSFMx; spf=pass (google.com: domain of linux-kernel+bounces-983-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-983-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id v20-20020a0561020f1400b00464924fb61csi3710857vss.701.2023.12.15.05.15.22 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 05:15:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-983-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@crapouillou.net header.s=mail header.b=DIiMSFMx; spf=pass (google.com: domain of linux-kernel+bounces-983-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-983-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=crapouillou.net Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id BDAAF1C2377D for ; Fri, 15 Dec 2023 13:15:22 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8F21F3158D; Fri, 15 Dec 2023 13:14:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=crapouillou.net header.i=@crapouillou.net header.b="DIiMSFMx" X-Original-To: linux-kernel@vger.kernel.org Received: from aposti.net (aposti.net [89.234.176.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C44693065B; Fri, 15 Dec 2023 13:14:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=crapouillou.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=crapouillou.net DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=crapouillou.net; s=mail; t=1702646031; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kAeMuIy1XyR2uI2Lp4H2je64QseMIBGkzrSyKbKGd4Y=; b=DIiMSFMxTT1aXlDXsy/PFjuLsM+XpoHYg0O5x93fbK0H4ObtJC8KMmoG5cWrdP7CLuV480 S8lKlFIBKgMSWMsyyCDK0whoT0VT03xXOTmqvGWPh3Z1LLgqLy5DfaF0oe8XdNfIl+nw6X aD/W6o2E4+ikDbHVRK0jH8lN3mUCMko= From: Paul Cercueil To: Vinod Koul Cc: Lars-Peter Clausen , =?utf-8?q?Nuno_S=C3=A1?= , Michael Hennerich , dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org, Paul Cercueil Subject: [PATCH v2 5/5] dmaengine: axi-dmac: Improve cyclic DMA transfers in SG mode Date: Fri, 15 Dec 2023 14:13:13 +0100 Message-ID: <20231215131313.23840-6-paul@crapouillou.net> In-Reply-To: <20231215131313.23840-1-paul@crapouillou.net> References: <20231215131313.23840-1-paul@crapouillou.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam: Yes X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785353861347004150 X-GMAIL-MSGID: 1785353861347004150 For cyclic transfers, chain the last descriptor to the first one, and disable IRQ generation if there is no callback registered with the cyclic transfer. Signed-off-by: Paul Cercueil --- v2: New patch --- drivers/dma/dma-axi-dmac.c | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/drivers/dma/dma-axi-dmac.c b/drivers/dma/dma-axi-dmac.c index f63acae511fb..4e339c04fc1e 100644 --- a/drivers/dma/dma-axi-dmac.c +++ b/drivers/dma/dma-axi-dmac.c @@ -285,12 +285,14 @@ static void axi_dmac_start_transfer(struct axi_dmac_chan *chan) /* * If the hardware supports cyclic transfers and there is no callback to - * call and only a single segment, enable hw cyclic mode to avoid - * unnecessary interrupts. + * call, enable hw cyclic mode to avoid unnecessary interrupts. */ - if (chan->hw_cyclic && desc->cyclic && !desc->vdesc.tx.callback && - desc->num_sgs == 1) - flags |= AXI_DMAC_FLAG_CYCLIC; + if (chan->hw_cyclic && desc->cyclic && !desc->vdesc.tx.callback) { + if (chan->hw_sg) + desc->sg[desc->num_sgs - 1].hw->flags &= ~AXI_DMAC_HW_FLAG_IRQ; + else if (desc->num_sgs == 1) + flags |= AXI_DMAC_FLAG_CYCLIC; + } if (chan->hw_partial_xfer) flags |= AXI_DMAC_FLAG_PARTIAL_REPORT; @@ -411,7 +413,6 @@ static bool axi_dmac_transfer_done(struct axi_dmac_chan *chan, if (chan->hw_sg) { if (active->cyclic) { vchan_cyclic_callback(&active->vdesc); - start_next = true; } else { list_del(&active->vdesc.node); vchan_cookie_complete(&active->vdesc); @@ -667,7 +668,7 @@ static struct dma_async_tx_descriptor *axi_dmac_prep_dma_cyclic( { struct axi_dmac_chan *chan = to_axi_dmac_chan(c); struct axi_dmac_desc *desc; - unsigned int num_periods, num_segments; + unsigned int num_periods, num_segments, num_sgs; if (direction != chan->direction) return NULL; @@ -681,11 +682,16 @@ static struct dma_async_tx_descriptor *axi_dmac_prep_dma_cyclic( num_periods = buf_len / period_len; num_segments = DIV_ROUND_UP(period_len, chan->max_length); + num_sgs = num_periods * num_segments; - desc = axi_dmac_alloc_desc(chan, num_periods * num_segments); + desc = axi_dmac_alloc_desc(chan, num_sgs); if (!desc) return NULL; + /* Chain the last descriptor to the first, and remove its "last" flag */ + desc->sg[num_sgs - 1].hw->next_sg_addr = desc->sg[0].hw_phys; + desc->sg[num_sgs - 1].hw->flags &= ~AXI_DMAC_HW_FLAG_LAST; + axi_dmac_fill_linear_sg(chan, direction, buf_addr, num_periods, period_len, desc->sg);