mbox series

[RESEND,V7,0/2] AMD QDMA driver

Message ID 1701277940-25645-1-git-send-email-lizhi.hou@amd.com
Headers
Series AMD QDMA driver |

Message

Lizhi Hou Nov. 29, 2023, 5:12 p.m. UTC 
  Hello,

The QDMA subsystem is used in conjunction with the PCI Express IP block
to provide high performance data transfer between host memory and the
card's DMA subsystem.

            +-------+       +-------+       +-----------+
   PCIe     |       |       |       |       |           |
   Tx/Rx    |       |       |       |  AXI  |           |
 <=======>  | PCIE  | <===> | QDMA  | <====>| User Logic|
            |       |       |       |       |           |
            +-------+       +-------+       +-----------+

Comparing to AMD/Xilinx XDMA subsystem,
    https://lore.kernel.org/lkml/Y+XeKt5yPr1nGGaq@matsya/
the QDMA subsystem is a queue based, configurable scatter-gather DMA
implementation which provides thousands of queues, support for multiple
physical/virtual functions with single-root I/O virtualization (SR-IOV),
and advanced interrupt support. In this mode the IP provides AXI4-MM and
AXI4-Stream user interfaces which may be configured on a per-queue basis.

The QDMA has been used for Xilinx Alveo PCIe devices.
    https://www.xilinx.com/applications/data-center/v70.html

This patch series is to provide the platform driver for AMD QDMA subsystem
to support AXI4-MM DMA transfers. More functions, such as AXI4-Stream
and SR-IOV, will be supported by future patches.

The device driver for any FPGA based PCIe device which leverages QDMA can
call the standard dmaengine APIs to discover and use the QDMA subsystem
without duplicating the QDMA driver code in its own driver.

Changes since v6:
- Added a patch to create amd/ and empty Kconfig/Makefile for AMD drivers
- Moved source code under amd/qdma/
- Minor changes for code review comments

Changes since v5:
- Add more in patch description.

Changes since v4:
- Convert to use platform driver callback .remove_new()

Changes since v3:
- Minor changes in Kconfig description.

Changes since v2:
- A minor change from code review comments.

Changes since v1:
- Minor changes from code review comments.
- Fixed kernel robot warning.

Lizhi Hou (2):
  dmaengine: amd: Add empty Kconfig and Makefile for AMD drivers
  dmaengine: amd: qdma: Add AMD QDMA driver

 MAINTAINERS                            |    8 +
 drivers/dma/Kconfig                    |    2 +
 drivers/dma/Makefile                   |    1 +
 drivers/dma/amd/Kconfig                |   14 +
 drivers/dma/amd/Makefile               |    6 +
 drivers/dma/amd/qdma/Makefile          |    8 +
 drivers/dma/amd/qdma/qdma-comm-regs.c  |   64 ++
 drivers/dma/amd/qdma/qdma.c            | 1185 ++++++++++++++++++++++++
 drivers/dma/amd/qdma/qdma.h            |  265 ++++++
 include/linux/platform_data/amd_qdma.h |   36 +
 10 files changed, 1589 insertions(+)
 create mode 100644 drivers/dma/amd/Kconfig
 create mode 100644 drivers/dma/amd/Makefile
 create mode 100644 drivers/dma/amd/qdma/Makefile
 create mode 100644 drivers/dma/amd/qdma/qdma-comm-regs.c
 create mode 100644 drivers/dma/amd/qdma/qdma.c
 create mode 100644 drivers/dma/amd/qdma/qdma.h
 create mode 100644 include/linux/platform_data/amd_qdma.h

Comments

Lizhi Hou Jan. 17, 2024, 7:58 p.m. UTC | #1 
Hi Mark,


The QDMA driver has been restructured for upstreaming. And the first 
QDMA patchset supports only AXI4-MM DMA transfers (mentioned in cover 
letter). Directly comparing code with QDMA git repo will not work well.

After QDMA driver patchset is merged, I believe you may submit a fix 
patch to dmaengine for any issue you encounter. We will review your 
patch and verify it with our hardware.


I do not work on the QDMA git repo or forum. I am not the right person 
for questions about QDMA git repo.


Hopefully, this helps.


Thanks,

Lizhi

On 1/14/24 09:54, Mark Harfouche wrote:
> On Wed, Nov 29, 2023 at 12:13 PM Lizhi Hou <lizhi.hou@amd.com> wrote:
>
>
>     Comparing to AMD/Xilinx XDMA subsystem,
>     https://lore.kernel.org/lkml/Y+XeKt5yPr1nGGaq@matsya/
>     the QDMA subsystem is a queue based, configurable scatter-gather DMA
>     implementation which provides thousands of queues, support for
>     multiple
>     physical/virtual functions with single-root I/O virtualization
>     (SR-IOV),
>     and advanced interrupt support. In this mode the IP provides
>     AXI4-MM and
>     AXI4-Stream user interfaces which may be configured on a per-queue
>     basis.
>     [...]
>     This patch series is to provide the platform driver for AMD QDMA
>     subsystem
>     to support AXI4-MM DMA transfers. More functions, such as AXI4-Stream
>     and SR-IOV, will be supported by future patches.
>
>
> Hello,
>
> My name is Mark Harfouche and I've been following the kernel mailing 
> for some time. I'm sorry if this message is coming months after the 
> initial post. Please let me know if there is a more appropriate venue 
> for this kind of message.
>
> Since 2018 we use QDMA, and continue to build the driver ourselves 
> (with some patches).
> I am really excited to see QDMA mature and become part of the linux 
> kernel.
>
> I wanted to share my experience with the QDMA driver and how I (and 
> other users) feel of how they react to feedback.
>
> Early in its development, the QDMA code base was constantly changing, 
> often breaking the userland code that we were writing to interface 
> with it.
> Beyond this, during our testing, we found it was full of unsafe usage 
> of string operations (for example, sprintf being used in the kernel 
> code without checking the validity of inputs).
> The most breaking changes often included the insertion of variables in 
> enum definitions in the netlink interface (which does not seem 
> included in this original set of patches).
>
> The community of users (beyond just me) has tried in many cases to 
> communicate small compatibility issues between kernel versions, and 
> suggested in many cases fixes in the form of small patches. All of 
> which were met with utter silence. Numerous examples on 
> https://github.com/Xilinx/dma_ip_drivers/pulls
> Some examples of breaking changes to enums (the interface between the 
> kernel code and the userland applications using netlink) can be found 
> between 2020 and today by looking at diffs in the qdma_nl.h (again 
> this file seems absent from the patches) 
> https://github.com/Xilinx/dma_ip_drivers/compare/2020.2...master#diff-4497b037af6c3a4f9ac917e639b58a1fa155feb4b62b7fe66741a42ed74cdfd6
>
> While I understand that in 2018-2022 the driver was not considered 
> stable, I am wondering what kind of mechanism for user feedback will 
> be available now that these patches are being submitted to the linux 
> kernel. Will the community be able to submit patches that resolve the 
> issues we find? Is AMD/Xilinx willing to stand behind their API moving 
> forward?
>
> Thank you for your time,
>
> Mark
>
> PS. I may have posted on various Xilinx/AMD/Github forums. My handle 
> there is often mark_ramona or hmaarrfk