From patchwork Fri Jul 28 13:23:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vikash Garodia X-Patchwork-Id: 12772 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp445984vqg; Fri, 28 Jul 2023 06:46:51 -0700 (PDT) X-Google-Smtp-Source: APBJJlGqhiWYkFmvZ01Ib09zJVGbWqW1jCUyNvFBhiPp5GJoqFh13+VjRJHopczjqdhuCJxRSiSl X-Received: by 2002:a05:6a00:22d3:b0:687:1604:39d2 with SMTP id f19-20020a056a0022d300b00687160439d2mr1207112pfj.10.1690552010880; Fri, 28 Jul 2023 06:46:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690552010; cv=none; d=google.com; s=arc-20160816; b=MLR5p7D9GHc00O3a3drg2ProQMoKxlpZ8SeLktPRFMxENaLvNtFhHru70pFmYTE9QD 17YdJj7sQBw17PIFYZynTDHhFFfDC2G5qNyZyhorM2+wOp5tR6viCkziE8cExc9QrsyC B32rpmFp/vmyzOfQHoXqswZmwmOgl9VCRIE16r0/+nlpnGFfUqbTWF9TEQfgJNERLF5u Bq1pa8id2H/9y4vFx0yNat5LM8QCdrBG4YeWxVgCqchUTv9MjIQTJEz19+55WWUUa1Dt yqhEmCX6Ii8ulWTOUPNuHZaGeA/f6vchh5MKslSfSnKcOrTEO2z65aMFnZzzv2BvcgqE B3Ig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:message-id:date:subject:cc:to:from :dkim-signature; bh=37MAkPsjaGX96mlWuIqcGnQpu5z2Xqn8aWxHFzyYNf0=; fh=iWlVk13gve725uc1TRs0OpylW0tqDNn46GJ41aL7BTI=; b=dkpS0Nzar1ZggR49LWsX2Yf2iM1IUMvJeAr36JwSOWVgQcYRhaune4aQtOjmiRTYBL LXuzmYCGWb4Qv9PtZFlX5sN5LYOW7XvyMaLxcKZedS/nRRnQfIo7cOxxzOyz/bVw16Q/ deLqVuJR/GnPecN/ITMI04xzsiumGFLkbCH8ddQNv1+EczQvOtFQp3tSq5MlwvDpmxcp qknWaJ1+3KhhLXWX/QCjKShcz9YovmUZd8ZxnfkT1KVu2dIl6gOCNV0oCnVN/aI3kCSK XJJwVxzKPfG+IVrWs5v12xTgsfrJ30/2EyPYdD1pTy12tyORNvgmB1YnrTn2Yp3n/28a CCSg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=g0m2otUr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 37-20020a631265000000b0055380fbd9e1si3036938pgs.483.2023.07.28.06.46.37; Fri, 28 Jul 2023 06:46:50 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=g0m2otUr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236170AbjG1NYx (ORCPT + 99 others); Fri, 28 Jul 2023 09:24:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41492 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232912AbjG1NYv (ORCPT ); Fri, 28 Jul 2023 09:24:51 -0400 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA446DB; Fri, 28 Jul 2023 06:24:49 -0700 (PDT) Received: from pps.filterd (m0279872.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 36S6170J028640; Fri, 28 Jul 2023 13:24:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : mime-version : content-type; s=qcppdkim1; bh=37MAkPsjaGX96mlWuIqcGnQpu5z2Xqn8aWxHFzyYNf0=; b=g0m2otUrw/GW1Sf1ZVfg7MBGup6eJ87cXtTLROQwKEAsGoPAkgPSHu+4lzeZ+KtMed15 1LSd6+T4GDxnCuf16D6QgDzSmXWdDpSOlBEBRyr9lv00sWqlr+SVd3anl31+koXkG9UX TK8oZexokoLxJ727EDl2CfZU+wPDmuLq8WkWvbO9GsgqN+hP/lee6raUsYJZuJ+tSzGM l5wIFMw2bEYuv6TA2Zmi6IOUj19AgyA1uQOMY8Jf5oQ1hx2CHxnnTlwoUBN3tfLxOMhm L6CYBrJjt2kMB2n83OqlbHWmYSDFWZK/+CP8FHVmjKkMOdNziT3p5A/rNt7Ud12WTWT5 jw== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3s416m9f7b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 28 Jul 2023 13:24:37 +0000 Received: from nasanex01a.na.qualcomm.com (nasanex01a.na.qualcomm.com [10.52.223.231]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 36SDOacS000681 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 28 Jul 2023 13:24:36 GMT Received: from hu-vgarodia-hyd.qualcomm.com (10.80.80.8) by nasanex01a.na.qualcomm.com (10.52.223.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Fri, 28 Jul 2023 06:24:32 -0700 From: Vikash Garodia To: , , , , , , , , CC: , Vikash Garodia Subject: [PATCH 00/33] Qualcomm video decoder/encoder driver Date: Fri, 28 Jul 2023 18:53:11 +0530 Message-ID: <1690550624-14642-1-git-send-email-quic_vgarodia@quicinc.com> X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01a.na.qualcomm.com (10.52.223.231) To nasanex01a.na.qualcomm.com (10.52.223.231) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: 9QZNoLJjWGzWoaOmvjyTKD2Hm4m8Kibm X-Proofpoint-ORIG-GUID: 9QZNoLJjWGzWoaOmvjyTKD2Hm4m8Kibm X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-07-27_10,2023-07-26_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 bulkscore=0 malwarescore=0 suspectscore=0 clxscore=1015 mlxlogscore=999 impostorscore=0 spamscore=0 mlxscore=0 adultscore=0 priorityscore=1501 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2306200000 definitions=main-2307280123 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772672265542939657 X-GMAIL-MSGID: 1772672265542939657 This patch series introduces support for Qualcomm new video acceleration hardware architecture, used for video stream decoding/encoding. This driver is based on new communication protocol between video hardware and application processor. This driver comes with below capabilities: - V4L2 complaint video driver with M2M and STREAMING capability. - Supports H264, H265, VP9 decoders. - Supports H264, H265 encoders. This driver comes with below features: - Centralized resource and memory management. - Centralized management of core and instance states. - Defines platform specific capabilities and features. As a results, it provides a single point of control to enable/disable a given feature depending on specific platform capabilities. - Handles hardware interdependent configurations. For a given configuration from client, the driver checks for hardware dependent configuration/s and updates the same. - Handles multiple complex video scenarios involving state transitions - DRC, Drain, Seek, back to back DRC, DRC during Drain sequence, DRC during Seek sequence. - Introduces a flexible way for driver to subscribe for any property with hardware. Hardware would inform driver with those subscribed property with any change in value. - Introduces performance (clock and bus) model based on new hardware architecture. - Introduces multi thread safe design to handle communication between client and hardware. - Adapts encoder quality improvements available in new hardware architecture. - Implements asynchronous communication with hardware to achieve better experience in low latency usecases. - Supports multi stage hardware architecture for encode/decode. - Output and capture planes are controlled independently. Thereby providing a way to reconfigure individual plane. - Hardware packetization layer supports synchronization between configuration packet and data packet. - Introduces a flexibility to receive a hardware response for a given command packet. - Native hardware support of LAST flag which is mandatory to align with port reconfiguration and DRAIN sequence as per V4L guidelines. - Native hardware support for drain sequence. I think that the driver is in good shape for mainline kernel, and I hope the review comments will help to improve it, so please do review, and make comments. Dikshita Agarwal (17): iris: vidc: add core functions iris: add vidc wrapper file iris: vidc: add vb2 ops iris: vidc: add helpers for memory management iris: vidc: add helper functions for resource management iris: vidc: add helper functions for power management iris: add helpers for media format iris: vidc: add PIL functionality for video firmware iris: platform: add platform files iris: platform: sm8550: add capability file for sm8550 iris: variant: add helper functions for register handling iris: variant: iris3: add iris3 specific ops iris: variant: iris3: add helpers for buffer size calculations iris: variant: iris3: add helper for bus and clock calculation iris: variant: iris: implement the logic to compute bus bandwidth iris: variant: iris3: implement logic to compute clock frequency iris: enable building of iris video driver Vikash Garodia (16): MAINTAINERS: Add Qualcomm Iris video accelerator driver iris: vidc: add v4l2 wrapper file iris: vidc: define video core and instance context iris: iris: add video encoder files iris: vidc: add video decoder files iris: vidc: add control files iris: vidc: add helper functions iris: vidc: add helpers for state management iris: add vidc buffer files iris: vidc: define various structures and enum iris: vidc: hfi: add Host Firmware Interface (HFI) iris: vidc: hfi: add Host Firmware Interface (HFI) response handling iris: vidc: hfi: add helpers for handling shared queues iris: vidc: hfi: Add packetization layer iris: vidc: hfi: defines HFI properties and enums iris: vidc: add debug files MAINTAINERS | 10 + drivers/media/platform/qcom/Kconfig | 1 + drivers/media/platform/qcom/Makefile | 1 + drivers/media/platform/qcom/iris/Kconfig | 15 + drivers/media/platform/qcom/iris/Makefile | 46 + .../iris/platform/common/inc/msm_vidc_platform.h | 305 ++ .../iris/platform/common/src/msm_vidc_platform.c | 2499 ++++++++++++ .../iris/platform/sm8550/inc/msm_vidc_sm8550.h | 14 + .../iris/platform/sm8550/src/msm_vidc_sm8550.c | 1727 ++++++++ .../iris/variant/common/inc/msm_vidc_variant.h | 22 + .../iris/variant/common/src/msm_vidc_variant.c | 163 + .../qcom/iris/variant/iris3/inc/hfi_buffer_iris3.h | 1481 +++++++ .../iris/variant/iris3/inc/msm_vidc_buffer_iris3.h | 19 + .../qcom/iris/variant/iris3/inc/msm_vidc_iris3.h | 15 + .../iris/variant/iris3/inc/msm_vidc_power_iris3.h | 17 + .../iris/variant/iris3/inc/perf_static_model.h | 229 ++ .../iris/variant/iris3/src/msm_vidc_buffer_iris3.c | 595 +++ .../iris/variant/iris3/src/msm_vidc_bus_iris3.c | 884 ++++ .../iris/variant/iris3/src/msm_vidc_clock_iris3.c | 627 +++ .../qcom/iris/variant/iris3/src/msm_vidc_iris3.c | 954 +++++ .../iris/variant/iris3/src/msm_vidc_power_iris3.c | 345 ++ .../media/platform/qcom/iris/vidc/inc/firmware.h | 18 + .../platform/qcom/iris/vidc/inc/hfi_command.h | 190 + .../media/platform/qcom/iris/vidc/inc/hfi_packet.h | 52 + .../platform/qcom/iris/vidc/inc/hfi_property.h | 666 +++ .../platform/qcom/iris/vidc/inc/msm_media_info.h | 599 +++ .../media/platform/qcom/iris/vidc/inc/msm_vdec.h | 40 + .../media/platform/qcom/iris/vidc/inc/msm_venc.h | 34 + .../media/platform/qcom/iris/vidc/inc/msm_vidc.h | 60 + .../platform/qcom/iris/vidc/inc/msm_vidc_buffer.h | 32 + .../platform/qcom/iris/vidc/inc/msm_vidc_control.h | 26 + .../platform/qcom/iris/vidc/inc/msm_vidc_core.h | 165 + .../platform/qcom/iris/vidc/inc/msm_vidc_debug.h | 186 + .../platform/qcom/iris/vidc/inc/msm_vidc_driver.h | 352 ++ .../platform/qcom/iris/vidc/inc/msm_vidc_inst.h | 207 + .../qcom/iris/vidc/inc/msm_vidc_internal.h | 787 ++++ .../platform/qcom/iris/vidc/inc/msm_vidc_memory.h | 83 + .../platform/qcom/iris/vidc/inc/msm_vidc_power.h | 94 + .../platform/qcom/iris/vidc/inc/msm_vidc_state.h | 102 + .../platform/qcom/iris/vidc/inc/msm_vidc_v4l2.h | 77 + .../platform/qcom/iris/vidc/inc/msm_vidc_vb2.h | 39 + .../media/platform/qcom/iris/vidc/inc/resources.h | 259 ++ .../media/platform/qcom/iris/vidc/inc/venus_hfi.h | 66 + .../platform/qcom/iris/vidc/inc/venus_hfi_queue.h | 89 + .../qcom/iris/vidc/inc/venus_hfi_response.h | 26 + .../media/platform/qcom/iris/vidc/src/firmware.c | 294 ++ .../media/platform/qcom/iris/vidc/src/hfi_packet.c | 657 +++ .../media/platform/qcom/iris/vidc/src/msm_vdec.c | 2091 ++++++++++ .../media/platform/qcom/iris/vidc/src/msm_venc.c | 1484 +++++++ .../media/platform/qcom/iris/vidc/src/msm_vidc.c | 841 ++++ .../platform/qcom/iris/vidc/src/msm_vidc_buffer.c | 290 ++ .../platform/qcom/iris/vidc/src/msm_vidc_control.c | 824 ++++ .../platform/qcom/iris/vidc/src/msm_vidc_debug.c | 581 +++ .../platform/qcom/iris/vidc/src/msm_vidc_driver.c | 4276 ++++++++++++++++++++ .../platform/qcom/iris/vidc/src/msm_vidc_memory.c | 448 ++ .../platform/qcom/iris/vidc/src/msm_vidc_power.c | 560 +++ .../platform/qcom/iris/vidc/src/msm_vidc_probe.c | 660 +++ .../platform/qcom/iris/vidc/src/msm_vidc_state.c | 1607 ++++++++ .../platform/qcom/iris/vidc/src/msm_vidc_v4l2.c | 953 +++++ .../platform/qcom/iris/vidc/src/msm_vidc_vb2.c | 605 +++ .../media/platform/qcom/iris/vidc/src/resources.c | 1321 ++++++ .../media/platform/qcom/iris/vidc/src/venus_hfi.c | 1503 +++++++ .../platform/qcom/iris/vidc/src/venus_hfi_queue.c | 537 +++ .../qcom/iris/vidc/src/venus_hfi_response.c | 1607 ++++++++ 64 files changed, 35357 insertions(+) create mode 100644 drivers/media/platform/qcom/iris/Kconfig create mode 100644 drivers/media/platform/qcom/iris/Makefile create mode 100644 drivers/media/platform/qcom/iris/platform/common/inc/msm_vidc_platform.h create mode 100644 drivers/media/platform/qcom/iris/platform/common/src/msm_vidc_platform.c create mode 100644 drivers/media/platform/qcom/iris/platform/sm8550/inc/msm_vidc_sm8550.h create mode 100644 drivers/media/platform/qcom/iris/platform/sm8550/src/msm_vidc_sm8550.c create mode 100644 drivers/media/platform/qcom/iris/variant/common/inc/msm_vidc_variant.h create mode 100644 drivers/media/platform/qcom/iris/variant/common/src/msm_vidc_variant.c create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/inc/hfi_buffer_iris3.h create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/inc/msm_vidc_buffer_iris3.h create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/inc/msm_vidc_iris3.h create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/inc/msm_vidc_power_iris3.h create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/inc/perf_static_model.h create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/src/msm_vidc_buffer_iris3.c create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/src/msm_vidc_bus_iris3.c create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/src/msm_vidc_clock_iris3.c create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/src/msm_vidc_iris3.c create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/src/msm_vidc_power_iris3.c create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/firmware.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/hfi_command.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/hfi_packet.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/hfi_property.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_media_info.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vdec.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_venc.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_buffer.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_control.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_core.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_debug.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_driver.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_inst.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_internal.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_memory.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_power.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_state.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_v4l2.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_vb2.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/resources.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/venus_hfi.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/venus_hfi_queue.h create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/venus_hfi_response.h create mode 100644 drivers/media/platform/qcom/iris/vidc/src/firmware.c create mode 100644 drivers/media/platform/qcom/iris/vidc/src/hfi_packet.c create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vdec.c create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_venc.c create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc.c create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_buffer.c create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_control.c create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_debug.c create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_driver.c create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_memory.c create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_power.c create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_probe.c create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_state.c create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_v4l2.c create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_vb2.c create mode 100644 drivers/media/platform/qcom/iris/vidc/src/resources.c create mode 100644 drivers/media/platform/qcom/iris/vidc/src/venus_hfi.c create mode 100644 drivers/media/platform/qcom/iris/vidc/src/venus_hfi_queue.c create mode 100644 drivers/media/platform/qcom/iris/vidc/src/venus_hfi_response.c