[RFC,0/6] H.264 stateless encoder RFC 0/6

  Dear All,

This series adds uAPI for stateless H.264 encoding and an
accompanying driver using it.

It has been tested on an stm32mp25 and there exists
a gstreamer user:

https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5676

example pipeline:

gst-launch-1.0 videotestsrc num-buffers=30 ! video/x-raw, format=YUY2 !
v4l2slh264enc ! filesink location=test.h264

Rebased onto v6.6 with:

- some patches from ST to actually run the hardware
- my previous VP8 statless encoding series
- VP8 support for H1 from Hugues Fruchet

In particular, this series depends on the latter, which can be
found here:

https://patchwork.linuxtv.org/project/linux-media/list/?series=11358

Here's a branch which contains everything needed to actually run:

https://gitlab.collabora.com/linux/for-upstream/-/tree/h264-enc-rfc-6.6

I kindly ask for comments.

Regards,

Andrzej Pietrasiewicz (6):
  media: verisilicon Correct a typo in
    H1_REG_ENC_CTRL2_DEBLOCKING_FILTER_MODE
  media: verisilicon: Correct a typo in H1_REG_MAD_CTRL_MAD_THRESHOLD
  media: verisilicon: Improve constant's name
  media: verisilicon: Update H1 register definitions
  media: uapi: Add H.264 stateless encoding uAPI
  media: verisilicon: Add H.264 stateless encoder

 drivers/media/platform/verisilicon/Makefile   |   1 +
 drivers/media/platform/verisilicon/hantro.h   |   3 +
 .../media/platform/verisilicon/hantro_drv.c   |  10 +
 .../platform/verisilicon/hantro_h1_h264_enc.c | 493 +++++++++++
 .../platform/verisilicon/hantro_h1_regs.h     |  20 +-
 .../platform/verisilicon/hantro_h1_vp8_enc.c  |   2 +-
 .../media/platform/verisilicon/hantro_h264.c  | 777 ++++++++++++++++++
 .../media/platform/verisilicon/hantro_hw.h    |  23 +
 .../platform/verisilicon/stm32mp25_venc_hw.c  |  22 +-
 drivers/media/v4l2-core/v4l2-ctrls-core.c     |  54 ++
 drivers/media/v4l2-core/v4l2-ctrls-defs.c     |   9 +
 include/uapi/linux/v4l2-controls.h            |  85 ++
 include/uapi/linux/videodev2.h                |   2 +
 13 files changed, 1496 insertions(+), 5 deletions(-)
 create mode 100644 drivers/media/platform/verisilicon/hantro_h1_h264_enc.c
  

On Thu, Nov 16, 2023 at 9:48 AM Andrzej Pietrasiewicz
<andrzej.p@collabora.com> wrote:
>
> Dear All,
>
> This series adds uAPI for stateless H.264 encoding and an
> accompanying driver using it.
>
> It has been tested on an stm32mp25 and there exists
> a gstreamer user:
>
> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5676
>
> example pipeline:
>
> gst-launch-1.0 videotestsrc num-buffers=30 ! video/x-raw, format=YUY2 !
> v4l2slh264enc ! filesink location=test.h264
>
> Rebased onto v6.6 with:
>
> - some patches from ST to actually run the hardware
> - my previous VP8 statless encoding series
> - VP8 support for H1 from Hugues Fruchet
>
> In particular, this series depends on the latter, which can be
> found here:
>
> https://patchwork.linuxtv.org/project/linux-media/list/?series=11358
>
> Here's a branch which contains everything needed to actually run:
>
> https://gitlab.collabora.com/linux/for-upstream/-/tree/h264-enc-rfc-6.6
>
> I kindly ask for comments.

I attempted to port this to an i.MX8M Mini without much success.  I
get the driver to enumerate, and the gst-inspect of the v4l2codecs
detects the device for the encoder, but I get t a bunch of errors.

Pipeline is PREROLLING ...
[  141.946608] f->type = 10
[  141.951595] f->type = 10
[  141.954331] f->type = 10
[  141.957027] f->type = 9
[  141.959512] trying format S264
[  141.962583] trying format NV12
[  141.965639] OUTPUT codec mode: -1
[  141.968962] fmt - w: 640, h: 480
[  141.972202] CAPTURE codec mode: 8
[  141.975529] fmt - w: 640, h: 480
[  141.978778] f->type = 10
[  141.981316] trying format YUYV
[  141.984384] OUTPUT codec mode: -1
[  141.987707] fmt - w: 640, h: 480
[  141.998500] Codec mode = 8
[  142.018876] plane 0 size: 4194304, sizeimage: 4194304
[  142.023986] plane 0 size: 614400, sizeimage: 614400
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...[  142.035697] plane 0 size: 4194304,
sizeimage: 4194304

Redistribute latency...
New cl[  142.042146] plane 0 size: 614400, sizeimage: 614400
ock: GstSystemClock
ERROR: from element
/GstPipeline:pipeline0/v4l2slh264enc:v4l2slh264enc0: Driver did not
ack the request.
Additional debug info:
../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codech264enc.c(1002):
gst_v4l2_codec_h264_enc_encode_frame ():
/GstPipeline:pipeline0/v4l2slh264enc:v4l2slh264enc0
Execution ended after 0:00:01.019990114
Setting pipeline to NULL ...
ERROR: from element
/GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0: Internal data
stream error.
Additional debug info:
../subprojects/gstreamer/libs/gst/base/gstbasesrc.c(3153):
gst_base_src_loop ():
/GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0:
streaming stopped, reason error (-5)
ERROR: from element /GstPipeline:pipeline0/GstFileSink:filesink0:
Error while writing to file "test.h264".
Additional debug info:
../subprojects/gstreamer/plugins/elements/gstfilesink.c(716):
gst_file_sink_event (): /GstPipeline:pipeline0/GstFileSink:filesink0
[  144.058389] hantro_watchdog:127: frame processing timed out!
Freeing pipeline ...
[h264-enc-st-dev] root@localhost:~/gstreamer#


If I can get this working, I can do more testing.  I don't know how
similar the H1 is on the STM32 vs the i.MX8M Mini.  I don't have a
datasheet for the STM32, but the registers that I checked seemed like
they matched, but I don't know enough about how this all works, so I'm
a bit at a loss.  I can post my own RFC if people are open to
reviewing it too.  I'll go through and review what I can.



adam
>
> Regards,
>
> Andrzej Pietrasiewicz (6):
>   media: verisilicon Correct a typo in
>     H1_REG_ENC_CTRL2_DEBLOCKING_FILTER_MODE
>   media: verisilicon: Correct a typo in H1_REG_MAD_CTRL_MAD_THRESHOLD
>   media: verisilicon: Improve constant's name
>   media: verisilicon: Update H1 register definitions
>   media: uapi: Add H.264 stateless encoding uAPI
>   media: verisilicon: Add H.264 stateless encoder
>
>  drivers/media/platform/verisilicon/Makefile   |   1 +
>  drivers/media/platform/verisilicon/hantro.h   |   3 +
>  .../media/platform/verisilicon/hantro_drv.c   |  10 +
>  .../platform/verisilicon/hantro_h1_h264_enc.c | 493 +++++++++++
>  .../platform/verisilicon/hantro_h1_regs.h     |  20 +-
>  .../platform/verisilicon/hantro_h1_vp8_enc.c  |   2 +-
>  .../media/platform/verisilicon/hantro_h264.c  | 777 ++++++++++++++++++
>  .../media/platform/verisilicon/hantro_hw.h    |  23 +
>  .../platform/verisilicon/stm32mp25_venc_hw.c  |  22 +-
>  drivers/media/v4l2-core/v4l2-ctrls-core.c     |  54 ++
>  drivers/media/v4l2-core/v4l2-ctrls-defs.c     |   9 +
>  include/uapi/linux/v4l2-controls.h            |  85 ++
>  include/uapi/linux/videodev2.h                |   2 +
>  13 files changed, 1496 insertions(+), 5 deletions(-)
>  create mode 100644 drivers/media/platform/verisilicon/hantro_h1_h264_enc.c
>
> --
> 2.25.1
>
>