[v3,00/14] media: rkisp1: Add support for i.MX8MP

  This series depends on v3 of "dt-bindings: media: Add macros for video
interface bus types" [1].

This series extends the rkisp1 driver to support the ISP found in the
NXP i.MX8MP SoC.

The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1")
and in the NXP i.MX8MP have the same origin, and have slightly diverged
over time as they are now independently developed (afaik) by Rockchip
and VeriSilicon. The latter is marketed under the name "ISP8000Nano",
and is close enough to the RK3399 ISP that it can easily be supported by
the same driver.

The last two patches add support for UYVY output format, which can be
implemented on the ISP version in the i.MX8MP but not in the one in the
RK3399.

This version of the series specifically has been tested on a Polyhex
Debix model A with an imx219 (Raspberry Pi cam v2).

[1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/

Laurent Pinchart (3):
  dt-bindings: media: rkisp1: Add i.MX8MP ISP example
  media: rkisp1: Add and use rkisp1_has_feature() macro
  media: rkisp1: Configure gasket on i.MX8MP

Paul Elder (11):
  dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible
  media: rkisp1: Add match data for i.MX8MP ISP
  media: rkisp1: Add and set registers for crop for i.MX8MP
  media: rkisp1: Add and set registers for output size config on i.MX8MP
  media: rkisp1: Add i.MX8MP-specific registers for MI and resizer
  media: rkisp1: Shift DMA buffer addresses on i.MX8MP
  media: rkisp1: Add register definitions for the test pattern generator
  media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP
  media: rkisp1: Support devices without self path
  media: rkisp1: Add YC swap capability
  media: rkisp1: Add UYVY as an output format

 .../bindings/media/rockchip-isp1.yaml         |  79 ++++++++++-
 .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++---
 .../platform/rockchip/rkisp1/rkisp1-common.h  |  32 +++++
 .../platform/rockchip/rkisp1/rkisp1-debug.c   |  14 +-
 .../platform/rockchip/rkisp1/rkisp1-dev.c     |  67 +++++++--
 .../platform/rockchip/rkisp1/rkisp1-isp.c     | 128 +++++++++++++++++-
 .../platform/rockchip/rkisp1/rkisp1-regs.h    |  90 ++++++++++++
 .../platform/rockchip/rkisp1/rkisp1-resizer.c |  35 ++++-
 include/uapi/linux/rkisp1-config.h            |   2 +
 9 files changed, 509 insertions(+), 40 deletions(-)
  

Hi Adam,

On Wed, Feb 15, 2023 at 07:57:53AM -0600, Adam Ford wrote:
> On Fri, Nov 18, 2022 at 3:44 AM Paul Elder wrote:
> 
> > This series depends on v3 of "dt-bindings: media: Add macros for video
> > interface bus types" [1].
> >
> > This series extends the rkisp1 driver to support the ISP found in the
> > NXP i.MX8MP SoC.
> 
> I'm going to spend some time testing this over the weekend.  Is there a V4
> pending, or should I just test whatever is in Laurent's repo?

I've updated all the v6.2-based branches on
https://gitlab.com/ideasonboard/nxp/linux.git (and renamed them to
v6.2/*). Beside a rebase, the v6.2/isp branch contains (at the bottom) 6
additional patches that I've previously posted to the linux-media
mailing list (feel free to review them ;-)).

My only concern with this series is with patch "media: rkisp1: Add match
data for i.MX8MP ISP", and in particular with the following hunk:

 enum rkisp1_cif_isp_version {
 	RKISP1_V10 = 10,
 	RKISP1_V11,
 	RKISP1_V12,
 	RKISP1_V13,
+	IMX8MP_V10,
 };

It's not a very nice versioning scheme :-S I'll see if I can find
something better, but regardless of that, I'll post v4 with the goal of
merging it in v6.4.

> I have an IMX219 camera with 4-lane support and an i.MX8M Plus kit from
> Beacon, and I want to test the RGGB bayer conversion to see how well it
> works.
> 
> > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1")
> > and in the NXP i.MX8MP have the same origin, and have slightly diverged
> > over time as they are now independently developed (afaik) by Rockchip
> > and VeriSilicon. The latter is marketed under the name "ISP8000Nano",
> > and is close enough to the RK3399 ISP that it can easily be supported by
> > the same driver.
>
> Is there a reason the driver cannot be renamed to a more generic name than
> rkisp1 if the Rockchip and VeriSilicon had similar origins?  Having the
> name Rockchip referenced from an NXP i.MX8M Plus seems odd to me.

The common roots of the IP core predate both Rockchip and VeriSilicon.
Those two implementations have now diverged (as with all forks), so
either name would be wrong in some cases :-S

> > The last two patches add support for UYVY output format, which can be
> > implemented on the ISP version in the i.MX8MP but not in the one in the
> > RK3399.
> >
> > This version of the series specifically has been tested on a Polyhex
> > Debix model A with an imx219 (Raspberry Pi cam v2).
> >
> > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/
> >
> > Laurent Pinchart (3):
> >   dt-bindings: media: rkisp1: Add i.MX8MP ISP example
> >   media: rkisp1: Add and use rkisp1_has_feature() macro
> >   media: rkisp1: Configure gasket on i.MX8MP
> >
> > Paul Elder (11):
> >   dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible
> >   media: rkisp1: Add match data for i.MX8MP ISP
> >   media: rkisp1: Add and set registers for crop for i.MX8MP
> >   media: rkisp1: Add and set registers for output size config on i.MX8MP
> >   media: rkisp1: Add i.MX8MP-specific registers for MI and resizer
> >   media: rkisp1: Shift DMA buffer addresses on i.MX8MP
> >   media: rkisp1: Add register definitions for the test pattern generator
> >   media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP
> >   media: rkisp1: Support devices without self path
> >   media: rkisp1: Add YC swap capability
> >   media: rkisp1: Add UYVY as an output format
> >
> >  .../bindings/media/rockchip-isp1.yaml         |  79 ++++++++++-
> >  .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++---
> >  .../platform/rockchip/rkisp1/rkisp1-common.h  |  32 +++++
> >  .../platform/rockchip/rkisp1/rkisp1-debug.c   |  14 +-
> >  .../platform/rockchip/rkisp1/rkisp1-dev.c     |  67 +++++++--
> >  .../platform/rockchip/rkisp1/rkisp1-isp.c     | 128 +++++++++++++++++-
> >  .../platform/rockchip/rkisp1/rkisp1-regs.h    |  90 ++++++++++++
> >  .../platform/rockchip/rkisp1/rkisp1-resizer.c |  35 ++++-
> >  include/uapi/linux/rkisp1-config.h            |   2 +
> >  9 files changed, 509 insertions(+), 40 deletions(-)
  

On Wed, Feb 15, 2023 at 5:55 PM Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> Hi Adam,
>
> On Wed, Feb 15, 2023 at 07:57:53AM -0600, Adam Ford wrote:
> > On Fri, Nov 18, 2022 at 3:44 AM Paul Elder wrote:
> >
> > > This series depends on v3 of "dt-bindings: media: Add macros for video
> > > interface bus types" [1].
> > >
> > > This series extends the rkisp1 driver to support the ISP found in the
> > > NXP i.MX8MP SoC.
> >
> > I'm going to spend some time testing this over the weekend.  Is there a V4
> > pending, or should I just test whatever is in Laurent's repo?
>
> I've updated all the v6.2-based branches on
> https://gitlab.com/ideasonboard/nxp/linux.git (and renamed them to
> v6.2/*). Beside a rebase, the v6.2/isp branch contains (at the bottom) 6
> additional patches that I've previously posted to the linux-media
> mailing list (feel free to review them ;-)).

I grabbed your v6.2 series, and applied some updates to enable an
imx219 camera and routed it through the ISP and configured the camera
to SRGGB10_1X10/640x480 and had the ISP convert to YUYV8_2X8/640x480
and it captured just fine.

With that, I think you can add

Tested-by: Adam Ford <aford173@gmail.com> #imx8mp-beacon

I haven't experimented with the resizer yet,but  I did have some
questions on the AWB.  The AWB appears to be available on the 8MP per
the TRM, and  I see reference to AWB in the driver, but when I query
the subdev via yavta, I didn't see anything obvious on how to enable
it.  My pipeline (attached) shows klisp1_params as video2 and
rkisp1_stats as video1.  I attempted to query both without much
success.

root@beacon-imx8mp-kit:~# yavta -l /dev/video2
Device /dev/video2 opened.
Device `rkisp1_params' on `platform:rkisp1' (driver 'rkisp1') supports
meta-data, output, without mplanes.
unable to query control 0xc0000000: Inappropriate ioctl for device (25).
Meta-data format: RK1P (50314b52) buffer size 3048
root@beacon-imx8mp-kit:~# yavta -l /dev/video1
Device /dev/video1 opened.
Device `rkisp1_stats' on `platform:rkisp1' (driver 'rkisp1') supports
meta-data, capture, without mplanes.
unable to query control 0xc0000000: Inappropriate ioctl for device (25).
Meta-data format: RK1S (53314b52) buffer size 260
root@beacon-imx8mp-kit:~#

Is there documentation somewhere on where to test the AWB?  This is of
particular interest to me, because the RGGB format of the camera comes
across with a green tint.  I am able to remove this green-ness on a
different platform using some AWB on the ARM, but I'd rather do it in
hardware if possible.

Thanks

adam


>
> My only concern with this series is with patch "media: rkisp1: Add match
> data for i.MX8MP ISP", and in particular with the following hunk:
>
>  enum rkisp1_cif_isp_version {
>         RKISP1_V10 = 10,
>         RKISP1_V11,
>         RKISP1_V12,
>         RKISP1_V13,
> +       IMX8MP_V10,
>  };
>
> It's not a very nice versioning scheme :-S I'll see if I can find
> something better, but regardless of that, I'll post v4 with the goal of
> merging it in v6.4.
>
> > I have an IMX219 camera with 4-lane support and an i.MX8M Plus kit from
> > Beacon, and I want to test the RGGB bayer conversion to see how well it
> > works.
> >
> > > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1")
> > > and in the NXP i.MX8MP have the same origin, and have slightly diverged
> > > over time as they are now independently developed (afaik) by Rockchip
> > > and VeriSilicon. The latter is marketed under the name "ISP8000Nano",
> > > and is close enough to the RK3399 ISP that it can easily be supported by
> > > the same driver.
> >
> > Is there a reason the driver cannot be renamed to a more generic name than
> > rkisp1 if the Rockchip and VeriSilicon had similar origins?  Having the
> > name Rockchip referenced from an NXP i.MX8M Plus seems odd to me.
>
> The common roots of the IP core predate both Rockchip and VeriSilicon.
> Those two implementations have now diverged (as with all forks), so
> either name would be wrong in some cases :-S
>
> > > The last two patches add support for UYVY output format, which can be
> > > implemented on the ISP version in the i.MX8MP but not in the one in the
> > > RK3399.
> > >
> > > This version of the series specifically has been tested on a Polyhex
> > > Debix model A with an imx219 (Raspberry Pi cam v2).
> > >
> > > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/
> > >
> > > Laurent Pinchart (3):
> > >   dt-bindings: media: rkisp1: Add i.MX8MP ISP example
> > >   media: rkisp1: Add and use rkisp1_has_feature() macro
> > >   media: rkisp1: Configure gasket on i.MX8MP
> > >
> > > Paul Elder (11):
> > >   dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible
> > >   media: rkisp1: Add match data for i.MX8MP ISP
> > >   media: rkisp1: Add and set registers for crop for i.MX8MP
> > >   media: rkisp1: Add and set registers for output size config on i.MX8MP
> > >   media: rkisp1: Add i.MX8MP-specific registers for MI and resizer
> > >   media: rkisp1: Shift DMA buffer addresses on i.MX8MP
> > >   media: rkisp1: Add register definitions for the test pattern generator
> > >   media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP
> > >   media: rkisp1: Support devices without self path
> > >   media: rkisp1: Add YC swap capability
> > >   media: rkisp1: Add UYVY as an output format
> > >
> > >  .../bindings/media/rockchip-isp1.yaml         |  79 ++++++++++-
> > >  .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++---
> > >  .../platform/rockchip/rkisp1/rkisp1-common.h  |  32 +++++
> > >  .../platform/rockchip/rkisp1/rkisp1-debug.c   |  14 +-
> > >  .../platform/rockchip/rkisp1/rkisp1-dev.c     |  67 +++++++--
> > >  .../platform/rockchip/rkisp1/rkisp1-isp.c     | 128 +++++++++++++++++-
> > >  .../platform/rockchip/rkisp1/rkisp1-regs.h    |  90 ++++++++++++
> > >  .../platform/rockchip/rkisp1/rkisp1-resizer.c |  35 ++++-
> > >  include/uapi/linux/rkisp1-config.h            |   2 +
> > >  9 files changed, 509 insertions(+), 40 deletions(-)
>
> --
> Regards,
>
> Laurent Pinchart
  

On Fri, Nov 18, 2022 at 3:44 AM Paul Elder <paul.elder@ideasonboard.com> wrote:
>
> This series depends on v3 of "dt-bindings: media: Add macros for video
> interface bus types" [1].
>
> This series extends the rkisp1 driver to support the ISP found in the
> NXP i.MX8MP SoC.
>
> The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1")
> and in the NXP i.MX8MP have the same origin, and have slightly diverged
> over time as they are now independently developed (afaik) by Rockchip
> and VeriSilicon. The latter is marketed under the name "ISP8000Nano",
> and is close enough to the RK3399 ISP that it can easily be supported by
> the same driver.
>
> The last two patches add support for UYVY output format, which can be
> implemented on the ISP version in the i.MX8MP but not in the one in the
> RK3399.
>
> This version of the series specifically has been tested on a Polyhex
> Debix model A with an imx219 (Raspberry Pi cam v2).
>
> [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/
>
> Laurent Pinchart (3):
>   dt-bindings: media: rkisp1: Add i.MX8MP ISP example
>   media: rkisp1: Add and use rkisp1_has_feature() macro
>   media: rkisp1: Configure gasket on i.MX8MP
>
> Paul Elder (11):
>   dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible
>   media: rkisp1: Add match data for i.MX8MP ISP
>   media: rkisp1: Add and set registers for crop for i.MX8MP
>   media: rkisp1: Add and set registers for output size config on i.MX8MP
>   media: rkisp1: Add i.MX8MP-specific registers for MI and resizer
>   media: rkisp1: Shift DMA buffer addresses on i.MX8MP
>   media: rkisp1: Add register definitions for the test pattern generator
>   media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP
>   media: rkisp1: Support devices without self path
>   media: rkisp1: Add YC swap capability
>   media: rkisp1: Add UYVY as an output format
>

Paul / Laurent,

I noticed an unexpected behaviour on the imx8mp.

If I setup my pipeline for 640x480, it works just fine using an imx219
camera configured for SRGGB10_1X10.

However, when I try to configure the pipeline to use the same camera
at 1920x1080 (no resizing), the ISP source keeps defaulting to 640x480

Media device information
------------------------
driver          rkisp1
model           rkisp1
serial
bus info        platform:rkisp1
hw revision     0xe
driver version  6.2.0

Device topology
- entity 1: rkisp1_isp (4 pads, 4 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
pad0: Sink
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none
ycbcr:601 quantization:full-range
crop.bounds:(0,0)/1920x1080
crop:(0,0)/640x480]
<- "csis-32e40000.csi":1 [ENABLED]
pad1: Sink
[fmt:unknown/0x0 field:none]
<- "rkisp1_params":0 [ENABLED,IMMUTABLE]
pad2: Source
[fmt:YUYV8_2X8/640x480 field:none colorspace:raw xfer:none ycbcr:601
quantization:lim-range
crop.bounds:(0,0)/640x480
crop:(0,0)/640x480]
-> "rkisp1_resizer_mainpath":0 [ENABLED]
pad3: Source
[fmt:unknown/0x0 field:none]
-> "rkisp1_stats":0 [ENABLED,IMMUTABLE]

- entity 6: rkisp1_resizer_mainpath (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
pad0: Sink
[fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb
ycbcr:601 quantization:lim-range
crop.bounds:(0,0)/1920x1080
crop:(0,0)/640x480]
<- "rkisp1_isp":2 [ENABLED]
pad1: Source
[fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb
ycbcr:601 quantization:lim-range]
-> "rkisp1_mainpath":0 [ENABLED,IMMUTABLE]

- entity 9: rkisp1_mainpath (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
pad0: Sink
<- "rkisp1_resizer_mainpath":1 [ENABLED,IMMUTABLE]

- entity 13: rkisp1_stats (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video1
pad0: Sink
<- "rkisp1_isp":3 [ENABLED,IMMUTABLE]

- entity 17: rkisp1_params (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video2
pad0: Source
-> "rkisp1_isp":1 [ENABLED,IMMUTABLE]

- entity 29: csis-32e40000.csi (2 pads, 2 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev2
pad0: Sink
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb
ycbcr:601 quantization:full-range]
<- "imx219 1-0010":0 [ENABLED]
pad1: Source
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb
ycbcr:601 quantization:full-range]
-> "rkisp1_isp":0 [ENABLED]

- entity 34: imx219 1-0010 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev3
pad0: Source
[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb
ycbcr:601 quantization:full-range
crop.bounds:(8,8)/3280x2464
crop:(688,700)/1920x1080]
-> "csis-32e40000.csi":0 [ENABLED]

It's at this point that everything except the ISP source is 1920x1080.

When I try to set the ISP sink to 1080, it ends up being 640x480 and
the resizer sink is also changed to 640x480

root@beacon-imx8mp-kit:~# media-ctl -v -V "'rkisp1_isp':2
[fmt:YUYV8_2X8/1920x1080 field:none]"
Opening media device /dev/media0
Enumerating entities
looking up device: 81:3
looking up device: 81:4
looking up device: 81:0
looking up device: 81:1
looking up device: 81:2
looking up device: 81:5
looking up device: 81:6
Found 7 entities
Enumerating pads and links
Setting up format YUYV8_2X8 1920x1080 on pad rkisp1_isp/2
Format set: YUYV8_2X8 640x480
Setting up format YUYV8_2X8 640x480 on pad rkisp1_resizer_mainpath/0
Format set: YUYV8_2X8 640x480


It's my understanding that the ISP should be able to handle 1920x1080,
and the resizer sink should match the ISP source.

With the pipeline improperly setup, the capture fails.

adam


>  .../bindings/media/rockchip-isp1.yaml         |  79 ++++++++++-
>  .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++---
>  .../platform/rockchip/rkisp1/rkisp1-common.h  |  32 +++++
>  .../platform/rockchip/rkisp1/rkisp1-debug.c   |  14 +-
>  .../platform/rockchip/rkisp1/rkisp1-dev.c     |  67 +++++++--
>  .../platform/rockchip/rkisp1/rkisp1-isp.c     | 128 +++++++++++++++++-
>  .../platform/rockchip/rkisp1/rkisp1-regs.h    |  90 ++++++++++++
>  .../platform/rockchip/rkisp1/rkisp1-resizer.c |  35 ++++-
>  include/uapi/linux/rkisp1-config.h            |   2 +
>  9 files changed, 509 insertions(+), 40 deletions(-)
>
> --
> 2.35.1
>
  

Hi Adam
   sorry to jump up without being involved in the conversation

On Sat, Feb 18, 2023 at 10:14:08AM -0600, Adam Ford wrote:
> On Wed, Feb 15, 2023 at 5:55 PM Laurent Pinchart
> <laurent.pinchart@ideasonboard.com> wrote:
> >
> > Hi Adam,
> >
> > On Wed, Feb 15, 2023 at 07:57:53AM -0600, Adam Ford wrote:
> > > On Fri, Nov 18, 2022 at 3:44 AM Paul Elder wrote:
> > >
> > > > This series depends on v3 of "dt-bindings: media: Add macros for video
> > > > interface bus types" [1].
> > > >
> > > > This series extends the rkisp1 driver to support the ISP found in the
> > > > NXP i.MX8MP SoC.
> > >
> > > I'm going to spend some time testing this over the weekend.  Is there a V4
> > > pending, or should I just test whatever is in Laurent's repo?
> >
> > I've updated all the v6.2-based branches on
> > https://gitlab.com/ideasonboard/nxp/linux.git (and renamed them to
> > v6.2/*). Beside a rebase, the v6.2/isp branch contains (at the bottom) 6
> > additional patches that I've previously posted to the linux-media
> > mailing list (feel free to review them ;-)).
>
> I grabbed your v6.2 series, and applied some updates to enable an
> imx219 camera and routed it through the ISP and configured the camera
> to SRGGB10_1X10/640x480 and had the ISP convert to YUYV8_2X8/640x480
> and it captured just fine.
>
> With that, I think you can add
>
> Tested-by: Adam Ford <aford173@gmail.com> #imx8mp-beacon
>
> I haven't experimented with the resizer yet,but  I did have some
> questions on the AWB.  The AWB appears to be available on the 8MP per
> the TRM, and  I see reference to AWB in the driver, but when I query
> the subdev via yavta, I didn't see anything obvious on how to enable
> it.  My pipeline (attached) shows klisp1_params as video2 and
> rkisp1_stats as video1.  I attempted to query both without much
> success.

As you might be aware there's no magic button to "turn AWB on". The
ISP enables the implementation of AWB algorithms that consumes the
statistics the ISP produces on the raw images it is fed with and
allows to program the color gains to realize colors balancing.

The implementation of such algorithms doesn't live in the driver but
rather in a separate component usually running in user space. With
libcamera we're creating a userspace camera stack where it is possible
to implement such algorithms, and the i.MX8MP is fairly well supported
by the "rkisp1" component.
https://git.libcamera.org/libcamera/libcamera.git/tree/src/libcamera/pipeline/rkisp1
https://git.libcamera.org/libcamera/libcamera.git/tree/src/ipa/rkisp1/algorithms

There's probably one single patch still out of tree in libcamera to flip
the switch and enable i.MX8MP support through the RkISP1 component.

If you're willing to give it a spin let me know and I can try
support you in testing it.


>
> root@beacon-imx8mp-kit:~# yavta -l /dev/video2
> Device /dev/video2 opened.
> Device `rkisp1_params' on `platform:rkisp1' (driver 'rkisp1') supports
> meta-data, output, without mplanes.
> unable to query control 0xc0000000: Inappropriate ioctl for device (25).
> Meta-data format: RK1P (50314b52) buffer size 3048
> root@beacon-imx8mp-kit:~# yavta -l /dev/video1
> Device /dev/video1 opened.
> Device `rkisp1_stats' on `platform:rkisp1' (driver 'rkisp1') supports
> meta-data, capture, without mplanes.
> unable to query control 0xc0000000: Inappropriate ioctl for device (25).
> Meta-data format: RK1S (53314b52) buffer size 260
> root@beacon-imx8mp-kit:~#
>
> Is there documentation somewhere on where to test the AWB?  This is of
> particular interest to me, because the RGGB format of the camera comes
> across with a green tint.  I am able to remove this green-ness on a
> different platform using some AWB on the ARM, but I'd rather do it in
> hardware if possible.
>
> Thanks
>
> adam
>
>
> >
> > My only concern with this series is with patch "media: rkisp1: Add match
> > data for i.MX8MP ISP", and in particular with the following hunk:
> >
> >  enum rkisp1_cif_isp_version {
> >         RKISP1_V10 = 10,
> >         RKISP1_V11,
> >         RKISP1_V12,
> >         RKISP1_V13,
> > +       IMX8MP_V10,
> >  };
> >
> > It's not a very nice versioning scheme :-S I'll see if I can find
> > something better, but regardless of that, I'll post v4 with the goal of
> > merging it in v6.4.
> >
> > > I have an IMX219 camera with 4-lane support and an i.MX8M Plus kit from
> > > Beacon, and I want to test the RGGB bayer conversion to see how well it
> > > works.
> > >
> > > > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1")
> > > > and in the NXP i.MX8MP have the same origin, and have slightly diverged
> > > > over time as they are now independently developed (afaik) by Rockchip
> > > > and VeriSilicon. The latter is marketed under the name "ISP8000Nano",
> > > > and is close enough to the RK3399 ISP that it can easily be supported by
> > > > the same driver.
> > >
> > > Is there a reason the driver cannot be renamed to a more generic name than
> > > rkisp1 if the Rockchip and VeriSilicon had similar origins?  Having the
> > > name Rockchip referenced from an NXP i.MX8M Plus seems odd to me.
> >
> > The common roots of the IP core predate both Rockchip and VeriSilicon.
> > Those two implementations have now diverged (as with all forks), so
> > either name would be wrong in some cases :-S
> >
> > > > The last two patches add support for UYVY output format, which can be
> > > > implemented on the ISP version in the i.MX8MP but not in the one in the
> > > > RK3399.
> > > >
> > > > This version of the series specifically has been tested on a Polyhex
> > > > Debix model A with an imx219 (Raspberry Pi cam v2).
> > > >
> > > > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/
> > > >
> > > > Laurent Pinchart (3):
> > > >   dt-bindings: media: rkisp1: Add i.MX8MP ISP example
> > > >   media: rkisp1: Add and use rkisp1_has_feature() macro
> > > >   media: rkisp1: Configure gasket on i.MX8MP
> > > >
> > > > Paul Elder (11):
> > > >   dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible
> > > >   media: rkisp1: Add match data for i.MX8MP ISP
> > > >   media: rkisp1: Add and set registers for crop for i.MX8MP
> > > >   media: rkisp1: Add and set registers for output size config on i.MX8MP
> > > >   media: rkisp1: Add i.MX8MP-specific registers for MI and resizer
> > > >   media: rkisp1: Shift DMA buffer addresses on i.MX8MP
> > > >   media: rkisp1: Add register definitions for the test pattern generator
> > > >   media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP
> > > >   media: rkisp1: Support devices without self path
> > > >   media: rkisp1: Add YC swap capability
> > > >   media: rkisp1: Add UYVY as an output format
> > > >
> > > >  .../bindings/media/rockchip-isp1.yaml         |  79 ++++++++++-
> > > >  .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++---
> > > >  .../platform/rockchip/rkisp1/rkisp1-common.h  |  32 +++++
> > > >  .../platform/rockchip/rkisp1/rkisp1-debug.c   |  14 +-
> > > >  .../platform/rockchip/rkisp1/rkisp1-dev.c     |  67 +++++++--
> > > >  .../platform/rockchip/rkisp1/rkisp1-isp.c     | 128 +++++++++++++++++-
> > > >  .../platform/rockchip/rkisp1/rkisp1-regs.h    |  90 ++++++++++++
> > > >  .../platform/rockchip/rkisp1/rkisp1-resizer.c |  35 ++++-
> > > >  include/uapi/linux/rkisp1-config.h            |   2 +
> > > >  9 files changed, 509 insertions(+), 40 deletions(-)
> >
> > --
> > Regards,
> >
> > Laurent Pinchart
  

Hi Adam

On Wed, Feb 22, 2023 at 05:39:30PM -0600, Adam Ford wrote:
> On Fri, Nov 18, 2022 at 3:44 AM Paul Elder <paul.elder@ideasonboard.com> wrote:
> >
> > This series depends on v3 of "dt-bindings: media: Add macros for video
> > interface bus types" [1].
> >
> > This series extends the rkisp1 driver to support the ISP found in the
> > NXP i.MX8MP SoC.
> >
> > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1")
> > and in the NXP i.MX8MP have the same origin, and have slightly diverged
> > over time as they are now independently developed (afaik) by Rockchip
> > and VeriSilicon. The latter is marketed under the name "ISP8000Nano",
> > and is close enough to the RK3399 ISP that it can easily be supported by
> > the same driver.
> >
> > The last two patches add support for UYVY output format, which can be
> > implemented on the ISP version in the i.MX8MP but not in the one in the
> > RK3399.
> >
> > This version of the series specifically has been tested on a Polyhex
> > Debix model A with an imx219 (Raspberry Pi cam v2).
> >
> > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/
> >
> > Laurent Pinchart (3):
> >   dt-bindings: media: rkisp1: Add i.MX8MP ISP example
> >   media: rkisp1: Add and use rkisp1_has_feature() macro
> >   media: rkisp1: Configure gasket on i.MX8MP
> >
> > Paul Elder (11):
> >   dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible
> >   media: rkisp1: Add match data for i.MX8MP ISP
> >   media: rkisp1: Add and set registers for crop for i.MX8MP
> >   media: rkisp1: Add and set registers for output size config on i.MX8MP
> >   media: rkisp1: Add i.MX8MP-specific registers for MI and resizer
> >   media: rkisp1: Shift DMA buffer addresses on i.MX8MP
> >   media: rkisp1: Add register definitions for the test pattern generator
> >   media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP
> >   media: rkisp1: Support devices without self path
> >   media: rkisp1: Add YC swap capability
> >   media: rkisp1: Add UYVY as an output format
> >
>
> Paul / Laurent,
>
> I noticed an unexpected behaviour on the imx8mp.
>
> If I setup my pipeline for 640x480, it works just fine using an imx219
> camera configured for SRGGB10_1X10.
>
> However, when I try to configure the pipeline to use the same camera
> at 1920x1080 (no resizing), the ISP source keeps defaulting to 640x480
>
> Media device information
> ------------------------
> driver          rkisp1
> model           rkisp1
> serial
> bus info        platform:rkisp1
> hw revision     0xe
> driver version  6.2.0
>
> Device topology
> - entity 1: rkisp1_isp (4 pads, 4 links)
>             type V4L2 subdev subtype Unknown flags 0
>             device node name /dev/v4l-subdev0
> pad0: Sink
> [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none
> ycbcr:601 quantization:full-range
> crop.bounds:(0,0)/1920x1080
> crop:(0,0)/640x480]
> <- "csis-32e40000.csi":1 [ENABLED]
> pad1: Sink
> [fmt:unknown/0x0 field:none]
> <- "rkisp1_params":0 [ENABLED,IMMUTABLE]
> pad2: Source
> [fmt:YUYV8_2X8/640x480 field:none colorspace:raw xfer:none ycbcr:601
> quantization:lim-range
> crop.bounds:(0,0)/640x480
> crop:(0,0)/640x480]
> -> "rkisp1_resizer_mainpath":0 [ENABLED]
> pad3: Source
> [fmt:unknown/0x0 field:none]
> -> "rkisp1_stats":0 [ENABLED,IMMUTABLE]
>
> - entity 6: rkisp1_resizer_mainpath (2 pads, 2 links)
>             type V4L2 subdev subtype Unknown flags 0
>             device node name /dev/v4l-subdev1
> pad0: Sink
> [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb
> ycbcr:601 quantization:lim-range
> crop.bounds:(0,0)/1920x1080
> crop:(0,0)/640x480]
> <- "rkisp1_isp":2 [ENABLED]
> pad1: Source
> [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb
> ycbcr:601 quantization:lim-range]
> -> "rkisp1_mainpath":0 [ENABLED,IMMUTABLE]
>
> - entity 9: rkisp1_mainpath (1 pad, 1 link)
>             type Node subtype V4L flags 0
>             device node name /dev/video0
> pad0: Sink
> <- "rkisp1_resizer_mainpath":1 [ENABLED,IMMUTABLE]
>
> - entity 13: rkisp1_stats (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video1
> pad0: Sink
> <- "rkisp1_isp":3 [ENABLED,IMMUTABLE]
>
> - entity 17: rkisp1_params (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video2
> pad0: Source
> -> "rkisp1_isp":1 [ENABLED,IMMUTABLE]
>
> - entity 29: csis-32e40000.csi (2 pads, 2 links)
>              type V4L2 subdev subtype Unknown flags 0
>              device node name /dev/v4l-subdev2
> pad0: Sink
> [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb
> ycbcr:601 quantization:full-range]
> <- "imx219 1-0010":0 [ENABLED]
> pad1: Source
> [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb
> ycbcr:601 quantization:full-range]
> -> "rkisp1_isp":0 [ENABLED]
>
> - entity 34: imx219 1-0010 (1 pad, 1 link)
>              type V4L2 subdev subtype Sensor flags 0
>              device node name /dev/v4l-subdev3
> pad0: Source
> [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb
> ycbcr:601 quantization:full-range
> crop.bounds:(8,8)/3280x2464
> crop:(688,700)/1920x1080]
> -> "csis-32e40000.csi":0 [ENABLED]

Not sure if it's my email reader but I'm sorry I can't read this.
Could you use an online paste service next time if lines are long and
gets wrapped in this way ?

>
> It's at this point that everything except the ISP source is 1920x1080.
>
> When I try to set the ISP sink to 1080, it ends up being 640x480 and
> the resizer sink is also changed to 640x480
>
> root@beacon-imx8mp-kit:~# media-ctl -v -V "'rkisp1_isp':2
> [fmt:YUYV8_2X8/1920x1080 field:none]"
> Opening media device /dev/media0
> Enumerating entities
> looking up device: 81:3
> looking up device: 81:4
> looking up device: 81:0
> looking up device: 81:1
> looking up device: 81:2
> looking up device: 81:5
> looking up device: 81:6
> Found 7 entities
> Enumerating pads and links
> Setting up format YUYV8_2X8 1920x1080 on pad rkisp1_isp/2
> Format set: YUYV8_2X8 640x480
> Setting up format YUYV8_2X8 640x480 on pad rkisp1_resizer_mainpath/0
> Format set: YUYV8_2X8 640x480
>

I'm on https://gitlab.com/ideasonboard/nxp/linux/-/tree/v6.2/merge
with an additional patch to enable the ISP on the board I'm testing
with

I've tested capturing YUYV 1920x1080 with the following script

------------------------------------------------------------------------------
#!/bin/bash -x

media-ctl -r

# Link entities
media-ctl -l '"ar0521 1-0036":0 -> "csis-32e40000.csi":0'[1]
media-ctl -l '"csis-32e40000.csi":1 -> "rkisp1_isp":0'[1]
media-ctl -l '"rkisp1_isp":2 -> "rkisp1_resizer_mainpath":0'[1]

# Setup format
media-ctl -v -V '"ar0521 1-0036":0 [fmt:SGRBG8_1X8/2592x1944]'
media-ctl -v -V '"csis-32e40000.csi":1 [fmt:SGRBG8_1X8/2592x1944]'
media-ctl -v -V '"rkisp1_isp":2 [fmt:YUYV8_2X8/2592x1944]'
media-ctl  -v -V '"rkisp1_resizer_mainpath":1 [fmt:YUYV8_2X8/1920x1080]'

yavta -f YUYV -s 1920x1080 -c10 /dev/video0 --file=/tmp/frame-#.bin
------------------------------------------------------------------------------

You will have to adjust it for imx219

Unless you tweak the sensor's exposure and gains manually do not
expect nice images. You would need an auto-exposure and gain routine
like the one implemented in libcamera.

>
> It's my understanding that the ISP should be able to handle 1920x1080,
> and the resizer sink should match the ISP source.
>
> With the pipeline improperly setup, the capture fails.
>
> adam
>
>
> >  .../bindings/media/rockchip-isp1.yaml         |  79 ++++++++++-
> >  .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++---
> >  .../platform/rockchip/rkisp1/rkisp1-common.h  |  32 +++++
> >  .../platform/rockchip/rkisp1/rkisp1-debug.c   |  14 +-
> >  .../platform/rockchip/rkisp1/rkisp1-dev.c     |  67 +++++++--
> >  .../platform/rockchip/rkisp1/rkisp1-isp.c     | 128 +++++++++++++++++-
> >  .../platform/rockchip/rkisp1/rkisp1-regs.h    |  90 ++++++++++++
> >  .../platform/rockchip/rkisp1/rkisp1-resizer.c |  35 ++++-
> >  include/uapi/linux/rkisp1-config.h            |   2 +
> >  9 files changed, 509 insertions(+), 40 deletions(-)
> >
> > --
> > 2.35.1
> >
  

Hi Adam,

On Wed, Feb 22, 2023 at 05:39:30PM -0600, Adam Ford wrote:
> On Fri, Nov 18, 2022 at 3:44 AM Paul Elder wrote:
> >
> > This series depends on v3 of "dt-bindings: media: Add macros for video
> > interface bus types" [1].
> >
> > This series extends the rkisp1 driver to support the ISP found in the
> > NXP i.MX8MP SoC.
> >
> > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1")
> > and in the NXP i.MX8MP have the same origin, and have slightly diverged
> > over time as they are now independently developed (afaik) by Rockchip
> > and VeriSilicon. The latter is marketed under the name "ISP8000Nano",
> > and is close enough to the RK3399 ISP that it can easily be supported by
> > the same driver.
> >
> > The last two patches add support for UYVY output format, which can be
> > implemented on the ISP version in the i.MX8MP but not in the one in the
> > RK3399.
> >
> > This version of the series specifically has been tested on a Polyhex
> > Debix model A with an imx219 (Raspberry Pi cam v2).
> >
> > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/
> >
> > Laurent Pinchart (3):
> >   dt-bindings: media: rkisp1: Add i.MX8MP ISP example
> >   media: rkisp1: Add and use rkisp1_has_feature() macro
> >   media: rkisp1: Configure gasket on i.MX8MP
> >
> > Paul Elder (11):
> >   dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible
> >   media: rkisp1: Add match data for i.MX8MP ISP
> >   media: rkisp1: Add and set registers for crop for i.MX8MP
> >   media: rkisp1: Add and set registers for output size config on i.MX8MP
> >   media: rkisp1: Add i.MX8MP-specific registers for MI and resizer
> >   media: rkisp1: Shift DMA buffer addresses on i.MX8MP
> >   media: rkisp1: Add register definitions for the test pattern generator
> >   media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP
> >   media: rkisp1: Support devices without self path
> >   media: rkisp1: Add YC swap capability
> >   media: rkisp1: Add UYVY as an output format
> 
> Paul / Laurent,
> 
> I noticed an unexpected behaviour on the imx8mp.
> 
> If I setup my pipeline for 640x480, it works just fine using an imx219
> camera configured for SRGGB10_1X10.
> 
> However, when I try to configure the pipeline to use the same camera
> at 1920x1080 (no resizing), the ISP source keeps defaulting to 640x480
> 
> Media device information
> ------------------------
> driver          rkisp1
> model           rkisp1
> serial
> bus info        platform:rkisp1
> hw revision     0xe
> driver version  6.2.0
> 
> Device topology
> - entity 1: rkisp1_isp (4 pads, 4 links)
>             type V4L2 subdev subtype Unknown flags 0
>             device node name /dev/v4l-subdev0
> pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480]

You're cropping the image to 640x480 here. You need to set the crop
rectangle to 1920x1080.

As Jacopo mentioned, I wouldn't recommend exercising the ISP directly.
Not only do you need to setup the pipeline, but you would also need to
implement all the imaging algorithms in userspace. libcamera will do all
this for you.

> <- "csis-32e40000.csi":1 [ENABLED]
> pad1: Sink [fmt:unknown/0x0 field:none]
> <- "rkisp1_params":0 [ENABLED,IMMUTABLE]
> pad2: Source [fmt:YUYV8_2X8/640x480 field:none colorspace:raw xfer:none ycbcr:601 quantization:lim-range crop.bounds:(0,0)/640x480 crop:(0,0)/640x480]
> -> "rkisp1_resizer_mainpath":0 [ENABLED]
> pad3: Source [fmt:unknown/0x0 field:none]
> -> "rkisp1_stats":0 [ENABLED,IMMUTABLE]
> 
> - entity 6: rkisp1_resizer_mainpath (2 pads, 2 links)
>             type V4L2 subdev subtype Unknown flags 0
>             device node name /dev/v4l-subdev1
> pad0: Sink [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480]
> <- "rkisp1_isp":2 [ENABLED]
> pad1: Source [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> -> "rkisp1_mainpath":0 [ENABLED,IMMUTABLE]
> 
> - entity 9: rkisp1_mainpath (1 pad, 1 link)
>             type Node subtype V4L flags 0
>             device node name /dev/video0
> pad0: Sink
> <- "rkisp1_resizer_mainpath":1 [ENABLED,IMMUTABLE]
> 
> - entity 13: rkisp1_stats (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video1
> pad0: Sink
> <- "rkisp1_isp":3 [ENABLED,IMMUTABLE]
> 
> - entity 17: rkisp1_params (1 pad, 1 link)
>              type Node subtype V4L flags 0
>              device node name /dev/video2
> pad0: Source
> -> "rkisp1_isp":1 [ENABLED,IMMUTABLE]
> 
> - entity 29: csis-32e40000.csi (2 pads, 2 links)
>              type V4L2 subdev subtype Unknown flags 0
>              device node name /dev/v4l-subdev2
> pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
> <- "imx219 1-0010":0 [ENABLED]
> pad1: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
> -> "rkisp1_isp":0 [ENABLED]
> 
> - entity 34: imx219 1-0010 (1 pad, 1 link)
>              type V4L2 subdev subtype Sensor flags 0
>              device node name /dev/v4l-subdev3
> pad0: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range crop.bounds:(8,8)/3280x2464 crop:(688,700)/1920x1080]
> -> "csis-32e40000.csi":0 [ENABLED]
> 
> It's at this point that everything except the ISP source is 1920x1080.
> 
> When I try to set the ISP sink to 1080, it ends up being 640x480 and
> the resizer sink is also changed to 640x480
> 
> root@beacon-imx8mp-kit:~# media-ctl -v -V "'rkisp1_isp':2
> [fmt:YUYV8_2X8/1920x1080 field:none]"
> Opening media device /dev/media0
> Enumerating entities
> looking up device: 81:3
> looking up device: 81:4
> looking up device: 81:0
> looking up device: 81:1
> looking up device: 81:2
> looking up device: 81:5
> looking up device: 81:6
> Found 7 entities
> Enumerating pads and links
> Setting up format YUYV8_2X8 1920x1080 on pad rkisp1_isp/2
> Format set: YUYV8_2X8 640x480
> Setting up format YUYV8_2X8 640x480 on pad rkisp1_resizer_mainpath/0
> Format set: YUYV8_2X8 640x480
> 
> 
> It's my understanding that the ISP should be able to handle 1920x1080,
> and the resizer sink should match the ISP source.
> 
> With the pipeline improperly setup, the capture fails.
> 
> >  .../bindings/media/rockchip-isp1.yaml         |  79 ++++++++++-
> >  .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++---
> >  .../platform/rockchip/rkisp1/rkisp1-common.h  |  32 +++++
> >  .../platform/rockchip/rkisp1/rkisp1-debug.c   |  14 +-
> >  .../platform/rockchip/rkisp1/rkisp1-dev.c     |  67 +++++++--
> >  .../platform/rockchip/rkisp1/rkisp1-isp.c     | 128 +++++++++++++++++-
> >  .../platform/rockchip/rkisp1/rkisp1-regs.h    |  90 ++++++++++++
> >  .../platform/rockchip/rkisp1/rkisp1-resizer.c |  35 ++++-
> >  include/uapi/linux/rkisp1-config.h            |   2 +
> >  9 files changed, 509 insertions(+), 40 deletions(-)
  

On Thu, Feb 23, 2023 at 8:26 AM Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> Hi Adam,
>
> On Wed, Feb 22, 2023 at 05:39:30PM -0600, Adam Ford wrote:
> > On Fri, Nov 18, 2022 at 3:44 AM Paul Elder wrote:
> > >
> > > This series depends on v3 of "dt-bindings: media: Add macros for video
> > > interface bus types" [1].
> > >
> > > This series extends the rkisp1 driver to support the ISP found in the
> > > NXP i.MX8MP SoC.
> > >
> > > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1")
> > > and in the NXP i.MX8MP have the same origin, and have slightly diverged
> > > over time as they are now independently developed (afaik) by Rockchip
> > > and VeriSilicon. The latter is marketed under the name "ISP8000Nano",
> > > and is close enough to the RK3399 ISP that it can easily be supported by
> > > the same driver.
> > >
> > > The last two patches add support for UYVY output format, which can be
> > > implemented on the ISP version in the i.MX8MP but not in the one in the
> > > RK3399.
> > >
> > > This version of the series specifically has been tested on a Polyhex
> > > Debix model A with an imx219 (Raspberry Pi cam v2).
> > >
> > > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/
> > >
> > > Laurent Pinchart (3):
> > >   dt-bindings: media: rkisp1: Add i.MX8MP ISP example
> > >   media: rkisp1: Add and use rkisp1_has_feature() macro
> > >   media: rkisp1: Configure gasket on i.MX8MP
> > >
> > > Paul Elder (11):
> > >   dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible
> > >   media: rkisp1: Add match data for i.MX8MP ISP
> > >   media: rkisp1: Add and set registers for crop for i.MX8MP
> > >   media: rkisp1: Add and set registers for output size config on i.MX8MP
> > >   media: rkisp1: Add i.MX8MP-specific registers for MI and resizer
> > >   media: rkisp1: Shift DMA buffer addresses on i.MX8MP
> > >   media: rkisp1: Add register definitions for the test pattern generator
> > >   media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP
> > >   media: rkisp1: Support devices without self path
> > >   media: rkisp1: Add YC swap capability
> > >   media: rkisp1: Add UYVY as an output format
> >
> > Paul / Laurent,
> >
> > I noticed an unexpected behaviour on the imx8mp.
> >
> > If I setup my pipeline for 640x480, it works just fine using an imx219
> > camera configured for SRGGB10_1X10.
> >
> > However, when I try to configure the pipeline to use the same camera
> > at 1920x1080 (no resizing), the ISP source keeps defaulting to 640x480
> >
> > Media device information
> > ------------------------
> > driver          rkisp1
> > model           rkisp1
> > serial
> > bus info        platform:rkisp1
> > hw revision     0xe
> > driver version  6.2.0
> >
> > Device topology
> > - entity 1: rkisp1_isp (4 pads, 4 links)
> >             type V4L2 subdev subtype Unknown flags 0
> >             device node name /dev/v4l-subdev0
> > pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480]
>
> You're cropping the image to 640x480 here. You need to set the crop
> rectangle to 1920x1080.
>
> As Jacopo mentioned, I wouldn't recommend exercising the ISP directly.
> Not only do you need to setup the pipeline, but you would also need to
> implement all the imaging algorithms in userspace. libcamera will do all
> this for you.

I'll give that a try.  My current employer has a v4l2src requirement,
but I can likely make an argument to switch to libcamera.  I didn't
catch the cropping part. Thanks for that.

adam
>
> > <- "csis-32e40000.csi":1 [ENABLED]
> > pad1: Sink [fmt:unknown/0x0 field:none]
> > <- "rkisp1_params":0 [ENABLED,IMMUTABLE]
> > pad2: Source [fmt:YUYV8_2X8/640x480 field:none colorspace:raw xfer:none ycbcr:601 quantization:lim-range crop.bounds:(0,0)/640x480 crop:(0,0)/640x480]
> > -> "rkisp1_resizer_mainpath":0 [ENABLED]
> > pad3: Source [fmt:unknown/0x0 field:none]
> > -> "rkisp1_stats":0 [ENABLED,IMMUTABLE]
> >
> > - entity 6: rkisp1_resizer_mainpath (2 pads, 2 links)
> >             type V4L2 subdev subtype Unknown flags 0
> >             device node name /dev/v4l-subdev1
> > pad0: Sink [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480]
> > <- "rkisp1_isp":2 [ENABLED]
> > pad1: Source [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> > -> "rkisp1_mainpath":0 [ENABLED,IMMUTABLE]
> >
> > - entity 9: rkisp1_mainpath (1 pad, 1 link)
> >             type Node subtype V4L flags 0
> >             device node name /dev/video0
> > pad0: Sink
> > <- "rkisp1_resizer_mainpath":1 [ENABLED,IMMUTABLE]
> >
> > - entity 13: rkisp1_stats (1 pad, 1 link)
> >              type Node subtype V4L flags 0
> >              device node name /dev/video1
> > pad0: Sink
> > <- "rkisp1_isp":3 [ENABLED,IMMUTABLE]
> >
> > - entity 17: rkisp1_params (1 pad, 1 link)
> >              type Node subtype V4L flags 0
> >              device node name /dev/video2
> > pad0: Source
> > -> "rkisp1_isp":1 [ENABLED,IMMUTABLE]
> >
> > - entity 29: csis-32e40000.csi (2 pads, 2 links)
> >              type V4L2 subdev subtype Unknown flags 0
> >              device node name /dev/v4l-subdev2
> > pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
> > <- "imx219 1-0010":0 [ENABLED]
> > pad1: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
> > -> "rkisp1_isp":0 [ENABLED]
> >
> > - entity 34: imx219 1-0010 (1 pad, 1 link)
> >              type V4L2 subdev subtype Sensor flags 0
> >              device node name /dev/v4l-subdev3
> > pad0: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range crop.bounds:(8,8)/3280x2464 crop:(688,700)/1920x1080]
> > -> "csis-32e40000.csi":0 [ENABLED]
> >
> > It's at this point that everything except the ISP source is 1920x1080.
> >
> > When I try to set the ISP sink to 1080, it ends up being 640x480 and
> > the resizer sink is also changed to 640x480
> >
> > root@beacon-imx8mp-kit:~# media-ctl -v -V "'rkisp1_isp':2
> > [fmt:YUYV8_2X8/1920x1080 field:none]"
> > Opening media device /dev/media0
> > Enumerating entities
> > looking up device: 81:3
> > looking up device: 81:4
> > looking up device: 81:0
> > looking up device: 81:1
> > looking up device: 81:2
> > looking up device: 81:5
> > looking up device: 81:6
> > Found 7 entities
> > Enumerating pads and links
> > Setting up format YUYV8_2X8 1920x1080 on pad rkisp1_isp/2
> > Format set: YUYV8_2X8 640x480
> > Setting up format YUYV8_2X8 640x480 on pad rkisp1_resizer_mainpath/0
> > Format set: YUYV8_2X8 640x480
> >
> >
> > It's my understanding that the ISP should be able to handle 1920x1080,
> > and the resizer sink should match the ISP source.
> >
> > With the pipeline improperly setup, the capture fails.
> >
> > >  .../bindings/media/rockchip-isp1.yaml         |  79 ++++++++++-
> > >  .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++---
> > >  .../platform/rockchip/rkisp1/rkisp1-common.h  |  32 +++++
> > >  .../platform/rockchip/rkisp1/rkisp1-debug.c   |  14 +-
> > >  .../platform/rockchip/rkisp1/rkisp1-dev.c     |  67 +++++++--
> > >  .../platform/rockchip/rkisp1/rkisp1-isp.c     | 128 +++++++++++++++++-
> > >  .../platform/rockchip/rkisp1/rkisp1-regs.h    |  90 ++++++++++++
> > >  .../platform/rockchip/rkisp1/rkisp1-resizer.c |  35 ++++-
> > >  include/uapi/linux/rkisp1-config.h            |   2 +
> > >  9 files changed, 509 insertions(+), 40 deletions(-)
>
> --
> Regards,
>
> Laurent Pinchart
  

Hi Adam,

On Thu, Feb 23, 2023 at 10:10:28AM -0600, Adam Ford wrote:
> On Thu, Feb 23, 2023 at 8:26 AM Laurent Pinchart wrote:
> > On Wed, Feb 22, 2023 at 05:39:30PM -0600, Adam Ford wrote:
> > > On Fri, Nov 18, 2022 at 3:44 AM Paul Elder wrote:
> > > >
> > > > This series depends on v3 of "dt-bindings: media: Add macros for video
> > > > interface bus types" [1].
> > > >
> > > > This series extends the rkisp1 driver to support the ISP found in the
> > > > NXP i.MX8MP SoC.
> > > >
> > > > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1")
> > > > and in the NXP i.MX8MP have the same origin, and have slightly diverged
> > > > over time as they are now independently developed (afaik) by Rockchip
> > > > and VeriSilicon. The latter is marketed under the name "ISP8000Nano",
> > > > and is close enough to the RK3399 ISP that it can easily be supported by
> > > > the same driver.
> > > >
> > > > The last two patches add support for UYVY output format, which can be
> > > > implemented on the ISP version in the i.MX8MP but not in the one in the
> > > > RK3399.
> > > >
> > > > This version of the series specifically has been tested on a Polyhex
> > > > Debix model A with an imx219 (Raspberry Pi cam v2).
> > > >
> > > > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/
> > > >
> > > > Laurent Pinchart (3):
> > > >   dt-bindings: media: rkisp1: Add i.MX8MP ISP example
> > > >   media: rkisp1: Add and use rkisp1_has_feature() macro
> > > >   media: rkisp1: Configure gasket on i.MX8MP
> > > >
> > > > Paul Elder (11):
> > > >   dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible
> > > >   media: rkisp1: Add match data for i.MX8MP ISP
> > > >   media: rkisp1: Add and set registers for crop for i.MX8MP
> > > >   media: rkisp1: Add and set registers for output size config on i.MX8MP
> > > >   media: rkisp1: Add i.MX8MP-specific registers for MI and resizer
> > > >   media: rkisp1: Shift DMA buffer addresses on i.MX8MP
> > > >   media: rkisp1: Add register definitions for the test pattern generator
> > > >   media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP
> > > >   media: rkisp1: Support devices without self path
> > > >   media: rkisp1: Add YC swap capability
> > > >   media: rkisp1: Add UYVY as an output format
> > >
> > > Paul / Laurent,
> > >
> > > I noticed an unexpected behaviour on the imx8mp.
> > >
> > > If I setup my pipeline for 640x480, it works just fine using an imx219
> > > camera configured for SRGGB10_1X10.
> > >
> > > However, when I try to configure the pipeline to use the same camera
> > > at 1920x1080 (no resizing), the ISP source keeps defaulting to 640x480
> > >
> > > Media device information
> > > ------------------------
> > > driver          rkisp1
> > > model           rkisp1
> > > serial
> > > bus info        platform:rkisp1
> > > hw revision     0xe
> > > driver version  6.2.0
> > >
> > > Device topology
> > > - entity 1: rkisp1_isp (4 pads, 4 links)
> > >             type V4L2 subdev subtype Unknown flags 0
> > >             device node name /dev/v4l-subdev0
> > > pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480]
> >
> > You're cropping the image to 640x480 here. You need to set the crop
> > rectangle to 1920x1080.
> >
> > As Jacopo mentioned, I wouldn't recommend exercising the ISP directly.
> > Not only do you need to setup the pipeline, but you would also need to
> > implement all the imaging algorithms in userspace. libcamera will do all
> > this for you.
> 
> I'll give that a try.  My current employer has a v4l2src requirement,
> but I can likely make an argument to switch to libcamera.  I didn't
> catch the cropping part. Thanks for that.

libcamera has a GStreamer source element called libcamerasrc which is
aimed to be a drop-in replacement for v4l2src. Some features may be
missing, let us know and we can discuss how to add them.

> > > <- "csis-32e40000.csi":1 [ENABLED]
> > > pad1: Sink [fmt:unknown/0x0 field:none]
> > > <- "rkisp1_params":0 [ENABLED,IMMUTABLE]
> > > pad2: Source [fmt:YUYV8_2X8/640x480 field:none colorspace:raw xfer:none ycbcr:601 quantization:lim-range crop.bounds:(0,0)/640x480 crop:(0,0)/640x480]
> > > -> "rkisp1_resizer_mainpath":0 [ENABLED]
> > > pad3: Source [fmt:unknown/0x0 field:none]
> > > -> "rkisp1_stats":0 [ENABLED,IMMUTABLE]
> > >
> > > - entity 6: rkisp1_resizer_mainpath (2 pads, 2 links)
> > >             type V4L2 subdev subtype Unknown flags 0
> > >             device node name /dev/v4l-subdev1
> > > pad0: Sink [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480]
> > > <- "rkisp1_isp":2 [ENABLED]
> > > pad1: Source [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> > > -> "rkisp1_mainpath":0 [ENABLED,IMMUTABLE]
> > >
> > > - entity 9: rkisp1_mainpath (1 pad, 1 link)
> > >             type Node subtype V4L flags 0
> > >             device node name /dev/video0
> > > pad0: Sink
> > > <- "rkisp1_resizer_mainpath":1 [ENABLED,IMMUTABLE]
> > >
> > > - entity 13: rkisp1_stats (1 pad, 1 link)
> > >              type Node subtype V4L flags 0
> > >              device node name /dev/video1
> > > pad0: Sink
> > > <- "rkisp1_isp":3 [ENABLED,IMMUTABLE]
> > >
> > > - entity 17: rkisp1_params (1 pad, 1 link)
> > >              type Node subtype V4L flags 0
> > >              device node name /dev/video2
> > > pad0: Source
> > > -> "rkisp1_isp":1 [ENABLED,IMMUTABLE]
> > >
> > > - entity 29: csis-32e40000.csi (2 pads, 2 links)
> > >              type V4L2 subdev subtype Unknown flags 0
> > >              device node name /dev/v4l-subdev2
> > > pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
> > > <- "imx219 1-0010":0 [ENABLED]
> > > pad1: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
> > > -> "rkisp1_isp":0 [ENABLED]
> > >
> > > - entity 34: imx219 1-0010 (1 pad, 1 link)
> > >              type V4L2 subdev subtype Sensor flags 0
> > >              device node name /dev/v4l-subdev3
> > > pad0: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range crop.bounds:(8,8)/3280x2464 crop:(688,700)/1920x1080]
> > > -> "csis-32e40000.csi":0 [ENABLED]
> > >
> > > It's at this point that everything except the ISP source is 1920x1080.
> > >
> > > When I try to set the ISP sink to 1080, it ends up being 640x480 and
> > > the resizer sink is also changed to 640x480
> > >
> > > root@beacon-imx8mp-kit:~# media-ctl -v -V "'rkisp1_isp':2
> > > [fmt:YUYV8_2X8/1920x1080 field:none]"
> > > Opening media device /dev/media0
> > > Enumerating entities
> > > looking up device: 81:3
> > > looking up device: 81:4
> > > looking up device: 81:0
> > > looking up device: 81:1
> > > looking up device: 81:2
> > > looking up device: 81:5
> > > looking up device: 81:6
> > > Found 7 entities
> > > Enumerating pads and links
> > > Setting up format YUYV8_2X8 1920x1080 on pad rkisp1_isp/2
> > > Format set: YUYV8_2X8 640x480
> > > Setting up format YUYV8_2X8 640x480 on pad rkisp1_resizer_mainpath/0
> > > Format set: YUYV8_2X8 640x480
> > >
> > >
> > > It's my understanding that the ISP should be able to handle 1920x1080,
> > > and the resizer sink should match the ISP source.
> > >
> > > With the pipeline improperly setup, the capture fails.
> > >
> > > >  .../bindings/media/rockchip-isp1.yaml         |  79 ++++++++++-
> > > >  .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++---
> > > >  .../platform/rockchip/rkisp1/rkisp1-common.h  |  32 +++++
> > > >  .../platform/rockchip/rkisp1/rkisp1-debug.c   |  14 +-
> > > >  .../platform/rockchip/rkisp1/rkisp1-dev.c     |  67 +++++++--
> > > >  .../platform/rockchip/rkisp1/rkisp1-isp.c     | 128 +++++++++++++++++-
> > > >  .../platform/rockchip/rkisp1/rkisp1-regs.h    |  90 ++++++++++++
> > > >  .../platform/rockchip/rkisp1/rkisp1-resizer.c |  35 ++++-
> > > >  include/uapi/linux/rkisp1-config.h            |   2 +
> > > >  9 files changed, 509 insertions(+), 40 deletions(-)
  

Hi Adam,

Le jeudi 23 février 2023 à 10:10 -0600, Adam Ford a écrit :
> On Thu, Feb 23, 2023 at 8:26 AM Laurent Pinchart
> <laurent.pinchart@ideasonboard.com> wrote:
> > 
> > Hi Adam,
> > 
> > On Wed, Feb 22, 2023 at 05:39:30PM -0600, Adam Ford wrote:
> > > On Fri, Nov 18, 2022 at 3:44 AM Paul Elder wrote:
> > > > 
> > > > This series depends on v3 of "dt-bindings: media: Add macros for video
> > > > interface bus types" [1].
> > > > 
> > > > This series extends the rkisp1 driver to support the ISP found in the
> > > > NXP i.MX8MP SoC.
> > > > 
> > > > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1")
> > > > and in the NXP i.MX8MP have the same origin, and have slightly diverged
> > > > over time as they are now independently developed (afaik) by Rockchip
> > > > and VeriSilicon. The latter is marketed under the name "ISP8000Nano",
> > > > and is close enough to the RK3399 ISP that it can easily be supported by
> > > > the same driver.
> > > > 
> > > > The last two patches add support for UYVY output format, which can be
> > > > implemented on the ISP version in the i.MX8MP but not in the one in the
> > > > RK3399.
> > > > 
> > > > This version of the series specifically has been tested on a Polyhex
> > > > Debix model A with an imx219 (Raspberry Pi cam v2).
> > > > 
> > > > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/
> > > > 
> > > > Laurent Pinchart (3):
> > > >   dt-bindings: media: rkisp1: Add i.MX8MP ISP example
> > > >   media: rkisp1: Add and use rkisp1_has_feature() macro
> > > >   media: rkisp1: Configure gasket on i.MX8MP
> > > > 
> > > > Paul Elder (11):
> > > >   dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible
> > > >   media: rkisp1: Add match data for i.MX8MP ISP
> > > >   media: rkisp1: Add and set registers for crop for i.MX8MP
> > > >   media: rkisp1: Add and set registers for output size config on i.MX8MP
> > > >   media: rkisp1: Add i.MX8MP-specific registers for MI and resizer
> > > >   media: rkisp1: Shift DMA buffer addresses on i.MX8MP
> > > >   media: rkisp1: Add register definitions for the test pattern generator
> > > >   media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP
> > > >   media: rkisp1: Support devices without self path
> > > >   media: rkisp1: Add YC swap capability
> > > >   media: rkisp1: Add UYVY as an output format
> > > 
> > > Paul / Laurent,
> > > 
> > > I noticed an unexpected behaviour on the imx8mp.
> > > 
> > > If I setup my pipeline for 640x480, it works just fine using an imx219
> > > camera configured for SRGGB10_1X10.
> > > 
> > > However, when I try to configure the pipeline to use the same camera
> > > at 1920x1080 (no resizing), the ISP source keeps defaulting to 640x480
> > > 
> > > Media device information
> > > ------------------------
> > > driver          rkisp1
> > > model           rkisp1
> > > serial
> > > bus info        platform:rkisp1
> > > hw revision     0xe
> > > driver version  6.2.0
> > > 
> > > Device topology
> > > - entity 1: rkisp1_isp (4 pads, 4 links)
> > >             type V4L2 subdev subtype Unknown flags 0
> > >             device node name /dev/v4l-subdev0
> > > pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480]
> > 
> > You're cropping the image to 640x480 here. You need to set the crop
> > rectangle to 1920x1080.
> > 
> > As Jacopo mentioned, I wouldn't recommend exercising the ISP directly.
> > Not only do you need to setup the pipeline, but you would also need to
> > implement all the imaging algorithms in userspace. libcamera will do all
> > this for you.
> 
> I'll give that a try.  My current employer has a v4l2src requirement,
> but I can likely make an argument to switch to libcamera.  I didn't
> catch the cropping part. Thanks for that.

I'd hope you can transparently replace v4l2src with libcamerasrc, the plugins
currently lives inside the libcamera project. If not, I'd really like to know
why. We can work together on adding missing controls (this is something I'm
starting on soon).

regards,
Nicolas

> 
> adam
> > 
> > > <- "csis-32e40000.csi":1 [ENABLED]
> > > pad1: Sink [fmt:unknown/0x0 field:none]
> > > <- "rkisp1_params":0 [ENABLED,IMMUTABLE]
> > > pad2: Source [fmt:YUYV8_2X8/640x480 field:none colorspace:raw xfer:none ycbcr:601 quantization:lim-range crop.bounds:(0,0)/640x480 crop:(0,0)/640x480]
> > > -> "rkisp1_resizer_mainpath":0 [ENABLED]
> > > pad3: Source [fmt:unknown/0x0 field:none]
> > > -> "rkisp1_stats":0 [ENABLED,IMMUTABLE]
> > > 
> > > - entity 6: rkisp1_resizer_mainpath (2 pads, 2 links)
> > >             type V4L2 subdev subtype Unknown flags 0
> > >             device node name /dev/v4l-subdev1
> > > pad0: Sink [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480]
> > > <- "rkisp1_isp":2 [ENABLED]
> > > pad1: Source [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> > > -> "rkisp1_mainpath":0 [ENABLED,IMMUTABLE]
> > > 
> > > - entity 9: rkisp1_mainpath (1 pad, 1 link)
> > >             type Node subtype V4L flags 0
> > >             device node name /dev/video0
> > > pad0: Sink
> > > <- "rkisp1_resizer_mainpath":1 [ENABLED,IMMUTABLE]
> > > 
> > > - entity 13: rkisp1_stats (1 pad, 1 link)
> > >              type Node subtype V4L flags 0
> > >              device node name /dev/video1
> > > pad0: Sink
> > > <- "rkisp1_isp":3 [ENABLED,IMMUTABLE]
> > > 
> > > - entity 17: rkisp1_params (1 pad, 1 link)
> > >              type Node subtype V4L flags 0
> > >              device node name /dev/video2
> > > pad0: Source
> > > -> "rkisp1_isp":1 [ENABLED,IMMUTABLE]
> > > 
> > > - entity 29: csis-32e40000.csi (2 pads, 2 links)
> > >              type V4L2 subdev subtype Unknown flags 0
> > >              device node name /dev/v4l-subdev2
> > > pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
> > > <- "imx219 1-0010":0 [ENABLED]
> > > pad1: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
> > > -> "rkisp1_isp":0 [ENABLED]
> > > 
> > > - entity 34: imx219 1-0010 (1 pad, 1 link)
> > >              type V4L2 subdev subtype Sensor flags 0
> > >              device node name /dev/v4l-subdev3
> > > pad0: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range crop.bounds:(8,8)/3280x2464 crop:(688,700)/1920x1080]
> > > -> "csis-32e40000.csi":0 [ENABLED]
> > > 
> > > It's at this point that everything except the ISP source is 1920x1080.
> > > 
> > > When I try to set the ISP sink to 1080, it ends up being 640x480 and
> > > the resizer sink is also changed to 640x480
> > > 
> > > root@beacon-imx8mp-kit:~# media-ctl -v -V "'rkisp1_isp':2
> > > [fmt:YUYV8_2X8/1920x1080 field:none]"
> > > Opening media device /dev/media0
> > > Enumerating entities
> > > looking up device: 81:3
> > > looking up device: 81:4
> > > looking up device: 81:0
> > > looking up device: 81:1
> > > looking up device: 81:2
> > > looking up device: 81:5
> > > looking up device: 81:6
> > > Found 7 entities
> > > Enumerating pads and links
> > > Setting up format YUYV8_2X8 1920x1080 on pad rkisp1_isp/2
> > > Format set: YUYV8_2X8 640x480
> > > Setting up format YUYV8_2X8 640x480 on pad rkisp1_resizer_mainpath/0
> > > Format set: YUYV8_2X8 640x480
> > > 
> > > 
> > > It's my understanding that the ISP should be able to handle 1920x1080,
> > > and the resizer sink should match the ISP source.
> > > 
> > > With the pipeline improperly setup, the capture fails.
> > > 
> > > >  .../bindings/media/rockchip-isp1.yaml         |  79 ++++++++++-
> > > >  .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++---
> > > >  .../platform/rockchip/rkisp1/rkisp1-common.h  |  32 +++++
> > > >  .../platform/rockchip/rkisp1/rkisp1-debug.c   |  14 +-
> > > >  .../platform/rockchip/rkisp1/rkisp1-dev.c     |  67 +++++++--
> > > >  .../platform/rockchip/rkisp1/rkisp1-isp.c     | 128 +++++++++++++++++-
> > > >  .../platform/rockchip/rkisp1/rkisp1-regs.h    |  90 ++++++++++++
> > > >  .../platform/rockchip/rkisp1/rkisp1-resizer.c |  35 ++++-
> > > >  include/uapi/linux/rkisp1-config.h            |   2 +
> > > >  9 files changed, 509 insertions(+), 40 deletions(-)
> > 
> > --
> > Regards,
> > 
> > Laurent Pinchart
  

On Fri, Feb 24, 2023 at 12:24 PM Nicolas Dufresne <nicolas@ndufresne.ca> wrote:
>
> Hi Adam,
>
> Le jeudi 23 février 2023 à 10:10 -0600, Adam Ford a écrit :
> > On Thu, Feb 23, 2023 at 8:26 AM Laurent Pinchart
> > <laurent.pinchart@ideasonboard.com> wrote:
> > >
> > > Hi Adam,
> > >
> > > On Wed, Feb 22, 2023 at 05:39:30PM -0600, Adam Ford wrote:
> > > > On Fri, Nov 18, 2022 at 3:44 AM Paul Elder wrote:
> > > > >
> > > > > This series depends on v3 of "dt-bindings: media: Add macros for video
> > > > > interface bus types" [1].
> > > > >
> > > > > This series extends the rkisp1 driver to support the ISP found in the
> > > > > NXP i.MX8MP SoC.
> > > > >
> > > > > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1")
> > > > > and in the NXP i.MX8MP have the same origin, and have slightly diverged
> > > > > over time as they are now independently developed (afaik) by Rockchip
> > > > > and VeriSilicon. The latter is marketed under the name "ISP8000Nano",
> > > > > and is close enough to the RK3399 ISP that it can easily be supported by
> > > > > the same driver.
> > > > >
> > > > > The last two patches add support for UYVY output format, which can be
> > > > > implemented on the ISP version in the i.MX8MP but not in the one in the
> > > > > RK3399.
> > > > >
> > > > > This version of the series specifically has been tested on a Polyhex
> > > > > Debix model A with an imx219 (Raspberry Pi cam v2).
> > > > >
> > > > > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/
> > > > >
> > > > > Laurent Pinchart (3):
> > > > >   dt-bindings: media: rkisp1: Add i.MX8MP ISP example
> > > > >   media: rkisp1: Add and use rkisp1_has_feature() macro
> > > > >   media: rkisp1: Configure gasket on i.MX8MP
> > > > >
> > > > > Paul Elder (11):
> > > > >   dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible
> > > > >   media: rkisp1: Add match data for i.MX8MP ISP
> > > > >   media: rkisp1: Add and set registers for crop for i.MX8MP
> > > > >   media: rkisp1: Add and set registers for output size config on i.MX8MP
> > > > >   media: rkisp1: Add i.MX8MP-specific registers for MI and resizer
> > > > >   media: rkisp1: Shift DMA buffer addresses on i.MX8MP
> > > > >   media: rkisp1: Add register definitions for the test pattern generator
> > > > >   media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP
> > > > >   media: rkisp1: Support devices without self path
> > > > >   media: rkisp1: Add YC swap capability
> > > > >   media: rkisp1: Add UYVY as an output format
> > > >
> > > > Paul / Laurent,
> > > >
> > > > I noticed an unexpected behaviour on the imx8mp.
> > > >
> > > > If I setup my pipeline for 640x480, it works just fine using an imx219
> > > > camera configured for SRGGB10_1X10.
> > > >
> > > > However, when I try to configure the pipeline to use the same camera
> > > > at 1920x1080 (no resizing), the ISP source keeps defaulting to 640x480
> > > >
> > > > Media device information
> > > > ------------------------
> > > > driver          rkisp1
> > > > model           rkisp1
> > > > serial
> > > > bus info        platform:rkisp1
> > > > hw revision     0xe
> > > > driver version  6.2.0
> > > >
> > > > Device topology
> > > > - entity 1: rkisp1_isp (4 pads, 4 links)
> > > >             type V4L2 subdev subtype Unknown flags 0
> > > >             device node name /dev/v4l-subdev0
> > > > pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480]
> > >
> > > You're cropping the image to 640x480 here. You need to set the crop
> > > rectangle to 1920x1080.
> > >
> > > As Jacopo mentioned, I wouldn't recommend exercising the ISP directly.
> > > Not only do you need to setup the pipeline, but you would also need to
> > > implement all the imaging algorithms in userspace. libcamera will do all
> > > this for you.
> >
> > I'll give that a try.  My current employer has a v4l2src requirement,
> > but I can likely make an argument to switch to libcamera.  I didn't
> > catch the cropping part. Thanks for that.
>
> I'd hope you can transparently replace v4l2src with libcamerasrc, the plugins
> currently lives inside the libcamera project. If not, I'd really like to know
> why. We can work together on adding missing controls (this is something I'm
> starting on soon).

I plan to give it a try.  From what I've read it appears to be the
right thing to do.  I just need to carve out some time to get it
installed.  I mostly wanted to check out a camera adapter board my
company made, test some updates I pushed for the imx219 on a second
platform, and get more familiar with the ISP on the 8MP.

I'll open a separate thread if I have questions on the cameralib.
Thanks for all the feedback.  I look forward to seeing this driver
merged.

adam
>
> regards,
> Nicolas
>
> >
> > adam
> > >
> > > > <- "csis-32e40000.csi":1 [ENABLED]
> > > > pad1: Sink [fmt:unknown/0x0 field:none]
> > > > <- "rkisp1_params":0 [ENABLED,IMMUTABLE]
> > > > pad2: Source [fmt:YUYV8_2X8/640x480 field:none colorspace:raw xfer:none ycbcr:601 quantization:lim-range crop.bounds:(0,0)/640x480 crop:(0,0)/640x480]
> > > > -> "rkisp1_resizer_mainpath":0 [ENABLED]
> > > > pad3: Source [fmt:unknown/0x0 field:none]
> > > > -> "rkisp1_stats":0 [ENABLED,IMMUTABLE]
> > > >
> > > > - entity 6: rkisp1_resizer_mainpath (2 pads, 2 links)
> > > >             type V4L2 subdev subtype Unknown flags 0
> > > >             device node name /dev/v4l-subdev1
> > > > pad0: Sink [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480]
> > > > <- "rkisp1_isp":2 [ENABLED]
> > > > pad1: Source [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> > > > -> "rkisp1_mainpath":0 [ENABLED,IMMUTABLE]
> > > >
> > > > - entity 9: rkisp1_mainpath (1 pad, 1 link)
> > > >             type Node subtype V4L flags 0
> > > >             device node name /dev/video0
> > > > pad0: Sink
> > > > <- "rkisp1_resizer_mainpath":1 [ENABLED,IMMUTABLE]
> > > >
> > > > - entity 13: rkisp1_stats (1 pad, 1 link)
> > > >              type Node subtype V4L flags 0
> > > >              device node name /dev/video1
> > > > pad0: Sink
> > > > <- "rkisp1_isp":3 [ENABLED,IMMUTABLE]
> > > >
> > > > - entity 17: rkisp1_params (1 pad, 1 link)
> > > >              type Node subtype V4L flags 0
> > > >              device node name /dev/video2
> > > > pad0: Source
> > > > -> "rkisp1_isp":1 [ENABLED,IMMUTABLE]
> > > >
> > > > - entity 29: csis-32e40000.csi (2 pads, 2 links)
> > > >              type V4L2 subdev subtype Unknown flags 0
> > > >              device node name /dev/v4l-subdev2
> > > > pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
> > > > <- "imx219 1-0010":0 [ENABLED]
> > > > pad1: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
> > > > -> "rkisp1_isp":0 [ENABLED]
> > > >
> > > > - entity 34: imx219 1-0010 (1 pad, 1 link)
> > > >              type V4L2 subdev subtype Sensor flags 0
> > > >              device node name /dev/v4l-subdev3
> > > > pad0: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range crop.bounds:(8,8)/3280x2464 crop:(688,700)/1920x1080]
> > > > -> "csis-32e40000.csi":0 [ENABLED]
> > > >
> > > > It's at this point that everything except the ISP source is 1920x1080.
> > > >
> > > > When I try to set the ISP sink to 1080, it ends up being 640x480 and
> > > > the resizer sink is also changed to 640x480
> > > >
> > > > root@beacon-imx8mp-kit:~# media-ctl -v -V "'rkisp1_isp':2
> > > > [fmt:YUYV8_2X8/1920x1080 field:none]"
> > > > Opening media device /dev/media0
> > > > Enumerating entities
> > > > looking up device: 81:3
> > > > looking up device: 81:4
> > > > looking up device: 81:0
> > > > looking up device: 81:1
> > > > looking up device: 81:2
> > > > looking up device: 81:5
> > > > looking up device: 81:6
> > > > Found 7 entities
> > > > Enumerating pads and links
> > > > Setting up format YUYV8_2X8 1920x1080 on pad rkisp1_isp/2
> > > > Format set: YUYV8_2X8 640x480
> > > > Setting up format YUYV8_2X8 640x480 on pad rkisp1_resizer_mainpath/0
> > > > Format set: YUYV8_2X8 640x480
> > > >
> > > >
> > > > It's my understanding that the ISP should be able to handle 1920x1080,
> > > > and the resizer sink should match the ISP source.
> > > >
> > > > With the pipeline improperly setup, the capture fails.
> > > >
> > > > >  .../bindings/media/rockchip-isp1.yaml         |  79 ++++++++++-
> > > > >  .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++---
> > > > >  .../platform/rockchip/rkisp1/rkisp1-common.h  |  32 +++++
> > > > >  .../platform/rockchip/rkisp1/rkisp1-debug.c   |  14 +-
> > > > >  .../platform/rockchip/rkisp1/rkisp1-dev.c     |  67 +++++++--
> > > > >  .../platform/rockchip/rkisp1/rkisp1-isp.c     | 128 +++++++++++++++++-
> > > > >  .../platform/rockchip/rkisp1/rkisp1-regs.h    |  90 ++++++++++++
> > > > >  .../platform/rockchip/rkisp1/rkisp1-resizer.c |  35 ++++-
> > > > >  include/uapi/linux/rkisp1-config.h            |   2 +
> > > > >  9 files changed, 509 insertions(+), 40 deletions(-)
> > >
> > > --
> > > Regards,
> > >
> > > Laurent Pinchart
>
  

Hello Paul,

On Fri, Nov 18, 2022 at 06:39:17PM +0900, Paul Elder wrote:
> This series depends on v3 of "dt-bindings: media: Add macros for video
> interface bus types" [1].
> 
> This series extends the rkisp1 driver to support the ISP found in the
> NXP i.MX8MP SoC.
> 
> The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1")
> and in the NXP i.MX8MP have the same origin, and have slightly diverged
> over time as they are now independently developed (afaik) by Rockchip
> and VeriSilicon. The latter is marketed under the name "ISP8000Nano",
> and is close enough to the RK3399 ISP that it can easily be supported by
> the same driver.
> 
> The last two patches add support for UYVY output format, which can be
> implemented on the ISP version in the i.MX8MP but not in the one in the
> RK3399.
> 
> This version of the series specifically has been tested on a Polyhex
> Debix model A with an imx219 (Raspberry Pi cam v2).
> 
> [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/

I tested your series on imx274 on imx8mp-evk csi0.
All looks good on my side.
Thanks for your work!

Tested-by: Tommaso Merciai <tomm.merciai@gmail.com>

Regards,
Tommaso

> 
> Laurent Pinchart (3):
>   dt-bindings: media: rkisp1: Add i.MX8MP ISP example
>   media: rkisp1: Add and use rkisp1_has_feature() macro
>   media: rkisp1: Configure gasket on i.MX8MP
> 
> Paul Elder (11):
>   dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible
>   media: rkisp1: Add match data for i.MX8MP ISP
>   media: rkisp1: Add and set registers for crop for i.MX8MP
>   media: rkisp1: Add and set registers for output size config on i.MX8MP
>   media: rkisp1: Add i.MX8MP-specific registers for MI and resizer
>   media: rkisp1: Shift DMA buffer addresses on i.MX8MP
>   media: rkisp1: Add register definitions for the test pattern generator
>   media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP
>   media: rkisp1: Support devices without self path
>   media: rkisp1: Add YC swap capability
>   media: rkisp1: Add UYVY as an output format
> 
>  .../bindings/media/rockchip-isp1.yaml         |  79 ++++++++++-
>  .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++---
>  .../platform/rockchip/rkisp1/rkisp1-common.h  |  32 +++++
>  .../platform/rockchip/rkisp1/rkisp1-debug.c   |  14 +-
>  .../platform/rockchip/rkisp1/rkisp1-dev.c     |  67 +++++++--
>  .../platform/rockchip/rkisp1/rkisp1-isp.c     | 128 +++++++++++++++++-
>  .../platform/rockchip/rkisp1/rkisp1-regs.h    |  90 ++++++++++++
>  .../platform/rockchip/rkisp1/rkisp1-resizer.c |  35 ++++-
>  include/uapi/linux/rkisp1-config.h            |   2 +
>  9 files changed, 509 insertions(+), 40 deletions(-)
> 
> -- 
> 2.35.1
>
  

Hi Paul,

On 18/11/2022 10:39, Paul Elder wrote:
> This series depends on v3 of "dt-bindings: media: Add macros for video
> interface bus types" [1].
> 
> This series extends the rkisp1 driver to support the ISP found in the
> NXP i.MX8MP SoC.
> 
> The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1")
> and in the NXP i.MX8MP have the same origin, and have slightly diverged
> over time as they are now independently developed (afaik) by Rockchip
> and VeriSilicon. The latter is marketed under the name "ISP8000Nano",
> and is close enough to the RK3399 ISP that it can easily be supported by
> the same driver.
> 
> The last two patches add support for UYVY output format, which can be
> implemented on the ISP version in the i.MX8MP but not in the one in the
> RK3399.
> 
> This version of the series specifically has been tested on a Polyhex
> Debix model A with an imx219 (Raspberry Pi cam v2).

There were comments for the first few patches, but I haven't seen a v4.

I'm marking this series as 'Changes Requested' in patchwork, just so you
know.

Regards,

	Hans

> 
> [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/
> 
> Laurent Pinchart (3):
>   dt-bindings: media: rkisp1: Add i.MX8MP ISP example
>   media: rkisp1: Add and use rkisp1_has_feature() macro
>   media: rkisp1: Configure gasket on i.MX8MP
> 
> Paul Elder (11):
>   dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible
>   media: rkisp1: Add match data for i.MX8MP ISP
>   media: rkisp1: Add and set registers for crop for i.MX8MP
>   media: rkisp1: Add and set registers for output size config on i.MX8MP
>   media: rkisp1: Add i.MX8MP-specific registers for MI and resizer
>   media: rkisp1: Shift DMA buffer addresses on i.MX8MP
>   media: rkisp1: Add register definitions for the test pattern generator
>   media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP
>   media: rkisp1: Support devices without self path
>   media: rkisp1: Add YC swap capability
>   media: rkisp1: Add UYVY as an output format
> 
>  .../bindings/media/rockchip-isp1.yaml         |  79 ++++++++++-
>  .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++---
>  .../platform/rockchip/rkisp1/rkisp1-common.h  |  32 +++++
>  .../platform/rockchip/rkisp1/rkisp1-debug.c   |  14 +-
>  .../platform/rockchip/rkisp1/rkisp1-dev.c     |  67 +++++++--
>  .../platform/rockchip/rkisp1/rkisp1-isp.c     | 128 +++++++++++++++++-
>  .../platform/rockchip/rkisp1/rkisp1-regs.h    |  90 ++++++++++++
>  .../platform/rockchip/rkisp1/rkisp1-resizer.c |  35 ++++-
>  include/uapi/linux/rkisp1-config.h            |   2 +
>  9 files changed, 509 insertions(+), 40 deletions(-)
>