[v2,00/15] staging: vc04_services: bcm2835-isp support

  This series aims to upport bcm2835-isp from the RPi kernel.
It is developed on top of staging-next which comprises many
VC04 changes for it's de-staging. Hence, the merge of this
driver is targeted when VC04 is de-staged completely (which I
have been pushing), but it can be helped getting reviewed meanwhile.
Hence, the reason for posting the series.

Patch (01-02)/15  adds a new driver named vc-sm-cma to handle memory sharing
with the VC4 VPU. 

Patch 03/15 adds a small extension to videobuf2 to allow exporting as a
dma_buf instead of a file-descriptor.

Patch (04-05)/15 adds a couple of improvements/support for
bcm2835-isp(event callback and zero-copy) to vchiq-mmal.

Patch (06-10)/15 adds the core bcm2835-isp driver along with headers
and format defintions. 09/15 is a standalone patch which can be merged
independently I believe.

Patch (11-12)/15 deals with the colorspace support.

Patch 13/15 allows multiple instances of the ISP.

Patch 14/15 adds a admin-guide document on bcm2835-isp.

Patch 15/15 deals with driver registeration.

Dave Stevenson (6):
  staging: vc04_services: Add new vc-sm-cma driver
  media: videobuf2: Allow exporting of a struct dmabuf
  staging: mmal-vchiq: Add support for event callbacks
  staging: mmal-vchiq: Use vc-sm-cma to support zero copy
  staging: mmal_vchiq: Add image formats to be used by bcm2835-isp
  uapi: bcm2835-isp: Add bcm2835-isp uapi header file

David Plowman (2):
  vc04_services: bcm2835-isp: Allow formats with different colour spaces
  vc04_services: bcm2835-isp: Permit all sRGB colour spaces on ISP
    outputs

Naushir Patuck (4):
  media: uapi: v4l2-core: Add ISP statistics output V4L2 fourcc type
  staging: vc04_services: bcm2835-isp: Add a more complex ISP processing
    component
  staging: vc04_services: bcm2835_isp: Allow multiple users
  docs: admin-guide: media: bcm2835-isp: Add documentation for
    bcm2835-isp

Umang Jain (3):
  staging: vc04_services: vchiq_arm: Register vcsm-cma driver
  staging: vc04_services: Add helpers for vchiq driver data
  staging: vc04_services: vchiq: Register bcm2835-isp

 .../admin-guide/media/bcm2835-isp.rst         |  127 ++
 .../userspace-api/media/drivers/index.rst     |    1 +
 .../userspace-api/media/v4l/meta-formats.rst  |    1 +
 .../v4l/pixfmt-meta-bcm2835-isp-stats.rst     |   32 +
 MAINTAINERS                                   |    9 +
 .../media/common/videobuf2/videobuf2-core.c   |   36 +-
 drivers/media/v4l2-core/v4l2-ioctl.c          |    1 +
 drivers/staging/vc04_services/Kconfig         |    4 +
 drivers/staging/vc04_services/Makefile        |    3 +-
 .../staging/vc04_services/bcm2835-isp/Kconfig |   14 +
 .../vc04_services/bcm2835-isp/Makefile        |    4 +
 .../bcm2835-isp/bcm2835-isp-ctrls.h           |   72 +
 .../bcm2835-isp/bcm2835-isp-fmts.h            |  559 +++++
 .../bcm2835-isp/bcm2835-v4l2-isp.c            | 1822 +++++++++++++++++
 .../interface/vchiq_arm/vchiq_arm.c           |    6 +
 .../interface/vchiq_arm/vchiq_bus.h           |   10 +
 .../staging/vc04_services/vc-sm-cma/Kconfig   |   10 +
 .../staging/vc04_services/vc-sm-cma/Makefile  |    4 +
 .../staging/vc04_services/vc-sm-cma/vc_sm.c   |  817 ++++++++
 .../staging/vc04_services/vc-sm-cma/vc_sm.h   |   54 +
 .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c  |  507 +++++
 .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h  |   63 +
 .../vc04_services/vc-sm-cma/vc_sm_defs.h      |  187 ++
 .../vc04_services/vc-sm-cma/vc_sm_knl.h       |   28 +
 .../staging/vc04_services/vchiq-mmal/Kconfig  |    1 +
 .../vc04_services/vchiq-mmal/mmal-common.h    |    5 +
 .../vc04_services/vchiq-mmal/mmal-encodings.h |   66 +
 .../vc04_services/vchiq-mmal/mmal-msg.h       |   35 +
 .../vchiq-mmal/mmal-parameters.h              |  165 +-
 .../vc04_services/vchiq-mmal/mmal-vchiq.c     |  253 ++-
 .../vc04_services/vchiq-mmal/mmal-vchiq.h     |    5 +
 include/media/videobuf2-core.h                |   15 +
 include/uapi/linux/bcm2835-isp.h              |  347 ++++
 include/uapi/linux/v4l2-controls.h            |    5 +
 include/uapi/linux/videodev2.h                |    1 +
 35 files changed, 5235 insertions(+), 34 deletions(-)
 create mode 100644 Documentation/admin-guide/media/bcm2835-isp.rst
 create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
 create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
 create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
 create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-ctrls.h
 create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
 create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
 create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
 create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
 create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
 create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
 create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
 create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
 create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
 create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
 create mode 100644 include/uapi/linux/bcm2835-isp.h
  

Hi Umang,

Am 09.11.23 um 22:02 schrieb Umang Jain:
> This series aims to upport bcm2835-isp from the RPi kernel.
> It is developed on top of staging-next which comprises many
> VC04 changes for it's de-staging. Hence, the merge of this
> driver is targeted when VC04 is de-staged completely (which I
> have been pushing), but it can be helped getting reviewed meanwhile.
> Hence, the reason for posting the series.
>
> Patch (01-02)/15  adds a new driver named vc-sm-cma to handle memory sharing
> with the VC4 VPU.
>
> Patch 03/15 adds a small extension to videobuf2 to allow exporting as a
> dma_buf instead of a file-descriptor.
>
> Patch (04-05)/15 adds a couple of improvements/support for
> bcm2835-isp(event callback and zero-copy) to vchiq-mmal.
>
> Patch (06-10)/15 adds the core bcm2835-isp driver along with headers
> and format defintions. 09/15 is a standalone patch which can be merged
> independently I believe.
>
> Patch (11-12)/15 deals with the colorspace support.
>
> Patch 13/15 allows multiple instances of the ISP.
>
> Patch 14/15 adds a admin-guide document on bcm2835-isp.
>
> Patch 15/15 deals with driver registeration.

since this is V2, i couldn't find a changelog? The first one was one
year ago, so it's very hard to remember. In such a case a link to the
last version is also helpful.

Recently i saw you have a Github account. For such a complex series it's
also very helpful to provide a link to a working test branch with this
series applied. It's not a must, but appreciated.

Best regards

>
> Dave Stevenson (6):
>    staging: vc04_services: Add new vc-sm-cma driver
>    media: videobuf2: Allow exporting of a struct dmabuf
>    staging: mmal-vchiq: Add support for event callbacks
>    staging: mmal-vchiq: Use vc-sm-cma to support zero copy
>    staging: mmal_vchiq: Add image formats to be used by bcm2835-isp
>    uapi: bcm2835-isp: Add bcm2835-isp uapi header file
>
> David Plowman (2):
>    vc04_services: bcm2835-isp: Allow formats with different colour spaces
>    vc04_services: bcm2835-isp: Permit all sRGB colour spaces on ISP
>      outputs
>
> Naushir Patuck (4):
>    media: uapi: v4l2-core: Add ISP statistics output V4L2 fourcc type
>    staging: vc04_services: bcm2835-isp: Add a more complex ISP processing
>      component
>    staging: vc04_services: bcm2835_isp: Allow multiple users
>    docs: admin-guide: media: bcm2835-isp: Add documentation for
>      bcm2835-isp
>
> Umang Jain (3):
>    staging: vc04_services: vchiq_arm: Register vcsm-cma driver
>    staging: vc04_services: Add helpers for vchiq driver data
>    staging: vc04_services: vchiq: Register bcm2835-isp
>
>   .../admin-guide/media/bcm2835-isp.rst         |  127 ++
>   .../userspace-api/media/drivers/index.rst     |    1 +
>   .../userspace-api/media/v4l/meta-formats.rst  |    1 +
>   .../v4l/pixfmt-meta-bcm2835-isp-stats.rst     |   32 +
>   MAINTAINERS                                   |    9 +
>   .../media/common/videobuf2/videobuf2-core.c   |   36 +-
>   drivers/media/v4l2-core/v4l2-ioctl.c          |    1 +
>   drivers/staging/vc04_services/Kconfig         |    4 +
>   drivers/staging/vc04_services/Makefile        |    3 +-
>   .../staging/vc04_services/bcm2835-isp/Kconfig |   14 +
>   .../vc04_services/bcm2835-isp/Makefile        |    4 +
>   .../bcm2835-isp/bcm2835-isp-ctrls.h           |   72 +
>   .../bcm2835-isp/bcm2835-isp-fmts.h            |  559 +++++
>   .../bcm2835-isp/bcm2835-v4l2-isp.c            | 1822 +++++++++++++++++
>   .../interface/vchiq_arm/vchiq_arm.c           |    6 +
>   .../interface/vchiq_arm/vchiq_bus.h           |   10 +
>   .../staging/vc04_services/vc-sm-cma/Kconfig   |   10 +
>   .../staging/vc04_services/vc-sm-cma/Makefile  |    4 +
>   .../staging/vc04_services/vc-sm-cma/vc_sm.c   |  817 ++++++++
>   .../staging/vc04_services/vc-sm-cma/vc_sm.h   |   54 +
>   .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c  |  507 +++++
>   .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h  |   63 +
>   .../vc04_services/vc-sm-cma/vc_sm_defs.h      |  187 ++
>   .../vc04_services/vc-sm-cma/vc_sm_knl.h       |   28 +
>   .../staging/vc04_services/vchiq-mmal/Kconfig  |    1 +
>   .../vc04_services/vchiq-mmal/mmal-common.h    |    5 +
>   .../vc04_services/vchiq-mmal/mmal-encodings.h |   66 +
>   .../vc04_services/vchiq-mmal/mmal-msg.h       |   35 +
>   .../vchiq-mmal/mmal-parameters.h              |  165 +-
>   .../vc04_services/vchiq-mmal/mmal-vchiq.c     |  253 ++-
>   .../vc04_services/vchiq-mmal/mmal-vchiq.h     |    5 +
>   include/media/videobuf2-core.h                |   15 +
>   include/uapi/linux/bcm2835-isp.h              |  347 ++++
>   include/uapi/linux/v4l2-controls.h            |    5 +
>   include/uapi/linux/videodev2.h                |    1 +
>   35 files changed, 5235 insertions(+), 34 deletions(-)
>   create mode 100644 Documentation/admin-guide/media/bcm2835-isp.rst
>   create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
>   create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
>   create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
>   create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-ctrls.h
>   create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
>   create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
>   create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
>   create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
>   create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
>   create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
>   create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
>   create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
>   create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
>   create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
>   create mode 100644 include/uapi/linux/bcm2835-isp.h
>
  

Hi Stefan,

On 11/10/23 4:02 PM, Stefan Wahren wrote:
> Hi Umang,
>
> Am 09.11.23 um 22:02 schrieb Umang Jain:
>> This series aims to upport bcm2835-isp from the RPi kernel.
>> It is developed on top of staging-next which comprises many
>> VC04 changes for it's de-staging. Hence, the merge of this
>> driver is targeted when VC04 is de-staged completely (which I
>> have been pushing), but it can be helped getting reviewed meanwhile.
>> Hence, the reason for posting the series.
>>
>> Patch (01-02)/15  adds a new driver named vc-sm-cma to handle memory 
>> sharing
>> with the VC4 VPU.
>>
>> Patch 03/15 adds a small extension to videobuf2 to allow exporting as a
>> dma_buf instead of a file-descriptor.
>>
>> Patch (04-05)/15 adds a couple of improvements/support for
>> bcm2835-isp(event callback and zero-copy) to vchiq-mmal.
>>
>> Patch (06-10)/15 adds the core bcm2835-isp driver along with headers
>> and format defintions. 09/15 is a standalone patch which can be merged
>> independently I believe.
>>
>> Patch (11-12)/15 deals with the colorspace support.
>>
>> Patch 13/15 allows multiple instances of the ISP.
>>
>> Patch 14/15 adds a admin-guide document on bcm2835-isp.
>>
>> Patch 15/15 deals with driver registeration.
>
> since this is V2, i couldn't find a changelog? The first one was one
> year ago, so it's very hard to remember. In such a case a link to the
> last version is also helpful.

I am in your same situation. The changelog won't help here since the I 
have too have lost count of the iterations.

In brief, I would say, the patches have further cleaned up and aligned, 
rebased over the latest staging-next branch.
That majorly includes work I did for VC04 in general for de-staging.

>
> Recently i saw you have a Github account. For such a complex series it's
> also very helpful to provide a link to a working test branch with this
> series applied. It's not a must, but appreciated.

I actually was going to send test branch but at the last minute I 
screwed up my .git repo because of a failed branch checkout.

I had to recreate the branches over the weekend and here they are:

I have tested these branches:

On RPi-3-b 32-bit:
Git branch: 
https://git.uk.ideasonboard.com/uajain/linux/commits/branch/uajain/rpi3/staging-next/isp

On RPi-4-b 64-bit:
Git branch: 
https://git.uk.ideasonboard.com/uajain/linux/commits/branch/uajain/rpi4/staging-next/isp

Both these branches have been tested with IMX219 sensor (there is a DT 
overlay included) and libcamera. These branches have bcm2835-unicam with 
streams support (which is handled by Laurent Pinchart as I believe) and 
on top sits bcm2835-isp (this series).
You'll find DT and kernel configs I have tested with, in the respective 
branches.

As far as libcamera branch goes (it requires a special branch due to 
streams support):
https://git.uk.ideasonboard.com/uajain/libcamera/commits/branch/pinchartl/rpi/streams

Let me know how you get on if you are testing. I am happy to answer/help 
if you decide to test this.
>
> Best regards
>
>>
>> Dave Stevenson (6):
>>    staging: vc04_services: Add new vc-sm-cma driver
>>    media: videobuf2: Allow exporting of a struct dmabuf
>>    staging: mmal-vchiq: Add support for event callbacks
>>    staging: mmal-vchiq: Use vc-sm-cma to support zero copy
>>    staging: mmal_vchiq: Add image formats to be used by bcm2835-isp
>>    uapi: bcm2835-isp: Add bcm2835-isp uapi header file
>>
>> David Plowman (2):
>>    vc04_services: bcm2835-isp: Allow formats with different colour 
>> spaces
>>    vc04_services: bcm2835-isp: Permit all sRGB colour spaces on ISP
>>      outputs
>>
>> Naushir Patuck (4):
>>    media: uapi: v4l2-core: Add ISP statistics output V4L2 fourcc type
>>    staging: vc04_services: bcm2835-isp: Add a more complex ISP 
>> processing
>>      component
>>    staging: vc04_services: bcm2835_isp: Allow multiple users
>>    docs: admin-guide: media: bcm2835-isp: Add documentation for
>>      bcm2835-isp
>>
>> Umang Jain (3):
>>    staging: vc04_services: vchiq_arm: Register vcsm-cma driver
>>    staging: vc04_services: Add helpers for vchiq driver data
>>    staging: vc04_services: vchiq: Register bcm2835-isp
>>
>>   .../admin-guide/media/bcm2835-isp.rst         |  127 ++
>>   .../userspace-api/media/drivers/index.rst     |    1 +
>>   .../userspace-api/media/v4l/meta-formats.rst  |    1 +
>>   .../v4l/pixfmt-meta-bcm2835-isp-stats.rst     |   32 +
>>   MAINTAINERS                                   |    9 +
>>   .../media/common/videobuf2/videobuf2-core.c   |   36 +-
>>   drivers/media/v4l2-core/v4l2-ioctl.c          |    1 +
>>   drivers/staging/vc04_services/Kconfig         |    4 +
>>   drivers/staging/vc04_services/Makefile        |    3 +-
>>   .../staging/vc04_services/bcm2835-isp/Kconfig |   14 +
>>   .../vc04_services/bcm2835-isp/Makefile        |    4 +
>>   .../bcm2835-isp/bcm2835-isp-ctrls.h           |   72 +
>>   .../bcm2835-isp/bcm2835-isp-fmts.h            |  559 +++++
>>   .../bcm2835-isp/bcm2835-v4l2-isp.c            | 1822 +++++++++++++++++
>>   .../interface/vchiq_arm/vchiq_arm.c           |    6 +
>>   .../interface/vchiq_arm/vchiq_bus.h           |   10 +
>>   .../staging/vc04_services/vc-sm-cma/Kconfig   |   10 +
>>   .../staging/vc04_services/vc-sm-cma/Makefile  |    4 +
>>   .../staging/vc04_services/vc-sm-cma/vc_sm.c   |  817 ++++++++
>>   .../staging/vc04_services/vc-sm-cma/vc_sm.h   |   54 +
>>   .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c  |  507 +++++
>>   .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h  |   63 +
>>   .../vc04_services/vc-sm-cma/vc_sm_defs.h      |  187 ++
>>   .../vc04_services/vc-sm-cma/vc_sm_knl.h       |   28 +
>>   .../staging/vc04_services/vchiq-mmal/Kconfig  |    1 +
>>   .../vc04_services/vchiq-mmal/mmal-common.h    |    5 +
>>   .../vc04_services/vchiq-mmal/mmal-encodings.h |   66 +
>>   .../vc04_services/vchiq-mmal/mmal-msg.h       |   35 +
>>   .../vchiq-mmal/mmal-parameters.h              |  165 +-
>>   .../vc04_services/vchiq-mmal/mmal-vchiq.c     |  253 ++-
>>   .../vc04_services/vchiq-mmal/mmal-vchiq.h     |    5 +
>>   include/media/videobuf2-core.h                |   15 +
>>   include/uapi/linux/bcm2835-isp.h              |  347 ++++
>>   include/uapi/linux/v4l2-controls.h            |    5 +
>>   include/uapi/linux/videodev2.h                |    1 +
>>   35 files changed, 5235 insertions(+), 34 deletions(-)
>>   create mode 100644 Documentation/admin-guide/media/bcm2835-isp.rst
>>   create mode 100644 
>> Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
>>   create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
>>   create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
>>   create mode 100644 
>> drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-ctrls.h
>>   create mode 100644 
>> drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
>>   create mode 100644 
>> drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
>>   create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
>>   create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
>>   create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
>>   create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
>>   create mode 100644 
>> drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
>>   create mode 100644 
>> drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
>>   create mode 100644 
>> drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
>>   create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
>>   create mode 100644 include/uapi/linux/bcm2835-isp.h
>>
  

Hi Umang,

Am 13.11.23 um 14:38 schrieb Umang Jain:
> Hi Stefan,
>
> On 11/10/23 4:02 PM, Stefan Wahren wrote:
>> Hi Umang,
>>
>> Am 09.11.23 um 22:02 schrieb Umang Jain:
>>> This series aims to upport bcm2835-isp from the RPi kernel.
>>> It is developed on top of staging-next which comprises many
>>> VC04 changes for it's de-staging. Hence, the merge of this
>>> driver is targeted when VC04 is de-staged completely (which I
>>> have been pushing), but it can be helped getting reviewed meanwhile.
>>> Hence, the reason for posting the series.
>>>
>>> Patch (01-02)/15  adds a new driver named vc-sm-cma to handle memory
>>> sharing
>>> with the VC4 VPU.
>>>
>>> Patch 03/15 adds a small extension to videobuf2 to allow exporting as a
>>> dma_buf instead of a file-descriptor.
>>>
>>> Patch (04-05)/15 adds a couple of improvements/support for
>>> bcm2835-isp(event callback and zero-copy) to vchiq-mmal.
>>>
>>> Patch (06-10)/15 adds the core bcm2835-isp driver along with headers
>>> and format defintions. 09/15 is a standalone patch which can be merged
>>> independently I believe.
>>>
>>> Patch (11-12)/15 deals with the colorspace support.
>>>
>>> Patch 13/15 allows multiple instances of the ISP.
>>>
>>> Patch 14/15 adds a admin-guide document on bcm2835-isp.
>>>
>>> Patch 15/15 deals with driver registeration.
>>
>> since this is V2, i couldn't find a changelog? The first one was one
>> year ago, so it's very hard to remember. In such a case a link to the
>> last version is also helpful.
>
> I am in your same situation. The changelog won't help here since the I
> have too have lost count of the iterations.
>
> In brief, I would say, the patches have further cleaned up and
> aligned, rebased over the latest staging-next branch.
> That majorly includes work I did for VC04 in general for de-staging.
since this version
https://lore.kernel.org/lkml/910dbd8f-85f2-4979-49ee-6d760e89af84@ideasonboard.com/T/#mca165d5dcb1ba9f3ff3bf91ca2bc6ec2d3ae1c29

there were some feedback from other people like Laurent. I think it's
necessary to know if all the comments has been addressed or not. I think
there is no need to list every single change.

>
>>
>> Recently i saw you have a Github account. For such a complex series it's
>> also very helpful to provide a link to a working test branch with this
>> series applied. It's not a must, but appreciated.
>
> I actually was going to send test branch but at the last minute I
> screwed up my .git repo because of a failed branch checkout.
>
> I had to recreate the branches over the weekend and here they are:
>
> I have tested these branches:
>
> On RPi-3-b 32-bit:
> Git branch:
> https://git.uk.ideasonboard.com/uajain/linux/commits/branch/uajain/rpi3/staging-next/isp
>
> On RPi-4-b 64-bit:
> Git branch:
> https://git.uk.ideasonboard.com/uajain/linux/commits/branch/uajain/rpi4/staging-next/isp
>
> Both these branches have been tested with IMX219 sensor (there is a DT
> overlay included) and libcamera. These branches have bcm2835-unicam
> with streams support (which is handled by Laurent Pinchart as I
> believe) and on top sits bcm2835-isp (this series).
> You'll find DT and kernel configs I have tested with, in the
> respective branches.
>
> As far as libcamera branch goes (it requires a special branch due to
> streams support):
> https://git.uk.ideasonboard.com/uajain/libcamera/commits/branch/pinchartl/rpi/streams
>
>
> Let me know how you get on if you are testing. I am happy to
> answer/help if you decide to test this.
This is great, i don't know when i've the time to test, but it also give
others the chance to do tests.

Regards
>>
>> Best regards
>>
>>>
>>> Dave Stevenson (6):
>>>    staging: vc04_services: Add new vc-sm-cma driver
>>>    media: videobuf2: Allow exporting of a struct dmabuf
>>>    staging: mmal-vchiq: Add support for event callbacks
>>>    staging: mmal-vchiq: Use vc-sm-cma to support zero copy
>>>    staging: mmal_vchiq: Add image formats to be used by bcm2835-isp
>>>    uapi: bcm2835-isp: Add bcm2835-isp uapi header file
>>>
>>> David Plowman (2):
>>>    vc04_services: bcm2835-isp: Allow formats with different colour
>>> spaces
>>>    vc04_services: bcm2835-isp: Permit all sRGB colour spaces on ISP
>>>      outputs
>>>
>>> Naushir Patuck (4):
>>>    media: uapi: v4l2-core: Add ISP statistics output V4L2 fourcc type
>>>    staging: vc04_services: bcm2835-isp: Add a more complex ISP
>>> processing
>>>      component
>>>    staging: vc04_services: bcm2835_isp: Allow multiple users
>>>    docs: admin-guide: media: bcm2835-isp: Add documentation for
>>>      bcm2835-isp
>>>
>>> Umang Jain (3):
>>>    staging: vc04_services: vchiq_arm: Register vcsm-cma driver
>>>    staging: vc04_services: Add helpers for vchiq driver data
>>>    staging: vc04_services: vchiq: Register bcm2835-isp
>>>
>>>   .../admin-guide/media/bcm2835-isp.rst         |  127 ++
>>>   .../userspace-api/media/drivers/index.rst     |    1 +
>>>   .../userspace-api/media/v4l/meta-formats.rst  |    1 +
>>>   .../v4l/pixfmt-meta-bcm2835-isp-stats.rst     |   32 +
>>>   MAINTAINERS                                   |    9 +
>>>   .../media/common/videobuf2/videobuf2-core.c   |   36 +-
>>>   drivers/media/v4l2-core/v4l2-ioctl.c          |    1 +
>>>   drivers/staging/vc04_services/Kconfig         |    4 +
>>>   drivers/staging/vc04_services/Makefile        |    3 +-
>>>   .../staging/vc04_services/bcm2835-isp/Kconfig |   14 +
>>>   .../vc04_services/bcm2835-isp/Makefile        |    4 +
>>>   .../bcm2835-isp/bcm2835-isp-ctrls.h           |   72 +
>>>   .../bcm2835-isp/bcm2835-isp-fmts.h            |  559 +++++
>>>   .../bcm2835-isp/bcm2835-v4l2-isp.c            | 1822
>>> +++++++++++++++++
>>>   .../interface/vchiq_arm/vchiq_arm.c           |    6 +
>>>   .../interface/vchiq_arm/vchiq_bus.h           |   10 +
>>>   .../staging/vc04_services/vc-sm-cma/Kconfig   |   10 +
>>>   .../staging/vc04_services/vc-sm-cma/Makefile  |    4 +
>>>   .../staging/vc04_services/vc-sm-cma/vc_sm.c   |  817 ++++++++
>>>   .../staging/vc04_services/vc-sm-cma/vc_sm.h   |   54 +
>>>   .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c  |  507 +++++
>>>   .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h  |   63 +
>>>   .../vc04_services/vc-sm-cma/vc_sm_defs.h      |  187 ++
>>>   .../vc04_services/vc-sm-cma/vc_sm_knl.h       |   28 +
>>>   .../staging/vc04_services/vchiq-mmal/Kconfig  |    1 +
>>>   .../vc04_services/vchiq-mmal/mmal-common.h    |    5 +
>>>   .../vc04_services/vchiq-mmal/mmal-encodings.h |   66 +
>>>   .../vc04_services/vchiq-mmal/mmal-msg.h       |   35 +
>>>   .../vchiq-mmal/mmal-parameters.h              |  165 +-
>>>   .../vc04_services/vchiq-mmal/mmal-vchiq.c     |  253 ++-
>>>   .../vc04_services/vchiq-mmal/mmal-vchiq.h     |    5 +
>>>   include/media/videobuf2-core.h                |   15 +
>>>   include/uapi/linux/bcm2835-isp.h              |  347 ++++
>>>   include/uapi/linux/v4l2-controls.h            |    5 +
>>>   include/uapi/linux/videodev2.h                |    1 +
>>>   35 files changed, 5235 insertions(+), 34 deletions(-)
>>>   create mode 100644 Documentation/admin-guide/media/bcm2835-isp.rst
>>>   create mode 100644
>>> Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
>>>   create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
>>>   create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
>>>   create mode 100644
>>> drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-ctrls.h
>>>   create mode 100644
>>> drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
>>>   create mode 100644
>>> drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
>>>   create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
>>>   create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
>>>   create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
>>>   create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
>>>   create mode 100644
>>> drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
>>>   create mode 100644
>>> drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
>>>   create mode 100644
>>> drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
>>>   create mode 100644
>>> drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
>>>   create mode 100644 include/uapi/linux/bcm2835-isp.h
>>>
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
  

Hello,

On Thu, Nov 09, 2023 at 04:02:52PM -0500, Umang Jain wrote:
> This series aims to upport bcm2835-isp from the RPi kernel.
> It is developed on top of staging-next which comprises many
> VC04 changes for it's de-staging. Hence, the merge of this
> driver is targeted when VC04 is de-staged completely (which I
> have been pushing), but it can be helped getting reviewed meanwhile.
> Hence, the reason for posting the series.

Related question, what do people think about dropping the legacy
firmware-based bcm2385-camera driver once this gets merged ?
firmware-based camera operation is deprecated by Raspberry Pi, and
doesn't work on the Pi 5.

> Patch (01-02)/15  adds a new driver named vc-sm-cma to handle memory sharing
> with the VC4 VPU. 
> 
> Patch 03/15 adds a small extension to videobuf2 to allow exporting as a
> dma_buf instead of a file-descriptor.
> 
> Patch (04-05)/15 adds a couple of improvements/support for
> bcm2835-isp(event callback and zero-copy) to vchiq-mmal.
> 
> Patch (06-10)/15 adds the core bcm2835-isp driver along with headers
> and format defintions. 09/15 is a standalone patch which can be merged
> independently I believe.
> 
> Patch (11-12)/15 deals with the colorspace support.
> 
> Patch 13/15 allows multiple instances of the ISP.
> 
> Patch 14/15 adds a admin-guide document on bcm2835-isp.
> 
> Patch 15/15 deals with driver registeration.
> 
> Dave Stevenson (6):
>   staging: vc04_services: Add new vc-sm-cma driver
>   media: videobuf2: Allow exporting of a struct dmabuf
>   staging: mmal-vchiq: Add support for event callbacks
>   staging: mmal-vchiq: Use vc-sm-cma to support zero copy
>   staging: mmal_vchiq: Add image formats to be used by bcm2835-isp
>   uapi: bcm2835-isp: Add bcm2835-isp uapi header file
> 
> David Plowman (2):
>   vc04_services: bcm2835-isp: Allow formats with different colour spaces
>   vc04_services: bcm2835-isp: Permit all sRGB colour spaces on ISP
>     outputs
> 
> Naushir Patuck (4):
>   media: uapi: v4l2-core: Add ISP statistics output V4L2 fourcc type
>   staging: vc04_services: bcm2835-isp: Add a more complex ISP processing
>     component
>   staging: vc04_services: bcm2835_isp: Allow multiple users
>   docs: admin-guide: media: bcm2835-isp: Add documentation for
>     bcm2835-isp
> 
> Umang Jain (3):
>   staging: vc04_services: vchiq_arm: Register vcsm-cma driver
>   staging: vc04_services: Add helpers for vchiq driver data
>   staging: vc04_services: vchiq: Register bcm2835-isp
> 
>  .../admin-guide/media/bcm2835-isp.rst         |  127 ++
>  .../userspace-api/media/drivers/index.rst     |    1 +
>  .../userspace-api/media/v4l/meta-formats.rst  |    1 +
>  .../v4l/pixfmt-meta-bcm2835-isp-stats.rst     |   32 +
>  MAINTAINERS                                   |    9 +
>  .../media/common/videobuf2/videobuf2-core.c   |   36 +-
>  drivers/media/v4l2-core/v4l2-ioctl.c          |    1 +
>  drivers/staging/vc04_services/Kconfig         |    4 +
>  drivers/staging/vc04_services/Makefile        |    3 +-
>  .../staging/vc04_services/bcm2835-isp/Kconfig |   14 +
>  .../vc04_services/bcm2835-isp/Makefile        |    4 +
>  .../bcm2835-isp/bcm2835-isp-ctrls.h           |   72 +
>  .../bcm2835-isp/bcm2835-isp-fmts.h            |  559 +++++
>  .../bcm2835-isp/bcm2835-v4l2-isp.c            | 1822 +++++++++++++++++
>  .../interface/vchiq_arm/vchiq_arm.c           |    6 +
>  .../interface/vchiq_arm/vchiq_bus.h           |   10 +
>  .../staging/vc04_services/vc-sm-cma/Kconfig   |   10 +
>  .../staging/vc04_services/vc-sm-cma/Makefile  |    4 +
>  .../staging/vc04_services/vc-sm-cma/vc_sm.c   |  817 ++++++++
>  .../staging/vc04_services/vc-sm-cma/vc_sm.h   |   54 +
>  .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c  |  507 +++++
>  .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h  |   63 +
>  .../vc04_services/vc-sm-cma/vc_sm_defs.h      |  187 ++
>  .../vc04_services/vc-sm-cma/vc_sm_knl.h       |   28 +
>  .../staging/vc04_services/vchiq-mmal/Kconfig  |    1 +
>  .../vc04_services/vchiq-mmal/mmal-common.h    |    5 +
>  .../vc04_services/vchiq-mmal/mmal-encodings.h |   66 +
>  .../vc04_services/vchiq-mmal/mmal-msg.h       |   35 +
>  .../vchiq-mmal/mmal-parameters.h              |  165 +-
>  .../vc04_services/vchiq-mmal/mmal-vchiq.c     |  253 ++-
>  .../vc04_services/vchiq-mmal/mmal-vchiq.h     |    5 +
>  include/media/videobuf2-core.h                |   15 +
>  include/uapi/linux/bcm2835-isp.h              |  347 ++++
>  include/uapi/linux/v4l2-controls.h            |    5 +
>  include/uapi/linux/videodev2.h                |    1 +
>  35 files changed, 5235 insertions(+), 34 deletions(-)
>  create mode 100644 Documentation/admin-guide/media/bcm2835-isp.rst
>  create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
>  create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
>  create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
>  create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-ctrls.h
>  create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
>  create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
>  create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
>  create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
>  create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
>  create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
>  create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
>  create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
>  create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
>  create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
>  create mode 100644 include/uapi/linux/bcm2835-isp.h
  

Hi Laurent,

[add Ivan & Peter]

Am 15.11.23 um 20:59 schrieb Laurent Pinchart:
> Hello,
>
> On Thu, Nov 09, 2023 at 04:02:52PM -0500, Umang Jain wrote:
>> This series aims to upport bcm2835-isp from the RPi kernel.
>> It is developed on top of staging-next which comprises many
>> VC04 changes for it's de-staging. Hence, the merge of this
>> driver is targeted when VC04 is de-staged completely (which I
>> have been pushing), but it can be helped getting reviewed meanwhile.
>> Hence, the reason for posting the series.
>
> Related question, what do people think about dropping the legacy
> firmware-based bcm2385-camera driver once this gets merged ?
> firmware-based camera operation is deprecated by Raspberry Pi, and
> doesn't work on the Pi 5
i don't remember exactly, but wasn't the bcm2835-camera required for Pi
Camera V1.3?

At the end cannot speak for the users. AFAIK OpenSuSE and Fedora use the
driver.
  

On Wed, Nov 15, 2023 at 09:57:52PM +0100, Stefan Wahren wrote:
> Hi Laurent,
> 
> [add Ivan & Peter]
> 
> Am 15.11.23 um 20:59 schrieb Laurent Pinchart:
> > Hello,
> >
> > On Thu, Nov 09, 2023 at 04:02:52PM -0500, Umang Jain wrote:
> >> This series aims to upport bcm2835-isp from the RPi kernel.
> >> It is developed on top of staging-next which comprises many
> >> VC04 changes for it's de-staging. Hence, the merge of this
> >> driver is targeted when VC04 is de-staged completely (which I
> >> have been pushing), but it can be helped getting reviewed meanwhile.
> >> Hence, the reason for posting the series.
> >
> > Related question, what do people think about dropping the legacy
> > firmware-based bcm2385-camera driver once this gets merged ?
> > firmware-based camera operation is deprecated by Raspberry Pi, and
> > doesn't work on the Pi 5
>
> i don't remember exactly, but wasn't the bcm2835-camera required for Pi
> Camera V1.3?

If I'm not mistaken (Dave can correct me), the legacy camera stack works
only with the Raspberry Pi official camera v1, v2 and HQ modules.
Raspberry Pi has switched to a new camera stack based on libcamera,
which works on the Pi Zero 2, Pi 3, Pi 4 and Pi 5. This new stack
supports the same camera modules as the legacy stack, and many more. The
legacy stack doesn't work on Pi 5 at all.

> At the end cannot speak for the users. AFAIK OpenSuSE and Fedora use the
> driver.
  

Hi,

On 2023-11-16 02:50, Laurent Pinchart wrote:
> On Wed, Nov 15, 2023 at 09:57:52PM +0100, Stefan Wahren wrote:
>> Hi Laurent,
>> 
>> [add Ivan & Peter]
>> 
>> Am 15.11.23 um 20:59 schrieb Laurent Pinchart:
>> > Hello,
>> >
>> > On Thu, Nov 09, 2023 at 04:02:52PM -0500, Umang Jain wrote:
>> >> This series aims to upport bcm2835-isp from the RPi kernel.
>> >> It is developed on top of staging-next which comprises many
>> >> VC04 changes for it's de-staging. Hence, the merge of this
>> >> driver is targeted when VC04 is de-staged completely (which I
>> >> have been pushing), but it can be helped getting reviewed meanwhile.
>> >> Hence, the reason for posting the series.
>> >
>> > Related question, what do people think about dropping the legacy
>> > firmware-based bcm2385-camera driver once this gets merged ?
>> > firmware-based camera operation is deprecated by Raspberry Pi, and
>> > doesn't work on the Pi 5
>> 
>> i don't remember exactly, but wasn't the bcm2835-camera required for 
>> Pi
>> Camera V1.3?
> 
> If I'm not mistaken (Dave can correct me), the legacy camera stack 
> works
> only with the Raspberry Pi official camera v1, v2 and HQ modules.
> Raspberry Pi has switched to a new camera stack based on libcamera,
> which works on the Pi Zero 2, Pi 3, Pi 4 and Pi 5. This new stack
> supports the same camera modules as the legacy stack, and many more. 
> The
> legacy stack doesn't work on Pi 5 at all.
> 
>> At the end cannot speak for the users. AFAIK OpenSuSE and Fedora use 
>> the
>> driver.

Guillaume (in CC) is more authoritative about this topic, but as long we 
have
smooth migration plan I am more than happy to switch to proper camera 
driver.

Regards,
Ivan
  

> >> > On Thu, Nov 09, 2023 at 04:02:52PM -0500, Umang Jain wrote:
> >> >> This series aims to upport bcm2835-isp from the RPi kernel.
> >> >> It is developed on top of staging-next which comprises many
> >> >> VC04 changes for it's de-staging. Hence, the merge of this
> >> >> driver is targeted when VC04 is de-staged completely (which I
> >> >> have been pushing), but it can be helped getting reviewed meanwhile.
> >> >> Hence, the reason for posting the series.
> >> >
> >> > Related question, what do people think about dropping the legacy
> >> > firmware-based bcm2385-camera driver once this gets merged ?
> >> > firmware-based camera operation is deprecated by Raspberry Pi, and
> >> > doesn't work on the Pi 5
> >>
> >> i don't remember exactly, but wasn't the bcm2835-camera required for
> >> Pi
> >> Camera V1.3?
> >
> > If I'm not mistaken (Dave can correct me), the legacy camera stack
> > works
> > only with the Raspberry Pi official camera v1, v2 and HQ modules.
> > Raspberry Pi has switched to a new camera stack based on libcamera,
> > which works on the Pi Zero 2, Pi 3, Pi 4 and Pi 5. This new stack
> > supports the same camera modules as the legacy stack, and many more.
> > The
> > legacy stack doesn't work on Pi 5 at all.
> >
> >> At the end cannot speak for the users. AFAIK OpenSuSE and Fedora use
> >> the
> >> driver.
>
> Guillaume (in CC) is more authoritative about this topic, but as long we
> have
> smooth migration plan I am more than happy to switch to proper camera
> driver.

From a Fedora PoV I am more than happy to move to the new libcamera
based stack for the RPi camera, IMO sooner the better :)