Message ID | 20221223231736.2111774-1-kieran.bingham@ideasonboard.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:6000:1b02:0:0:0:0 with SMTP id f2csp28926wrz; Fri, 23 Dec 2022 15:20:55 -0800 (PST) X-Google-Smtp-Source: AMrXdXvarS6fnSROXanLsZB28z/xIw0J7biJxYn7fHIjPguA0jnKfWgc9SwifhH9A2gRGXsD7hpl X-Received: by 2002:a17:907:88cb:b0:7bf:1072:2246 with SMTP id rq11-20020a17090788cb00b007bf10722246mr9295509ejc.38.1671837655164; Fri, 23 Dec 2022 15:20:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671837655; cv=none; d=google.com; s=arc-20160816; b=AqGTy8Ar18gRkvFGd/5qxjmFo9jSR+mWYf6yfuwALv/4uw7YBosQEAQC9YRA4H1zZI kpInkJdLzo8S/5zvcxmi7nfwzPcsnVZ4gLdveroFj4F5pKoh25/idQFx1LUuhR/F+uiq AM3s10BdBTJDu/so/mTeMw8OgNo3PFey/fy1noZtXOX8bKm7cMWCXcwvMfJ4hr5gCfhy Nz8CYC/GkK/rU9Wwx/nlJqeKkr7RM8Bbv+5qTQoWFcHRXP7NrZP93xDTCwORw8Y2wqXH rzD0zrbUkyLPOWXvn4OCObij6WJUBd2XP6t9eIybinvVeSSg5HUG+gAey4ru789JUaY8 k72Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=lAjj7zvOJmAQ0j4HHmfrbPuappDw3FuikEq7PoN3lLI=; b=agNkgePCwfXFl4BCqQdrx4+3CgDDA+/qdJPsVhtgoQKE2lPaQoSsNxdD9zlhFqx7SR sjx+cDbAHwcthpjMWVNh7DV2L9Da4pf2u+9UmnEIcqqsV5gXgIcGWiHFgpXKn9v66Tjx s53DFhhZr/tbJ7BJGRHFjIg+HhVV8N61x57V2s8thXhjsP7r4RzMLrWVBdEApk+gsqzL ZXBTgxCyyHKqiMpK4DiAv7CV0cTkgzm5zXPunETDxlVSHZBYb+8FAjQdfzdjbpDnlhri 1By3tZPCdmf7bBHPrfg2wR9+LZzg5o8VjH6sg5McxEuW/4Ek7D2IqqTs4hyZXgMPKnkI AVHg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass (test mode) header.i=@ideasonboard.com header.s=mail header.b=hC47oqIh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id gn38-20020a1709070d2600b007c10ac7fe46si3792185ejc.19.2022.12.23.15.20.31; Fri, 23 Dec 2022 15:20:55 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass (test mode) header.i=@ideasonboard.com header.s=mail header.b=hC47oqIh; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231512AbiLWXRs (ORCPT <rfc822;pacteraone@gmail.com> + 99 others); Fri, 23 Dec 2022 18:17:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50010 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231192AbiLWXRq (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Fri, 23 Dec 2022 18:17:46 -0500 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 589BF11808; Fri, 23 Dec 2022 15:17:44 -0800 (PST) Received: from Monstersaurus.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 396ED4DD; Sat, 24 Dec 2022 00:17:41 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1671837461; bh=DRCjQnT/kw18dafqbNf3dcNvaDSQIX28MrmBKL7Y7TQ=; h=From:To:Cc:Subject:Date:From; b=hC47oqIhAEYWcviBHXg2RF+C8g6m7SBgmrS0DtMi2M2r6EKR/rMKH8+7oFOEKbJqQ GvHoxHKc9JH5/sZF0GRAtG/sxYPDbdAWGD1RIn9CxXlWGTBxDZpEtsERp9oMstbqHY kUWgOTNjULRfrdw17imkJFaTmpTQjqeTOmF69c9c= From: Kieran Bingham <kieran.bingham@ideasonboard.com> To: linux-media@vger.kernel.org Cc: Kieran Bingham <kieran.bingham@ideasonboard.com>, Mauro Carvalho Chehab <mchehab@kernel.org>, Sakari Ailus <sakari.ailus@linux.intel.com>, Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>, Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>, Hans Verkuil <hverkuil-cisco@xs4all.nl>, Christophe JAILLET <christophe.jaillet@wanadoo.fr>, linux-kernel@vger.kernel.org (open list) Subject: [RFC PATCH] media: v4l2-dev: sysfs: Support streaming attribute Date: Fri, 23 Dec 2022 23:17:35 +0000 Message-Id: <20221223231736.2111774-1-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_PASS,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1753048841271870957?= X-GMAIL-MSGID: =?utf-8?q?1753048841271870957?= |
Series |
[RFC] media: v4l2-dev: sysfs: Support streaming attribute
|
|
Commit Message
Kieran Bingham
Dec. 23, 2022, 11:17 p.m. UTC
Provide a streaming attribute to allow userspace to interogate if a device
is actively streaming or not.
This will allow desktop notifications to report if a camera or device
is active on the system, rather than just 'open' which can occur when
configuring the device.
Bug: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
---
This is a quick POC to see if such a facility makes sense.
I'm weary that not all video devices may have the queues registered on
the struct video_device, but this seems like an effective way to be able
to determine if a device is actively streaming on a system.
Documentation/ABI/stable/sysfs-class-video | 9 +++++++++
MAINTAINERS | 1 +
drivers/media/v4l2-core/v4l2-dev.c | 13 +++++++++++++
3 files changed, 23 insertions(+)
create mode 100644 Documentation/ABI/stable/sysfs-class-video
Comments
Hi Kieran, Thank you for the patch. On Fri, Dec 23, 2022 at 11:17:35PM +0000, Kieran Bingham wrote: > Provide a streaming attribute to allow userspace to interogate if a device > is actively streaming or not. > > This will allow desktop notifications to report if a camera or device > is active on the system, rather than just 'open' which can occur when > configuring the device. > > Bug: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669 > Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com> > --- > > This is a quick POC to see if such a facility makes sense. > I'm weary that not all video devices may have the queues registered on > the struct video_device, but this seems like an effective way to be able > to determine if a device is actively streaming on a system. I can imagine multiple problems, from race conditions to permissions and privacy. In order to comment on the fitness of this solution to address the problem you're trying to solve, could you describe the actual problem ? > Documentation/ABI/stable/sysfs-class-video | 9 +++++++++ > MAINTAINERS | 1 + > drivers/media/v4l2-core/v4l2-dev.c | 13 +++++++++++++ > 3 files changed, 23 insertions(+) > create mode 100644 Documentation/ABI/stable/sysfs-class-video > > diff --git a/Documentation/ABI/stable/sysfs-class-video b/Documentation/ABI/stable/sysfs-class-video > new file mode 100644 > index 000000000000..99dd27475a92 > --- /dev/null > +++ b/Documentation/ABI/stable/sysfs-class-video > @@ -0,0 +1,9 @@ > +What: /sys/class/video4linux/video<n>/streaming > +Date: January 2023 > +KernelVersion: 6.3 > +Contact: Kieran Bingham <kieran.bingham@ideasonboard.com> > +Description: > + Indicates if the video device has an actively streaming queue. > + This may indicate that the device is capturing or outputing > + video data. > +Users: > diff --git a/MAINTAINERS b/MAINTAINERS > index 11987154eeee..8449f5a6c0da 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -12867,6 +12867,7 @@ S: Maintained > W: https://linuxtv.org > Q: http://patchwork.kernel.org/project/linux-media/list/ > T: git git://linuxtv.org/media_tree.git > +F: Documentation/ABI/stable/sysfs-class-video > F: Documentation/admin-guide/media/ > F: Documentation/devicetree/bindings/media/ > F: Documentation/driver-api/media/ > diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c > index 397d553177fa..7d800309d076 100644 > --- a/drivers/media/v4l2-core/v4l2-dev.c > +++ b/drivers/media/v4l2-core/v4l2-dev.c > @@ -30,6 +30,7 @@ > #include <media/v4l2-device.h> > #include <media/v4l2-ioctl.h> > #include <media/v4l2-event.h> > +#include <media/videobuf2-core.h> > > #define VIDEO_NUM_DEVICES 256 > #define VIDEO_NAME "video4linux" > @@ -85,7 +86,19 @@ static ssize_t name_show(struct device *cd, > } > static DEVICE_ATTR_RO(name); > > +static ssize_t streaming_show(struct device *cd, > + struct device_attribute *attr, char *buf) > +{ > + struct video_device *vdev = to_video_device(cd); > + > + int active = vdev->queue ? vb2_is_streaming(vdev->queue) : false; > + > + return sprintf(buf, "%s\n", active ? "active" : "inactive"); > +} > +static DEVICE_ATTR_RO(streaming); > + > static struct attribute *video_device_attrs[] = { > + &dev_attr_streaming.attr, > &dev_attr_name.attr, > &dev_attr_dev_debug.attr, > &dev_attr_index.attr,
Hi On 2022. december 26., hétfő 10:52, Laurent Pinchart wrote: > Hi Kieran, > > Thank you for the patch. > > On Fri, Dec 23, 2022 at 11:17:35PM +0000, Kieran Bingham wrote: > > > Provide a streaming attribute to allow userspace to interogate if a device > > is actively streaming or not. > > > > This will allow desktop notifications to report if a camera or device > > is active on the system, rather than just 'open' which can occur when > > configuring the device. > > > > Bug: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669 > > Signed-off-by: Kieran Bingham kieran.bingham@ideasonboard.com > > --- > > > > This is a quick POC to see if such a facility makes sense. > > I'm weary that not all video devices may have the queues registered on > > the struct video_device, but this seems like an effective way to be able > > to determine if a device is actively streaming on a system. > > > I can imagine multiple problems, from race conditions to permissions and > privacy. In order to comment on the fitness of this solution to address > the problem you're trying to solve, could you describe the actual > problem ? The issue is explained in the following thread: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669#note_1697388 In short, the user wants to show a "camera-in-use" indicator when the laptop camera is used. The script that the user previously had only checked if /dev/video0 was open in any process, if it was, the indicator was shown. However, libcamera - at least at the moment - keeps the file descriptor open as long as the Camera object exists, which pipewire keeps alive for the entire lifetime of the device, therefore the "camera-in-use" indicator is always shown. > [...] Regards, Barnabás Pőcze
Hi Barnabás, others, On Wed, Dec 28, 2022 at 01:44:38AM +0000, Barnabás Pőcze wrote: > Hi > > > On 2022. december 26., hétfő 10:52, Laurent Pinchart wrote: > > > Hi Kieran, > > > > Thank you for the patch. > > > > On Fri, Dec 23, 2022 at 11:17:35PM +0000, Kieran Bingham wrote: > > > > > Provide a streaming attribute to allow userspace to interogate if a device > > > is actively streaming or not. > > > > > > This will allow desktop notifications to report if a camera or device > > > is active on the system, rather than just 'open' which can occur when > > > configuring the device. > > > > > > Bug: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669 > > > Signed-off-by: Kieran Bingham kieran.bingham@ideasonboard.com > > > --- > > > > > > This is a quick POC to see if such a facility makes sense. > > > I'm weary that not all video devices may have the queues registered on > > > the struct video_device, but this seems like an effective way to be able > > > to determine if a device is actively streaming on a system. > > > > > > I can imagine multiple problems, from race conditions to permissions and > > privacy. In order to comment on the fitness of this solution to address > > the problem you're trying to solve, could you describe the actual > > problem ? > > > The issue is explained in the following thread: > https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669#note_1697388 > > In short, the user wants to show a "camera-in-use" indicator when the laptop camera > is used. The script that the user previously had only checked if /dev/video0 > was open in any process, if it was, the indicator was shown. However, libcamera > - at least at the moment - keeps the file descriptor open as long as the Camera > object exists, which pipewire keeps alive for the entire lifetime of the device, > therefore the "camera-in-use" indicator is always shown. A sysfs attribute is not a great way to address this. libcamera certainly has information on whether streaming is ongoing. The information should come from there. Or Pipewire. Dbus perhaps? Alternatively libcamera could close the video devices while not streaming but that would involve e.g. releasing possible video buffer allocations as well, increasing streaming start latency.
Hello, On Mon, Jan 02, 2023 at 01:14:15PM +0000, Sakari Ailus wrote: > On Wed, Dec 28, 2022 at 01:44:38AM +0000, Barnabás Pőcze wrote: > > On 2022. december 26., hétfő 10:52, Laurent Pinchart wrote: > > > On Fri, Dec 23, 2022 at 11:17:35PM +0000, Kieran Bingham wrote: > > > > > > > Provide a streaming attribute to allow userspace to interogate if a device > > > > is actively streaming or not. > > > > > > > > This will allow desktop notifications to report if a camera or device > > > > is active on the system, rather than just 'open' which can occur when > > > > configuring the device. > > > > > > > > Bug: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669 > > > > Signed-off-by: Kieran Bingham kieran.bingham@ideasonboard.com > > > > --- > > > > > > > > This is a quick POC to see if such a facility makes sense. > > > > I'm weary that not all video devices may have the queues registered on > > > > the struct video_device, but this seems like an effective way to be able > > > > to determine if a device is actively streaming on a system. > > > > > > I can imagine multiple problems, from race conditions to permissions and > > > privacy. In order to comment on the fitness of this solution to address > > > the problem you're trying to solve, could you describe the actual > > > problem ? > > > > The issue is explained in the following thread: > > https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669#note_1697388 > > > > In short, the user wants to show a "camera-in-use" indicator when the laptop camera > > is used. The script that the user previously had only checked if /dev/video0 > > was open in any process, if it was, the indicator was shown. However, libcamera > > - at least at the moment - keeps the file descriptor open as long as the Camera > > object exists, which pipewire keeps alive for the entire lifetime of the device, > > therefore the "camera-in-use" indicator is always shown. > > A sysfs attribute is not a great way to address this. > > libcamera certainly has information on whether streaming is ongoing. The > information should come from there. Or Pipewire. Dbus perhaps? I tend to agree, I think this is best solved in userspace where PipeWire can have a centralized view of all cameras in the system, and of their users. > Alternatively libcamera could close the video devices while not streaming > but that would involve e.g. releasing possible video buffer allocations as > well, increasing streaming start latency. Closing video (and subdev) nodes when the camera is not in use would be good I think. It doesn't mean we have to open them when starting capture, explicit open/close operation (or similar, maybe introducing a capture session object in the libcamera API would also make sense, it should be considered as part of the same issue) could help with this.
Quoting Laurent Pinchart (2023-01-02 13:35:33) > Hello, > > On Mon, Jan 02, 2023 at 01:14:15PM +0000, Sakari Ailus wrote: > > On Wed, Dec 28, 2022 at 01:44:38AM +0000, Barnabás Pőcze wrote: > > > On 2022. december 26., hétfő 10:52, Laurent Pinchart wrote: > > > > On Fri, Dec 23, 2022 at 11:17:35PM +0000, Kieran Bingham wrote: > > > > > > > > > Provide a streaming attribute to allow userspace to interogate if a device > > > > > is actively streaming or not. > > > > > > > > > > This will allow desktop notifications to report if a camera or device > > > > > is active on the system, rather than just 'open' which can occur when > > > > > configuring the device. > > > > > > > > > > Bug: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669 > > > > > Signed-off-by: Kieran Bingham kieran.bingham@ideasonboard.com > > > > > --- > > > > > > > > > > This is a quick POC to see if such a facility makes sense. > > > > > I'm weary that not all video devices may have the queues registered on > > > > > the struct video_device, but this seems like an effective way to be able > > > > > to determine if a device is actively streaming on a system. > > > > > > > > I can imagine multiple problems, from race conditions to permissions and > > > > privacy. In order to comment on the fitness of this solution to address > > > > the problem you're trying to solve, could you describe the actual > > > > problem ? > > > > > > The issue is explained in the following thread: > > > https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669#note_1697388 > > > > > > In short, the user wants to show a "camera-in-use" indicator when the laptop camera > > > is used. The script that the user previously had only checked if /dev/video0 > > > was open in any process, if it was, the indicator was shown. However, libcamera > > > - at least at the moment - keeps the file descriptor open as long as the Camera > > > object exists, which pipewire keeps alive for the entire lifetime of the device, > > > therefore the "camera-in-use" indicator is always shown. > > > > A sysfs attribute is not a great way to address this. > > > > libcamera certainly has information on whether streaming is ongoing. The > > information should come from there. Or Pipewire. Dbus perhaps? > > I tend to agree, I think this is best solved in userspace where PipeWire > can have a centralized view of all cameras in the system, and of their > users. I fear that misses the entire point I was trying to make. Lets say pipewire 'is' available and in use and can be used to capture video streams for video calls, that's fine. But what happens if a user runs a gstreamer pipeline without using the pipewire source, or a suspcious process runs "yavta" and captures an image or stream discreetly... Only the kernel has a true centralised view of what devices are in use. > > Alternatively libcamera could close the video devices while not streaming > > but that would involve e.g. releasing possible video buffer allocations as > > well, increasing streaming start latency. Or is it just that in that case 'lsof' should be sufficient? The problem I have with that is - just like with the issue when the Privacy LED comes on during power up/probe - then any time a device is opened to identify the device and not necessarily use it - the 'camera in use' notification would get flashed... > Closing video (and subdev) nodes when the camera is not in use would be > good I think. It doesn't mean we have to open them when starting > capture, explicit open/close operation (or similar, maybe introducing a > capture session object in the libcamera API would also make sense, it > should be considered as part of the same issue) could help with this. I'm not talking about libcamera in this thread. It's how does a user correctly identify when a camera is in use globally in a system. -- Kieran
Hi Kieran, On Tue, Jan 03, 2023 at 09:33:22AM +0000, Kieran Bingham wrote: > Quoting Laurent Pinchart (2023-01-02 13:35:33) > > On Mon, Jan 02, 2023 at 01:14:15PM +0000, Sakari Ailus wrote: > > > On Wed, Dec 28, 2022 at 01:44:38AM +0000, Barnabás Pőcze wrote: > > > > On 2022. december 26., hétfő 10:52, Laurent Pinchart wrote: > > > > > On Fri, Dec 23, 2022 at 11:17:35PM +0000, Kieran Bingham wrote: > > > > > > > > > > > Provide a streaming attribute to allow userspace to interogate if a device > > > > > > is actively streaming or not. > > > > > > > > > > > > This will allow desktop notifications to report if a camera or device > > > > > > is active on the system, rather than just 'open' which can occur when > > > > > > configuring the device. > > > > > > > > > > > > Bug: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669 > > > > > > Signed-off-by: Kieran Bingham kieran.bingham@ideasonboard.com > > > > > > --- > > > > > > > > > > > > This is a quick POC to see if such a facility makes sense. > > > > > > I'm weary that not all video devices may have the queues registered on > > > > > > the struct video_device, but this seems like an effective way to be able > > > > > > to determine if a device is actively streaming on a system. > > > > > > > > > > I can imagine multiple problems, from race conditions to permissions and > > > > > privacy. In order to comment on the fitness of this solution to address > > > > > the problem you're trying to solve, could you describe the actual > > > > > problem ? > > > > > > > > The issue is explained in the following thread: > > > > https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669#note_1697388 > > > > > > > > In short, the user wants to show a "camera-in-use" indicator when the laptop camera > > > > is used. The script that the user previously had only checked if /dev/video0 > > > > was open in any process, if it was, the indicator was shown. However, libcamera > > > > - at least at the moment - keeps the file descriptor open as long as the Camera > > > > object exists, which pipewire keeps alive for the entire lifetime of the device, > > > > therefore the "camera-in-use" indicator is always shown. > > > > > > A sysfs attribute is not a great way to address this. > > > > > > libcamera certainly has information on whether streaming is ongoing. The > > > information should come from there. Or Pipewire. Dbus perhaps? > > > > I tend to agree, I think this is best solved in userspace where PipeWire > > can have a centralized view of all cameras in the system, and of their > > users. > > I fear that misses the entire point I was trying to make. > > Lets say pipewire 'is' available and in use and can be used to capture > video streams for video calls, that's fine. But what happens if a user > runs a gstreamer pipeline without using the pipewire source, or a > suspcious process runs "yavta" and captures an image or stream > discreetly... > > Only the kernel has a true centralised view of what devices are in use. That's right, but at the same time, the kernel as little view of what a "camera" is. At the beginning of V4L a video capture node was a TV capture card (soon with a few exceptions), then it also modelled a camera, for the past ten years at least it's "just" a DMA engine in many cases, and relatively recently even evolved to simply model a data flow endpoint with the addition of metadata video nodes. This doesn't even mention usage of video capture nodes in codecs or other memory to memory devices. Video devices are now in many cases just one of the many components in a camera pipeline. In most cases drivers can reasonably decide which video devices most likely represent a "camera", but that an approximation in any case, and not a general guarantee. In userspace the situation is worse, the link between a video device and a camera has been long lost. We started recovering it with libcamera, which is, today, the only open-source component available in Linux systems that has knowledge of cameras, not just video device nodes. > > > Alternatively libcamera could close the video devices while not streaming > > > but that would involve e.g. releasing possible video buffer allocations as > > > well, increasing streaming start latency. > > Or is it just that in that case 'lsof' should be sufficient? > > The problem I have with that is - just like with the issue when the > Privacy LED comes on during power up/probe - then any time a device is > opened to identify the device and not necessarily use it - the 'camera > in use' notification would get flashed... Regardless of whether an open device node indication or a streaming status is used, you don't want to indicate a camera is used because the user is watching a movie and the V4L2-based codec is in use. You thus need to at least filter out unrelated video devices in userspace, and if you want to do so for privacy reasons, hardcoding in PipeWire (or anywhere else) a heuristic will be prone to false positives or false negatives. That isn't a good idea in my opinion, I believe this problem can only be solved by handling the concept of "camera" in userspace. > > Closing video (and subdev) nodes when the camera is not in use would be > > good I think. It doesn't mean we have to open them when starting > > capture, explicit open/close operation (or similar, maybe introducing a > > capture session object in the libcamera API would also make sense, it > > should be considered as part of the same issue) could help with this. > > I'm not talking about libcamera in this thread. It's how does a user > correctly identify when a camera is in use globally in a system.
Quoting Laurent Pinchart (2023-01-03 13:31:28) > Hi Kieran, > > On Tue, Jan 03, 2023 at 09:33:22AM +0000, Kieran Bingham wrote: > > Quoting Laurent Pinchart (2023-01-02 13:35:33) > > > On Mon, Jan 02, 2023 at 01:14:15PM +0000, Sakari Ailus wrote: > > > > On Wed, Dec 28, 2022 at 01:44:38AM +0000, Barnabás Pőcze wrote: > > > > > On 2022. december 26., hétfő 10:52, Laurent Pinchart wrote: > > > > > > On Fri, Dec 23, 2022 at 11:17:35PM +0000, Kieran Bingham wrote: > > > > > > > > > > > > > Provide a streaming attribute to allow userspace to interogate if a device > > > > > > > is actively streaming or not. > > > > > > > > > > > > > > This will allow desktop notifications to report if a camera or device > > > > > > > is active on the system, rather than just 'open' which can occur when > > > > > > > configuring the device. > > > > > > > > > > > > > > Bug: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669 > > > > > > > Signed-off-by: Kieran Bingham kieran.bingham@ideasonboard.com > > > > > > > --- > > > > > > > > > > > > > > This is a quick POC to see if such a facility makes sense. > > > > > > > I'm weary that not all video devices may have the queues registered on > > > > > > > the struct video_device, but this seems like an effective way to be able > > > > > > > to determine if a device is actively streaming on a system. > > > > > > > > > > > > I can imagine multiple problems, from race conditions to permissions and > > > > > > privacy. In order to comment on the fitness of this solution to address > > > > > > the problem you're trying to solve, could you describe the actual > > > > > > problem ? > > > > > > > > > > The issue is explained in the following thread: > > > > > https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669#note_1697388 > > > > > > > > > > In short, the user wants to show a "camera-in-use" indicator when the laptop camera > > > > > is used. The script that the user previously had only checked if /dev/video0 > > > > > was open in any process, if it was, the indicator was shown. However, libcamera > > > > > - at least at the moment - keeps the file descriptor open as long as the Camera > > > > > object exists, which pipewire keeps alive for the entire lifetime of the device, > > > > > therefore the "camera-in-use" indicator is always shown. > > > > > > > > A sysfs attribute is not a great way to address this. > > > > > > > > libcamera certainly has information on whether streaming is ongoing. The > > > > information should come from there. Or Pipewire. Dbus perhaps? > > > > > > I tend to agree, I think this is best solved in userspace where PipeWire > > > can have a centralized view of all cameras in the system, and of their > > > users. > > > > I fear that misses the entire point I was trying to make. > > > > Lets say pipewire 'is' available and in use and can be used to capture > > video streams for video calls, that's fine. But what happens if a user > > runs a gstreamer pipeline without using the pipewire source, or a > > suspcious process runs "yavta" and captures an image or stream > > discreetly... > > > > Only the kernel has a true centralised view of what devices are in use. > > That's right, but at the same time, the kernel as little view of what a > "camera" is. > > At the beginning of V4L a video capture node was a TV capture card (soon > with a few exceptions), then it also modelled a camera, for the past ten > years at least it's "just" a DMA engine in many cases, and relatively > recently even evolved to simply model a data flow endpoint with the > addition of metadata video nodes. This doesn't even mention usage of > video capture nodes in codecs or other memory to memory devices. Video > devices are now in many cases just one of the many components in a > camera pipeline. > > In most cases drivers can reasonably decide which video devices most > likely represent a "camera", but that an approximation in any case, and > not a general guarantee. In userspace the situation is worse, the link > between a video device and a camera has been long lost. We started > recovering it with libcamera, which is, today, the only open-source > component available in Linux systems that has knowledge of cameras, not > just video device nodes. > > > > > Alternatively libcamera could close the video devices while not streaming > > > > but that would involve e.g. releasing possible video buffer allocations as > > > > well, increasing streaming start latency. > > > > Or is it just that in that case 'lsof' should be sufficient? > > > > The problem I have with that is - just like with the issue when the > > Privacy LED comes on during power up/probe - then any time a device is > > opened to identify the device and not necessarily use it - the 'camera > > in use' notification would get flashed... > > Regardless of whether an open device node indication or a streaming > status is used, you don't want to indicate a camera is used because the > user is watching a movie and the V4L2-based codec is in use. You thus > need to at least filter out unrelated video devices in userspace, and if > you want to do so for privacy reasons, hardcoding in PipeWire (or > anywhere else) a heuristic will be prone to false positives or false > negatives. That isn't a good idea in my opinion, I believe this problem > can only be solved by handling the concept of "camera" in userspace. Indeed, with v4l2 based codecs, this gets far more complex. I think it's a shame we can't get this state globally, in a way that can't be 'hidden' or 'subverted' but I'll just consider this patch dropped. -- Kieran > > > > Closing video (and subdev) nodes when the camera is not in use would be > > > good I think. It doesn't mean we have to open them when starting > > > capture, explicit open/close operation (or similar, maybe introducing a > > > capture session object in the libcamera API would also make sense, it > > > should be considered as part of the same issue) could help with this. > > > > I'm not talking about libcamera in this thread. It's how does a user > > correctly identify when a camera is in use globally in a system. > > -- > Regards, > > Laurent Pinchart
diff --git a/Documentation/ABI/stable/sysfs-class-video b/Documentation/ABI/stable/sysfs-class-video new file mode 100644 index 000000000000..99dd27475a92 --- /dev/null +++ b/Documentation/ABI/stable/sysfs-class-video @@ -0,0 +1,9 @@ +What: /sys/class/video4linux/video<n>/streaming +Date: January 2023 +KernelVersion: 6.3 +Contact: Kieran Bingham <kieran.bingham@ideasonboard.com> +Description: + Indicates if the video device has an actively streaming queue. + This may indicate that the device is capturing or outputing + video data. +Users: diff --git a/MAINTAINERS b/MAINTAINERS index 11987154eeee..8449f5a6c0da 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -12867,6 +12867,7 @@ S: Maintained W: https://linuxtv.org Q: http://patchwork.kernel.org/project/linux-media/list/ T: git git://linuxtv.org/media_tree.git +F: Documentation/ABI/stable/sysfs-class-video F: Documentation/admin-guide/media/ F: Documentation/devicetree/bindings/media/ F: Documentation/driver-api/media/ diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c index 397d553177fa..7d800309d076 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -30,6 +30,7 @@ #include <media/v4l2-device.h> #include <media/v4l2-ioctl.h> #include <media/v4l2-event.h> +#include <media/videobuf2-core.h> #define VIDEO_NUM_DEVICES 256 #define VIDEO_NAME "video4linux" @@ -85,7 +86,19 @@ static ssize_t name_show(struct device *cd, } static DEVICE_ATTR_RO(name); +static ssize_t streaming_show(struct device *cd, + struct device_attribute *attr, char *buf) +{ + struct video_device *vdev = to_video_device(cd); + + int active = vdev->queue ? vb2_is_streaming(vdev->queue) : false; + + return sprintf(buf, "%s\n", active ? "active" : "inactive"); +} +static DEVICE_ATTR_RO(streaming); + static struct attribute *video_device_attrs[] = { + &dev_attr_streaming.attr, &dev_attr_name.attr, &dev_attr_dev_debug.attr, &dev_attr_index.attr,