Message ID | 1705581128-4604-11-git-send-email-shengjiu.wang@nxp.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel+bounces-30136-ouuuleilei=gmail.com@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2bc4:b0:101:a8e8:374 with SMTP id hx4csp330433dyb; Thu, 18 Jan 2024 05:16:17 -0800 (PST) X-Google-Smtp-Source: AGHT+IFG2kR8jELh8NN2spKIu90MFcytvsWZOR3Fzc+sqMtK+U0+AECeKrguT81L8E6XGjEhfwe+ X-Received: by 2002:a9d:6859:0:b0:6dd:ee5f:5021 with SMTP id c25-20020a9d6859000000b006ddee5f5021mr748815oto.54.1705583776988; Thu, 18 Jan 2024 05:16:16 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1705583776; cv=pass; d=google.com; s=arc-20160816; b=iZf29xI8zS/KrNK1M35JjwWfM5EW0ERRdZ2q+zcLhdUDZSqevszBWUwjFOmZ0Hu9PQ wpAQI6JzhBcyCTTXV8DpLvG79UsCogU/zK9IC0Xx19SJujwPgqyBxUAozMxX5mAOmCGj jU90tRH2/wO4t9orOuzgO3ywaQy2TmCMJ9D6QFkM+/AL5NpJhQup5KG15+E6RPo8Qlfe cHizPMCVvWUW1CPaLqUHcrNoFZvw7h9xHIjHAAnMoLGo1PhXM2Pn2FOvBCF7FN0BnuRm 6Ua3cCvMfFy8TGNigvM6GB/MZlOD4EqCKj//tyaLF6UxmmDtbtPVK/a1Flabk5xejiGY NwGw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-id:precedence:references :in-reply-to:message-id:date:subject:to:from; bh=UYU+8RpbLVUT7ufDmDitdVwUZwydFsQn+0DK9biK8tc=; fh=ufiESpW7Hvn5oNmN0l6/qCUiQrKfLLrOwb5uR1Dftds=; b=cVjzIbTn24FqKil8mi5HbF/5ESmWGX364VslPiErg/dfY7MOc+ds+V5Jw9bFFUnctz mKD5kWvVxtxNtyrX7MpJ556R5dcj7vqAmqtx94TspHmXJdNbDqB39bwjV0Gp+Ty3fKCc BjyCzEkMZmkazbUrtjnu50YuXgCmEqoc40DGk8KX2tDnfEM8ew+DXhbkTBkYFzncLvht D58WI87YlxUcX1iUhciebySG35tv4SMAt1ZF+8a0AipaeSZGZ8NuRF6SDs24BSRN4toi HksM+xWkftZcLSGwZUJd7i5v6bVjKqevjXh5py9uLvHBZsNDm1PHH3wvwkPFrrQA87iC L11g== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=nxp.com dmarc=pass fromdomain=nxp.com); spf=pass (google.com: domain of linux-kernel+bounces-30136-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-30136-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=nxp.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id k29-20020a67ef5d000000b00466f50e73a9si1514492vsr.105.2024.01.18.05.16.16 for <ouuuleilei@gmail.com> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Jan 2024 05:16:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-30136-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=nxp.com dmarc=pass fromdomain=nxp.com); spf=pass (google.com: domain of linux-kernel+bounces-30136-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-30136-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=nxp.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 1F2A91C24A71 for <ouuuleilei@gmail.com>; Thu, 18 Jan 2024 13:16:12 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CDFCD28DDA; Thu, 18 Jan 2024 13:15:09 +0000 (UTC) Received: from inva020.nxp.com (inva020.nxp.com [92.121.34.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A5AE2560F; Thu, 18 Jan 2024 13:15:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=92.121.34.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705583708; cv=none; b=AsI/BtSgoz2zxMRNYAEnCrdgwItB8vE6nNpMEwxoJGEKkRfjNlpe1/1duidWgMkc7ilHfSJUo2NYTWnvuonAm1lLWJ1zdPEjpqNPE4orZlCu9c3BWmMVta5rHc5y36xuS5iqj2kX9/FnkzCD6gYQljQe5KZND6CtetnswcM7MKs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705583708; c=relaxed/simple; bh=fpWAagc9TBPC2nK/rsoLD6Cght7/uGfO3bRO+j3x+wo=; h=Received:Received:Received:From:To:Subject:Date:Message-Id: X-Mailer:In-Reply-To:References:X-Virus-Scanned; b=Ex6lVsu5rSnZ2Yh8fBnosI5tl0QydeytizZRCbtY3HBaAYcH3jdHmHZXplXF5rDZWQE+D6r3MfCk+gMsymdXHAHpuyp1VONnwswIpp4MoSBre89StXneT2BoJmHA5/pcIAVd3neN6+8AtDmA3WqnGCdP/zjSVM8FZe55sUbAFAo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com; spf=pass smtp.mailfrom=nxp.com; arc=none smtp.client-ip=92.121.34.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=nxp.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=nxp.com Received: from inva020.nxp.com (localhost [127.0.0.1]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id F18A01A228F; Thu, 18 Jan 2024 14:15:04 +0100 (CET) Received: from aprdc01srsp001v.ap-rdc01.nxp.com (aprdc01srsp001v.ap-rdc01.nxp.com [165.114.16.16]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 900CD1A22A0; Thu, 18 Jan 2024 14:15:04 +0100 (CET) Received: from localhost.localdomain (shlinux2.ap.freescale.net [10.192.224.44]) by aprdc01srsp001v.ap-rdc01.nxp.com (Postfix) with ESMTP id DA2F6183AC0A; Thu, 18 Jan 2024 21:15:02 +0800 (+08) From: Shengjiu Wang <shengjiu.wang@nxp.com> To: hverkuil@xs4all.nl, sakari.ailus@iki.fi, tfiga@chromium.org, m.szyprowski@samsung.com, mchehab@kernel.org, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, shengjiu.wang@gmail.com, Xiubo.Lee@gmail.com, festevam@gmail.com, nicoleotsuka@gmail.com, lgirdwood@gmail.com, broonie@kernel.org, perex@perex.cz, tiwai@suse.com, alsa-devel@alsa-project.org, linuxppc-dev@lists.ozlabs.org Subject: [PATCH v12 10/15] media: uapi: Add audio rate controls support Date: Thu, 18 Jan 2024 20:32:03 +0800 Message-Id: <1705581128-4604-11-git-send-email-shengjiu.wang@nxp.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1705581128-4604-1-git-send-email-shengjiu.wang@nxp.com> References: <1705581128-4604-1-git-send-email-shengjiu.wang@nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: <linux-kernel.vger.kernel.org> List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org> List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org> X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1788434214494370409 X-GMAIL-MSGID: 1788434214494370409 |
Series |
Add audio support in v4l2 framework
|
|
Commit Message
Shengjiu Wang
Jan. 18, 2024, 12:32 p.m. UTC
Add V4L2_CID_M2M_AUDIO_SOURCE_RATE and V4L2_CID_M2M_AUDIO_DEST_RATE
new IDs for rate control.
Add V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET and
V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET for clock drift.
Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com>
---
.../media/v4l/ext-ctrls-audio-m2m.rst | 20 +++++++++++++++++++
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 6 ++++++
include/uapi/linux/v4l2-controls.h | 5 +++++
3 files changed, 31 insertions(+)
Comments
Em Thu, 18 Jan 2024 20:32:03 +0800 Shengjiu Wang <shengjiu.wang@nxp.com> escreveu: > Add V4L2_CID_M2M_AUDIO_SOURCE_RATE and V4L2_CID_M2M_AUDIO_DEST_RATE > new IDs for rate control. > > Add V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET and > V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET for clock drift. > > Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com> > --- > .../media/v4l/ext-ctrls-audio-m2m.rst | 20 +++++++++++++++++++ > drivers/media/v4l2-core/v4l2-ctrls-defs.c | 6 ++++++ > include/uapi/linux/v4l2-controls.h | 5 +++++ > 3 files changed, 31 insertions(+) > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst > index 82d2ecedbfee..de579ab8fb94 100644 > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst > @@ -19,3 +19,23 @@ Audio M2M Control IDs > The Audio M2M class descriptor. Calling > :ref:`VIDIOC_QUERYCTRL` for this control will > return a description of this control class. > + > +.. _v4l2-audio-asrc: > + > +``V4L2_CID_M2M_AUDIO_SOURCE_RATE (integer menu)`` > + Sets the audio source sample rate, unit is Hz > + > +``V4L2_CID_M2M_AUDIO_DEST_RATE (integer menu)`` > + Sets the audio destination sample rate, unit is Hz > + > +``V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET (fixed point)`` > + Sets the offset from the audio source sample rate, unit is Hz. > + The offset compensates for any clock drift. The actual source audio > + sample rate is the ideal source audio sample rate from > + ``V4L2_CID_M2M_AUDIO_SOURCE_RATE`` plus this fixed point offset. > + > +``V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET (fixed point)`` > + Sets the offset from the audio destination sample rate, unit is Hz. > + The offset compensates for any clock drift. The actual destination audio > + sample rate is the ideal source audio sample rate from > + ``V4L2_CID_M2M_AUDIO_DEST_RATE`` plus this fixed point offset. Hmm... first of all, controls on V4L2 API can either be get or set. So, starting the sentence with "Set" sounds an assumption that may be wrong. Also, I would explain a little bit more about the frequency offset values, as clock drift adjustment on PCM streams is something that can be done using different approaches. I'm assuming that what you wanted here is to use it to check if the video and audio clocks have some drift, and reducing or increasing the audio sample rate dynamically to ensure that such drift will stay constraint to a maximum allowed drift measured in mili or nano seconds. So, userspace would be expected to be monitoring such drift and increasing/decreasing the sample frequency as needed to maintain such constraint. Is that the way such uAPI is expected to work? > diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c > index 2a85ea3dc92f..91e1f5348c23 100644 > --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c > +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c > @@ -1245,6 +1245,8 @@ const char *v4l2_ctrl_get_name(u32 id) > > /* Audio M2M controls */ > case V4L2_CID_M2M_AUDIO_CLASS: return "Audio M2M Controls"; > + case V4L2_CID_M2M_AUDIO_SOURCE_RATE: return "Audio Source Sample Rate"; > + case V4L2_CID_M2M_AUDIO_DEST_RATE: return "Audio Destination Sample Rate"; > default: > return NULL; > } > @@ -1606,6 +1608,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, > case V4L2_CID_COLORIMETRY_HDR10_MASTERING_DISPLAY: > *type = V4L2_CTRL_TYPE_HDR10_MASTERING_DISPLAY; > break; > + case V4L2_CID_M2M_AUDIO_SOURCE_RATE: > + case V4L2_CID_M2M_AUDIO_DEST_RATE: > + *type = V4L2_CTRL_TYPE_INTEGER_MENU; > + break; > default: > *type = V4L2_CTRL_TYPE_INTEGER; > break; > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h > index a8b4b830c757..30129ccdc282 100644 > --- a/include/uapi/linux/v4l2-controls.h > +++ b/include/uapi/linux/v4l2-controls.h > @@ -3495,6 +3495,11 @@ struct v4l2_ctrl_av1_film_grain { > #define V4L2_CID_M2M_AUDIO_CLASS_BASE (V4L2_CTRL_CLASS_M2M_AUDIO | 0x900) > #define V4L2_CID_M2M_AUDIO_CLASS (V4L2_CTRL_CLASS_M2M_AUDIO | 1) > > +#define V4L2_CID_M2M_AUDIO_SOURCE_RATE (V4L2_CID_M2M_AUDIO_CLASS_BASE + 0) > +#define V4L2_CID_M2M_AUDIO_DEST_RATE (V4L2_CID_M2M_AUDIO_CLASS_BASE + 1) > +#define V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET (V4L2_CID_M2M_AUDIO_CLASS_BASE + 2) > +#define V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET (V4L2_CID_M2M_AUDIO_CLASS_BASE + 3) > + > /* MPEG-compression definitions kept for backwards compatibility */ > #ifndef __KERNEL__ > #define V4L2_CTRL_CLASS_MPEG V4L2_CTRL_CLASS_CODEC Thanks, Mauro
On Sat, Feb 17, 2024 at 5:57 PM Mauro Carvalho Chehab <mchehab@kernel.org> wrote: > > Em Thu, 18 Jan 2024 20:32:03 +0800 > Shengjiu Wang <shengjiu.wang@nxp.com> escreveu: > > > Add V4L2_CID_M2M_AUDIO_SOURCE_RATE and V4L2_CID_M2M_AUDIO_DEST_RATE > > new IDs for rate control. > > > > Add V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET and > > V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET for clock drift. > > > > Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com> > > --- > > .../media/v4l/ext-ctrls-audio-m2m.rst | 20 +++++++++++++++++++ > > drivers/media/v4l2-core/v4l2-ctrls-defs.c | 6 ++++++ > > include/uapi/linux/v4l2-controls.h | 5 +++++ > > 3 files changed, 31 insertions(+) > > > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst > > index 82d2ecedbfee..de579ab8fb94 100644 > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst > > @@ -19,3 +19,23 @@ Audio M2M Control IDs > > The Audio M2M class descriptor. Calling > > :ref:`VIDIOC_QUERYCTRL` for this control will > > return a description of this control class. > > + > > +.. _v4l2-audio-asrc: > > + > > +``V4L2_CID_M2M_AUDIO_SOURCE_RATE (integer menu)`` > > + Sets the audio source sample rate, unit is Hz > > + > > +``V4L2_CID_M2M_AUDIO_DEST_RATE (integer menu)`` > > + Sets the audio destination sample rate, unit is Hz > > + > > +``V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET (fixed point)`` > > + Sets the offset from the audio source sample rate, unit is Hz. > > + The offset compensates for any clock drift. The actual source audio > > + sample rate is the ideal source audio sample rate from > > + ``V4L2_CID_M2M_AUDIO_SOURCE_RATE`` plus this fixed point offset. > > + > > +``V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET (fixed point)`` > > + Sets the offset from the audio destination sample rate, unit is Hz. > > + The offset compensates for any clock drift. The actual destination audio > > + sample rate is the ideal source audio sample rate from > > + ``V4L2_CID_M2M_AUDIO_DEST_RATE`` plus this fixed point offset. > > Hmm... first of all, controls on V4L2 API can either be get or set. > So, starting the sentence with "Set" sounds an assumption that may > be wrong. Ok, will update the description. > > Also, I would explain a little bit more about the frequency offset values, > as clock drift adjustment on PCM streams is something that can be done > using different approaches. > > I'm assuming that what you wanted here is to use it to check if the > video and audio clocks have some drift, and reducing or increasing > the audio sample rate dynamically to ensure that such drift will > stay constraint to a maximum allowed drift measured in mili or nano > seconds. So, userspace would be expected to be monitoring such drift > and increasing/decreasing the sample frequency as needed to maintain > such constraint. > > Is that the way such uAPI is expected to work? Yes. Userspace should monitor the drift, get the offset based on the common sample rate (8k, 44.1k, 48k...) then send it to the driver. The offset is a fixed point. it is base on the patch: https://patchwork.kernel.org/project/linux-media/patch/cec82507-ced9-4e7d-802c-04a40f84a4b4@xs4all.nl/ Best regards Shengjiu Wang > > > diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c > > index 2a85ea3dc92f..91e1f5348c23 100644 > > --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c > > +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c > > @@ -1245,6 +1245,8 @@ const char *v4l2_ctrl_get_name(u32 id) > > > > /* Audio M2M controls */ > > case V4L2_CID_M2M_AUDIO_CLASS: return "Audio M2M Controls"; > > + case V4L2_CID_M2M_AUDIO_SOURCE_RATE: return "Audio Source Sample Rate"; > > + case V4L2_CID_M2M_AUDIO_DEST_RATE: return "Audio Destination Sample Rate"; > > default: > > return NULL; > > } > > @@ -1606,6 +1608,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, > > case V4L2_CID_COLORIMETRY_HDR10_MASTERING_DISPLAY: > > *type = V4L2_CTRL_TYPE_HDR10_MASTERING_DISPLAY; > > break; > > + case V4L2_CID_M2M_AUDIO_SOURCE_RATE: > > + case V4L2_CID_M2M_AUDIO_DEST_RATE: > > + *type = V4L2_CTRL_TYPE_INTEGER_MENU; > > + break; > > default: > > *type = V4L2_CTRL_TYPE_INTEGER; > > break; > > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h > > index a8b4b830c757..30129ccdc282 100644 > > --- a/include/uapi/linux/v4l2-controls.h > > +++ b/include/uapi/linux/v4l2-controls.h > > @@ -3495,6 +3495,11 @@ struct v4l2_ctrl_av1_film_grain { > > #define V4L2_CID_M2M_AUDIO_CLASS_BASE (V4L2_CTRL_CLASS_M2M_AUDIO | 0x900) > > #define V4L2_CID_M2M_AUDIO_CLASS (V4L2_CTRL_CLASS_M2M_AUDIO | 1) > > > > +#define V4L2_CID_M2M_AUDIO_SOURCE_RATE (V4L2_CID_M2M_AUDIO_CLASS_BASE + 0) > > +#define V4L2_CID_M2M_AUDIO_DEST_RATE (V4L2_CID_M2M_AUDIO_CLASS_BASE + 1) > > +#define V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET (V4L2_CID_M2M_AUDIO_CLASS_BASE + 2) > > +#define V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET (V4L2_CID_M2M_AUDIO_CLASS_BASE + 3) > > + > > /* MPEG-compression definitions kept for backwards compatibility */ > > #ifndef __KERNEL__ > > #define V4L2_CTRL_CLASS_MPEG V4L2_CTRL_CLASS_CODEC > > > > Thanks, > Mauro
Em Mon, 19 Feb 2024 14:03:37 +0800 Shengjiu Wang <shengjiu.wang@gmail.com> escreveu: > On Sat, Feb 17, 2024 at 5:57 PM Mauro Carvalho Chehab > <mchehab@kernel.org> wrote: > > > > Em Thu, 18 Jan 2024 20:32:03 +0800 > > Shengjiu Wang <shengjiu.wang@nxp.com> escreveu: > > > > > Add V4L2_CID_M2M_AUDIO_SOURCE_RATE and V4L2_CID_M2M_AUDIO_DEST_RATE > > > new IDs for rate control. > > > > > > Add V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET and > > > V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET for clock drift. > > > > > > Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com> > > > --- > > > .../media/v4l/ext-ctrls-audio-m2m.rst | 20 +++++++++++++++++++ > > > drivers/media/v4l2-core/v4l2-ctrls-defs.c | 6 ++++++ > > > include/uapi/linux/v4l2-controls.h | 5 +++++ > > > 3 files changed, 31 insertions(+) > > > > > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst > > > index 82d2ecedbfee..de579ab8fb94 100644 > > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst > > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst > > > @@ -19,3 +19,23 @@ Audio M2M Control IDs > > > The Audio M2M class descriptor. Calling > > > :ref:`VIDIOC_QUERYCTRL` for this control will > > > return a description of this control class. > > > + > > > +.. _v4l2-audio-asrc: > > > + > > > +``V4L2_CID_M2M_AUDIO_SOURCE_RATE (integer menu)`` > > > + Sets the audio source sample rate, unit is Hz > > > + > > > +``V4L2_CID_M2M_AUDIO_DEST_RATE (integer menu)`` > > > + Sets the audio destination sample rate, unit is Hz > > > + > > > +``V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET (fixed point)`` > > > + Sets the offset from the audio source sample rate, unit is Hz. > > > + The offset compensates for any clock drift. The actual source audio > > > + sample rate is the ideal source audio sample rate from > > > + ``V4L2_CID_M2M_AUDIO_SOURCE_RATE`` plus this fixed point offset. > > > + > > > +``V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET (fixed point)`` > > > + Sets the offset from the audio destination sample rate, unit is Hz. > > > + The offset compensates for any clock drift. The actual destination audio > > > + sample rate is the ideal source audio sample rate from > > > + ``V4L2_CID_M2M_AUDIO_DEST_RATE`` plus this fixed point offset. > > > > Hmm... first of all, controls on V4L2 API can either be get or set. > > So, starting the sentence with "Set" sounds an assumption that may > > be wrong. > > Ok, will update the description. > > > > Also, I would explain a little bit more about the frequency offset values, > > as clock drift adjustment on PCM streams is something that can be done > > using different approaches. > > > > I'm assuming that what you wanted here is to use it to check if the > > video and audio clocks have some drift, and reducing or increasing > > the audio sample rate dynamically to ensure that such drift will > > stay constraint to a maximum allowed drift measured in mili or nano > > seconds. So, userspace would be expected to be monitoring such drift > > and increasing/decreasing the sample frequency as needed to maintain > > such constraint. > > > > Is that the way such uAPI is expected to work? > > Yes. Userspace should monitor the drift, get the offset based on the > common sample rate (8k, 44.1k, 48k...) then send it to the driver. > The offset is a fixed point. it is base on the patch: > https://patchwork.kernel.org/project/linux-media/patch/cec82507-ced9-4e7d-802c-04a40f84a4b4@xs4all.nl/ Ok, so write a description about how this should be used (like the test I written),to help userspace developers to better understand how this uAPI should be used. See, one of the main goals of the uAPI documentation is to ensure that userspace programs will implement the uAPI bits the right way, behaving the right way when using it. That's why we even have some userspace code examples for some ioctls. IMO frequence drift is one of such cases where more explanation is needed. > > Best regards > Shengjiu Wang > > > > > diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c > > > index 2a85ea3dc92f..91e1f5348c23 100644 > > > --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c > > > +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c > > > @@ -1245,6 +1245,8 @@ const char *v4l2_ctrl_get_name(u32 id) > > > > > > /* Audio M2M controls */ > > > case V4L2_CID_M2M_AUDIO_CLASS: return "Audio M2M Controls"; > > > + case V4L2_CID_M2M_AUDIO_SOURCE_RATE: return "Audio Source Sample Rate"; > > > + case V4L2_CID_M2M_AUDIO_DEST_RATE: return "Audio Destination Sample Rate"; > > > default: > > > return NULL; > > > } > > > @@ -1606,6 +1608,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, > > > case V4L2_CID_COLORIMETRY_HDR10_MASTERING_DISPLAY: > > > *type = V4L2_CTRL_TYPE_HDR10_MASTERING_DISPLAY; > > > break; > > > + case V4L2_CID_M2M_AUDIO_SOURCE_RATE: > > > + case V4L2_CID_M2M_AUDIO_DEST_RATE: > > > + *type = V4L2_CTRL_TYPE_INTEGER_MENU; > > > + break; > > > default: > > > *type = V4L2_CTRL_TYPE_INTEGER; > > > break; > > > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h > > > index a8b4b830c757..30129ccdc282 100644 > > > --- a/include/uapi/linux/v4l2-controls.h > > > +++ b/include/uapi/linux/v4l2-controls.h > > > @@ -3495,6 +3495,11 @@ struct v4l2_ctrl_av1_film_grain { > > > #define V4L2_CID_M2M_AUDIO_CLASS_BASE (V4L2_CTRL_CLASS_M2M_AUDIO | 0x900) > > > #define V4L2_CID_M2M_AUDIO_CLASS (V4L2_CTRL_CLASS_M2M_AUDIO | 1) > > > > > > +#define V4L2_CID_M2M_AUDIO_SOURCE_RATE (V4L2_CID_M2M_AUDIO_CLASS_BASE + 0) > > > +#define V4L2_CID_M2M_AUDIO_DEST_RATE (V4L2_CID_M2M_AUDIO_CLASS_BASE + 1) > > > +#define V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET (V4L2_CID_M2M_AUDIO_CLASS_BASE + 2) > > > +#define V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET (V4L2_CID_M2M_AUDIO_CLASS_BASE + 3) > > > + > > > /* MPEG-compression definitions kept for backwards compatibility */ > > > #ifndef __KERNEL__ > > > #define V4L2_CTRL_CLASS_MPEG V4L2_CTRL_CLASS_CODEC > > > > > > > > Thanks, > > Mauro Thanks, Mauro
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst index 82d2ecedbfee..de579ab8fb94 100644 --- a/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst @@ -19,3 +19,23 @@ Audio M2M Control IDs The Audio M2M class descriptor. Calling :ref:`VIDIOC_QUERYCTRL` for this control will return a description of this control class. + +.. _v4l2-audio-asrc: + +``V4L2_CID_M2M_AUDIO_SOURCE_RATE (integer menu)`` + Sets the audio source sample rate, unit is Hz + +``V4L2_CID_M2M_AUDIO_DEST_RATE (integer menu)`` + Sets the audio destination sample rate, unit is Hz + +``V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET (fixed point)`` + Sets the offset from the audio source sample rate, unit is Hz. + The offset compensates for any clock drift. The actual source audio + sample rate is the ideal source audio sample rate from + ``V4L2_CID_M2M_AUDIO_SOURCE_RATE`` plus this fixed point offset. + +``V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET (fixed point)`` + Sets the offset from the audio destination sample rate, unit is Hz. + The offset compensates for any clock drift. The actual destination audio + sample rate is the ideal source audio sample rate from + ``V4L2_CID_M2M_AUDIO_DEST_RATE`` plus this fixed point offset. diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c index 2a85ea3dc92f..91e1f5348c23 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c @@ -1245,6 +1245,8 @@ const char *v4l2_ctrl_get_name(u32 id) /* Audio M2M controls */ case V4L2_CID_M2M_AUDIO_CLASS: return "Audio M2M Controls"; + case V4L2_CID_M2M_AUDIO_SOURCE_RATE: return "Audio Source Sample Rate"; + case V4L2_CID_M2M_AUDIO_DEST_RATE: return "Audio Destination Sample Rate"; default: return NULL; } @@ -1606,6 +1608,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, case V4L2_CID_COLORIMETRY_HDR10_MASTERING_DISPLAY: *type = V4L2_CTRL_TYPE_HDR10_MASTERING_DISPLAY; break; + case V4L2_CID_M2M_AUDIO_SOURCE_RATE: + case V4L2_CID_M2M_AUDIO_DEST_RATE: + *type = V4L2_CTRL_TYPE_INTEGER_MENU; + break; default: *type = V4L2_CTRL_TYPE_INTEGER; break; diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h index a8b4b830c757..30129ccdc282 100644 --- a/include/uapi/linux/v4l2-controls.h +++ b/include/uapi/linux/v4l2-controls.h @@ -3495,6 +3495,11 @@ struct v4l2_ctrl_av1_film_grain { #define V4L2_CID_M2M_AUDIO_CLASS_BASE (V4L2_CTRL_CLASS_M2M_AUDIO | 0x900) #define V4L2_CID_M2M_AUDIO_CLASS (V4L2_CTRL_CLASS_M2M_AUDIO | 1) +#define V4L2_CID_M2M_AUDIO_SOURCE_RATE (V4L2_CID_M2M_AUDIO_CLASS_BASE + 0) +#define V4L2_CID_M2M_AUDIO_DEST_RATE (V4L2_CID_M2M_AUDIO_CLASS_BASE + 1) +#define V4L2_CID_M2M_AUDIO_SOURCE_RATE_OFFSET (V4L2_CID_M2M_AUDIO_CLASS_BASE + 2) +#define V4L2_CID_M2M_AUDIO_DEST_RATE_OFFSET (V4L2_CID_M2M_AUDIO_CLASS_BASE + 3) + /* MPEG-compression definitions kept for backwards compatibility */ #ifndef __KERNEL__ #define V4L2_CTRL_CLASS_MPEG V4L2_CTRL_CLASS_CODEC