Message ID | 20230126031424.14582-21-quic_wcheng@quicinc.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp74923wrn; Wed, 25 Jan 2023 19:35:05 -0800 (PST) X-Google-Smtp-Source: AK7set9rJT+gpo7Kb2n5MB7ZhRyW1LAGHXLXI9GTO39dm8CE6dkifLu2nlbBg7k5k8GvjT0n50dP X-Received: by 2002:a17:906:1e92:b0:878:4cc9:18a8 with SMTP id e18-20020a1709061e9200b008784cc918a8mr1916596ejj.29.1674704105475; Wed, 25 Jan 2023 19:35:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674704105; cv=none; d=google.com; s=arc-20160816; b=mNuUo3/sphNAWvHk3WhSOxeb5zPgpXKY+XLLLqZY17w8HVV8PO//8l76iJ0poxJ7TV NcFUI/qDlHnlkNmeYl5FCeyTBm5/VFCZd0Cls7jh/vDIsV4nCVHqzuqMWnZJ0251fEnL 5RGLGahWtC9qRTWKJMl3JE0ymCXIOxrjUCq4T9E74+xeb2xXxd9l6dItUyV/fVvj6r7o oG3pAU0XFc9i3DDbqdZOcDNBbN6mRsqeJGSakaFtq/SEoKJLXkziwqj9efnzJQgna+Vr soqAARUP074WY8z94Yp79wlUOCil+9xCECusTMa5PrqCvFSeyU4ECJLF3PszmNQCHcDy 81Ow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=uk5UGMqs/ei1kDHwf+Z178yIlngwmaPizli7rDmuuVo=; b=dCdEEYOXSCyl2+F2ANy6wMEgVGzCwj1ma+SieaIIKSUVV8V1k72QJQ3fRshoSVUHNK wAhbNBqCDeqsRTxBLT3UKqcwQymzpCz8GkfS3+e4xhhl6xdRSZI/H3d4yvDSCAvN7sxp MY9EU2VVWb77PbB11EaBM69/ZuGqnXEovTL2ZqjWlwm+M7q5yQ6+RwgH6481MRfXfhiC iTsineXK2C1vc2E3OzupMBbO7q1ki3T2ElobVPClz6jOXfOCAP8tD/wT7s4Vy4yCzqNJ ETZv8xBvm1/vj4Gghegxqks80PAZmEK6XiLxrBittZwVvjgWQIAz2ydJwBXvmjCbZMIF 26ew== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=dMm7C3Yg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id wl14-20020a170907310e00b00871a4e8d7bdsi7473973ejb.1009.2023.01.25.19.34.38; Wed, 25 Jan 2023 19:35:05 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=dMm7C3Yg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236266AbjAZD0e (ORCPT <rfc822;lekhanya01809@gmail.com> + 99 others); Wed, 25 Jan 2023 22:26:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56840 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229481AbjAZD0d (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Wed, 25 Jan 2023 22:26:33 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E43599D; Wed, 25 Jan 2023 19:26:32 -0800 (PST) Received: from pps.filterd (m0279865.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 30Q32xNn004834; Thu, 26 Jan 2023 03:14:51 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=uk5UGMqs/ei1kDHwf+Z178yIlngwmaPizli7rDmuuVo=; b=dMm7C3Yg4rMZfpZ5dyB37WaG6RY1RtWh0frAKZjsL4n/U+4R9Yy/j/khISLZ0LbiI5A3 xrKS+bSTexJk1ZUhRdqw9jdCy4mfp8ONw0zmOiZyye1Bxoj/d0PipdiU4GdEpEbG0DJJ iGUvOb3C2MrkrynJgiXYVcBd0DOvfcJHkxSUkUefjgNrvozV7gkB358QIUvSUOFmgIrN vFjZf+dSF42mqAFOJu4xCHcFQ+lRtN2ZOC8JQ0RqEXg81mHEIJA1B6hAZ7TWRXCHEg6n B4q5R03hDMtU0wlowlECnCVeK//aH2OExU4A45vF+fY3dwlIDz/LBhucZDFQ6Lg95Le1 3w== Received: from nasanppmta01.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3nbd7arbmb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 26 Jan 2023 03:14:50 +0000 Received: from nasanex01c.na.qualcomm.com (nasanex01c.na.qualcomm.com [10.45.79.139]) by NASANPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 30Q3EopC008329 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 26 Jan 2023 03:14:50 GMT Received: from hu-wcheng-lv.qualcomm.com (10.49.16.6) by nasanex01c.na.qualcomm.com (10.45.79.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 25 Jan 2023 19:14:49 -0800 From: Wesley Cheng <quic_wcheng@quicinc.com> To: <srinivas.kandagatla@linaro.org>, <mathias.nyman@intel.com>, <perex@perex.cz>, <lgirdwood@gmail.com>, <andersson@kernel.org>, <krzysztof.kozlowski+dt@linaro.org>, <gregkh@linuxfoundation.org>, <Thinh.Nguyen@synopsys.com>, <broonie@kernel.org>, <bgoswami@quicinc.com>, <tiwai@suse.com>, <robh+dt@kernel.org>, <agross@kernel.org> CC: <linux-kernel@vger.kernel.org>, <linux-arm-msm@vger.kernel.org>, <alsa-devel@alsa-project.org>, <devicetree@vger.kernel.org>, <linux-usb@vger.kernel.org>, <quic_jackp@quicinc.com>, <quic_plai@quicinc.com>, Wesley Cheng <quic_wcheng@quicinc.com> Subject: [RFC PATCH v2 20/22] sound: usb: Prevent starting of audio stream if in use Date: Wed, 25 Jan 2023 19:14:22 -0800 Message-ID: <20230126031424.14582-21-quic_wcheng@quicinc.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230126031424.14582-1-quic_wcheng@quicinc.com> References: <20230126031424.14582-1-quic_wcheng@quicinc.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01c.na.qualcomm.com (10.47.97.35) To nasanex01c.na.qualcomm.com (10.45.79.139) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: CNJn2OyEhsGz6hbWYzCWCNzTbT0Ll6uu X-Proofpoint-GUID: CNJn2OyEhsGz6hbWYzCWCNzTbT0Ll6uu X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.122.1 definitions=2023-01-25_14,2023-01-25_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 lowpriorityscore=0 malwarescore=0 spamscore=0 mlxlogscore=898 impostorscore=0 adultscore=0 clxscore=1015 priorityscore=1501 phishscore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2301260028 X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756054531754515300?= X-GMAIL-MSGID: =?utf-8?q?1756054531754515300?= |
Series |
Introduce QC USB SND audio offloading support
|
|
Commit Message
Wesley Cheng
Jan. 26, 2023, 3:14 a.m. UTC
With USB audio offloading, an audio session is started from the ASoC
platform sound card and PCM devices. Likewise, the USB SND path is still
readily available for use, in case the non-offload path is desired. In
order to prevent the two entities from attempting to use the USB bus,
introduce a flag that determines when either paths are in use.
If a PCM device is already in use, the check will return an error to
userspace notifying that the stream is currently busy. This ensures that
only one path is using the USB substream.
Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
---
sound/usb/card.h | 1 +
sound/usb/pcm.c | 19 +++++++++++++++++--
sound/usb/qcom/qc_audio_offload.c | 15 ++++++++++++++-
3 files changed, 32 insertions(+), 3 deletions(-)
Comments
On 1/25/23 21:14, Wesley Cheng wrote: > With USB audio offloading, an audio session is started from the ASoC > platform sound card and PCM devices. Likewise, the USB SND path is still > readily available for use, in case the non-offload path is desired. In > order to prevent the two entities from attempting to use the USB bus, > introduce a flag that determines when either paths are in use. > > If a PCM device is already in use, the check will return an error to > userspace notifying that the stream is currently busy. This ensures that > only one path is using the USB substream. It's good to maintain mutual exclusion, but it's still very hard for an application to figure out which card can be used when. Returning -EBUSY is not super helpful. There should be something like a notification or connection status so that routing decisions can be made without trial-and-error. > Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com> > --- > sound/usb/card.h | 1 + > sound/usb/pcm.c | 19 +++++++++++++++++-- > sound/usb/qcom/qc_audio_offload.c | 15 ++++++++++++++- > 3 files changed, 32 insertions(+), 3 deletions(-) > > diff --git a/sound/usb/card.h b/sound/usb/card.h > index 410a4ffad98e..ff6d4695e727 100644 > --- a/sound/usb/card.h > +++ b/sound/usb/card.h > @@ -163,6 +163,7 @@ struct snd_usb_substream { > unsigned int pkt_offset_adj; /* Bytes to drop from beginning of packets (for non-compliant devices) */ > unsigned int stream_offset_adj; /* Bytes to drop from beginning of stream (for non-compliant devices) */ > > + unsigned int opened:1; /* pcm device opened */ > unsigned int running: 1; /* running status */ > unsigned int period_elapsed_pending; /* delay period handling */ > > diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c > index 0b01a5dfcb73..8946f8ddb892 100644 > --- a/sound/usb/pcm.c > +++ b/sound/usb/pcm.c > @@ -1114,8 +1114,15 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream) > struct snd_usb_stream *as = snd_pcm_substream_chip(substream); > struct snd_pcm_runtime *runtime = substream->runtime; > struct snd_usb_substream *subs = &as->substream[direction]; > + struct snd_usb_audio *chip = subs->stream->chip; > int ret; > > + mutex_lock(&chip->mutex); > + if (subs->opened) { > + mutex_unlock(&chip->mutex); > + return -EBUSY; > + } > + > runtime->hw = snd_usb_hardware; > /* need an explicit sync to catch applptr update in low-latency mode */ > if (direction == SNDRV_PCM_STREAM_PLAYBACK && > @@ -1132,13 +1139,17 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream) > > ret = setup_hw_info(runtime, subs); > if (ret < 0) > - return ret; > + goto out; > ret = snd_usb_autoresume(subs->stream->chip); > if (ret < 0) > - return ret; > + goto out; > ret = snd_media_stream_init(subs, as->pcm, direction); > if (ret < 0) > snd_usb_autosuspend(subs->stream->chip); > + subs->opened = 1; > +out: > + mutex_unlock(&chip->mutex); > + > return ret; > } > > @@ -1147,6 +1158,7 @@ static int snd_usb_pcm_close(struct snd_pcm_substream *substream) > int direction = substream->stream; > struct snd_usb_stream *as = snd_pcm_substream_chip(substream); > struct snd_usb_substream *subs = &as->substream[direction]; > + struct snd_usb_audio *chip = subs->stream->chip; > int ret; > > snd_media_stop_pipeline(subs); > @@ -1160,6 +1172,9 @@ static int snd_usb_pcm_close(struct snd_pcm_substream *substream) > > subs->pcm_substream = NULL; > snd_usb_autosuspend(subs->stream->chip); > + mutex_lock(&chip->mutex); > + subs->opened = 0; > + mutex_unlock(&chip->mutex); > > return 0; > } > diff --git a/sound/usb/qcom/qc_audio_offload.c b/sound/usb/qcom/qc_audio_offload.c > index c1254d5f680d..9bd09282e70d 100644 > --- a/sound/usb/qcom/qc_audio_offload.c > +++ b/sound/usb/qcom/qc_audio_offload.c > @@ -1365,12 +1365,17 @@ static void handle_uaudio_stream_req(struct qmi_handle *handle, > goto response; > } > > + mutex_lock(&chip->mutex); > if (req_msg->enable) { > - if (info_idx < 0 || chip->system_suspend) { > + if (info_idx < 0 || chip->system_suspend || subs->opened) { > ret = -EBUSY; > + mutex_unlock(&chip->mutex); > + > goto response; > } > + subs->opened = 1; > } > + mutex_unlock(&chip->mutex); > > if (req_msg->service_interval_valid) { > ret = get_data_interval_from_si(subs, > @@ -1392,6 +1397,11 @@ static void handle_uaudio_stream_req(struct qmi_handle *handle, > if (!ret) > ret = prepare_qmi_response(subs, req_msg, &resp, > info_idx); > + if (ret < 0) { > + mutex_lock(&chip->mutex); > + subs->opened = 0; > + mutex_unlock(&chip->mutex); > + } > } else { > info = &uadev[pcm_card_num].info[info_idx]; > if (info->data_ep_pipe) { > @@ -1413,6 +1423,9 @@ static void handle_uaudio_stream_req(struct qmi_handle *handle, > } > > disable_audio_stream(subs); > + mutex_lock(&chip->mutex); > + subs->opened = 0; > + mutex_unlock(&chip->mutex); > } > > response:
Hi Pierre, On 1/26/2023 8:12 AM, Pierre-Louis Bossart wrote: > > > On 1/25/23 21:14, Wesley Cheng wrote: >> With USB audio offloading, an audio session is started from the ASoC >> platform sound card and PCM devices. Likewise, the USB SND path is still >> readily available for use, in case the non-offload path is desired. In >> order to prevent the two entities from attempting to use the USB bus, >> introduce a flag that determines when either paths are in use. >> >> If a PCM device is already in use, the check will return an error to >> userspace notifying that the stream is currently busy. This ensures that >> only one path is using the USB substream. > > It's good to maintain mutual exclusion, but it's still very hard for an > application to figure out which card can be used when. > > Returning -EBUSY is not super helpful. There should be something like a > notification or connection status so that routing decisions can be made > without trial-and-error. > The USB offload driver does have access to the USB substream that is being utilized/offloaded. Maybe in addition to this check, we can also set the PCM runtime state as well (for that particular substream)? That way userspace can fetch information about if the stream is busy or not. Thanks Wesley Cheng
On 2/6/23 19:15, Wesley Cheng wrote: > Hi Pierre, > > On 1/26/2023 8:12 AM, Pierre-Louis Bossart wrote: >> >> >> On 1/25/23 21:14, Wesley Cheng wrote: >>> With USB audio offloading, an audio session is started from the ASoC >>> platform sound card and PCM devices. Likewise, the USB SND path is >>> still >>> readily available for use, in case the non-offload path is desired. In >>> order to prevent the two entities from attempting to use the USB bus, >>> introduce a flag that determines when either paths are in use. >>> >>> If a PCM device is already in use, the check will return an error to >>> userspace notifying that the stream is currently busy. This ensures >>> that >>> only one path is using the USB substream. >> >> It's good to maintain mutual exclusion, but it's still very hard for an >> application to figure out which card can be used when. >> >> Returning -EBUSY is not super helpful. There should be something like a >> notification or connection status so that routing decisions can be made >> without trial-and-error. >> > > The USB offload driver does have access to the USB substream that is > being utilized/offloaded. Maybe in addition to this check, we can also > set the PCM runtime state as well (for that particular substream)? That > way userspace can fetch information about if the stream is busy or not. You're missing the point. When a card is exposed but the PCM devices may or may not be usable (consuming data with no sound rendered or returning an error), it's much better to provide a clear connection status to userspace. Let me give you an example. Intel drivers can expose 3 HDMI/DP PCM devices. Userspace has no idea which one to use, so there's a jack control that tells userspace whether there is a receiver connected so that the audio server can use the relevant PCM device. Audio routing based on trial and error is really problematic, errors can happen but they should be exceptional (e.g. xruns), not a means of driver-userspace communication on the device status.
Hi Pierre, On 2/7/2023 5:29 AM, Pierre-Louis Bossart wrote: > > > On 2/6/23 19:15, Wesley Cheng wrote: >> Hi Pierre, >> >> On 1/26/2023 8:12 AM, Pierre-Louis Bossart wrote: >>> >>> >>> On 1/25/23 21:14, Wesley Cheng wrote: >>>> With USB audio offloading, an audio session is started from the ASoC >>>> platform sound card and PCM devices. Likewise, the USB SND path is >>>> still >>>> readily available for use, in case the non-offload path is desired. In >>>> order to prevent the two entities from attempting to use the USB bus, >>>> introduce a flag that determines when either paths are in use. >>>> >>>> If a PCM device is already in use, the check will return an error to >>>> userspace notifying that the stream is currently busy. This ensures >>>> that >>>> only one path is using the USB substream. >>> >>> It's good to maintain mutual exclusion, but it's still very hard for an >>> application to figure out which card can be used when. >>> >>> Returning -EBUSY is not super helpful. There should be something like a >>> notification or connection status so that routing decisions can be made >>> without trial-and-error. >>> >> >> The USB offload driver does have access to the USB substream that is >> being utilized/offloaded. Maybe in addition to this check, we can also >> set the PCM runtime state as well (for that particular substream)? That >> way userspace can fetch information about if the stream is busy or not. > > You're missing the point. When a card is exposed but the PCM devices may > or may not be usable (consuming data with no sound rendered or returning > an error), it's much better to provide a clear connection status to > userspace. > > Let me give you an example. Intel drivers can expose 3 HDMI/DP PCM > devices. Userspace has no idea which one to use, so there's a jack > control that tells userspace whether there is a receiver connected so > that the audio server can use the relevant PCM device. > > Audio routing based on trial and error is really problematic, errors can > happen but they should be exceptional (e.g. xruns), not a means of > driver-userspace communication on the device status. Thanks for clarifying. The example helped me understand a bit more on how the potential use of the SND control interface. Since we're dealing with multiple sound cards here (platform sound card (offload) and USB SND card (legacy)), what do you think about creating a SND control on both the USB backend (platform card) and the USB SND card listing the PCM device status? That way at least userspace can have the information about which PCM dev (USB substream) is available (and not offloaded, or vice versa). So the USB SND control will contain the PCM devices (exposed by the card) and if any are offloaded (if so mark them as unavailable). Likewise, for the USB backend, if the legacy path is being used, mark them as unavailable for offloading. Thanks Wesley Cheng
On 2/11/23 03:52, Wesley Cheng wrote: > Hi Pierre, > > On 2/7/2023 5:29 AM, Pierre-Louis Bossart wrote: >> >> >> On 2/6/23 19:15, Wesley Cheng wrote: >>> Hi Pierre, >>> >>> On 1/26/2023 8:12 AM, Pierre-Louis Bossart wrote: >>>> >>>> >>>> On 1/25/23 21:14, Wesley Cheng wrote: >>>>> With USB audio offloading, an audio session is started from the ASoC >>>>> platform sound card and PCM devices. Likewise, the USB SND path is >>>>> still >>>>> readily available for use, in case the non-offload path is >>>>> desired. In >>>>> order to prevent the two entities from attempting to use the USB bus, >>>>> introduce a flag that determines when either paths are in use. >>>>> >>>>> If a PCM device is already in use, the check will return an error to >>>>> userspace notifying that the stream is currently busy. This ensures >>>>> that >>>>> only one path is using the USB substream. >>>> >>>> It's good to maintain mutual exclusion, but it's still very hard for an >>>> application to figure out which card can be used when. >>>> >>>> Returning -EBUSY is not super helpful. There should be something like a >>>> notification or connection status so that routing decisions can be made >>>> without trial-and-error. >>>> >>> >>> The USB offload driver does have access to the USB substream that is >>> being utilized/offloaded. Maybe in addition to this check, we can also >>> set the PCM runtime state as well (for that particular substream)? That >>> way userspace can fetch information about if the stream is busy or not. >> >> You're missing the point. When a card is exposed but the PCM devices may >> or may not be usable (consuming data with no sound rendered or returning >> an error), it's much better to provide a clear connection status to >> userspace. >> >> Let me give you an example. Intel drivers can expose 3 HDMI/DP PCM >> devices. Userspace has no idea which one to use, so there's a jack >> control that tells userspace whether there is a receiver connected so >> that the audio server can use the relevant PCM device. >> >> Audio routing based on trial and error is really problematic, errors can >> happen but they should be exceptional (e.g. xruns), not a means of >> driver-userspace communication on the device status. > > Thanks for clarifying. The example helped me understand a bit more on > how the potential use of the SND control interface. Since we're dealing > with multiple sound cards here (platform sound card (offload) and USB > SND card (legacy)), what do you think about creating a SND control on > both the USB backend (platform card) and the USB SND card listing the > PCM device status? > > That way at least userspace can have the information about which PCM dev > (USB substream) is available (and not offloaded, or vice versa). So the > USB SND control will contain the PCM devices (exposed by the card) and > if any are offloaded (if so mark them as unavailable). Likewise, for > the USB backend, if the legacy path is being used, mark them as > unavailable for offloading. We definitively need a control to indicate that a PCM offload device is available or not. There's still a very large open with the notion of having separate cards for the same audio device. Not only would it duplicate the control parts for e.g. volume control, but it would introduce the need to tag devices across two cards are being the same physical device. I still think the least-bad option is to have a single card and an optional PCM device for offload.
Hi Pierre, On 2/13/2023 7:22 AM, Pierre-Louis Bossart wrote: > > > On 2/11/23 03:52, Wesley Cheng wrote: >> Hi Pierre, >> >> On 2/7/2023 5:29 AM, Pierre-Louis Bossart wrote: >>> >>> >>> On 2/6/23 19:15, Wesley Cheng wrote: >>>> Hi Pierre, >>>> >>>> On 1/26/2023 8:12 AM, Pierre-Louis Bossart wrote: >>>>> >>>>> >>>>> On 1/25/23 21:14, Wesley Cheng wrote: >>>>>> With USB audio offloading, an audio session is started from the ASoC >>>>>> platform sound card and PCM devices. Likewise, the USB SND path is >>>>>> still >>>>>> readily available for use, in case the non-offload path is >>>>>> desired. In >>>>>> order to prevent the two entities from attempting to use the USB bus, >>>>>> introduce a flag that determines when either paths are in use. >>>>>> >>>>>> If a PCM device is already in use, the check will return an error to >>>>>> userspace notifying that the stream is currently busy. This ensures >>>>>> that >>>>>> only one path is using the USB substream. >>>>> >>>>> It's good to maintain mutual exclusion, but it's still very hard for an >>>>> application to figure out which card can be used when. >>>>> >>>>> Returning -EBUSY is not super helpful. There should be something like a >>>>> notification or connection status so that routing decisions can be made >>>>> without trial-and-error. >>>>> >>>> >>>> The USB offload driver does have access to the USB substream that is >>>> being utilized/offloaded. Maybe in addition to this check, we can also >>>> set the PCM runtime state as well (for that particular substream)? That >>>> way userspace can fetch information about if the stream is busy or not. >>> >>> You're missing the point. When a card is exposed but the PCM devices may >>> or may not be usable (consuming data with no sound rendered or returning >>> an error), it's much better to provide a clear connection status to >>> userspace. >>> >>> Let me give you an example. Intel drivers can expose 3 HDMI/DP PCM >>> devices. Userspace has no idea which one to use, so there's a jack >>> control that tells userspace whether there is a receiver connected so >>> that the audio server can use the relevant PCM device. >>> >>> Audio routing based on trial and error is really problematic, errors can >>> happen but they should be exceptional (e.g. xruns), not a means of >>> driver-userspace communication on the device status. >> >> Thanks for clarifying. The example helped me understand a bit more on >> how the potential use of the SND control interface. Since we're dealing >> with multiple sound cards here (platform sound card (offload) and USB >> SND card (legacy)), what do you think about creating a SND control on >> both the USB backend (platform card) and the USB SND card listing the >> PCM device status? >> >> That way at least userspace can have the information about which PCM dev >> (USB substream) is available (and not offloaded, or vice versa). So the >> USB SND control will contain the PCM devices (exposed by the card) and >> if any are offloaded (if so mark them as unavailable). Likewise, for >> the USB backend, if the legacy path is being used, mark them as >> unavailable for offloading. > > We definitively need a control to indicate that a PCM offload device is > available or not. > There's still a very large open with the notion of having separate cards > for the same audio device. Not only would it duplicate the control parts > for e.g. volume control, but it would introduce the need to tag devices > across two cards are being the same physical device. The volume control would still be done through the card that is exposed by the USB SND card (even for the offload path)[no vol control option for the USB device on the platform card]. In the last discussion, you did mention that maybe we can tag the offload path as the "power saving" option for a particular USB stream. Although I'm not sure how intricate the logic is, but if userspace marks to use the power saving path, then would it already know which card and PCM devices are involved? Although, that part is missing, ie to select the card and pcm device that we want to offload. It may be possible to do this with another control on the USB ASoC backend driver. I believe the audio DSP can support device selection. > I still think the least-bad option is to have a single card and an > optional PCM device for offload. This is most likely the end goal, but as mentioned previously, its going to be a large effort to slowly decouple some of the PCM related operations from USB SND. IMO, that would most likely be another significant patch series in itself. Thanks Wesley Cheng
diff --git a/sound/usb/card.h b/sound/usb/card.h index 410a4ffad98e..ff6d4695e727 100644 --- a/sound/usb/card.h +++ b/sound/usb/card.h @@ -163,6 +163,7 @@ struct snd_usb_substream { unsigned int pkt_offset_adj; /* Bytes to drop from beginning of packets (for non-compliant devices) */ unsigned int stream_offset_adj; /* Bytes to drop from beginning of stream (for non-compliant devices) */ + unsigned int opened:1; /* pcm device opened */ unsigned int running: 1; /* running status */ unsigned int period_elapsed_pending; /* delay period handling */ diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c index 0b01a5dfcb73..8946f8ddb892 100644 --- a/sound/usb/pcm.c +++ b/sound/usb/pcm.c @@ -1114,8 +1114,15 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream) struct snd_usb_stream *as = snd_pcm_substream_chip(substream); struct snd_pcm_runtime *runtime = substream->runtime; struct snd_usb_substream *subs = &as->substream[direction]; + struct snd_usb_audio *chip = subs->stream->chip; int ret; + mutex_lock(&chip->mutex); + if (subs->opened) { + mutex_unlock(&chip->mutex); + return -EBUSY; + } + runtime->hw = snd_usb_hardware; /* need an explicit sync to catch applptr update in low-latency mode */ if (direction == SNDRV_PCM_STREAM_PLAYBACK && @@ -1132,13 +1139,17 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream) ret = setup_hw_info(runtime, subs); if (ret < 0) - return ret; + goto out; ret = snd_usb_autoresume(subs->stream->chip); if (ret < 0) - return ret; + goto out; ret = snd_media_stream_init(subs, as->pcm, direction); if (ret < 0) snd_usb_autosuspend(subs->stream->chip); + subs->opened = 1; +out: + mutex_unlock(&chip->mutex); + return ret; } @@ -1147,6 +1158,7 @@ static int snd_usb_pcm_close(struct snd_pcm_substream *substream) int direction = substream->stream; struct snd_usb_stream *as = snd_pcm_substream_chip(substream); struct snd_usb_substream *subs = &as->substream[direction]; + struct snd_usb_audio *chip = subs->stream->chip; int ret; snd_media_stop_pipeline(subs); @@ -1160,6 +1172,9 @@ static int snd_usb_pcm_close(struct snd_pcm_substream *substream) subs->pcm_substream = NULL; snd_usb_autosuspend(subs->stream->chip); + mutex_lock(&chip->mutex); + subs->opened = 0; + mutex_unlock(&chip->mutex); return 0; } diff --git a/sound/usb/qcom/qc_audio_offload.c b/sound/usb/qcom/qc_audio_offload.c index c1254d5f680d..9bd09282e70d 100644 --- a/sound/usb/qcom/qc_audio_offload.c +++ b/sound/usb/qcom/qc_audio_offload.c @@ -1365,12 +1365,17 @@ static void handle_uaudio_stream_req(struct qmi_handle *handle, goto response; } + mutex_lock(&chip->mutex); if (req_msg->enable) { - if (info_idx < 0 || chip->system_suspend) { + if (info_idx < 0 || chip->system_suspend || subs->opened) { ret = -EBUSY; + mutex_unlock(&chip->mutex); + goto response; } + subs->opened = 1; } + mutex_unlock(&chip->mutex); if (req_msg->service_interval_valid) { ret = get_data_interval_from_si(subs, @@ -1392,6 +1397,11 @@ static void handle_uaudio_stream_req(struct qmi_handle *handle, if (!ret) ret = prepare_qmi_response(subs, req_msg, &resp, info_idx); + if (ret < 0) { + mutex_lock(&chip->mutex); + subs->opened = 0; + mutex_unlock(&chip->mutex); + } } else { info = &uadev[pcm_card_num].info[info_idx]; if (info->data_ep_pipe) { @@ -1413,6 +1423,9 @@ static void handle_uaudio_stream_req(struct qmi_handle *handle, } disable_audio_stream(subs); + mutex_lock(&chip->mutex); + subs->opened = 0; + mutex_unlock(&chip->mutex); } response: