[v2,00/11] arm64: qcom: add and enable SHM Bridge support

  From: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>

This is technically the second iteration of the SHM Bridge enablement on
QCom platforms but in practice it's a complete rewrite.

During the internal discussion with QCom the requirement has been established
as an SHM Bridge pool manager with the assumption that there will be multiple
users, each with their own pool. The problem with this is that we don't have
many potential users of SHM pools upstream at the moment which was rightfully
pointed out in the reviews under v1 (which even had some unused symbols etc.).

Moreover: after some investigating, it turns out that there simply isn't any
need for multiple pools as the core SCM only allocates a buffer if given call
requires more than 4 arguments and there are only a handful of SCM calls that
need to pass a physical address to a buffer as argument to the trustzone.

Additionally all but one SCM call allocate buffers passed to the TZ only for
the duration of the call and then free it right aftr it returns. The one
exception is called once and the buffer it uses can remain in memory until
released by the user.

This all makes using multiple pools wasteful as most of that memory will be
reserved but remain unused 99% of the time. What we need is a single pool of
SCM memory that deals out chunks of suitable format (coherent and
page-aligned) that fulfills the requirements of all calls.

As not all architectures support SHM bridge, it makes sense to first unify the
memory operations in SCM calls. All users do some kind of DMA mapping to obtain
buffers, most using dma_alloc_coherent() which impacts performance.

Genalloc pools are very fast so let's use them instead. Create a custom
allocator that also deals with the mapping and use it across all SCM calls.

Then once this is done, let's extend the allocator to use the SHM bridge
functionality if available on the given platform.

While at it: let's create a qcom specific directory in drivers/firmware/ and
move relevant code in there.

I couldn't test all SCM calls but tested with the inline crypto engine on
sm8550 and sa8775p that runs most of new code paths (with and without SHM
bridge). At least qseecom would need some Tested-by.

v1 -> v2:
- too many changes to list, it's a complete rewrite as explained above

Bartosz Golaszewski (11):
  firmware: qcom: move Qualcomm code into its own directory
  firmware: qcom: scm: add a dedicated SCM memory allocator
  firmware: qcom: scm: switch to using the SCM allocator
  firmware: qcom: scm: make qcom_scm_assign_mem() use the SCM allocator
  firmware: qcom: scm: make qcom_scm_ice_set_key() use the SCM allocator
  firmware: qcom: scm: make qcom_scm_pas_init_image() use the SCM
    allocator
  firmware: qcom: scm: make qcom_scm_lmh_dcvsh() use the SCM allocator
  firmware: qcom: scm: make qcom_scm_qseecom_app_get_id() use the SCM
    allocator
  firmware: qcom: qseecom: convert to using the SCM allocator
  firmware: qcom-scm: add support for SHM bridge operations
  firmware: qcom: scm: enable SHM bridge

 MAINTAINERS                                   |   4 +-
 drivers/firmware/Kconfig                      |  48 +---
 drivers/firmware/Makefile                     |   5 +-
 drivers/firmware/qcom/Kconfig                 |  56 ++++
 drivers/firmware/qcom/Makefile                |   9 +
 drivers/firmware/{ => qcom}/qcom_qseecom.c    |   0
 .../{ => qcom}/qcom_qseecom_uefisecapp.c      | 251 ++++++------------
 drivers/firmware/{ => qcom}/qcom_scm-legacy.c |   0
 drivers/firmware/qcom/qcom_scm-mem.c          | 170 ++++++++++++
 drivers/firmware/{ => qcom}/qcom_scm-smc.c    |  21 +-
 drivers/firmware/{ => qcom}/qcom_scm.c        | 149 ++++++-----
 drivers/firmware/{ => qcom}/qcom_scm.h        |   9 +
 include/linux/firmware/qcom/qcom_qseecom.h    |   4 +-
 include/linux/firmware/qcom/qcom_scm.h        |  13 +
 14 files changed, 427 insertions(+), 312 deletions(-)
 create mode 100644 drivers/firmware/qcom/Kconfig
 create mode 100644 drivers/firmware/qcom/Makefile
 rename drivers/firmware/{ => qcom}/qcom_qseecom.c (100%)
 rename drivers/firmware/{ => qcom}/qcom_qseecom_uefisecapp.c (84%)
 rename drivers/firmware/{ => qcom}/qcom_scm-legacy.c (100%)
 create mode 100644 drivers/firmware/qcom/qcom_scm-mem.c
 rename drivers/firmware/{ => qcom}/qcom_scm-smc.c (92%)
 rename drivers/firmware/{ => qcom}/qcom_scm.c (94%)
 rename drivers/firmware/{ => qcom}/qcom_scm.h (95%)
  

On 9/28/2023 2:20 AM, Bartosz Golaszewski wrote:
> From: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
> 
> We need to allocate, map and pass a buffer to the trustzone if we have
> more than 4 arguments for a given SCM calls. Let's use the new SCM
> allocator for that memory and shrink the code in process.
> 
> Signed-off-by: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>

Reviewed-by: Elliot Berman <quic_eberman@quicinc.com>

> ---
>  drivers/firmware/qcom/qcom_scm-smc.c | 21 ++++-----------------
>  1 file changed, 4 insertions(+), 17 deletions(-)
> 
> diff --git a/drivers/firmware/qcom/qcom_scm-smc.c b/drivers/firmware/qcom/qcom_scm-smc.c
> index 16cf88acfa8e..0d5554df1321 100644
> --- a/drivers/firmware/qcom/qcom_scm-smc.c
> +++ b/drivers/firmware/qcom/qcom_scm-smc.c
> @@ -2,6 +2,7 @@
>  /* Copyright (c) 2015,2019 The Linux Foundation. All rights reserved.
>   */
>  
> +#include <linux/cleanup.h>
>  #include <linux/io.h>
>  #include <linux/errno.h>
>  #include <linux/delay.h>
> @@ -152,8 +153,7 @@ int __scm_smc_call(struct device *dev, const struct qcom_scm_desc *desc,
>  {
>  	int arglen = desc->arginfo & 0xf;
>  	int i, ret;
> -	dma_addr_t args_phys = 0;
> -	void *args_virt = NULL;
> +	void *args_virt __free(qcom_scm_mem) = NULL;
>  	size_t alloc_len;
>  	gfp_t flag = atomic ? GFP_ATOMIC : GFP_KERNEL;
>  	u32 smccc_call_type = atomic ? ARM_SMCCC_FAST_CALL : ARM_SMCCC_STD_CALL;
> @@ -173,7 +173,7 @@ int __scm_smc_call(struct device *dev, const struct qcom_scm_desc *desc,
>  
>  	if (unlikely(arglen > SCM_SMC_N_REG_ARGS)) {
>  		alloc_len = SCM_SMC_N_EXT_ARGS * sizeof(u64);
> -		args_virt = kzalloc(PAGE_ALIGN(alloc_len), flag);
> +		args_virt = qcom_scm_mem_alloc(PAGE_ALIGN(alloc_len), flag);
>  
>  		if (!args_virt)
>  			return -ENOMEM;
> @@ -192,25 +192,12 @@ int __scm_smc_call(struct device *dev, const struct qcom_scm_desc *desc,
>  						      SCM_SMC_FIRST_EXT_IDX]);
>  		}
>  
> -		args_phys = dma_map_single(dev, args_virt, alloc_len,
> -					   DMA_TO_DEVICE);
> -
> -		if (dma_mapping_error(dev, args_phys)) {
> -			kfree(args_virt);
> -			return -ENOMEM;
> -		}
> -
> -		smc.args[SCM_SMC_LAST_REG_IDX] = args_phys;
> +		smc.args[SCM_SMC_LAST_REG_IDX] = qcom_scm_mem_to_phys(args_virt);
>  	}
>  
>  	/* ret error check follows after args_virt cleanup*/
>  	ret = __scm_smc_do(dev, &smc, &smc_res, atomic);
>  
> -	if (args_virt) {
> -		dma_unmap_single(dev, args_phys, alloc_len, DMA_TO_DEVICE);
> -		kfree(args_virt);
> -	}
> -
>  	if (ret)
>  		return ret;
>
  

On Thu, Sep 28, 2023 at 11:20:29AM +0200, Bartosz Golaszewski wrote:
> From: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
>
> This is technically the second iteration of the SHM Bridge enablement on
> QCom platforms but in practice it's a complete rewrite.
>
> During the internal discussion with QCom the requirement has been established
> as an SHM Bridge pool manager with the assumption that there will be multiple
> users, each with their own pool. The problem with this is that we don't have
> many potential users of SHM pools upstream at the moment which was rightfully
> pointed out in the reviews under v1 (which even had some unused symbols etc.).
>
> Moreover: after some investigating, it turns out that there simply isn't any
> need for multiple pools as the core SCM only allocates a buffer if given call
> requires more than 4 arguments and there are only a handful of SCM calls that
> need to pass a physical address to a buffer as argument to the trustzone.
>
> Additionally all but one SCM call allocate buffers passed to the TZ only for
> the duration of the call and then free it right aftr it returns. The one
> exception is called once and the buffer it uses can remain in memory until
> released by the user.
>
> This all makes using multiple pools wasteful as most of that memory will be
> reserved but remain unused 99% of the time. What we need is a single pool of
> SCM memory that deals out chunks of suitable format (coherent and
> page-aligned) that fulfills the requirements of all calls.
>
> As not all architectures support SHM bridge, it makes sense to first unify the
> memory operations in SCM calls. All users do some kind of DMA mapping to obtain
> buffers, most using dma_alloc_coherent() which impacts performance.
>
> Genalloc pools are very fast so let's use them instead. Create a custom
> allocator that also deals with the mapping and use it across all SCM calls.
>
> Then once this is done, let's extend the allocator to use the SHM bridge
> functionality if available on the given platform.
>
> While at it: let's create a qcom specific directory in drivers/firmware/ and
> move relevant code in there.
>
> I couldn't test all SCM calls but tested with the inline crypto engine on
> sm8550 and sa8775p that runs most of new code paths (with and without SHM
> bridge). At least qseecom would need some Tested-by.

I have been playing around with this on my x13s (sc8280xp). It seems
that efivars works ok (and therefore the qseecom stuff I believe), but
with these patches applied I'm getting -22 when loading any firmware.

I'll try to dig a little more, but thought I'd report that before I
context switch for a little bit.

    dmesg | grep "firmware\|-22"
    [    0.000000] psci: PSCIv1.1 detected in firmware.
    [    2.351999] qcom_scm firmware:scm: SHM Bridge enabled
    [    2.353002] qcom_scm firmware:scm: qseecom: found qseecom with version 0x1402000
    [    6.727604] systemd[1]: systemd-pcrmachine.service - TPM2 PCR Machine ID Measurement was skipped because of an unmet condition check (ConditionPathExists=/sys/firmware/efi/efivars/StubPcrKernelImage-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f).
    [    8.198381] qcom_q6v5_pas 1b300000.remoteproc: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qccdsp8280.mbn
    [    8.198418] remoteproc remoteproc1: can't start rproc 1b300000.remoteproc: -22
    [    8.407641] qcom_q6v5_pas 3000000.remoteproc: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcadsp8280.mbn
    [    8.407742] remoteproc remoteproc0: can't start rproc 3000000.remoteproc: -22
    [    8.588496] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [    8.588509] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [    9.392185] ath11k_pci 0006:01:00.0: fw_version 0x110b196e fw_build_timestamp 2022-12-22 12:54 fw_build_id WLAN.HSP.1.1-03125-QCAHSPSWPL_V1_V2_SILICONZ_LITE-3.6510.23
    [   10.229367] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   10.229383] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   11.041385] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   11.041399] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   11.070144] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   11.070160] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   11.321999] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   11.322015] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   11.340989] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   11.341002] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   11.576180] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   11.576198] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   11.593647] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   11.593661] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   11.854212] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   11.854226] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   11.879925] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   11.879937] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   12.157090] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   12.157106] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
    [   12.183074] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
    [   12.183088] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22

Thanks,
Andrew
>
> v1 -> v2:
> - too many changes to list, it's a complete rewrite as explained above
>
> Bartosz Golaszewski (11):
>   firmware: qcom: move Qualcomm code into its own directory
>   firmware: qcom: scm: add a dedicated SCM memory allocator
>   firmware: qcom: scm: switch to using the SCM allocator
>   firmware: qcom: scm: make qcom_scm_assign_mem() use the SCM allocator
>   firmware: qcom: scm: make qcom_scm_ice_set_key() use the SCM allocator
>   firmware: qcom: scm: make qcom_scm_pas_init_image() use the SCM
>     allocator
>   firmware: qcom: scm: make qcom_scm_lmh_dcvsh() use the SCM allocator
>   firmware: qcom: scm: make qcom_scm_qseecom_app_get_id() use the SCM
>     allocator
>   firmware: qcom: qseecom: convert to using the SCM allocator
>   firmware: qcom-scm: add support for SHM bridge operations
>   firmware: qcom: scm: enable SHM bridge
>
>  MAINTAINERS                                   |   4 +-
>  drivers/firmware/Kconfig                      |  48 +---
>  drivers/firmware/Makefile                     |   5 +-
>  drivers/firmware/qcom/Kconfig                 |  56 ++++
>  drivers/firmware/qcom/Makefile                |   9 +
>  drivers/firmware/{ => qcom}/qcom_qseecom.c    |   0
>  .../{ => qcom}/qcom_qseecom_uefisecapp.c      | 251 ++++++------------
>  drivers/firmware/{ => qcom}/qcom_scm-legacy.c |   0
>  drivers/firmware/qcom/qcom_scm-mem.c          | 170 ++++++++++++
>  drivers/firmware/{ => qcom}/qcom_scm-smc.c    |  21 +-
>  drivers/firmware/{ => qcom}/qcom_scm.c        | 149 ++++++-----
>  drivers/firmware/{ => qcom}/qcom_scm.h        |   9 +
>  include/linux/firmware/qcom/qcom_qseecom.h    |   4 +-
>  include/linux/firmware/qcom/qcom_scm.h        |  13 +
>  14 files changed, 427 insertions(+), 312 deletions(-)
>  create mode 100644 drivers/firmware/qcom/Kconfig
>  create mode 100644 drivers/firmware/qcom/Makefile
>  rename drivers/firmware/{ => qcom}/qcom_qseecom.c (100%)
>  rename drivers/firmware/{ => qcom}/qcom_qseecom_uefisecapp.c (84%)
>  rename drivers/firmware/{ => qcom}/qcom_scm-legacy.c (100%)
>  create mode 100644 drivers/firmware/qcom/qcom_scm-mem.c
>  rename drivers/firmware/{ => qcom}/qcom_scm-smc.c (92%)
>  rename drivers/firmware/{ => qcom}/qcom_scm.c (94%)
>  rename drivers/firmware/{ => qcom}/qcom_scm.h (95%)
>
> --
> 2.39.2
>
  

On Fri, Sep 29, 2023 at 5:29 PM Andrew Halaney <ahalaney@redhat.com> wrote:
>
> On Thu, Sep 28, 2023 at 11:20:29AM +0200, Bartosz Golaszewski wrote:
> > From: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
> >
> > This is technically the second iteration of the SHM Bridge enablement on
> > QCom platforms but in practice it's a complete rewrite.
> >
> > During the internal discussion with QCom the requirement has been established
> > as an SHM Bridge pool manager with the assumption that there will be multiple
> > users, each with their own pool. The problem with this is that we don't have
> > many potential users of SHM pools upstream at the moment which was rightfully
> > pointed out in the reviews under v1 (which even had some unused symbols etc.).
> >
> > Moreover: after some investigating, it turns out that there simply isn't any
> > need for multiple pools as the core SCM only allocates a buffer if given call
> > requires more than 4 arguments and there are only a handful of SCM calls that
> > need to pass a physical address to a buffer as argument to the trustzone.
> >
> > Additionally all but one SCM call allocate buffers passed to the TZ only for
> > the duration of the call and then free it right aftr it returns. The one
> > exception is called once and the buffer it uses can remain in memory until
> > released by the user.
> >
> > This all makes using multiple pools wasteful as most of that memory will be
> > reserved but remain unused 99% of the time. What we need is a single pool of
> > SCM memory that deals out chunks of suitable format (coherent and
> > page-aligned) that fulfills the requirements of all calls.
> >
> > As not all architectures support SHM bridge, it makes sense to first unify the
> > memory operations in SCM calls. All users do some kind of DMA mapping to obtain
> > buffers, most using dma_alloc_coherent() which impacts performance.
> >
> > Genalloc pools are very fast so let's use them instead. Create a custom
> > allocator that also deals with the mapping and use it across all SCM calls.
> >
> > Then once this is done, let's extend the allocator to use the SHM bridge
> > functionality if available on the given platform.
> >
> > While at it: let's create a qcom specific directory in drivers/firmware/ and
> > move relevant code in there.
> >
> > I couldn't test all SCM calls but tested with the inline crypto engine on
> > sm8550 and sa8775p that runs most of new code paths (with and without SHM
> > bridge). At least qseecom would need some Tested-by.
>
> I have been playing around with this on my x13s (sc8280xp). It seems
> that efivars works ok (and therefore the qseecom stuff I believe), but
> with these patches applied I'm getting -22 when loading any firmware.
>
> I'll try to dig a little more, but thought I'd report that before I
> context switch for a little bit.
>
>     dmesg | grep "firmware\|-22"
>     [    0.000000] psci: PSCIv1.1 detected in firmware.
>     [    2.351999] qcom_scm firmware:scm: SHM Bridge enabled
>     [    2.353002] qcom_scm firmware:scm: qseecom: found qseecom with version 0x1402000
>     [    6.727604] systemd[1]: systemd-pcrmachine.service - TPM2 PCR Machine ID Measurement was skipped because of an unmet condition check (ConditionPathExists=/sys/firmware/efi/efivars/StubPcrKernelImage-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f).
>     [    8.198381] qcom_q6v5_pas 1b300000.remoteproc: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qccdsp8280.mbn
>     [    8.198418] remoteproc remoteproc1: can't start rproc 1b300000.remoteproc: -22
>     [    8.407641] qcom_q6v5_pas 3000000.remoteproc: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcadsp8280.mbn
>     [    8.407742] remoteproc remoteproc0: can't start rproc 3000000.remoteproc: -22
>     [    8.588496] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
>     [    8.588509] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
>     [    9.392185] ath11k_pci 0006:01:00.0: fw_version 0x110b196e fw_build_timestamp 2022-12-22 12:54 fw_build_id WLAN.HSP.1.1-03125-QCAHSPSWPL_V1_V2_SILICONZ_LITE-3.6510.23
>     [   10.229367] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
>     [   10.229383] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
>     [   11.041385] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
>     [   11.041399] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
>     [   11.070144] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
>     [   11.070160] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
>     [   11.321999] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
>     [   11.322015] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
>     [   11.340989] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
>     [   11.341002] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
>     [   11.576180] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
>     [   11.576198] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
>     [   11.593647] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
>     [   11.593661] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
>     [   11.854212] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
>     [   11.854226] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
>     [   11.879925] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
>     [   11.879937] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
>     [   12.157090] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
>     [   12.157106] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
>     [   12.183074] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
>     [   12.183088] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
>
> Thanks,
> Andrew

Huh remoteproc seems to work fine on sm8550. Can you post the full
kernel log? Do you know which SCM calls fails?

Bart

> >
> > v1 -> v2:
> > - too many changes to list, it's a complete rewrite as explained above
> >
> > Bartosz Golaszewski (11):
> >   firmware: qcom: move Qualcomm code into its own directory
> >   firmware: qcom: scm: add a dedicated SCM memory allocator
> >   firmware: qcom: scm: switch to using the SCM allocator
> >   firmware: qcom: scm: make qcom_scm_assign_mem() use the SCM allocator
> >   firmware: qcom: scm: make qcom_scm_ice_set_key() use the SCM allocator
> >   firmware: qcom: scm: make qcom_scm_pas_init_image() use the SCM
> >     allocator
> >   firmware: qcom: scm: make qcom_scm_lmh_dcvsh() use the SCM allocator
> >   firmware: qcom: scm: make qcom_scm_qseecom_app_get_id() use the SCM
> >     allocator
> >   firmware: qcom: qseecom: convert to using the SCM allocator
> >   firmware: qcom-scm: add support for SHM bridge operations
> >   firmware: qcom: scm: enable SHM bridge
> >
> >  MAINTAINERS                                   |   4 +-
> >  drivers/firmware/Kconfig                      |  48 +---
> >  drivers/firmware/Makefile                     |   5 +-
> >  drivers/firmware/qcom/Kconfig                 |  56 ++++
> >  drivers/firmware/qcom/Makefile                |   9 +
> >  drivers/firmware/{ => qcom}/qcom_qseecom.c    |   0
> >  .../{ => qcom}/qcom_qseecom_uefisecapp.c      | 251 ++++++------------
> >  drivers/firmware/{ => qcom}/qcom_scm-legacy.c |   0
> >  drivers/firmware/qcom/qcom_scm-mem.c          | 170 ++++++++++++
> >  drivers/firmware/{ => qcom}/qcom_scm-smc.c    |  21 +-
> >  drivers/firmware/{ => qcom}/qcom_scm.c        | 149 ++++++-----
> >  drivers/firmware/{ => qcom}/qcom_scm.h        |   9 +
> >  include/linux/firmware/qcom/qcom_qseecom.h    |   4 +-
> >  include/linux/firmware/qcom/qcom_scm.h        |  13 +
> >  14 files changed, 427 insertions(+), 312 deletions(-)
> >  create mode 100644 drivers/firmware/qcom/Kconfig
> >  create mode 100644 drivers/firmware/qcom/Makefile
> >  rename drivers/firmware/{ => qcom}/qcom_qseecom.c (100%)
> >  rename drivers/firmware/{ => qcom}/qcom_qseecom_uefisecapp.c (84%)
> >  rename drivers/firmware/{ => qcom}/qcom_scm-legacy.c (100%)
> >  create mode 100644 drivers/firmware/qcom/qcom_scm-mem.c
> >  rename drivers/firmware/{ => qcom}/qcom_scm-smc.c (92%)
> >  rename drivers/firmware/{ => qcom}/qcom_scm.c (94%)
> >  rename drivers/firmware/{ => qcom}/qcom_scm.h (95%)
> >
> > --
> > 2.39.2
> >
>
  

On Fri, Sep 29, 2023 at 08:56:57PM +0200, Bartosz Golaszewski wrote:
> On Fri, Sep 29, 2023 at 5:29 PM Andrew Halaney <ahalaney@redhat.com> wrote:
> >
> > On Thu, Sep 28, 2023 at 11:20:29AM +0200, Bartosz Golaszewski wrote:
> > > From: Bartosz Golaszewski <bartosz.golaszewski@linaro.org>
> > >
> > > This is technically the second iteration of the SHM Bridge enablement on
> > > QCom platforms but in practice it's a complete rewrite.
> > >
> > > During the internal discussion with QCom the requirement has been established
> > > as an SHM Bridge pool manager with the assumption that there will be multiple
> > > users, each with their own pool. The problem with this is that we don't have
> > > many potential users of SHM pools upstream at the moment which was rightfully
> > > pointed out in the reviews under v1 (which even had some unused symbols etc.).
> > >
> > > Moreover: after some investigating, it turns out that there simply isn't any
> > > need for multiple pools as the core SCM only allocates a buffer if given call
> > > requires more than 4 arguments and there are only a handful of SCM calls that
> > > need to pass a physical address to a buffer as argument to the trustzone.
> > >
> > > Additionally all but one SCM call allocate buffers passed to the TZ only for
> > > the duration of the call and then free it right aftr it returns. The one
> > > exception is called once and the buffer it uses can remain in memory until
> > > released by the user.
> > >
> > > This all makes using multiple pools wasteful as most of that memory will be
> > > reserved but remain unused 99% of the time. What we need is a single pool of
> > > SCM memory that deals out chunks of suitable format (coherent and
> > > page-aligned) that fulfills the requirements of all calls.
> > >
> > > As not all architectures support SHM bridge, it makes sense to first unify the
> > > memory operations in SCM calls. All users do some kind of DMA mapping to obtain
> > > buffers, most using dma_alloc_coherent() which impacts performance.
> > >
> > > Genalloc pools are very fast so let's use them instead. Create a custom
> > > allocator that also deals with the mapping and use it across all SCM calls.
> > >
> > > Then once this is done, let's extend the allocator to use the SHM bridge
> > > functionality if available on the given platform.
> > >
> > > While at it: let's create a qcom specific directory in drivers/firmware/ and
> > > move relevant code in there.
> > >
> > > I couldn't test all SCM calls but tested with the inline crypto engine on
> > > sm8550 and sa8775p that runs most of new code paths (with and without SHM
> > > bridge). At least qseecom would need some Tested-by.
> >
> > I have been playing around with this on my x13s (sc8280xp). It seems
> > that efivars works ok (and therefore the qseecom stuff I believe), but
> > with these patches applied I'm getting -22 when loading any firmware.
> >
> > I'll try to dig a little more, but thought I'd report that before I
> > context switch for a little bit.
> >
> >     dmesg | grep "firmware\|-22"
> >     [    0.000000] psci: PSCIv1.1 detected in firmware.
> >     [    2.351999] qcom_scm firmware:scm: SHM Bridge enabled
> >     [    2.353002] qcom_scm firmware:scm: qseecom: found qseecom with version 0x1402000
> >     [    6.727604] systemd[1]: systemd-pcrmachine.service - TPM2 PCR Machine ID Measurement was skipped because of an unmet condition check (ConditionPathExists=/sys/firmware/efi/efivars/StubPcrKernelImage-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f).
> >     [    8.198381] qcom_q6v5_pas 1b300000.remoteproc: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qccdsp8280.mbn
> >     [    8.198418] remoteproc remoteproc1: can't start rproc 1b300000.remoteproc: -22
> >     [    8.407641] qcom_q6v5_pas 3000000.remoteproc: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcadsp8280.mbn
> >     [    8.407742] remoteproc remoteproc0: can't start rproc 3000000.remoteproc: -22
> >     [    8.588496] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [    8.588509] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [    9.392185] ath11k_pci 0006:01:00.0: fw_version 0x110b196e fw_build_timestamp 2022-12-22 12:54 fw_build_id WLAN.HSP.1.1-03125-QCAHSPSWPL_V1_V2_SILICONZ_LITE-3.6510.23
> >     [   10.229367] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   10.229383] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   11.041385] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   11.041399] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   11.070144] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   11.070160] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   11.321999] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   11.322015] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   11.340989] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   11.341002] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   11.576180] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   11.576198] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   11.593647] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   11.593661] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   11.854212] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   11.854226] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   11.879925] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   11.879937] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   12.157090] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   12.157106] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >     [   12.183074] adreno 3d00000.gpu: error -22 initializing firmware qcom/sc8280xp/LENOVO/21BX/qcdxkmsuc8280.mbn
> >     [   12.183088] msm_dpu ae01000.display-controller: [drm:adreno_load_gpu [msm]] *ERROR* gpu hw init failed: -22
> >
> > Thanks,
> > Andrew
> 
> Huh remoteproc seems to work fine on sm8550. Can you post the full
> kernel log? Do you know which SCM calls fails?
> 
> Bart

See my response on patch 6, and please let me know if you I messed up
the logic there and need to dig a little more. The log didn't have
anything useful outside what is shown here unfortunately, but I can
gather more if you refute that comment on patch 6.

Thanks,
Andrew