Message ID | 20221222133123.50676-4-manivannan.sadhasivam@linaro.org |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp327357wrn; Thu, 22 Dec 2022 05:35:12 -0800 (PST) X-Google-Smtp-Source: AMrXdXuGBss0/Oi0yAtJVeNTFRN+Vrjztr3uCIC9ABDgsict62k5QQP9JxUBZCDFfSMyD9HQzMW9 X-Received: by 2002:a17:906:99d1:b0:7c1:12ef:bf52 with SMTP id s17-20020a17090699d100b007c112efbf52mr4249586ejn.3.1671716112486; Thu, 22 Dec 2022 05:35:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671716112; cv=none; d=google.com; s=arc-20160816; b=MMJGusUS5VDTulJIxsJbrGejMNepynW2ICilSqeeIuX5CQ8vNf3RKmiiyN4Cem237k tg9nhU8br+Fc0ivz4YDunvdlmVlXsdLBlRmcW/HQSv+KBoAf1JlH/LDOulFH7F0gI7cm v4wPNhhHbQ+YbQDz5t8CBqz49UNfKTZyRhnIueeqQ0Fqfa/UZsjtrpaQW820yQ7l/gIH t5/MqGvCqDuz+RxLrz7VDH7scL70uaDXuwqo1dk6z3MTME7D/zG8ATgfSP4QNg1JUdM/ BfXNkxOf8jYPjttx5vbHQvdXkYmwgYIrw/nJcsbybzt8V7LrWnoK8qay0jX6m6NO4Rll 3cXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=2xuFcNG+Lmd6sCAcx37A72Ddu+m83mu9NNvxLdsptSw=; b=egkgE5tAk7hYYYbMyE5S1NCFeCmcX7hpRwUFJeW4mHnXOFlp9lPd6m3kEL9lxs8ELE qGRrbo/fhozj0fukeZlfgJ4EfKWNXvPMNTrERoF2p1uG4W55kPHJ4acoTMxeqbkChaCs NI5L0f1GLUKbqs/ETbghL1wG2K2XdCcTgpTbIhcK05rtHfTWn9clUPCwJaT2icY2MUA5 8cZErZ/rocauTtyN6K3jacTe3WSLIDYPcAQlhP52W1aO/8tglvLNQ4k78vTAY3o/2cW1 ChO9sPLA3IJDBbgbxGfppY9V7pHpqyuBsLKK9rphdZrmhtsswEBoTVgkW7MDirbNbBWf ebNw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=nA0ejjbc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q30-20020a50cc9e000000b004604906b23csi665581edi.545.2022.12.22.05.34.48; Thu, 22 Dec 2022 05:35:12 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=nA0ejjbc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235212AbiLVNcA (ORCPT <rfc822;pacteraone@gmail.com> + 99 others); Thu, 22 Dec 2022 08:32:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57334 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234727AbiLVNbw (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Thu, 22 Dec 2022 08:31:52 -0500 Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5E55A13FAB for <linux-kernel@vger.kernel.org>; Thu, 22 Dec 2022 05:31:51 -0800 (PST) Received: by mail-pj1-x1035.google.com with SMTP id w4-20020a17090ac98400b002186f5d7a4cso5757496pjt.0 for <linux-kernel@vger.kernel.org>; Thu, 22 Dec 2022 05:31:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2xuFcNG+Lmd6sCAcx37A72Ddu+m83mu9NNvxLdsptSw=; b=nA0ejjbc4ytOQVnxKuefw8MAuoRRUqjMaGNYnMY+ffjK/YbyiaNBpzJcU+FMdCNPv9 qzJ+NCcvT5bJcFZsC+3gSV9hOTT10rdAe7l12HO/wqnJ3E5e1Iz1Oyo7zDYY6iJfo2Yr abNHgiimpIb4/+OxWWJ83YaQciW8gx1jdh/CiWuA4yoRQ9JXoZU/J6WLVLCG9eWXpYqZ Pzssu2lDki9ge53ut13Cr6nDhJJvEng5+Gt0G0Yd6CJh9ZXuScFZdiV+socMh4eDvQWG qyXqE6bmxNqbqhIaRZK4pZDCEg7UDXXXZzL53S0KezKK9Mr5F6o/kxJiNUQTQKE37WxJ UAAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2xuFcNG+Lmd6sCAcx37A72Ddu+m83mu9NNvxLdsptSw=; b=FITo8zn0/W8urRfx9WpC94B0RhhWRapGqN+WF2zk+3qoRlreEHIPusZHr9cx3pGjec OBek7YgDe0KDr9OvjuJectC4C/BBHmAapRE7VqkRhVJthYjYYgDh/L166xaMKwSEKkm5 +fvBipGuDxiXHpHk/x6D8V960tGacyCYdyLTAdkmRQc6Ywj+0WIcPezyFir3V8xCI9WV Q5QQEJOjIHPg+GeG/sG/nvoCML17uXZNChzCeQq6uDEewAKi6mcQav45EiwMB6J+Mrsx kQZKmgDoSwTACmZqjriTGqGFsIcKCJEnbt8y5XtRspx7u8+/fhwW22v3PUWvgrgGUpGv 3n7w== X-Gm-Message-State: AFqh2kqaPRqT5MHY4ibPWfgLjp+LTZFz0r2p+voX1H/hrQeR+FuD7z/y VSxicHDdbEdgpXIiVMaL5Y0u X-Received: by 2002:a05:6a20:3d14:b0:af:b909:2b3f with SMTP id y20-20020a056a203d1400b000afb9092b3fmr8965864pzi.34.1671715910863; Thu, 22 Dec 2022 05:31:50 -0800 (PST) Received: from localhost.localdomain ([117.217.177.99]) by smtp.gmail.com with ESMTPSA id f66-20020a623845000000b00573a9d13e9esm737467pfa.36.2022.12.22.05.31.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Dec 2022 05:31:50 -0800 (PST) From: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> To: andersson@kernel.org, robh+dt@kernel.org, krzysztof.kozlowski+dt@linaro.org Cc: bhelgaas@google.com, konrad.dybcio@linaro.org, linux-arm-msm@vger.kernel.org, linux-pci@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> Subject: [PATCH v2 3/3] arm64: dts: qcom: sm8450: Use GIC-ITS for PCIe0 and PCIe1 Date: Thu, 22 Dec 2022 19:01:23 +0530 Message-Id: <20221222133123.50676-4-manivannan.sadhasivam@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221222133123.50676-1-manivannan.sadhasivam@linaro.org> References: <20221222133123.50676-1-manivannan.sadhasivam@linaro.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752921394275995956?= X-GMAIL-MSGID: =?utf-8?q?1752921394275995956?= |
Series |
Qcom: Add GIC-ITS support to SM8450 PCIe controllers
|
|
Commit Message
Manivannan Sadhasivam
Dec. 22, 2022, 1:31 p.m. UTC
Both PCIe0 and PCIe1 controllers are capable of receiving MSIs from
endpoint devices using GIC-ITS MSI controller. Add support for it.
Currently, BDF (0:0.0) and BDF (1:0.0) are enabled and with the
msi-map-mask of 0xff00, all the 32 devices under these two busses can
share the same Device ID.
The GIC-ITS MSI implementation provides an advantage over internal MSI
implementation using Locality-specific Peripheral Interrupts (LPI) that
would allow MSIs to be targeted for each CPU core.
It should be noted that the MSIs for BDF (1:0.0) only works with Device
ID of 0x5980 and 0x5a00. Hence, the IDs are swapped.
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
---
arch/arm64/boot/dts/qcom/sm8450.dtsi | 20 ++++++++++++++------
1 file changed, 14 insertions(+), 6 deletions(-)
Comments
On 22/12/2022 15:31, Manivannan Sadhasivam wrote: > Both PCIe0 and PCIe1 controllers are capable of receiving MSIs from > endpoint devices using GIC-ITS MSI controller. Add support for it. > > Currently, BDF (0:0.0) and BDF (1:0.0) are enabled and with the > msi-map-mask of 0xff00, all the 32 devices under these two busses can > share the same Device ID. > > The GIC-ITS MSI implementation provides an advantage over internal MSI > implementation using Locality-specific Peripheral Interrupts (LPI) that > would allow MSIs to be targeted for each CPU core. > > It should be noted that the MSIs for BDF (1:0.0) only works with Device > ID of 0x5980 and 0x5a00. Hence, the IDs are swapped. > > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> > --- > arch/arm64/boot/dts/qcom/sm8450.dtsi | 20 ++++++++++++++------ > 1 file changed, 14 insertions(+), 6 deletions(-) > > diff --git a/arch/arm64/boot/dts/qcom/sm8450.dtsi b/arch/arm64/boot/dts/qcom/sm8450.dtsi > index 570475040d95..c4dd5838fac6 100644 > --- a/arch/arm64/boot/dts/qcom/sm8450.dtsi > +++ b/arch/arm64/boot/dts/qcom/sm8450.dtsi > @@ -1733,9 +1733,13 @@ pcie0: pci@1c00000 { > ranges = <0x01000000 0x0 0x60200000 0 0x60200000 0x0 0x100000>, > <0x02000000 0x0 0x60300000 0 0x60300000 0x0 0x3d00000>; > > - interrupts = <GIC_SPI 141 IRQ_TYPE_LEVEL_HIGH>; > - interrupt-names = "msi"; > - #interrupt-cells = <1>; > + /* > + * MSIs for BDF (1:0.0) only works with Device ID 0x5980. > + * Hence, the IDs are swapped. > + */ > + msi-map = <0x0 &gic_its 0x5981 0x1>, > + <0x100 &gic_its 0x5980 0x1>; This definitely doesn't match what has been used in the downstream. Also if I understand correctly this change would prevent us from using multiple MSI interrupts for the connected device, as the last value of the 0x100 mapping is 0x1, while the vendor kernel uses <0x100 &its 0x5981 0x20>. Do you know by chance, why do we differ from the vendor dtsi? > + msi-map-mask = <0xff00>; > interrupt-map-mask = <0 0 0 0x7>; > interrupt-map = <0 0 0 1 &intc 0 0 0 149 IRQ_TYPE_LEVEL_HIGH>, /* int_a */ > <0 0 0 2 &intc 0 0 0 150 IRQ_TYPE_LEVEL_HIGH>, /* int_b */ > @@ -1842,9 +1846,13 @@ pcie1: pci@1c08000 { > ranges = <0x01000000 0x0 0x40200000 0 0x40200000 0x0 0x100000>, > <0x02000000 0x0 0x40300000 0 0x40300000 0x0 0x1fd00000>; > > - interrupts = <GIC_SPI 307 IRQ_TYPE_LEVEL_HIGH>; > - interrupt-names = "msi"; > - #interrupt-cells = <1>; > + /* > + * MSIs for BDF (1:0.0) only works with Device ID 0x5a00. > + * Hence, the IDs are swapped. > + */ > + msi-map = <0x0 &gic_its 0x5a01 0x1>, > + <0x100 &gic_its 0x5a00 0x1>; > + msi-map-mask = <0xff00>; > interrupt-map-mask = <0 0 0 0x7>; > interrupt-map = <0 0 0 1 &intc 0 0 0 434 IRQ_TYPE_LEVEL_HIGH>, /* int_a */ > <0 0 0 2 &intc 0 0 0 435 IRQ_TYPE_LEVEL_HIGH>, /* int_b */
On Fri, Dec 23, 2022 at 07:18:32PM +0200, Dmitry Baryshkov wrote: > On 22/12/2022 15:31, Manivannan Sadhasivam wrote: > > Both PCIe0 and PCIe1 controllers are capable of receiving MSIs from > > endpoint devices using GIC-ITS MSI controller. Add support for it. > > > > Currently, BDF (0:0.0) and BDF (1:0.0) are enabled and with the > > msi-map-mask of 0xff00, all the 32 devices under these two busses can > > share the same Device ID. > > > > The GIC-ITS MSI implementation provides an advantage over internal MSI > > implementation using Locality-specific Peripheral Interrupts (LPI) that > > would allow MSIs to be targeted for each CPU core. > > > > It should be noted that the MSIs for BDF (1:0.0) only works with Device > > ID of 0x5980 and 0x5a00. Hence, the IDs are swapped. > > > > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> > > --- > > arch/arm64/boot/dts/qcom/sm8450.dtsi | 20 ++++++++++++++------ > > 1 file changed, 14 insertions(+), 6 deletions(-) > > > > diff --git a/arch/arm64/boot/dts/qcom/sm8450.dtsi b/arch/arm64/boot/dts/qcom/sm8450.dtsi > > index 570475040d95..c4dd5838fac6 100644 > > --- a/arch/arm64/boot/dts/qcom/sm8450.dtsi > > +++ b/arch/arm64/boot/dts/qcom/sm8450.dtsi > > @@ -1733,9 +1733,13 @@ pcie0: pci@1c00000 { > > ranges = <0x01000000 0x0 0x60200000 0 0x60200000 0x0 0x100000>, > > <0x02000000 0x0 0x60300000 0 0x60300000 0x0 0x3d00000>; > > - interrupts = <GIC_SPI 141 IRQ_TYPE_LEVEL_HIGH>; > > - interrupt-names = "msi"; > > - #interrupt-cells = <1>; > > + /* > > + * MSIs for BDF (1:0.0) only works with Device ID 0x5980. > > + * Hence, the IDs are swapped. > > + */ > > + msi-map = <0x0 &gic_its 0x5981 0x1>, > > + <0x100 &gic_its 0x5980 0x1>; > > This definitely doesn't match what has been used in the downstream. > Yes, I do not know why the downstream Device ID doesn't work. I tried finding the answer within Qcom but didn't get any answer so far :/ So I just went with the value that works on multiple boards. > Also if I understand correctly this change would prevent us from using > multiple MSI interrupts for the connected device, as the last value of the > 0x100 mapping is 0x1, while the vendor kernel uses <0x100 &its 0x5981 0x20>. > Not true. The controller can still support multiple MSIs for the endpoint devices but the only difference is, it would use the same Device ID for all. The Qcom GIC-ITS implementation could only support 32 Device IDs. By specifying the size of 0x20, a separate Device ID would be used for each devices of bus 1. But if a PCIe switch is connected and the bus count becomes > 1, then the MSI allocation would fail because Device IDs are exhausted. The downstream implementation just assumes that there will be only bus 1 and I do not want to follow that assumption. That's why I used "msi-map-mask" property of value "0xff00" here, as that will allow all the devices under the bus 1 to share the same Device ID. For now I only mapped bus 1, but extending that in the future for other busses is simple. Thanks, Mani > Do you know by chance, why do we differ from the vendor dtsi? > > > + msi-map-mask = <0xff00>; > > interrupt-map-mask = <0 0 0 0x7>; > > interrupt-map = <0 0 0 1 &intc 0 0 0 149 IRQ_TYPE_LEVEL_HIGH>, /* int_a */ > > <0 0 0 2 &intc 0 0 0 150 IRQ_TYPE_LEVEL_HIGH>, /* int_b */ > > @@ -1842,9 +1846,13 @@ pcie1: pci@1c08000 { > > ranges = <0x01000000 0x0 0x40200000 0 0x40200000 0x0 0x100000>, > > <0x02000000 0x0 0x40300000 0 0x40300000 0x0 0x1fd00000>; > > - interrupts = <GIC_SPI 307 IRQ_TYPE_LEVEL_HIGH>; > > - interrupt-names = "msi"; > > - #interrupt-cells = <1>; > > + /* > > + * MSIs for BDF (1:0.0) only works with Device ID 0x5a00. > > + * Hence, the IDs are swapped. > > + */ > > + msi-map = <0x0 &gic_its 0x5a01 0x1>, > > + <0x100 &gic_its 0x5a00 0x1>; > > + msi-map-mask = <0xff00>; > > interrupt-map-mask = <0 0 0 0x7>; > > interrupt-map = <0 0 0 1 &intc 0 0 0 434 IRQ_TYPE_LEVEL_HIGH>, /* int_a */ > > <0 0 0 2 &intc 0 0 0 435 IRQ_TYPE_LEVEL_HIGH>, /* int_b */ > > -- > With best wishes > Dmitry >
On 23/12/2022 19:45, Manivannan Sadhasivam wrote: > On Fri, Dec 23, 2022 at 07:18:32PM +0200, Dmitry Baryshkov wrote: >> On 22/12/2022 15:31, Manivannan Sadhasivam wrote: >>> Both PCIe0 and PCIe1 controllers are capable of receiving MSIs from >>> endpoint devices using GIC-ITS MSI controller. Add support for it. >>> >>> Currently, BDF (0:0.0) and BDF (1:0.0) are enabled and with the >>> msi-map-mask of 0xff00, all the 32 devices under these two busses can >>> share the same Device ID. >>> >>> The GIC-ITS MSI implementation provides an advantage over internal MSI >>> implementation using Locality-specific Peripheral Interrupts (LPI) that >>> would allow MSIs to be targeted for each CPU core. >>> >>> It should be noted that the MSIs for BDF (1:0.0) only works with Device >>> ID of 0x5980 and 0x5a00. Hence, the IDs are swapped. >>> >>> Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> >>> --- >>> arch/arm64/boot/dts/qcom/sm8450.dtsi | 20 ++++++++++++++------ >>> 1 file changed, 14 insertions(+), 6 deletions(-) >>> >>> diff --git a/arch/arm64/boot/dts/qcom/sm8450.dtsi b/arch/arm64/boot/dts/qcom/sm8450.dtsi >>> index 570475040d95..c4dd5838fac6 100644 >>> --- a/arch/arm64/boot/dts/qcom/sm8450.dtsi >>> +++ b/arch/arm64/boot/dts/qcom/sm8450.dtsi >>> @@ -1733,9 +1733,13 @@ pcie0: pci@1c00000 { >>> ranges = <0x01000000 0x0 0x60200000 0 0x60200000 0x0 0x100000>, >>> <0x02000000 0x0 0x60300000 0 0x60300000 0x0 0x3d00000>; >>> - interrupts = <GIC_SPI 141 IRQ_TYPE_LEVEL_HIGH>; >>> - interrupt-names = "msi"; >>> - #interrupt-cells = <1>; >>> + /* >>> + * MSIs for BDF (1:0.0) only works with Device ID 0x5980. >>> + * Hence, the IDs are swapped. >>> + */ >>> + msi-map = <0x0 &gic_its 0x5981 0x1>, >>> + <0x100 &gic_its 0x5980 0x1>; >> >> This definitely doesn't match what has been used in the downstream. >> > > Yes, I do not know why the downstream Device ID doesn't work. I tried finding > the answer within Qcom but didn't get any answer so far :/ So I just went with > the value that works on multiple boards. Ugh :-( > >> Also if I understand correctly this change would prevent us from using >> multiple MSI interrupts for the connected device, as the last value of the >> 0x100 mapping is 0x1, while the vendor kernel uses <0x100 &its 0x5981 0x20>. >> > > Not true. The controller can still support multiple MSIs for the endpoint > devices but the only difference is, it would use the same Device ID for all. I see, please excuse me then. But don't we have to define multiple MSI vectors here too? > > The Qcom GIC-ITS implementation could only support 32 Device IDs. By specifying > the size of 0x20, a separate Device ID would be used for each devices of bus 1. > But if a PCIe switch is connected and the bus count becomes > 1, then the MSI > allocation would fail because Device IDs are exhausted. > > The downstream implementation just assumes that there will be only bus 1 and I > do not want to follow that assumption. > > That's why I used "msi-map-mask" property of value "0xff00" here, as that will > allow all the devices under the bus 1 to share the same Device ID. For now I > only mapped bus 1, but extending that in the future for other busses is simple. > > Thanks, > Mani > >> Do you know by chance, why do we differ from the vendor dtsi? >> >>> + msi-map-mask = <0xff00>; >>> interrupt-map-mask = <0 0 0 0x7>; >>> interrupt-map = <0 0 0 1 &intc 0 0 0 149 IRQ_TYPE_LEVEL_HIGH>, /* int_a */ >>> <0 0 0 2 &intc 0 0 0 150 IRQ_TYPE_LEVEL_HIGH>, /* int_b */ >>> @@ -1842,9 +1846,13 @@ pcie1: pci@1c08000 { >>> ranges = <0x01000000 0x0 0x40200000 0 0x40200000 0x0 0x100000>, >>> <0x02000000 0x0 0x40300000 0 0x40300000 0x0 0x1fd00000>; >>> - interrupts = <GIC_SPI 307 IRQ_TYPE_LEVEL_HIGH>; >>> - interrupt-names = "msi"; >>> - #interrupt-cells = <1>; >>> + /* >>> + * MSIs for BDF (1:0.0) only works with Device ID 0x5a00. >>> + * Hence, the IDs are swapped. >>> + */ >>> + msi-map = <0x0 &gic_its 0x5a01 0x1>, >>> + <0x100 &gic_its 0x5a00 0x1>; >>> + msi-map-mask = <0xff00>; >>> interrupt-map-mask = <0 0 0 0x7>; >>> interrupt-map = <0 0 0 1 &intc 0 0 0 434 IRQ_TYPE_LEVEL_HIGH>, /* int_a */ >>> <0 0 0 2 &intc 0 0 0 435 IRQ_TYPE_LEVEL_HIGH>, /* int_b */ >> >> -- >> With best wishes >> Dmitry >> >
On Fri, Dec 23, 2022 at 08:15:32PM +0200, Dmitry Baryshkov wrote: > On 23/12/2022 19:45, Manivannan Sadhasivam wrote: > > On Fri, Dec 23, 2022 at 07:18:32PM +0200, Dmitry Baryshkov wrote: > > > On 22/12/2022 15:31, Manivannan Sadhasivam wrote: > > > > Both PCIe0 and PCIe1 controllers are capable of receiving MSIs from > > > > endpoint devices using GIC-ITS MSI controller. Add support for it. > > > > > > > > Currently, BDF (0:0.0) and BDF (1:0.0) are enabled and with the > > > > msi-map-mask of 0xff00, all the 32 devices under these two busses can > > > > share the same Device ID. > > > > > > > > The GIC-ITS MSI implementation provides an advantage over internal MSI > > > > implementation using Locality-specific Peripheral Interrupts (LPI) that > > > > would allow MSIs to be targeted for each CPU core. > > > > > > > > It should be noted that the MSIs for BDF (1:0.0) only works with Device > > > > ID of 0x5980 and 0x5a00. Hence, the IDs are swapped. > > > > > > > > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> > > > > --- > > > > arch/arm64/boot/dts/qcom/sm8450.dtsi | 20 ++++++++++++++------ > > > > 1 file changed, 14 insertions(+), 6 deletions(-) > > > > > > > > diff --git a/arch/arm64/boot/dts/qcom/sm8450.dtsi b/arch/arm64/boot/dts/qcom/sm8450.dtsi > > > > index 570475040d95..c4dd5838fac6 100644 > > > > --- a/arch/arm64/boot/dts/qcom/sm8450.dtsi > > > > +++ b/arch/arm64/boot/dts/qcom/sm8450.dtsi > > > > @@ -1733,9 +1733,13 @@ pcie0: pci@1c00000 { > > > > ranges = <0x01000000 0x0 0x60200000 0 0x60200000 0x0 0x100000>, > > > > <0x02000000 0x0 0x60300000 0 0x60300000 0x0 0x3d00000>; > > > > - interrupts = <GIC_SPI 141 IRQ_TYPE_LEVEL_HIGH>; > > > > - interrupt-names = "msi"; > > > > - #interrupt-cells = <1>; > > > > + /* > > > > + * MSIs for BDF (1:0.0) only works with Device ID 0x5980. > > > > + * Hence, the IDs are swapped. > > > > + */ > > > > + msi-map = <0x0 &gic_its 0x5981 0x1>, > > > > + <0x100 &gic_its 0x5980 0x1>; > > > > > > This definitely doesn't match what has been used in the downstream. > > > > > > > Yes, I do not know why the downstream Device ID doesn't work. I tried finding > > the answer within Qcom but didn't get any answer so far :/ So I just went with > > the value that works on multiple boards. > > Ugh :-( > > > > > > Also if I understand correctly this change would prevent us from using > > > multiple MSI interrupts for the connected device, as the last value of the > > > 0x100 mapping is 0x1, while the vendor kernel uses <0x100 &its 0x5981 0x20>. > > > > > > > Not true. The controller can still support multiple MSIs for the endpoint > > devices but the only difference is, it would use the same Device ID for all. > > I see, please excuse me then. But don't we have to define multiple MSI > vectors here too? > No, it is not required. GIC-ITS driver will handle the MSI mapping internally and devicetree only needs to specify the Device ID for each PCIe device. Thanks, Mani > > > > The Qcom GIC-ITS implementation could only support 32 Device IDs. By specifying > > the size of 0x20, a separate Device ID would be used for each devices of bus 1. > > But if a PCIe switch is connected and the bus count becomes > 1, then the MSI > > allocation would fail because Device IDs are exhausted. > > > > The downstream implementation just assumes that there will be only bus 1 and I > > do not want to follow that assumption. > > > > That's why I used "msi-map-mask" property of value "0xff00" here, as that will > > allow all the devices under the bus 1 to share the same Device ID. For now I > > only mapped bus 1, but extending that in the future for other busses is simple. > > > > Thanks, > > Mani > > > > > Do you know by chance, why do we differ from the vendor dtsi? > > > > > > > + msi-map-mask = <0xff00>; > > > > interrupt-map-mask = <0 0 0 0x7>; > > > > interrupt-map = <0 0 0 1 &intc 0 0 0 149 IRQ_TYPE_LEVEL_HIGH>, /* int_a */ > > > > <0 0 0 2 &intc 0 0 0 150 IRQ_TYPE_LEVEL_HIGH>, /* int_b */ > > > > @@ -1842,9 +1846,13 @@ pcie1: pci@1c08000 { > > > > ranges = <0x01000000 0x0 0x40200000 0 0x40200000 0x0 0x100000>, > > > > <0x02000000 0x0 0x40300000 0 0x40300000 0x0 0x1fd00000>; > > > > - interrupts = <GIC_SPI 307 IRQ_TYPE_LEVEL_HIGH>; > > > > - interrupt-names = "msi"; > > > > - #interrupt-cells = <1>; > > > > + /* > > > > + * MSIs for BDF (1:0.0) only works with Device ID 0x5a00. > > > > + * Hence, the IDs are swapped. > > > > + */ > > > > + msi-map = <0x0 &gic_its 0x5a01 0x1>, > > > > + <0x100 &gic_its 0x5a00 0x1>; > > > > + msi-map-mask = <0xff00>; > > > > interrupt-map-mask = <0 0 0 0x7>; > > > > interrupt-map = <0 0 0 1 &intc 0 0 0 434 IRQ_TYPE_LEVEL_HIGH>, /* int_a */ > > > > <0 0 0 2 &intc 0 0 0 435 IRQ_TYPE_LEVEL_HIGH>, /* int_b */ > > > > > > -- > > > With best wishes > > > Dmitry > > > > > > > -- > With best wishes > Dmitry >
On Thu, Dec 22, 2022 at 07:01:23PM +0530, Manivannan Sadhasivam wrote: > Both PCIe0 and PCIe1 controllers are capable of receiving MSIs from > endpoint devices using GIC-ITS MSI controller. Add support for it. Nit: the PCI controllers don't receive MSIs using the GIC-ITS MSI controller, they signal MSIs interrupts using the GIC-ITS controller. > > Currently, BDF (0:0.0) and BDF (1:0.0) are enabled and with the > msi-map-mask of 0xff00, all the 32 devices under these two busses can > share the same Device ID. > > The GIC-ITS MSI implementation provides an advantage over internal MSI > implementation using Locality-specific Peripheral Interrupts (LPI) that > would allow MSIs to be targeted for each CPU core. > > It should be noted that the MSIs for BDF (1:0.0) only works with Device > ID of 0x5980 and 0x5a00. Hence, the IDs are swapped. > > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> > --- > arch/arm64/boot/dts/qcom/sm8450.dtsi | 20 ++++++++++++++------ > 1 file changed, 14 insertions(+), 6 deletions(-) > > diff --git a/arch/arm64/boot/dts/qcom/sm8450.dtsi b/arch/arm64/boot/dts/qcom/sm8450.dtsi > index 570475040d95..c4dd5838fac6 100644 > --- a/arch/arm64/boot/dts/qcom/sm8450.dtsi > +++ b/arch/arm64/boot/dts/qcom/sm8450.dtsi > @@ -1733,9 +1733,13 @@ pcie0: pci@1c00000 { > ranges = <0x01000000 0x0 0x60200000 0 0x60200000 0x0 0x100000>, > <0x02000000 0x0 0x60300000 0 0x60300000 0x0 0x3d00000>; > > - interrupts = <GIC_SPI 141 IRQ_TYPE_LEVEL_HIGH>; > - interrupt-names = "msi"; > - #interrupt-cells = <1>; > + /* > + * MSIs for BDF (1:0.0) only works with Device ID 0x5980. > + * Hence, the IDs are swapped. > + */ > + msi-map = <0x0 &gic_its 0x5981 0x1>, > + <0x100 &gic_its 0x5980 0x1>; > + msi-map-mask = <0xff00>; > interrupt-map-mask = <0 0 0 0x7>; > interrupt-map = <0 0 0 1 &intc 0 0 0 149 IRQ_TYPE_LEVEL_HIGH>, /* int_a */ > <0 0 0 2 &intc 0 0 0 150 IRQ_TYPE_LEVEL_HIGH>, /* int_b */ > @@ -1842,9 +1846,13 @@ pcie1: pci@1c08000 { > ranges = <0x01000000 0x0 0x40200000 0 0x40200000 0x0 0x100000>, > <0x02000000 0x0 0x40300000 0 0x40300000 0x0 0x1fd00000>; > > - interrupts = <GIC_SPI 307 IRQ_TYPE_LEVEL_HIGH>; > - interrupt-names = "msi"; > - #interrupt-cells = <1>; > + /* > + * MSIs for BDF (1:0.0) only works with Device ID 0x5a00. > + * Hence, the IDs are swapped. > + */ > + msi-map = <0x0 &gic_its 0x5a01 0x1>, > + <0x100 &gic_its 0x5a00 0x1>; > + msi-map-mask = <0xff00>; > interrupt-map-mask = <0 0 0 0x7>; > interrupt-map = <0 0 0 1 &intc 0 0 0 434 IRQ_TYPE_LEVEL_HIGH>, /* int_a */ > <0 0 0 2 &intc 0 0 0 435 IRQ_TYPE_LEVEL_HIGH>, /* int_b */ > -- > 2.25.1 >
diff --git a/arch/arm64/boot/dts/qcom/sm8450.dtsi b/arch/arm64/boot/dts/qcom/sm8450.dtsi index 570475040d95..c4dd5838fac6 100644 --- a/arch/arm64/boot/dts/qcom/sm8450.dtsi +++ b/arch/arm64/boot/dts/qcom/sm8450.dtsi @@ -1733,9 +1733,13 @@ pcie0: pci@1c00000 { ranges = <0x01000000 0x0 0x60200000 0 0x60200000 0x0 0x100000>, <0x02000000 0x0 0x60300000 0 0x60300000 0x0 0x3d00000>; - interrupts = <GIC_SPI 141 IRQ_TYPE_LEVEL_HIGH>; - interrupt-names = "msi"; - #interrupt-cells = <1>; + /* + * MSIs for BDF (1:0.0) only works with Device ID 0x5980. + * Hence, the IDs are swapped. + */ + msi-map = <0x0 &gic_its 0x5981 0x1>, + <0x100 &gic_its 0x5980 0x1>; + msi-map-mask = <0xff00>; interrupt-map-mask = <0 0 0 0x7>; interrupt-map = <0 0 0 1 &intc 0 0 0 149 IRQ_TYPE_LEVEL_HIGH>, /* int_a */ <0 0 0 2 &intc 0 0 0 150 IRQ_TYPE_LEVEL_HIGH>, /* int_b */ @@ -1842,9 +1846,13 @@ pcie1: pci@1c08000 { ranges = <0x01000000 0x0 0x40200000 0 0x40200000 0x0 0x100000>, <0x02000000 0x0 0x40300000 0 0x40300000 0x0 0x1fd00000>; - interrupts = <GIC_SPI 307 IRQ_TYPE_LEVEL_HIGH>; - interrupt-names = "msi"; - #interrupt-cells = <1>; + /* + * MSIs for BDF (1:0.0) only works with Device ID 0x5a00. + * Hence, the IDs are swapped. + */ + msi-map = <0x0 &gic_its 0x5a01 0x1>, + <0x100 &gic_its 0x5a00 0x1>; + msi-map-mask = <0xff00>; interrupt-map-mask = <0 0 0 0x7>; interrupt-map = <0 0 0 1 &intc 0 0 0 434 IRQ_TYPE_LEVEL_HIGH>, /* int_a */ <0 0 0 2 &intc 0 0 0 435 IRQ_TYPE_LEVEL_HIGH>, /* int_b */