Message ID | 1686030570-5439-1-git-send-email-quic_krichai@quicinc.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp3171639vqr; Mon, 5 Jun 2023 23:04:27 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7JDEEmZTrIfsivYketiYclzL2wOu+VS5RO4PNKu+O7uuS/TgqiCeEw4+GoSdWg5L+yAkG2 X-Received: by 2002:a17:902:c14a:b0:1af:ea40:34e6 with SMTP id 10-20020a170902c14a00b001afea4034e6mr452134plj.60.1686031467402; Mon, 05 Jun 2023 23:04:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686031467; cv=none; d=google.com; s=arc-20160816; b=pR6EIOWeIcsC2heTl1wuHory78PdwjuZsbAPE/7S03+wiN6eq+Su4c6RMxacFhM4SY HsTDJZunwcvv3cTCjXrb+HV94CmhXWrCYUPIe7rNEAFptvQOLndArMS/CQIRoNKurbpo ovEBHudbMTLDTi7PwktbuV6KDqDo6nezka433FQdKW+MykI9oNpevsA9Oc76AxaRewsF bTBSvQ1511ry+6ZqtY6BKJMhzmrKth5MEKZ5Ew+6HbtEHk0fzeDSP407AVefQL3TrvSh yLxC/+eBK4a1hirRNETRmYu2ZbxmvvIo0QaCkA1BOd7uL8jnx6VKqc35Su8hKfQAwRVR /Jww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:message-id:date:subject:cc:to:from :dkim-signature; bh=aQ6KnwOt1mwRRlCCHY+UFOi0CAj60uT8JU+m6VHJT9I=; b=Zn/N/+2nfyXCxsUVuUlirVz41peDgi7Lbb4Bt+EESZ+mrZwJ6M+Wv55jnTVw6TIyxx mrj1DVbO733s+2Qr0VZnA9L6O6/CiNubvbOFIoMawPyfA07lsmFbtzE15N/s8cuVAGLf pFmqUhoNWeqlu9b0hSKib/0b3bWKbCJRBSjIaJ4fh7Gs02MhI1/5Cw7+uClnBl9KOyli BCprG5cw5q4oXSMN92OOzuj3g/keP96idl9pPFN4z4uvatpzbcB5+Xdt9gdnU+v64wFo u4XFuVenwxxB6R7qVRWJfB3/5dL9eKgm5pb5tyOU9XfbWCYhrAkkFdbEgXePZJ/KZBn7 K8JQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=Xu44K9su; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a10-20020a170902ecca00b001b184b4bb75si6521671plh.381.2023.06.05.23.04.14; Mon, 05 Jun 2023 23:04:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@quicinc.com header.s=qcppdkim1 header.b=Xu44K9su; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=quicinc.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233765AbjFFFuB (ORCPT <rfc822;xxoosimple@gmail.com> + 99 others); Tue, 6 Jun 2023 01:50:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50490 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231670AbjFFFt7 (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Tue, 6 Jun 2023 01:49:59 -0400 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 77EDF9E; Mon, 5 Jun 2023 22:49:58 -0700 (PDT) Received: from pps.filterd (m0279865.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3561Jvq2007753; Tue, 6 Jun 2023 05:49:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id; s=qcppdkim1; bh=aQ6KnwOt1mwRRlCCHY+UFOi0CAj60uT8JU+m6VHJT9I=; b=Xu44K9su7X2iH4DmyKH+Lu6InkbauADcpZSlksSjsLeD8/F936yL5xjM4SNchIaTiZW6 9vhu8Ydx04eGWEAHw9/nUcKYQGtK4g7ro7jpvI4fpkFWPYhVym/I6dBtWFPJr2Rt8Av7 vflwolp4VIsJXXmZtyMh5jnXAlbSa6JO0/AlVLhjgkNokIf6QZBzDSAoEmUpUvjsYWVT 9w13oIkyxhRTngI4LymJ1rcRZRuwO6anaHdwR32ZvZhuSXECxjwGih2EcGzdmLDQDPw2 Mj3u9B0PI3iMELW5ZIv22YOdzFYmrP9dUswHpHzbmPTY+gWMTF6QiEBJnxkc/xsena3J YA== Received: from apblrppmta02.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3r1d4et5vx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 06 Jun 2023 05:49:45 +0000 Received: from pps.filterd (APBLRPPMTA02.qualcomm.com [127.0.0.1]) by APBLRPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3565nef8022823; Tue, 6 Jun 2023 05:49:40 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA02.qualcomm.com (PPS) with ESMTP id 3qyxkka702-1; Tue, 06 Jun 2023 05:49:40 +0000 Received: from APBLRPPMTA02.qualcomm.com (APBLRPPMTA02.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3565neGi022817; Tue, 6 Jun 2023 05:49:40 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-krichai-hyd.qualcomm.com [10.213.110.112]) by APBLRPPMTA02.qualcomm.com (PPS) with ESMTP id 3565ndPT022816; Tue, 06 Jun 2023 05:49:40 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 4058933) id 2CD50376D; Tue, 6 Jun 2023 11:19:39 +0530 (+0530) From: Krishna chaitanya chundru <quic_krichai@quicinc.com> To: manivannan.sadhasivam@linaro.org Cc: quic_vbadigan@quicinc.com, quic_ramkri@quicinc.com, Krishna chaitanya chundru <quic_krichai@quicinc.com>, Manivannan Sadhasivam <mani@kernel.org>, Lorenzo Pieralisi <lpieralisi@kernel.org>, =?utf-8?q?Krzysztof_Wilczy=C5=84?= =?utf-8?q?ski?= <kw@linux.com>, Rob Herring <robh@kernel.org>, Bjorn Helgaas <bhelgaas@google.com>, linux-pci@vger.kernel.org (open list:PCIE ENDPOINT DRIVER FOR QUALCOMM), linux-arm-msm@vger.kernel.org (open list:PCIE ENDPOINT DRIVER FOR QUALCOMM), linux-kernel@vger.kernel.org (open list) Subject: [PATCH] PCI: qcom-ep: Add ICC bandwidth voting support Date: Tue, 6 Jun 2023 11:19:29 +0530 Message-Id: <1686030570-5439-1-git-send-email-quic_krichai@quicinc.com> X-Mailer: git-send-email 2.7.4 X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: 1DPZgL_PPjqpcx1wvrIyr-YKE5TnYmq_ X-Proofpoint-GUID: 1DPZgL_PPjqpcx1wvrIyr-YKE5TnYmq_ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-06_03,2023-06-05_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 phishscore=0 clxscore=1011 priorityscore=1501 malwarescore=0 mlxscore=0 lowpriorityscore=0 adultscore=0 impostorscore=0 spamscore=0 mlxlogscore=964 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2306060049 X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_LOW,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767932131769844106?= X-GMAIL-MSGID: =?utf-8?q?1767932131769844106?= |
Series |
PCI: qcom-ep: Add ICC bandwidth voting support
|
|
Commit Message
Krishna chaitanya chundru
June 6, 2023, 5:49 a.m. UTC
Add support to vote for ICC bandwidth based up on the link
speed and width.
Signed-off-by: Krishna chaitanya chundru <quic_krichai@quicinc.com>
---
drivers/pci/controller/dwc/pcie-qcom-ep.c | 73 +++++++++++++++++++++++++++++++
1 file changed, 73 insertions(+)
Comments
Hi Krishna, Thanks for the patch! On 6.06.23 8:49, Krishna chaitanya chundru wrote: > Add support to vote for ICC bandwidth based up on the link > speed and width. > > Signed-off-by: Krishna chaitanya chundru <quic_krichai@quicinc.com> > --- > drivers/pci/controller/dwc/pcie-qcom-ep.c | 73 +++++++++++++++++++++++++++++++ > 1 file changed, 73 insertions(+) > > diff --git a/drivers/pci/controller/dwc/pcie-qcom-ep.c b/drivers/pci/controller/dwc/pcie-qcom-ep.c > index 19b3283..79e7559 100644 > --- a/drivers/pci/controller/dwc/pcie-qcom-ep.c > +++ b/drivers/pci/controller/dwc/pcie-qcom-ep.c > @@ -17,6 +17,7 @@ > #include <linux/phy/pcie.h> > #include <linux/phy/phy.h> > #include <linux/platform_device.h> > +#include <linux/interconnect.h> > #include <linux/pm_domain.h> > #include <linux/regmap.h> > #include <linux/reset.h> > @@ -28,6 +29,7 @@ > #define PARF_SYS_CTRL 0x00 > #define PARF_DB_CTRL 0x10 > #define PARF_PM_CTRL 0x20 > +#define PARF_PM_STTS 0x24 > #define PARF_MHI_CLOCK_RESET_CTRL 0x174 > #define PARF_MHI_BASE_ADDR_LOWER 0x178 > #define PARF_MHI_BASE_ADDR_UPPER 0x17c > @@ -128,6 +130,9 @@ > /* DBI register fields */ > #define DBI_CON_STATUS_POWER_STATE_MASK GENMASK(1, 0) > > +#define DBI_LINKCTRLSTATUS 0x80 > +#define DBI_LINKCTRKSTATUS_SHIFT 16 > + > #define XMLH_LINK_UP 0x400 > #define CORE_RESET_TIME_US_MIN 1000 > #define CORE_RESET_TIME_US_MAX 1005 > @@ -187,6 +192,8 @@ struct qcom_pcie_ep { > enum qcom_pcie_ep_link_status link_status; > int global_irq; > int perst_irq; > + > + struct icc_path *icc; > }; > > static int qcom_pcie_ep_core_reset(struct qcom_pcie_ep *pcie_ep) > @@ -253,9 +260,56 @@ static void qcom_pcie_dw_stop_link(struct dw_pcie *pci) > disable_irq(pcie_ep->perst_irq); > } > > +static void qcom_pcie_icc_update(struct qcom_pcie_ep *pcie_ep) > +{ > + struct dw_pcie *pci = &pcie_ep->pci; > + u32 val, bw; > + int speed, width; > + int ret; > + > + if (!pcie_ep->icc) > + return; > + > + val = dw_pcie_readl_dbi(pci, DBI_LINKCTRLSTATUS); > + val = val >> DBI_LINKCTRKSTATUS_SHIFT; > + > + speed = FIELD_GET(PCI_EXP_LNKSTA_CLS, val); > + width = FIELD_GET(PCI_EXP_LNKSTA_NLW, val); > + > + /* > + * ICC needs avg bw in KBps. > + * > + * For example for 2Gbps the avg BW = 2x1000x1000x1000/8*1000 = 250000 > + */ > + switch (speed) { > + case 1: > + bw = 250000; /* avg bw for GEN1 per lane: 2Gbps, peak bw: no vote */ > + break; > + case 2: > + bw = 500000; /* avg bw for GEN2 per lane: 4Gbps, peak bw no vote */ > + break; > + case 3: > + bw = 1000000; /* avg bw for GEN3 per lane: 8Gbps, peak bw no vote */ > + break; > + default: > + WARN_ON_ONCE(1); > + fallthrough; > + case 4: > + bw = 2000000; /* avg bw for GEN4 per lane: 16Gbps, peak bw no vote */ > + break; > + } > + > + ret = icc_set_bw(pcie_ep->icc, width * bw, 0); Here you should use a non-zero value for peak bandwidth. You can use the average value also as peak. There are some existing macros like GBps_to_icc(). Please use them. > + if (ret) { > + dev_err(pci->dev, "failed to set interconnect bandwidth: %d\n", > + ret); > + } > +} > + > static int qcom_pcie_enable_resources(struct qcom_pcie_ep *pcie_ep) > { > int ret; > + struct dw_pcie *pci = &pcie_ep->pci; > > ret = clk_bulk_prepare_enable(pcie_ep->num_clks, pcie_ep->clks); > if (ret) > @@ -277,6 +331,20 @@ static int qcom_pcie_enable_resources(struct qcom_pcie_ep *pcie_ep) > if (ret) > goto err_phy_exit; > > + /* > + * Some Qualcomm platforms require interconnect bandwidth constraints > + * to be set before enabling interconnect clocks. > + * > + * Set an initial average bandwidth corresponding to single-lane Gen 1 > + * for the pcie to mem path. > + */ > + ret = icc_set_bw(pcie_ep->icc, 250000, 0); /* avg bw: 2Gbps, peak bw: no vote */ Ditto. > + if (ret) { > + dev_err(pci->dev, "failed to set interconnect bandwidth: %d\n", > + ret); > + goto err_phy_exit; > + } > + > return 0; > > err_phy_exit: > @@ -550,6 +618,10 @@ static int qcom_pcie_ep_get_resources(struct platform_device *pdev, > if (IS_ERR(pcie_ep->phy)) > ret = PTR_ERR(pcie_ep->phy); > > + pcie_ep->icc = devm_of_icc_get(dev, "pci"); Is this "pci" path documented in the bindings? Thanks, Georgi > + if (IS_ERR(pcie_ep->icc)) > + ret = PTR_ERR(pcie_ep->icc); > + > return ret; > } > > @@ -572,6 +644,7 @@ static irqreturn_t qcom_pcie_ep_global_irq_thread(int irq, void *data) > } else if (FIELD_GET(PARF_INT_ALL_BME, status)) { > dev_dbg(dev, "Received BME event. Link is enabled!\n"); > pcie_ep->link_status = QCOM_PCIE_EP_LINK_ENABLED; > + qcom_pcie_icc_update(pcie_ep); > } else if (FIELD_GET(PARF_INT_ALL_PM_TURNOFF, status)) { > dev_dbg(dev, "Received PM Turn-off event! Entering L23\n"); > val = readl_relaxed(pcie_ep->parf + PARF_PM_CTRL);
On Tue, Jun 06, 2023 at 11:19:29AM +0530, Krishna chaitanya chundru wrote: > Add support to vote for ICC bandwidth based up on the link based on > speed and width. > Looks like the code got inspiration from pcie-qcom driver. So it should be mentioned in the commit message. > Signed-off-by: Krishna chaitanya chundru <quic_krichai@quicinc.com> Devicetree bindings update should precede this patch. > --- > drivers/pci/controller/dwc/pcie-qcom-ep.c | 73 +++++++++++++++++++++++++++++++ > 1 file changed, 73 insertions(+) > > diff --git a/drivers/pci/controller/dwc/pcie-qcom-ep.c b/drivers/pci/controller/dwc/pcie-qcom-ep.c > index 19b3283..79e7559 100644 > --- a/drivers/pci/controller/dwc/pcie-qcom-ep.c > +++ b/drivers/pci/controller/dwc/pcie-qcom-ep.c > @@ -17,6 +17,7 @@ > #include <linux/phy/pcie.h> > #include <linux/phy/phy.h> > #include <linux/platform_device.h> > +#include <linux/interconnect.h> Includes are sorted alphabetically > #include <linux/pm_domain.h> > #include <linux/regmap.h> > #include <linux/reset.h> > @@ -28,6 +29,7 @@ > #define PARF_SYS_CTRL 0x00 > #define PARF_DB_CTRL 0x10 > #define PARF_PM_CTRL 0x20 > +#define PARF_PM_STTS 0x24 > #define PARF_MHI_CLOCK_RESET_CTRL 0x174 > #define PARF_MHI_BASE_ADDR_LOWER 0x178 > #define PARF_MHI_BASE_ADDR_UPPER 0x17c > @@ -128,6 +130,9 @@ > /* DBI register fields */ > #define DBI_CON_STATUS_POWER_STATE_MASK GENMASK(1, 0) > > +#define DBI_LINKCTRLSTATUS 0x80 > +#define DBI_LINKCTRKSTATUS_SHIFT 16 Use GENMASK macro > + > #define XMLH_LINK_UP 0x400 > #define CORE_RESET_TIME_US_MIN 1000 > #define CORE_RESET_TIME_US_MAX 1005 > @@ -187,6 +192,8 @@ struct qcom_pcie_ep { > enum qcom_pcie_ep_link_status link_status; > int global_irq; > int perst_irq; > + > + struct icc_path *icc; Place this under debugfs entry. > }; > > static int qcom_pcie_ep_core_reset(struct qcom_pcie_ep *pcie_ep) > @@ -253,9 +260,56 @@ static void qcom_pcie_dw_stop_link(struct dw_pcie *pci) > disable_irq(pcie_ep->perst_irq); > } > > +static void qcom_pcie_icc_update(struct qcom_pcie_ep *pcie_ep) qcom_pcie_ep_icc_update > +{ > + struct dw_pcie *pci = &pcie_ep->pci; > + u32 val, bw; > + int speed, width; > + int ret; > + Follow reverse Xmas tree order for local variables. > + if (!pcie_ep->icc) > + return; > + > + val = dw_pcie_readl_dbi(pci, DBI_LINKCTRLSTATUS); > + val = val >> DBI_LINKCTRKSTATUS_SHIFT; > + Use FIELD_GET macro combined with GENMASK > + speed = FIELD_GET(PCI_EXP_LNKSTA_CLS, val); > + width = FIELD_GET(PCI_EXP_LNKSTA_NLW, val); > + > + /* > + * ICC needs avg bw in KBps. s/avg bw/BW ...everywhere > + * > + * For example for 2Gbps the avg BW = 2x1000x1000x1000/8*1000 = 250000 > + */ > + switch (speed) { > + case 1: > + bw = 250000; /* avg bw for GEN1 per lane: 2Gbps, peak bw: no vote */ To align with pcie-qcom driver, specify the value in MBps. Also, use the MBps_to_icc() macro. > + break; > + case 2: > + bw = 500000; /* avg bw for GEN2 per lane: 4Gbps, peak bw no vote */ > + break; > + case 3: > + bw = 1000000; /* avg bw for GEN3 per lane: 8Gbps, peak bw no vote */ > + break; > + default: > + WARN_ON_ONCE(1); > + fallthrough; > + case 4: > + bw = 2000000; /* avg bw for GEN4 per lane: 16Gbps, peak bw no vote */ > + break; > + } > + > + ret = icc_set_bw(pcie_ep->icc, width * bw, 0); AFAIU, avg bandwidth should be less than peak bandwidth. So use the vote for peak bandwidth, leaving 0 as avg. Also, the comment above should be adjusted accordingly. > + if (ret) { > + dev_err(pci->dev, "failed to set interconnect bandwidth: %d\n", > + ret); > + } > +} > + > static int qcom_pcie_enable_resources(struct qcom_pcie_ep *pcie_ep) > { > int ret; > + struct dw_pcie *pci = &pcie_ep->pci; > > ret = clk_bulk_prepare_enable(pcie_ep->num_clks, pcie_ep->clks); > if (ret) > @@ -277,6 +331,20 @@ static int qcom_pcie_enable_resources(struct qcom_pcie_ep *pcie_ep) > if (ret) > goto err_phy_exit; > > + /* > + * Some Qualcomm platforms require interconnect bandwidth constraints > + * to be set before enabling interconnect clocks. > + * > + * Set an initial average bandwidth corresponding to single-lane Gen 1 > + * for the pcie to mem path. > + */ > + ret = icc_set_bw(pcie_ep->icc, 250000, 0); /* avg bw: 2Gbps, peak bw: no vote */ Same as above > + if (ret) { > + dev_err(pci->dev, "failed to set interconnect bandwidth: %d\n", > + ret); > + goto err_phy_exit; > + } > + > return 0; > > err_phy_exit: > @@ -550,6 +618,10 @@ static int qcom_pcie_ep_get_resources(struct platform_device *pdev, > if (IS_ERR(pcie_ep->phy)) > ret = PTR_ERR(pcie_ep->phy); > > + pcie_ep->icc = devm_of_icc_get(dev, "pci"); This should specify the icc path like pcie-mem as specified in pcie-qcom driver. This helps in adding other icc paths if required in the future. - Mani > + if (IS_ERR(pcie_ep->icc)) > + ret = PTR_ERR(pcie_ep->icc); > + > return ret; > } > > @@ -572,6 +644,7 @@ static irqreturn_t qcom_pcie_ep_global_irq_thread(int irq, void *data) > } else if (FIELD_GET(PARF_INT_ALL_BME, status)) { > dev_dbg(dev, "Received BME event. Link is enabled!\n"); > pcie_ep->link_status = QCOM_PCIE_EP_LINK_ENABLED; > + qcom_pcie_icc_update(pcie_ep); > } else if (FIELD_GET(PARF_INT_ALL_PM_TURNOFF, status)) { > dev_dbg(dev, "Received PM Turn-off event! Entering L23\n"); > val = readl_relaxed(pcie_ep->parf + PARF_PM_CTRL); > -- > 2.7.4 >
On 6/6/2023 4:18 PM, Georgi Djakov wrote: > Hi Krishna, > > Thanks for the patch! > > On 6.06.23 8:49, Krishna chaitanya chundru wrote: >> Add support to vote for ICC bandwidth based up on the link >> speed and width. >> >> Signed-off-by: Krishna chaitanya chundru <quic_krichai@quicinc.com> >> --- >> drivers/pci/controller/dwc/pcie-qcom-ep.c | 73 >> +++++++++++++++++++++++++++++++ >> 1 file changed, 73 insertions(+) >> >> diff --git a/drivers/pci/controller/dwc/pcie-qcom-ep.c >> b/drivers/pci/controller/dwc/pcie-qcom-ep.c >> index 19b3283..79e7559 100644 >> --- a/drivers/pci/controller/dwc/pcie-qcom-ep.c >> +++ b/drivers/pci/controller/dwc/pcie-qcom-ep.c >> @@ -17,6 +17,7 @@ >> #include <linux/phy/pcie.h> >> #include <linux/phy/phy.h> >> #include <linux/platform_device.h> >> +#include <linux/interconnect.h> >> #include <linux/pm_domain.h> >> #include <linux/regmap.h> >> #include <linux/reset.h> >> @@ -28,6 +29,7 @@ >> #define PARF_SYS_CTRL 0x00 >> #define PARF_DB_CTRL 0x10 >> #define PARF_PM_CTRL 0x20 >> +#define PARF_PM_STTS 0x24 >> #define PARF_MHI_CLOCK_RESET_CTRL 0x174 >> #define PARF_MHI_BASE_ADDR_LOWER 0x178 >> #define PARF_MHI_BASE_ADDR_UPPER 0x17c >> @@ -128,6 +130,9 @@ >> /* DBI register fields */ >> #define DBI_CON_STATUS_POWER_STATE_MASK GENMASK(1, 0) >> +#define DBI_LINKCTRLSTATUS 0x80 >> +#define DBI_LINKCTRKSTATUS_SHIFT 16 >> + >> #define XMLH_LINK_UP 0x400 >> #define CORE_RESET_TIME_US_MIN 1000 >> #define CORE_RESET_TIME_US_MAX 1005 >> @@ -187,6 +192,8 @@ struct qcom_pcie_ep { >> enum qcom_pcie_ep_link_status link_status; >> int global_irq; >> int perst_irq; >> + >> + struct icc_path *icc; >> }; >> static int qcom_pcie_ep_core_reset(struct qcom_pcie_ep *pcie_ep) >> @@ -253,9 +260,56 @@ static void qcom_pcie_dw_stop_link(struct >> dw_pcie *pci) >> disable_irq(pcie_ep->perst_irq); >> } >> +static void qcom_pcie_icc_update(struct qcom_pcie_ep *pcie_ep) >> +{ >> + struct dw_pcie *pci = &pcie_ep->pci; >> + u32 val, bw; >> + int speed, width; >> + int ret; >> + >> + if (!pcie_ep->icc) >> + return; >> + >> + val = dw_pcie_readl_dbi(pci, DBI_LINKCTRLSTATUS); >> + val = val >> DBI_LINKCTRKSTATUS_SHIFT; >> + >> + speed = FIELD_GET(PCI_EXP_LNKSTA_CLS, val); >> + width = FIELD_GET(PCI_EXP_LNKSTA_NLW, val); >> + >> + /* >> + * ICC needs avg bw in KBps. >> + * >> + * For example for 2Gbps the avg BW = 2x1000x1000x1000/8*1000 = >> 250000 >> + */ >> + switch (speed) { >> + case 1: >> + bw = 250000; /* avg bw for GEN1 per lane: 2Gbps, peak bw: >> no vote */ >> + break; >> + case 2: >> + bw = 500000; /* avg bw for GEN2 per lane: 4Gbps, peak bw >> no vote */ >> + break; >> + case 3: >> + bw = 1000000; /* avg bw for GEN3 per lane: 8Gbps, peak bw >> no vote */ >> + break; >> + default: >> + WARN_ON_ONCE(1); >> + fallthrough; >> + case 4: >> + bw = 2000000; /* avg bw for GEN4 per lane: 16Gbps, peak >> bw no vote */ >> + break; >> + } >> + >> + ret = icc_set_bw(pcie_ep->icc, width * bw, 0); > > Here you should use a non-zero value for peak bandwidth. You can use > the average value also as peak. There are some existing macros like > GBps_to_icc(). Please use them. updated the logic in the new patch. > >> + if (ret) { >> + dev_err(pci->dev, "failed to set interconnect bandwidth: %d\n", >> + ret); >> + } >> +} >> + >> static int qcom_pcie_enable_resources(struct qcom_pcie_ep *pcie_ep) >> { >> int ret; >> + struct dw_pcie *pci = &pcie_ep->pci; >> ret = clk_bulk_prepare_enable(pcie_ep->num_clks, pcie_ep->clks); >> if (ret) >> @@ -277,6 +331,20 @@ static int qcom_pcie_enable_resources(struct >> qcom_pcie_ep *pcie_ep) >> if (ret) >> goto err_phy_exit; >> + /* >> + * Some Qualcomm platforms require interconnect bandwidth >> constraints >> + * to be set before enabling interconnect clocks. >> + * >> + * Set an initial average bandwidth corresponding to single-lane >> Gen 1 >> + * for the pcie to mem path. >> + */ >> + ret = icc_set_bw(pcie_ep->icc, 250000, 0); /* avg bw: 2Gbps, >> peak bw: no vote */ > > Ditto. > done >> + if (ret) { >> + dev_err(pci->dev, "failed to set interconnect bandwidth: %d\n", >> + ret); >> + goto err_phy_exit; >> + } >> + >> return 0; >> err_phy_exit: >> @@ -550,6 +618,10 @@ static int qcom_pcie_ep_get_resources(struct >> platform_device *pdev, >> if (IS_ERR(pcie_ep->phy)) >> ret = PTR_ERR(pcie_ep->phy); >> + pcie_ep->icc = devm_of_icc_get(dev, "pci"); > > Is this "pci" path documented in the bindings? > > Thanks, > Georgi > Added the bindings patch in new patch Thanks, Krishna Chaitanya. >> + if (IS_ERR(pcie_ep->icc)) >> + ret = PTR_ERR(pcie_ep->icc); >> + >> return ret; >> } >> @@ -572,6 +644,7 @@ static irqreturn_t >> qcom_pcie_ep_global_irq_thread(int irq, void *data) >> } else if (FIELD_GET(PARF_INT_ALL_BME, status)) { >> dev_dbg(dev, "Received BME event. Link is enabled!\n"); >> pcie_ep->link_status = QCOM_PCIE_EP_LINK_ENABLED; >> + qcom_pcie_icc_update(pcie_ep); >> } else if (FIELD_GET(PARF_INT_ALL_PM_TURNOFF, status)) { >> dev_dbg(dev, "Received PM Turn-off event! Entering L23\n"); >> val = readl_relaxed(pcie_ep->parf + PARF_PM_CTRL); >
On 6/6/2023 4:54 PM, Manivannan Sadhasivam wrote: > On Tue, Jun 06, 2023 at 11:19:29AM +0530, Krishna chaitanya chundru wrote: >> Add support to vote for ICC bandwidth based up on the link > based on done > >> speed and width. >> > Looks like the code got inspiration from pcie-qcom driver. So it should be > mentioned in the commit message. done >> Signed-off-by: Krishna chaitanya chundru <quic_krichai@quicinc.com> > Devicetree bindings update should precede this patch. done >> --- >> drivers/pci/controller/dwc/pcie-qcom-ep.c | 73 +++++++++++++++++++++++++++++++ >> 1 file changed, 73 insertions(+) >> >> diff --git a/drivers/pci/controller/dwc/pcie-qcom-ep.c b/drivers/pci/controller/dwc/pcie-qcom-ep.c >> index 19b3283..79e7559 100644 >> --- a/drivers/pci/controller/dwc/pcie-qcom-ep.c >> +++ b/drivers/pci/controller/dwc/pcie-qcom-ep.c >> @@ -17,6 +17,7 @@ >> #include <linux/phy/pcie.h> >> #include <linux/phy/phy.h> >> #include <linux/platform_device.h> >> +#include <linux/interconnect.h> > Includes are sorted alphabetically > >> #include <linux/pm_domain.h> >> #include <linux/regmap.h> >> #include <linux/reset.h> >> @@ -28,6 +29,7 @@ >> #define PARF_SYS_CTRL 0x00 >> #define PARF_DB_CTRL 0x10 >> #define PARF_PM_CTRL 0x20 >> +#define PARF_PM_STTS 0x24 >> #define PARF_MHI_CLOCK_RESET_CTRL 0x174 >> #define PARF_MHI_BASE_ADDR_LOWER 0x178 >> #define PARF_MHI_BASE_ADDR_UPPER 0x17c >> @@ -128,6 +130,9 @@ >> /* DBI register fields */ >> #define DBI_CON_STATUS_POWER_STATE_MASK GENMASK(1, 0) >> >> +#define DBI_LINKCTRLSTATUS 0x80 >> +#define DBI_LINKCTRKSTATUS_SHIFT 16 > Use GENMASK macro > >> + >> #define XMLH_LINK_UP 0x400 >> #define CORE_RESET_TIME_US_MIN 1000 >> #define CORE_RESET_TIME_US_MAX 1005 >> @@ -187,6 +192,8 @@ struct qcom_pcie_ep { >> enum qcom_pcie_ep_link_status link_status; >> int global_irq; >> int perst_irq; >> + >> + struct icc_path *icc; > Place this under debugfs entry. done >> }; >> >> static int qcom_pcie_ep_core_reset(struct qcom_pcie_ep *pcie_ep) >> @@ -253,9 +260,56 @@ static void qcom_pcie_dw_stop_link(struct dw_pcie *pci) >> disable_irq(pcie_ep->perst_irq); >> } >> >> +static void qcom_pcie_icc_update(struct qcom_pcie_ep *pcie_ep) > qcom_pcie_ep_icc_update done >> +{ >> + struct dw_pcie *pci = &pcie_ep->pci; >> + u32 val, bw; >> + int speed, width; >> + int ret; >> + > Follow reverse Xmas tree order for local variables. > >> + if (!pcie_ep->icc) >> + return; >> + >> + val = dw_pcie_readl_dbi(pci, DBI_LINKCTRLSTATUS); >> + val = val >> DBI_LINKCTRKSTATUS_SHIFT; >> + > Use FIELD_GET macro combined with GENMASK I didn't get this logic you are suggesting can you please elaborate >> + speed = FIELD_GET(PCI_EXP_LNKSTA_CLS, val); >> + width = FIELD_GET(PCI_EXP_LNKSTA_NLW, val); >> + >> + /* >> + * ICC needs avg bw in KBps. > s/avg bw/BW > ...everywhere done >> + * >> + * For example for 2Gbps the avg BW = 2x1000x1000x1000/8*1000 = 250000 >> + */ >> + switch (speed) { >> + case 1: >> + bw = 250000; /* avg bw for GEN1 per lane: 2Gbps, peak bw: no vote */ > To align with pcie-qcom driver, specify the value in MBps. Also, use the > MBps_to_icc() macro. done >> + break; >> + case 2: >> + bw = 500000; /* avg bw for GEN2 per lane: 4Gbps, peak bw no vote */ >> + break; >> + case 3: >> + bw = 1000000; /* avg bw for GEN3 per lane: 8Gbps, peak bw no vote */ >> + break; >> + default: >> + WARN_ON_ONCE(1); >> + fallthrough; >> + case 4: >> + bw = 2000000; /* avg bw for GEN4 per lane: 16Gbps, peak bw no vote */ >> + break; >> + } >> + >> + ret = icc_set_bw(pcie_ep->icc, width * bw, 0); > AFAIU, avg bandwidth should be less than peak bandwidth. So use the vote for > peak bandwidth, leaving 0 as avg. Also, the comment above should be adjusted > accordingly. done >> + if (ret) { >> + dev_err(pci->dev, "failed to set interconnect bandwidth: %d\n", >> + ret); >> + } >> +} >> + >> static int qcom_pcie_enable_resources(struct qcom_pcie_ep *pcie_ep) >> { >> int ret; >> + struct dw_pcie *pci = &pcie_ep->pci; >> >> ret = clk_bulk_prepare_enable(pcie_ep->num_clks, pcie_ep->clks); >> if (ret) >> @@ -277,6 +331,20 @@ static int qcom_pcie_enable_resources(struct qcom_pcie_ep *pcie_ep) >> if (ret) >> goto err_phy_exit; >> >> + /* >> + * Some Qualcomm platforms require interconnect bandwidth constraints >> + * to be set before enabling interconnect clocks. >> + * >> + * Set an initial average bandwidth corresponding to single-lane Gen 1 >> + * for the pcie to mem path. >> + */ >> + ret = icc_set_bw(pcie_ep->icc, 250000, 0); /* avg bw: 2Gbps, peak bw: no vote */ > Same as above done >> + if (ret) { >> + dev_err(pci->dev, "failed to set interconnect bandwidth: %d\n", >> + ret); >> + goto err_phy_exit; >> + } >> + >> return 0; >> >> err_phy_exit: >> @@ -550,6 +618,10 @@ static int qcom_pcie_ep_get_resources(struct platform_device *pdev, >> if (IS_ERR(pcie_ep->phy)) >> ret = PTR_ERR(pcie_ep->phy); >> >> + pcie_ep->icc = devm_of_icc_get(dev, "pci"); > This should specify the icc path like pcie-mem as specified in pcie-qcom driver. > This helps in adding other icc paths if required in the future. > > - Mani done Thanks, Krishna chaitanya. >> + if (IS_ERR(pcie_ep->icc)) >> + ret = PTR_ERR(pcie_ep->icc); >> + >> return ret; >> } >> >> @@ -572,6 +644,7 @@ static irqreturn_t qcom_pcie_ep_global_irq_thread(int irq, void *data) >> } else if (FIELD_GET(PARF_INT_ALL_BME, status)) { >> dev_dbg(dev, "Received BME event. Link is enabled!\n"); >> pcie_ep->link_status = QCOM_PCIE_EP_LINK_ENABLED; >> + qcom_pcie_icc_update(pcie_ep); >> } else if (FIELD_GET(PARF_INT_ALL_PM_TURNOFF, status)) { >> dev_dbg(dev, "Received PM Turn-off event! Entering L23\n"); >> val = readl_relaxed(pcie_ep->parf + PARF_PM_CTRL); >> -- >> 2.7.4 >>
diff --git a/drivers/pci/controller/dwc/pcie-qcom-ep.c b/drivers/pci/controller/dwc/pcie-qcom-ep.c index 19b3283..79e7559 100644 --- a/drivers/pci/controller/dwc/pcie-qcom-ep.c +++ b/drivers/pci/controller/dwc/pcie-qcom-ep.c @@ -17,6 +17,7 @@ #include <linux/phy/pcie.h> #include <linux/phy/phy.h> #include <linux/platform_device.h> +#include <linux/interconnect.h> #include <linux/pm_domain.h> #include <linux/regmap.h> #include <linux/reset.h> @@ -28,6 +29,7 @@ #define PARF_SYS_CTRL 0x00 #define PARF_DB_CTRL 0x10 #define PARF_PM_CTRL 0x20 +#define PARF_PM_STTS 0x24 #define PARF_MHI_CLOCK_RESET_CTRL 0x174 #define PARF_MHI_BASE_ADDR_LOWER 0x178 #define PARF_MHI_BASE_ADDR_UPPER 0x17c @@ -128,6 +130,9 @@ /* DBI register fields */ #define DBI_CON_STATUS_POWER_STATE_MASK GENMASK(1, 0) +#define DBI_LINKCTRLSTATUS 0x80 +#define DBI_LINKCTRKSTATUS_SHIFT 16 + #define XMLH_LINK_UP 0x400 #define CORE_RESET_TIME_US_MIN 1000 #define CORE_RESET_TIME_US_MAX 1005 @@ -187,6 +192,8 @@ struct qcom_pcie_ep { enum qcom_pcie_ep_link_status link_status; int global_irq; int perst_irq; + + struct icc_path *icc; }; static int qcom_pcie_ep_core_reset(struct qcom_pcie_ep *pcie_ep) @@ -253,9 +260,56 @@ static void qcom_pcie_dw_stop_link(struct dw_pcie *pci) disable_irq(pcie_ep->perst_irq); } +static void qcom_pcie_icc_update(struct qcom_pcie_ep *pcie_ep) +{ + struct dw_pcie *pci = &pcie_ep->pci; + u32 val, bw; + int speed, width; + int ret; + + if (!pcie_ep->icc) + return; + + val = dw_pcie_readl_dbi(pci, DBI_LINKCTRLSTATUS); + val = val >> DBI_LINKCTRKSTATUS_SHIFT; + + speed = FIELD_GET(PCI_EXP_LNKSTA_CLS, val); + width = FIELD_GET(PCI_EXP_LNKSTA_NLW, val); + + /* + * ICC needs avg bw in KBps. + * + * For example for 2Gbps the avg BW = 2x1000x1000x1000/8*1000 = 250000 + */ + switch (speed) { + case 1: + bw = 250000; /* avg bw for GEN1 per lane: 2Gbps, peak bw: no vote */ + break; + case 2: + bw = 500000; /* avg bw for GEN2 per lane: 4Gbps, peak bw no vote */ + break; + case 3: + bw = 1000000; /* avg bw for GEN3 per lane: 8Gbps, peak bw no vote */ + break; + default: + WARN_ON_ONCE(1); + fallthrough; + case 4: + bw = 2000000; /* avg bw for GEN4 per lane: 16Gbps, peak bw no vote */ + break; + } + + ret = icc_set_bw(pcie_ep->icc, width * bw, 0); + if (ret) { + dev_err(pci->dev, "failed to set interconnect bandwidth: %d\n", + ret); + } +} + static int qcom_pcie_enable_resources(struct qcom_pcie_ep *pcie_ep) { int ret; + struct dw_pcie *pci = &pcie_ep->pci; ret = clk_bulk_prepare_enable(pcie_ep->num_clks, pcie_ep->clks); if (ret) @@ -277,6 +331,20 @@ static int qcom_pcie_enable_resources(struct qcom_pcie_ep *pcie_ep) if (ret) goto err_phy_exit; + /* + * Some Qualcomm platforms require interconnect bandwidth constraints + * to be set before enabling interconnect clocks. + * + * Set an initial average bandwidth corresponding to single-lane Gen 1 + * for the pcie to mem path. + */ + ret = icc_set_bw(pcie_ep->icc, 250000, 0); /* avg bw: 2Gbps, peak bw: no vote */ + if (ret) { + dev_err(pci->dev, "failed to set interconnect bandwidth: %d\n", + ret); + goto err_phy_exit; + } + return 0; err_phy_exit: @@ -550,6 +618,10 @@ static int qcom_pcie_ep_get_resources(struct platform_device *pdev, if (IS_ERR(pcie_ep->phy)) ret = PTR_ERR(pcie_ep->phy); + pcie_ep->icc = devm_of_icc_get(dev, "pci"); + if (IS_ERR(pcie_ep->icc)) + ret = PTR_ERR(pcie_ep->icc); + return ret; } @@ -572,6 +644,7 @@ static irqreturn_t qcom_pcie_ep_global_irq_thread(int irq, void *data) } else if (FIELD_GET(PARF_INT_ALL_BME, status)) { dev_dbg(dev, "Received BME event. Link is enabled!\n"); pcie_ep->link_status = QCOM_PCIE_EP_LINK_ENABLED; + qcom_pcie_icc_update(pcie_ep); } else if (FIELD_GET(PARF_INT_ALL_PM_TURNOFF, status)) { dev_dbg(dev, "Received PM Turn-off event! Entering L23\n"); val = readl_relaxed(pcie_ep->parf + PARF_PM_CTRL);