Message ID | 20221107081327.336239-1-nathan@nathanrossi.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp1908054wru; Mon, 7 Nov 2022 00:16:14 -0800 (PST) X-Google-Smtp-Source: AMsMyM5UxT3tC3e2bV9kd/HEySaoOKE1ygXl6uvqudkNWv4amcAyyQVVk2qG8qTo5Hh0s+JdUOyj X-Received: by 2002:a65:464d:0:b0:46f:7896:e223 with SMTP id k13-20020a65464d000000b0046f7896e223mr41006398pgr.421.1667808974417; Mon, 07 Nov 2022 00:16:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1667808974; cv=none; d=google.com; s=arc-20160816; b=v4wNZqj3ig8DM56Ag2TkDxhtklNWzwxNk0WdGxGFPtvr7DTVpWBUXE1r63YHzzsPKO LFLcuECl3mkqeHzxCy321QBMzrTNYnUfGrrNeX67F4VaO3+vdA0lFuUWMHdHemRIwT6d 0h9i/pA8BdOsrGpedk3fbktsfW9YJE3o90mUxbVLOcJubH33vjH5fupdzstawoHXiTSQ HOPWYCXDW6j2l/VC1cqnZ352BuBrO7UN2LyBEEPnSfAif80v347yFIfNPahWk2sRkXZm l88/DuH0p2L1T6wEqFRnKj9A9qSc2Sv4kxcZUx0jbDbs4+43hUKcPt1dOb6ayzf3woCY ZjyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:content-transfer-encoding:subject :cc:to:from:message-id:date:dkim-signature; bh=y71pLMvooIRecOUj8AXrDoMOrUDoDubP/fQuwH79cVQ=; b=bijKqhfBhJveUEFaLP3vN/wvaEOFOgdVlX6pei91LEtVdrfK6NGA7J28tbg6Ht5gTg WSvBWHJKybpb3n9/6qdK3KSet98qD6OPrK/HjU5BrQqRmvFa89MVe3//wGSzo+eXEUal kfenRMe/yPLldfsbYKdH49bxP/KcG4YuOsw/A4zK2elHLRz1ofkCZeLCB6KEiZsWTtUo 6eSWCJsYOOazpGL4DpRTqb0DuYzZ393JJx+7RugQbwSrbEOatLjdAc7sLSQ2L4uMmzF9 DnM9shwAAoUzIn/zt8lifLJXW8YRMUcFkywUq0jIoyITdhyQYViejqB8NirTA+TzoeT+ z2lw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@nathanrossi.com header.s=google header.b="W7E7bB/v"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nathanrossi.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id lb8-20020a17090b4a4800b0021391d90dadsi17508767pjb.107.2022.11.07.00.15.59; Mon, 07 Nov 2022 00:16:14 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@nathanrossi.com header.s=google header.b="W7E7bB/v"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nathanrossi.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231440AbiKGINq (ORCPT <rfc822;hjfbswb@gmail.com> + 99 others); Mon, 7 Nov 2022 03:13:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32974 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231329AbiKGINo (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Mon, 7 Nov 2022 03:13:44 -0500 Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C9750B7FF for <linux-kernel@vger.kernel.org>; Mon, 7 Nov 2022 00:13:42 -0800 (PST) Received: by mail-pf1-x42d.google.com with SMTP id z26so9917988pff.1 for <linux-kernel@vger.kernel.org>; Mon, 07 Nov 2022 00:13:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nathanrossi.com; s=google; h=mime-version:content-transfer-encoding:subject:cc:to:from :message-id:date:from:to:cc:subject:date:message-id:reply-to; bh=y71pLMvooIRecOUj8AXrDoMOrUDoDubP/fQuwH79cVQ=; b=W7E7bB/v9cZQW93GKuGYJEJvY7ThtLs55R2ykpJdCI1aALO6M5vSiJd0lvjmxfNMMg wZiIJhDNqe6+tHHHtPW4Ue3sQK55dTfTuAdMeWJ+XEuZnGQMVC80Gly9j5tk+RbDSZdl f1LCtU+1QqqFUsLdQcxRpAOircqMCAGCIjTjPxWAN6LbvM5bP9pJswTbyDACmLYqslaA teX0SHTXTMOzq71cnehss6sz08a8Mtlw8zi9w9zfvE1E2xUvB59DVnNmvOOqsDq3oGFY PVdvNe5EPnKbO1QBUQXEKNX5Rm4kCfp3AUF8jLznBJvhtDrzKN2Hvx8v6iKI2Ih4FEce K0zg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:content-transfer-encoding:subject:cc:to:from :message-id:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=y71pLMvooIRecOUj8AXrDoMOrUDoDubP/fQuwH79cVQ=; b=KinB52zgOhzmJsmwunJuOtPJIddAFf7NehjmkK7QpLLCICEgX2loI+KGwFtrr3K2C4 3VPdY/jGwawUKVHj5nCh0SiG04cny9zOgn2Lq6INhQdjuRl6ru7PgljPYKP6YX8GIaOa CdVJWM0aWDFOqXA4MTDvdp9DqrxJLMZWEUNFg6T1y7TaAJ65gSELiIL2gRwlmMLsomlY 9/EMyB1aJqTcl5mYJMOiosFp8vpfrpaGfl9nm3VzsZ+rWnST/X+JERkr5yKHcY5R7850 +YKy4x4X28+KaNh0hD7eoitCODZKTp4raInipVtGgEc2fdacMwJyVrxhcM7vPVnHe15a IyCw== X-Gm-Message-State: ACrzQf36z+skz3T48BafyDCnvDAdumAwnxo4rNfU6alsVR6e+eIFLibz e8lDzkb7BYAn7l3EShVLNhv3fGiqSgOylyIr X-Received: by 2002:a63:dd4f:0:b0:46f:fae3:373d with SMTP id g15-20020a63dd4f000000b0046ffae3373dmr26459600pgj.31.1667808822337; Mon, 07 Nov 2022 00:13:42 -0800 (PST) Received: from [127.0.1.1] (117-20-68-146.751444.bne.nbn.aussiebb.net. [117.20.68.146]) by smtp.gmail.com with UTF8SMTPSA id pc3-20020a17090b3b8300b00212cf2fe8c3sm17579487pjb.1.2022.11.07.00.13.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 00:13:41 -0800 (PST) Date: Mon, 07 Nov 2022 08:13:27 +0000 Message-Id: <20221107081327.336239-1-nathan@nathanrossi.com> From: Nathan Rossi <nathan@nathanrossi.com> To: linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Nathan Rossi <nathan@nathanrossi.com>, Nathan Rossi <nathan.rossi@digi.com>, Thomas Petazzoni <thomas.petazzoni@bootlin.com>, Lorenzo Pieralisi <lpieralisi@kernel.org>, Bjorn Helgaas <bhelgaas@google.com>, Pali =?utf-8?b?Um9ow6Fy?= <pali@kernel.org> Subject: [PATCH] PCI: mvebu: Set Target Link Speed for 2.5GT downstream devices Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748824463202037374?= X-GMAIL-MSGID: =?utf-8?q?1748824463202037374?= |
Series |
PCI: mvebu: Set Target Link Speed for 2.5GT downstream devices
|
|
Commit Message
Nathan Rossi
Nov. 7, 2022, 8:13 a.m. UTC
From: Nathan Rossi <nathan.rossi@digi.com> There is a known issue with the mvebu PCIe controller when triggering retraining of the link (via Link Control) where the link is dropped completely causing significant delay in the renegotiation of the link. This occurs only when the downstream device is 2.5GT and the upstream port is configured to support both 2.5GT and 5GT. It is possible to prevent this link dropping by setting the associated link speed in Target Link Speed of the Link Control 2 register. This only needs to be done when the downstream is specifically 2.5GT. This change applies the required Target Link Speed value during mvebu_pcie_setup_hw conditionally depending on the current link speed from the Link Status register, only applying the change when the link is configured to 2.5GT already. Signed-off-by: Nathan Rossi <nathan.rossi@digi.com> --- drivers/pci/controller/pci-mvebu.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) --- 2.37.2
Comments
On Monday 07 November 2022 08:13:27 Nathan Rossi wrote: > From: Nathan Rossi <nathan.rossi@digi.com> > > There is a known issue with the mvebu PCIe controller when triggering > retraining of the link (via Link Control) where the link is dropped > completely causing significant delay in the renegotiation of the link. > This occurs only when the downstream device is 2.5GT and the upstream > port is configured to support both 2.5GT and 5GT. > > It is possible to prevent this link dropping by setting the associated > link speed in Target Link Speed of the Link Control 2 register. This > only needs to be done when the downstream is specifically 2.5GT. > > This change applies the required Target Link Speed value during > mvebu_pcie_setup_hw conditionally depending on the current link speed > from the Link Status register, only applying the change when the link > is configured to 2.5GT already. > > Signed-off-by: Nathan Rossi <nathan.rossi@digi.com> > --- > drivers/pci/controller/pci-mvebu.c | 18 +++++++++++++++++- > 1 file changed, 17 insertions(+), 1 deletion(-) > > diff --git a/drivers/pci/controller/pci-mvebu.c b/drivers/pci/controller/pci-mvebu.c > index 1ced73726a..6a869a33ba 100644 > --- a/drivers/pci/controller/pci-mvebu.c > +++ b/drivers/pci/controller/pci-mvebu.c > @@ -248,7 +248,7 @@ static void mvebu_pcie_setup_wins(struct mvebu_pcie_port *port) > > static void mvebu_pcie_setup_hw(struct mvebu_pcie_port *port) > { > - u32 ctrl, lnkcap, cmd, dev_rev, unmask, sspl; > + u32 ctrl, lnkcap, cmd, dev_rev, unmask, sspl, lnksta, lnkctl2; > > /* Setup PCIe controller to Root Complex mode. */ > ctrl = mvebu_readl(port, PCIE_CTRL_OFF); > @@ -339,6 +339,22 @@ static void mvebu_pcie_setup_hw(struct mvebu_pcie_port *port) > unmask |= PCIE_INT_INTX(0) | PCIE_INT_INTX(1) | > PCIE_INT_INTX(2) | PCIE_INT_INTX(3); > mvebu_writel(port, unmask, PCIE_INT_UNMASK_OFF); > + > + /* > + * Set Target Link Speed within the Link Control 2 register when the > + * linked downstream device is connected at 2.5GT. This is configured > + * in order to avoid issues with the controller when the upstream port > + * is configured to support 2.5GT and 5GT and the downstream device is > + * linked at 2.5GT, retraining the link in this case causes the link to > + * drop taking significant time to retrain. > + */ > + lnksta = mvebu_readl(port, PCIE_CAP_PCIEXP + PCI_EXP_LNKCTL) >> 16; > + if ((lnksta & PCI_EXP_LNKSTA_CLS) == PCI_EXP_LNKSTA_CLS_2_5GB) { This code does not work because at this stage endpoint device does not have to be ready and therefore link is not established yet. Also this code is not running when kernel issue PCIe Hot Reset via PCI Secondary Bus Reset bit. And it does not handle possible hot-plug situation. That check that code below has to be done _after_ kernel enumerate device. PCI core code has already logic to handle delays for "slow" devices. And reverse operation (setting lnkctl2 target speed to original value) has to be called after unplugging device - when link goes down. If you want to work on this stuff, I can try to find my notes which I done during investigation of this issue... where is probably the best place in kernel pci core code for handling this issue. > + lnkctl2 = mvebu_readl(port, PCIE_CAP_PCIEXP + PCI_EXP_LNKCTL2); > + lnkctl2 &= ~PCI_EXP_LNKCTL2_TLS; > + lnkctl2 |= PCI_EXP_LNKCTL2_TLS_2_5GT; > + mvebu_writel(port, lnkctl2, PCIE_CAP_PCIEXP + PCI_EXP_LNKCTL2); > + } > } > > static struct mvebu_pcie_port *mvebu_pcie_find_port(struct mvebu_pcie *pcie, > --- > 2.37.2
On Mon, 7 Nov 2022 at 18:43, Pali Rohár <pali@kernel.org> wrote: > > On Monday 07 November 2022 08:13:27 Nathan Rossi wrote: > > From: Nathan Rossi <nathan.rossi@digi.com> > > > > There is a known issue with the mvebu PCIe controller when triggering > > retraining of the link (via Link Control) where the link is dropped > > completely causing significant delay in the renegotiation of the link. > > This occurs only when the downstream device is 2.5GT and the upstream > > port is configured to support both 2.5GT and 5GT. > > > > It is possible to prevent this link dropping by setting the associated > > link speed in Target Link Speed of the Link Control 2 register. This > > only needs to be done when the downstream is specifically 2.5GT. > > > > This change applies the required Target Link Speed value during > > mvebu_pcie_setup_hw conditionally depending on the current link speed > > from the Link Status register, only applying the change when the link > > is configured to 2.5GT already. > > > > Signed-off-by: Nathan Rossi <nathan.rossi@digi.com> > > --- > > drivers/pci/controller/pci-mvebu.c | 18 +++++++++++++++++- > > 1 file changed, 17 insertions(+), 1 deletion(-) > > > > diff --git a/drivers/pci/controller/pci-mvebu.c b/drivers/pci/controller/pci-mvebu.c > > index 1ced73726a..6a869a33ba 100644 > > --- a/drivers/pci/controller/pci-mvebu.c > > +++ b/drivers/pci/controller/pci-mvebu.c > > @@ -248,7 +248,7 @@ static void mvebu_pcie_setup_wins(struct mvebu_pcie_port *port) > > > > static void mvebu_pcie_setup_hw(struct mvebu_pcie_port *port) > > { > > - u32 ctrl, lnkcap, cmd, dev_rev, unmask, sspl; > > + u32 ctrl, lnkcap, cmd, dev_rev, unmask, sspl, lnksta, lnkctl2; > > > > /* Setup PCIe controller to Root Complex mode. */ > > ctrl = mvebu_readl(port, PCIE_CTRL_OFF); > > @@ -339,6 +339,22 @@ static void mvebu_pcie_setup_hw(struct mvebu_pcie_port *port) > > unmask |= PCIE_INT_INTX(0) | PCIE_INT_INTX(1) | > > PCIE_INT_INTX(2) | PCIE_INT_INTX(3); > > mvebu_writel(port, unmask, PCIE_INT_UNMASK_OFF); > > + > > + /* > > + * Set Target Link Speed within the Link Control 2 register when the > > + * linked downstream device is connected at 2.5GT. This is configured > > + * in order to avoid issues with the controller when the upstream port > > + * is configured to support 2.5GT and 5GT and the downstream device is > > + * linked at 2.5GT, retraining the link in this case causes the link to > > + * drop taking significant time to retrain. > > + */ > > + lnksta = mvebu_readl(port, PCIE_CAP_PCIEXP + PCI_EXP_LNKCTL) >> 16; > > + if ((lnksta & PCI_EXP_LNKSTA_CLS) == PCI_EXP_LNKSTA_CLS_2_5GB) { > > This code does not work because at this stage endpoint device does not > have to be ready and therefore link is not established yet. > > Also this code is not running when kernel issue PCIe Hot Reset via > PCI Secondary Bus Reset bit. > > And it does not handle possible hot-plug situation. > > That check that code below has to be done _after_ kernel enumerate > device. PCI core code has already logic to handle delays for "slow" > devices. > > And reverse operation (setting lnkctl2 target speed to original value) > has to be called after unplugging device - when link goes down. > > If you want to work on this stuff, I can try to find my notes which I > done during investigation of this issue... where is probably the best > place in kernel pci core code for handling this issue. Some notes/direction for implementation would be very appreciated. I am not particularly familiar with the pci core code, so I don't have a good idea on how to best implement this workaround. Thanks, Nathan > > > + lnkctl2 = mvebu_readl(port, PCIE_CAP_PCIEXP + PCI_EXP_LNKCTL2); > > + lnkctl2 &= ~PCI_EXP_LNKCTL2_TLS; > > + lnkctl2 |= PCI_EXP_LNKCTL2_TLS_2_5GT; > > + mvebu_writel(port, lnkctl2, PCIE_CAP_PCIEXP + PCI_EXP_LNKCTL2); > > + } > > } > > > > static struct mvebu_pcie_port *mvebu_pcie_find_port(struct mvebu_pcie *pcie, > > --- > > 2.37.2
On Monday 07 November 2022 19:10:02 Nathan Rossi wrote: > On Mon, 7 Nov 2022 at 18:43, Pali Rohár <pali@kernel.org> wrote: > > > > On Monday 07 November 2022 08:13:27 Nathan Rossi wrote: > > > From: Nathan Rossi <nathan.rossi@digi.com> > > > > > > There is a known issue with the mvebu PCIe controller when triggering > > > retraining of the link (via Link Control) where the link is dropped > > > completely causing significant delay in the renegotiation of the link. > > > This occurs only when the downstream device is 2.5GT and the upstream > > > port is configured to support both 2.5GT and 5GT. > > > > > > It is possible to prevent this link dropping by setting the associated > > > link speed in Target Link Speed of the Link Control 2 register. This > > > only needs to be done when the downstream is specifically 2.5GT. > > > > > > This change applies the required Target Link Speed value during > > > mvebu_pcie_setup_hw conditionally depending on the current link speed > > > from the Link Status register, only applying the change when the link > > > is configured to 2.5GT already. > > > > > > Signed-off-by: Nathan Rossi <nathan.rossi@digi.com> > > > --- > > > drivers/pci/controller/pci-mvebu.c | 18 +++++++++++++++++- > > > 1 file changed, 17 insertions(+), 1 deletion(-) > > > > > > diff --git a/drivers/pci/controller/pci-mvebu.c b/drivers/pci/controller/pci-mvebu.c > > > index 1ced73726a..6a869a33ba 100644 > > > --- a/drivers/pci/controller/pci-mvebu.c > > > +++ b/drivers/pci/controller/pci-mvebu.c > > > @@ -248,7 +248,7 @@ static void mvebu_pcie_setup_wins(struct mvebu_pcie_port *port) > > > > > > static void mvebu_pcie_setup_hw(struct mvebu_pcie_port *port) > > > { > > > - u32 ctrl, lnkcap, cmd, dev_rev, unmask, sspl; > > > + u32 ctrl, lnkcap, cmd, dev_rev, unmask, sspl, lnksta, lnkctl2; > > > > > > /* Setup PCIe controller to Root Complex mode. */ > > > ctrl = mvebu_readl(port, PCIE_CTRL_OFF); > > > @@ -339,6 +339,22 @@ static void mvebu_pcie_setup_hw(struct mvebu_pcie_port *port) > > > unmask |= PCIE_INT_INTX(0) | PCIE_INT_INTX(1) | > > > PCIE_INT_INTX(2) | PCIE_INT_INTX(3); > > > mvebu_writel(port, unmask, PCIE_INT_UNMASK_OFF); > > > + > > > + /* > > > + * Set Target Link Speed within the Link Control 2 register when the > > > + * linked downstream device is connected at 2.5GT. This is configured > > > + * in order to avoid issues with the controller when the upstream port > > > + * is configured to support 2.5GT and 5GT and the downstream device is > > > + * linked at 2.5GT, retraining the link in this case causes the link to > > > + * drop taking significant time to retrain. > > > + */ > > > + lnksta = mvebu_readl(port, PCIE_CAP_PCIEXP + PCI_EXP_LNKCTL) >> 16; > > > + if ((lnksta & PCI_EXP_LNKSTA_CLS) == PCI_EXP_LNKSTA_CLS_2_5GB) { > > > > This code does not work because at this stage endpoint device does not > > have to be ready and therefore link is not established yet. > > > > Also this code is not running when kernel issue PCIe Hot Reset via > > PCI Secondary Bus Reset bit. > > > > And it does not handle possible hot-plug situation. > > > > That check that code below has to be done _after_ kernel enumerate > > device. PCI core code has already logic to handle delays for "slow" > > devices. > > > > And reverse operation (setting lnkctl2 target speed to original value) > > has to be called after unplugging device - when link goes down. > > > > If you want to work on this stuff, I can try to find my notes which I > > done during investigation of this issue... where is probably the best > > place in kernel pci core code for handling this issue. > > Some notes/direction for implementation would be very appreciated. I > am not particularly familiar with the pci core code, so I don't have a > good idea on how to best implement this workaround. Ok, I have checked and seems that I have removed my notes :-( So trying to reconstruct information from my memory... Target link speed in Root port's lnkctl2 register must be set to _correct_ value before configuring ASPM. Because link retraining (part of ASPM configuration) fails. ASPM is initialized by calling function pcie_aspm_init_link_state() from _non-endpoint_ device and it is called at the end of function pci_scan_slot(). Look also at the tree-traversal functions pci_scan_child_bus_extend() and pci_scan_bridge_extend() and try to find the best place where should be this "fix" called. Because same issue as you are trying to fix is also in pci-aardvark.c hardware (Marvell too), I think that you can introduce some flag in struct pci_host_bridge, set it in pci-mvebu.c (later I can do same in pci-aardvark.c) and then in core pci code (in some of above mentioned function when you find the proper place in tree traversal) add code which "fixes" lnkctl2 register. Because both pci hotplug and static initialization calls those pci core scan functions, this should fix init-probe part. Second thing is fixing unplugging part. Because in hotplug setup you can connect 2.5GT/s GEN1 card (which requires this workaround), then disconnect it and connect some 5GT/s GEN2 card, it is needed to set target link back to 5GT/s to use full speed of GEN2 card. For this second part, I think that it is needed to change target link speed back to 5GT/s after card is disconnected. As a good candidates where to do it is probably pci_stop_dev() or pci_destroy_dev() function. Beware that it is needed to change link speed of device on the other end of link - not the device which is being removed/unregistered. And check if it is the last kernel device being unregistered from the bus (endpoint card may be multifunction device). I hope that this information would help you. I'm really sorry that I do not have my notes about this issue where I documented it. Anyway I would try to provide other information if needed. > Thanks, > Nathan > > > > > > + lnkctl2 = mvebu_readl(port, PCIE_CAP_PCIEXP + PCI_EXP_LNKCTL2); > > > + lnkctl2 &= ~PCI_EXP_LNKCTL2_TLS; > > > + lnkctl2 |= PCI_EXP_LNKCTL2_TLS_2_5GT; > > > + mvebu_writel(port, lnkctl2, PCIE_CAP_PCIEXP + PCI_EXP_LNKCTL2); > > > + } > > > } > > > > > > static struct mvebu_pcie_port *mvebu_pcie_find_port(struct mvebu_pcie *pcie, > > > --- > > > 2.37.2
diff --git a/drivers/pci/controller/pci-mvebu.c b/drivers/pci/controller/pci-mvebu.c index 1ced73726a..6a869a33ba 100644 --- a/drivers/pci/controller/pci-mvebu.c +++ b/drivers/pci/controller/pci-mvebu.c @@ -248,7 +248,7 @@ static void mvebu_pcie_setup_wins(struct mvebu_pcie_port *port) static void mvebu_pcie_setup_hw(struct mvebu_pcie_port *port) { - u32 ctrl, lnkcap, cmd, dev_rev, unmask, sspl; + u32 ctrl, lnkcap, cmd, dev_rev, unmask, sspl, lnksta, lnkctl2; /* Setup PCIe controller to Root Complex mode. */ ctrl = mvebu_readl(port, PCIE_CTRL_OFF); @@ -339,6 +339,22 @@ static void mvebu_pcie_setup_hw(struct mvebu_pcie_port *port) unmask |= PCIE_INT_INTX(0) | PCIE_INT_INTX(1) | PCIE_INT_INTX(2) | PCIE_INT_INTX(3); mvebu_writel(port, unmask, PCIE_INT_UNMASK_OFF); + + /* + * Set Target Link Speed within the Link Control 2 register when the + * linked downstream device is connected at 2.5GT. This is configured + * in order to avoid issues with the controller when the upstream port + * is configured to support 2.5GT and 5GT and the downstream device is + * linked at 2.5GT, retraining the link in this case causes the link to + * drop taking significant time to retrain. + */ + lnksta = mvebu_readl(port, PCIE_CAP_PCIEXP + PCI_EXP_LNKCTL) >> 16; + if ((lnksta & PCI_EXP_LNKSTA_CLS) == PCI_EXP_LNKSTA_CLS_2_5GB) { + lnkctl2 = mvebu_readl(port, PCIE_CAP_PCIEXP + PCI_EXP_LNKCTL2); + lnkctl2 &= ~PCI_EXP_LNKCTL2_TLS; + lnkctl2 |= PCI_EXP_LNKCTL2_TLS_2_5GT; + mvebu_writel(port, lnkctl2, PCIE_CAP_PCIEXP + PCI_EXP_LNKCTL2); + } } static struct mvebu_pcie_port *mvebu_pcie_find_port(struct mvebu_pcie *pcie,