Message ID | alpine.DEB.2.21.2305310024400.59226@angie.orcam.me.uk |
---|---|
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2150320vqr; Sun, 11 Jun 2023 10:46:21 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4ZFmZbALDL7WPuC4WHpKH9/0q1WFteIKItY/c0H7NutiNDooIQq1UiT559mtyjE9LJcu89 X-Received: by 2002:aa7:ccce:0:b0:514:a4da:408e with SMTP id y14-20020aa7ccce000000b00514a4da408emr3764873edt.2.1686505581657; Sun, 11 Jun 2023 10:46:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686505581; cv=none; d=google.com; s=arc-20160816; b=ex7AbGhXSQVV8/aA5mlGX5c/bf/vgcSyZAmaDYubofNmaQ5GSd/rCdTyqTeaFC7kgg 141hqx5FBplfxG+Id6NWT0SMTYEhML6e7au8dsLTBsrzmyV1vvHYF3g2e7K5Ykw/UL/j sW+3X+jyZngel4jXDwqW0yqKZA5iCUiRehZ+tL+isZtCsWPnv/p2frmjFylwgIrqmYNJ r7ZHtghKP5euGtq8qHpyMWQsTmYTAE5eyYaDCHaFADkvIYDKM17lCzhUhN+3ZrOsR2bJ VOA+XSqUTmprnyH8mRe7RLznLlzj7coVzdWCsOpCoeEGkOpSmfJ7yUkNRV87DFlTqBjT wzCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:user-agent:message-id:subject:cc:to :from:date; bh=E3g1vQAHT/4Wbs/ixyekLAny06+i1sWxVERzlHQ1UfY=; b=JB5cwWrWASO5Hl9h6k2wQ6gmZIi0ehCO1iqsiokP/o/aigkX7Ixz+8gY0gWw8siIy1 iNw1G6MQ2aYlTf7IM6HfUOJkigvy/abBzkFCe4ZdqR/3TilPzHj2kjNuZsNrZpboUO27 TrvyhG5fWiMRBVF1K5rNloH+5UshT0so1SQxb80r7jNPaERFBgCFwX0lo/Oejn1aEcg8 z4ui8fTHOKaBB4Y0xE6sYCABjuvwFm9BdvVCBDQjVDoXHUAiSBgv4xytw5a3Kp/+s7Q8 vcVXjBlvw+DBaoHQEzoNrMRIxH83lZiJ2hT/KXTvX8IWHkN5dmYD8tHwxbINu9rAZU3X mzpg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ka22-20020a170907991600b00977d0db2c6asi3914374ejc.63.2023.06.11.10.45.57; Sun, 11 Jun 2023 10:46:21 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232887AbjFKRTM (ORCPT <rfc822;liningstudo@gmail.com> + 99 others); Sun, 11 Jun 2023 13:19:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43290 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229636AbjFKRTK (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Sun, 11 Jun 2023 13:19:10 -0400 Received: from angie.orcam.me.uk (angie.orcam.me.uk [IPv6:2001:4190:8020::34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 1EE00E5F; Sun, 11 Jun 2023 10:19:10 -0700 (PDT) Received: by angie.orcam.me.uk (Postfix, from userid 500) id 1966992009E; Sun, 11 Jun 2023 19:19:09 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id 134EB92009D; Sun, 11 Jun 2023 18:19:09 +0100 (BST) Date: Sun, 11 Jun 2023 18:19:08 +0100 (BST) From: "Maciej W. Rozycki" <macro@orcam.me.uk> To: Bjorn Helgaas <bhelgaas@google.com>, Mahesh J Salgaonkar <mahesh@linux.ibm.com>, Oliver O'Halloran <oohall@gmail.com>, Michael Ellerman <mpe@ellerman.id.au>, Nicholas Piggin <npiggin@gmail.com>, Christophe Leroy <christophe.leroy@csgroup.eu>, Saeed Mahameed <saeedm@nvidia.com>, Leon Romanovsky <leon@kernel.org>, "David S. Miller" <davem@davemloft.net>, Eric Dumazet <edumazet@google.com>, Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com> cc: Alex Williamson <alex.williamson@redhat.com>, Lukas Wunner <lukas@wunner.de>, Mika Westerberg <mika.westerberg@linux.intel.com>, Stefan Roese <sr@denx.de>, Jim Wilson <wilson@tuliptree.org>, David Abdurachmanov <david.abdurachmanov@gmail.com>, =?utf-8?q?Pali_Roh?= =?utf-8?q?=C3=A1r?= <pali@kernel.org>, linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v9 00/14] pci: Work around ASMedia ASM2824 PCIe link training failures Message-ID: <alpine.DEB.2.21.2305310024400.59226@angie.orcam.me.uk> User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,HDRS_LCASE, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768429277133278721?= X-GMAIL-MSGID: =?utf-8?q?1768429277133278721?= |
Series |
pci: Work around ASMedia ASM2824 PCIe link training failures
|
|
Message
Maciej W. Rozycki
June 11, 2023, 5:19 p.m. UTC
Hi, This is v9 of the change to work around a PCIe link training phenomenon where a pair of devices both capable of operating at a link speed above 2.5GT/s seems unable to negotiate the link speed and continues training indefinitely with the Link Training bit switching on and off repeatedly and the data link layer never reaching the active state. With several requests addressed and a few extra issues spotted this version has now grown to 14 patches. It has been verified for device enumeration with and without PCI_QUIRKS enabled, using the same piece of RISC-V hardware as previously. Hot plug or reset events have not been verified, as this is difficult if at all feasible with hardware in question. Last iteration: <https://lore.kernel.org/r/alpine.DEB.2.21.2304060100160.13659@angie.orcam.me.uk/>, and my input to it: <https://lore.kernel.org/r/alpine.DEB.2.21.2306080224280.36323@angie.orcam.me.uk/>. Maciej
Comments
On Sun, Jun 11, 2023 at 06:19:08PM +0100, Maciej W. Rozycki wrote: > Hi, > > This is v9 of the change to work around a PCIe link training phenomenon > where a pair of devices both capable of operating at a link speed above > 2.5GT/s seems unable to negotiate the link speed and continues training > indefinitely with the Link Training bit switching on and off repeatedly > and the data link layer never reaching the active state. > > With several requests addressed and a few extra issues spotted this > version has now grown to 14 patches. It has been verified for device > enumeration with and without PCI_QUIRKS enabled, using the same piece of > RISC-V hardware as previously. Hot plug or reset events have not been > verified, as this is difficult if at all feasible with hardware in > question. > > Last iteration: > <https://lore.kernel.org/r/alpine.DEB.2.21.2304060100160.13659@angie.orcam.me.uk/>, > and my input to it: > <https://lore.kernel.org/r/alpine.DEB.2.21.2306080224280.36323@angie.orcam.me.uk/>. Thanks, I applied these to pci/enumeration for v6.5. I tweaked a few things, so double-check to be sure I didn't break something: - Moved dev->link_active_reporting init to set_pcie_port_type() because it does other PCIe-related stuff. - Reordered to keep all the link_active_reporting things together. - Reordered to clean up & factor pcie_retrain_link() before exposing it to the rest of the PCI core. - Moved pcie_retrain_link() a little earlier to keep it next to pcie_wait_for_link_status(). - Squashed the stubs into the actual quirk so we don't have the intermediate state where we call the stubs but they never do anything (let me know if there's a reason we need your order). - Inline pcie_parent_link_retrain(), which seemed like it didn't add enough to be worthwhile. Interdiff below: diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 80694e2574b8..f11268924c8f 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -1153,27 +1153,16 @@ void pci_resume_bus(struct pci_bus *bus) pci_walk_bus(bus, pci_resume_one, NULL); } -/** - * pcie_parent_link_retrain - Check and retrain link we are downstream from - * @dev: PCI device to handle. - * - * Return TRUE if the link was retrained, FALSE otherwise. - */ -static bool pcie_parent_link_retrain(struct pci_dev *dev) -{ - struct pci_dev *bridge; - - bridge = pci_upstream_bridge(dev); - if (bridge) - return pcie_failed_link_retrain(bridge); - else - return false; -} - static int pci_dev_wait(struct pci_dev *dev, char *reset_type, int timeout) { - bool retrain = true; int delay = 1; + bool retrain = false; + struct pci_dev *bridge; + + if (pci_is_pcie(dev)) { + retrain = true; + bridge = pci_upstream_bridge(dev); + } /* * After reset, the device should not silently discard config @@ -1201,9 +1190,9 @@ static int pci_dev_wait(struct pci_dev *dev, char *reset_type, int timeout) } if (delay > PCI_RESET_WAIT) { - if (retrain) { + if (retrain && bridge) { retrain = false; - if (pcie_parent_link_retrain(dev)) { + if (pcie_failed_link_retrain(bridge)) { delay = 1; continue; } @@ -4914,6 +4903,38 @@ static bool pcie_wait_for_link_status(struct pci_dev *pdev, return (lnksta & lnksta_mask) == lnksta_match; } +/** + * pcie_retrain_link - Request a link retrain and wait for it to complete + * @pdev: Device whose link to retrain. + * @use_lt: Use the LT bit if TRUE, or the DLLLA bit if FALSE, for status. + * + * Retrain completion status is retrieved from the Link Status Register + * according to @use_lt. It is not verified whether the use of the DLLLA + * bit is valid. + * + * Return TRUE if successful, or FALSE if training has not completed + * within PCIE_LINK_RETRAIN_TIMEOUT_MS milliseconds. + */ +bool pcie_retrain_link(struct pci_dev *pdev, bool use_lt) +{ + u16 lnkctl; + + pcie_capability_read_word(pdev, PCI_EXP_LNKCTL, &lnkctl); + lnkctl |= PCI_EXP_LNKCTL_RL; + pcie_capability_write_word(pdev, PCI_EXP_LNKCTL, lnkctl); + if (pdev->clear_retrain_link) { + /* + * Due to an erratum in some devices the Retrain Link bit + * needs to be cleared again manually to allow the link + * training to succeed. + */ + lnkctl &= ~PCI_EXP_LNKCTL_RL; + pcie_capability_write_word(pdev, PCI_EXP_LNKCTL, lnkctl); + } + + return pcie_wait_for_link_status(pdev, use_lt, !use_lt); +} + /** * pcie_wait_for_link_delay - Wait until link is active or inactive * @pdev: Bridge device @@ -4968,37 +4989,6 @@ bool pcie_wait_for_link(struct pci_dev *pdev, bool active) return pcie_wait_for_link_delay(pdev, active, 100); } -/** - * pcie_retrain_link - Request a link retrain and wait for it to complete - * @pdev: Device whose link to retrain. - * @use_lt: Use the LT bit if TRUE, or the DLLLA bit if FALSE, for status. - * - * Retrain completion status is retrieved from the Link Status Register - * according to @use_lt. It is not verified whether the use of the DLLLA - * bit is valid. - * - * Return TRUE if successful, or FALSE if training has not completed. - */ -bool pcie_retrain_link(struct pci_dev *pdev, bool use_lt) -{ - u16 lnkctl; - - pcie_capability_read_word(pdev, PCI_EXP_LNKCTL, &lnkctl); - lnkctl |= PCI_EXP_LNKCTL_RL; - pcie_capability_write_word(pdev, PCI_EXP_LNKCTL, lnkctl); - if (pdev->clear_retrain_link) { - /* - * Due to an erratum in some devices the Retrain Link bit - * needs to be cleared again manually to allow the link - * training to succeed. - */ - lnkctl &= ~PCI_EXP_LNKCTL_RL; - pcie_capability_write_word(pdev, PCI_EXP_LNKCTL, lnkctl); - } - - return pcie_wait_for_link_status(pdev, use_lt, !use_lt); -} - /* * Find maximum D3cold delay required by all the devices on the bus. The * spec says 100 ms, but firmware can lower it and we allow drivers to diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 016a9d4a61f7..f547db0a728f 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -1526,6 +1526,7 @@ void set_pcie_port_type(struct pci_dev *pdev) { int pos; u16 reg16; + u32 reg32; int type; struct pci_dev *parent; @@ -1539,6 +1540,10 @@ void set_pcie_port_type(struct pci_dev *pdev) pci_read_config_dword(pdev, pos + PCI_EXP_DEVCAP, &pdev->devcap); pdev->pcie_mpss = FIELD_GET(PCI_EXP_DEVCAP_PAYLOAD, pdev->devcap); + pcie_capability_read_dword(pdev, PCI_EXP_LNKCAP, ®32); + if (reg32 & PCI_EXP_LNKCAP_DLLLARC) + pdev->link_active_reporting = 1; + parent = pci_upstream_bridge(pdev); if (!parent) return; @@ -1828,7 +1833,6 @@ int pci_setup_device(struct pci_dev *dev) int err, pos = 0; struct pci_bus_region region; struct resource *res; - u32 linkcap; hdr_type = pci_hdr_type(dev); @@ -1876,10 +1880,6 @@ int pci_setup_device(struct pci_dev *dev) /* "Unknown power state" */ dev->current_state = PCI_UNKNOWN; - /* Set it early to make it available to fixups, etc. */ - pcie_capability_read_dword(dev, PCI_EXP_LNKCAP, &linkcap); - dev->link_active_reporting = !!(linkcap & PCI_EXP_LNKCAP_DLLLARC); - /* Early fixups, before probing the BARs */ pci_fixup_device(pci_fixup_early, dev);
On Wed, 14 Jun 2023, Bjorn Helgaas wrote: > > This is v9 of the change to work around a PCIe link training phenomenon > > where a pair of devices both capable of operating at a link speed above > > 2.5GT/s seems unable to negotiate the link speed and continues training > > indefinitely with the Link Training bit switching on and off repeatedly > > and the data link layer never reaching the active state. > > > > With several requests addressed and a few extra issues spotted this > > version has now grown to 14 patches. It has been verified for device > > enumeration with and without PCI_QUIRKS enabled, using the same piece of > > RISC-V hardware as previously. Hot plug or reset events have not been > > verified, as this is difficult if at all feasible with hardware in > > question. > > > > Last iteration: > > <https://lore.kernel.org/r/alpine.DEB.2.21.2304060100160.13659@angie.orcam.me.uk/>, > > and my input to it: > > <https://lore.kernel.org/r/alpine.DEB.2.21.2306080224280.36323@angie.orcam.me.uk/>. > > Thanks, I applied these to pci/enumeration for v6.5. Great, thanks! > I tweaked a few things, so double-check to be sure I didn't break > something: > > - Moved dev->link_active_reporting init to set_pcie_port_type() > because it does other PCIe-related stuff. > > - Reordered to keep all the link_active_reporting things together. > > - Reordered to clean up & factor pcie_retrain_link() before exposing > it to the rest of the PCI core. > > - Moved pcie_retrain_link() a little earlier to keep it next to > pcie_wait_for_link_status(). > > - Squashed the stubs into the actual quirk so we don't have the > intermediate state where we call the stubs but they never do > anything (let me know if there's a reason we need your order). > > - Inline pcie_parent_link_retrain(), which seemed like it didn't add > enough to be worthwhile. Ack, I'll double-check and report back. A minor nit I've spotted below: > static int pci_dev_wait(struct pci_dev *dev, char *reset_type, int timeout) > { > - bool retrain = true; > int delay = 1; > + bool retrain = false; > + struct pci_dev *bridge; > + > + if (pci_is_pcie(dev)) { > + retrain = true; > + bridge = pci_upstream_bridge(dev); > + } If doing it this way, which I actually like, I think it would be a little bit better performance- and style-wise if this was written as: if (pci_is_pcie(dev)) { bridge = pci_upstream_bridge(dev); retrain = !!bridge; } (or "retrain = bridge != NULL" if you prefer this style), and then we don't have to repeatedly check two variables iff (pcie && !bridge) in the loop below: > @@ -1201,9 +1190,9 @@ static int pci_dev_wait(struct pci_dev *dev, char *reset_type, int timeout) > } > > if (delay > PCI_RESET_WAIT) { > - if (retrain) { > + if (retrain && bridge) { -- i.e. code can stay then as: if (retrain) { here. I hope you find this observation rather obvious, so will you amend your tree, or shall I send an incremental update? Otherwise I don't find anything suspicious with the interdiff itself (thanks for posting it, that's really useful indeed!), but as I say I'll yet double-check how things look and work with your tree. Hopefully tomorrow (Thu), as I have other stuff yet to complete tonight. Maciej
On Thu, Jun 15, 2023 at 01:41:10AM +0100, Maciej W. Rozycki wrote: > On Wed, 14 Jun 2023, Bjorn Helgaas wrote: > > > > This is v9 of the change to work around a PCIe link training phenomenon > > > where a pair of devices both capable of operating at a link speed above > > > 2.5GT/s seems unable to negotiate the link speed and continues training > > > indefinitely with the Link Training bit switching on and off repeatedly > > > and the data link layer never reaching the active state. > > > > > > With several requests addressed and a few extra issues spotted this > > > version has now grown to 14 patches. It has been verified for device > > > enumeration with and without PCI_QUIRKS enabled, using the same piece of > > > RISC-V hardware as previously. Hot plug or reset events have not been > > > verified, as this is difficult if at all feasible with hardware in > > > question. > > static int pci_dev_wait(struct pci_dev *dev, char *reset_type, int timeout) > > { > > - bool retrain = true; > > int delay = 1; > > + bool retrain = false; > > + struct pci_dev *bridge; > > + > > + if (pci_is_pcie(dev)) { > > + retrain = true; > > + bridge = pci_upstream_bridge(dev); > > + } > > If doing it this way, which I actually like, I think it would be a little > bit better performance- and style-wise if this was written as: > > if (pci_is_pcie(dev)) { > bridge = pci_upstream_bridge(dev); > retrain = !!bridge; > } > > (or "retrain = bridge != NULL" if you prefer this style), and then we > don't have to repeatedly check two variables iff (pcie && !bridge) in the > loop below: Done, thanks, I do like that better. I did: bridge = pci_upstream_bridge(dev); if (bridge) retrain = true; because it seems like it flows more naturally when reading. Bjorn
On Thu, 15 Jun 2023, Bjorn Helgaas wrote: > > If doing it this way, which I actually like, I think it would be a little > > bit better performance- and style-wise if this was written as: > > > > if (pci_is_pcie(dev)) { > > bridge = pci_upstream_bridge(dev); > > retrain = !!bridge; > > } > > > > (or "retrain = bridge != NULL" if you prefer this style), and then we > > don't have to repeatedly check two variables iff (pcie && !bridge) in the > > loop below: > > Done, thanks, I do like that better. I did: > > bridge = pci_upstream_bridge(dev); > if (bridge) > retrain = true; > > because it seems like it flows more naturally when reading. Perfect, and good timing too, as I have just started checking your tree as your message arrived. I ran my usual tests with and w/o PCI_QUIRKS enabled and results were as expected. As before I didn't check hot plug and reset paths as these features are awkward with the HiFive Unmatched system involved. I have skimmed over the changes as committed to pci/enumeration and found nothing suspicious. I have verified that the tree builds as at each of them with my configuration. As per my earlier remark: > I think making a system halfway-fixed would make little sense, but with > the actual fix actually made last as you suggested I think this can be > split off, because it'll make no functional change by itself. I am not perfectly happy with your rearrangement to fold the !PCI_QUIRKS stub into the change carrying the actual workaround and then have the reset path update with a follow-up change only, but I won't fight over it. It's only one tree revision that will be in this halfway-fixed state and I'll trust your judgement here. Let me know if anything pops up related to these changes anytime and I'll be happy to look into it. The system involved is nearing two years since its deployment already, but hopefully it has many years to go yet and will continue being ready to verify things. It's not that there's lots of real RISC-V hardware available, let alone with PCI/e connectivity. Thank you for staying with me and reviewing this patch series through all the iterations. Maciej
On Fri, Jun 16, 2023 at 01:27:52PM +0100, Maciej W. Rozycki wrote: > On Thu, 15 Jun 2023, Bjorn Helgaas wrote: > As per my earlier remark: > > > I think making a system halfway-fixed would make little sense, but with > > the actual fix actually made last as you suggested I think this can be > > split off, because it'll make no functional change by itself. > > I am not perfectly happy with your rearrangement to fold the !PCI_QUIRKS > stub into the change carrying the actual workaround and then have the > reset path update with a follow-up change only, but I won't fight over it. > It's only one tree revision that will be in this halfway-fixed state and > I'll trust your judgement here. Thanks for raising this. Here's my thought process: 12 PCI: Provide stub failed link recovery for device probing and hot plug 13 PCI: Add failed link recovery for device reset events 14 PCI: Work around PCIe link training failures Patch 12 [1] adds calls to pcie_failed_link_retrain(), which does nothing and returns false. Functionally, it's a no-op, but the structure is important later. Patch 13 [2] claims to request failed link recovery after resets, but actually doesn't do anything yet because pcie_failed_link_retrain() is still a no-op, so this was a bit confusing. Patch 14 [3] implements pcie_failed_link_retrain(), so the recovery mentioned in 12 and 13 actually happens. But this patch doesn't add the call to pcie_failed_link_retrain(), so it's a little bit hard to connect the dots. I agree that as I rearranged it, the workaround doesn't apply in all cases simultaneously. Maybe not ideal, but maybe not terrible either. Looking at it again, maybe it would have made more sense to move the pcie_wait_for_link_delay() change to the last patch along with the pci_dev_wait() change. I dunno. Bjorn [1] 12 https://lore.kernel.org/r/alpine.DEB.2.21.2306111619570.64925@angie.orcam.me.uk [2] 13 https://lore.kernel.org/r/alpine.DEB.2.21.2306111631050.64925@angie.orcam.me.uk [3] 14 https://lore.kernel.org/r/alpine.DEB.2.21.2305310038540.59226@angie.orcam.me.uk
On Fri, 16 Jun 2023, Bjorn Helgaas wrote: > I agree that as I rearranged it, the workaround doesn't apply in all > cases simultaneously. Maybe not ideal, but maybe not terrible either. > Looking at it again, maybe it would have made more sense to move the > pcie_wait_for_link_delay() change to the last patch along with the > pci_dev_wait() change. I dunno. I think the order of the changes is not important enough to justify spending a lot of time and mental effort on it. You decided, so be it. Thank you for your effort made with this review. With this series out of the way I have now posted a small clean-up for SBR code duplication between PCI core and an InfiniBand driver I came across in the course of working on this series. See <https://lore.kernel.org/r/alpine.DEB.2.21.2306200153110.14084@angie.orcam.me.uk/>. Please have a look at your convenience. Maciej