From patchwork Sun Jun 11 17:19:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Maciej W. Rozycki" X-Patchwork-Id: 106145 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2151453vqr; Sun, 11 Jun 2023 10:49:50 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4MpdHOCsQLHwD1yVaoEXQYl9UA1wpYL1zdXbXCsdiKVogChpfchw5MUOX4zu/x9wlVoT46 X-Received: by 2002:a17:907:6e12:b0:977:ab43:731f with SMTP id sd18-20020a1709076e1200b00977ab43731fmr9345678ejc.66.1686505790481; Sun, 11 Jun 2023 10:49:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686505790; cv=none; d=google.com; s=arc-20160816; b=ta/pkXf8wWMsyA30KElIaQl0NvEPgVlVz5neh9xzR6wyxC6AOeNz5b+55rlPB8hnx5 qouNsonaAuGsVoY9i6rizbO0hXjbZNpdaT6F/MLBkuXkDa4v0Awb9awBk8VxZv6t+q4X OuTrsQ7km9u9jiNL0z3/W53q0pquoyfpjMRq9yaw16Jn7RrgzdmWEeewJpRZ+XK4C06t 6Oh1nTMi1vOmk0PZdGiXL7GLcxNpEVBZeF03tgHKxGB73QpT0WOvCet/AEhokFa/Iw94 M6zkYLDKm/RU4Hd5MwvGwQIDfkEa7ph/tLpjZfEWKoRsTn9rFLh0qbqjj6CvZKGhXQL2 4yrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:user-agent:references:message-id :in-reply-to:subject:cc:to:from:date; bh=lIlFy+sUklVWA07f2WsNkrJoBRqOCpMxq4gcyXJh+Fw=; b=izg6AFb5K9+cNBXZwaacYLd0v+fAOaO9/GriBL+Zy1NwrIqLXFgB2XBmpWSG1REejE 0V8PYXCwMEKutxUBXX7uxxp8iDxGmAO1KUDqqcQIP2XzSzmnK55Qr7cxaIhCFxkcns9x Z/qHuI8zHZRWQuCc+fxL6SydbEkpDhREKqDo9A9rGUF385yTmaaiGLqo/6k4sZktLFZa QdxHLxdw++oeWk16Jrlph/jOSXPc/U8zMvkaQPepYRVlzngQvu/95vOqDqRgec/1G+nT FfVwW2z30eCaN+2wgEl7HJB9ui3CrAtwD2b/JwhOEdHHK0/qx2shbv7zJuMaLRyfCgze Sl/Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ka22-20020a170907991600b00977d0db2c6asi3914374ejc.63.2023.06.11.10.49.27; Sun, 11 Jun 2023 10:49:50 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233077AbjFKRTT (ORCPT + 99 others); Sun, 11 Jun 2023 13:19:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43328 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229636AbjFKRTR (ORCPT ); Sun, 11 Jun 2023 13:19:17 -0400 Received: from angie.orcam.me.uk (angie.orcam.me.uk [78.133.224.34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 0981BE5F for ; Sun, 11 Jun 2023 10:19:15 -0700 (PDT) Received: by angie.orcam.me.uk (Postfix, from userid 500) id 0EA1A9200B4; Sun, 11 Jun 2023 19:19:15 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id 0B5499200B3; Sun, 11 Jun 2023 18:19:15 +0100 (BST) Date: Sun, 11 Jun 2023 18:19:14 +0100 (BST) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?utf-8?q?Pali_Roh?= =?utf-8?q?=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v9 01/14] PCI: pciehp: Rely on `link_active_reporting' In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768429495698369344?= X-GMAIL-MSGID: =?utf-8?q?1768429495698369344?= Use `link_active_reporting' to determine whether Data Link Layer Link Active Reporting is available rather than re-retrieving the capability. Signed-off-by: Maciej W. Rozycki Reviewed-by: Lukas Wunner --- NB this has been compile-tested only with PPC64LE and x86-64 configurations. No change from v8. Changes from v7: - Add Reviewed-by: tag by Lukas Wunner. - Reorder from 6/7. No change from v6. New change in v6. --- drivers/pci/hotplug/pciehp_hpc.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) linux-pcie-link-active-reporting-hpc.diff Index: linux-macro/drivers/pci/hotplug/pciehp_hpc.c =================================================================== --- linux-macro.orig/drivers/pci/hotplug/pciehp_hpc.c +++ linux-macro/drivers/pci/hotplug/pciehp_hpc.c @@ -984,7 +984,7 @@ static inline int pcie_hotplug_depth(str struct controller *pcie_init(struct pcie_device *dev) { struct controller *ctrl; - u32 slot_cap, slot_cap2, link_cap; + u32 slot_cap, slot_cap2; u8 poweron; struct pci_dev *pdev = dev->port; struct pci_bus *subordinate = pdev->subordinate; @@ -1030,9 +1030,6 @@ struct controller *pcie_init(struct pcie if (dmi_first_match(inband_presence_disabled_dmi_table)) ctrl->inband_presence_disabled = 1; - /* Check if Data Link Layer Link Active Reporting is implemented */ - pcie_capability_read_dword(pdev, PCI_EXP_LNKCAP, &link_cap); - /* Clear all remaining event bits in Slot Status register. */ pcie_capability_write_word(pdev, PCI_EXP_SLTSTA, PCI_EXP_SLTSTA_ABP | PCI_EXP_SLTSTA_PFD | @@ -1051,7 +1048,7 @@ struct controller *pcie_init(struct pcie FLAG(slot_cap, PCI_EXP_SLTCAP_EIP), FLAG(slot_cap, PCI_EXP_SLTCAP_NCCS), FLAG(slot_cap2, PCI_EXP_SLTCAP2_IBPD), - FLAG(link_cap, PCI_EXP_LNKCAP_DLLLARC), + FLAG(pdev->link_active_reporting, true), pdev->broken_cmd_compl ? " (with Cmd Compl erratum)" : ""); /* From patchwork Sun Jun 11 17:19:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Maciej W. Rozycki" X-Patchwork-Id: 106142 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2151125vqr; Sun, 11 Jun 2023 10:48:45 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6IRmgSOtAeTyzrWeypJD/cMH4U3BA6nc+hSkbhdjXTnBfd9NsQNbnjCnpdSUAcBcunX/d2 X-Received: by 2002:a05:6402:150e:b0:50d:dba8:c64a with SMTP id f14-20020a056402150e00b0050ddba8c64amr3457482edw.18.1686505725639; Sun, 11 Jun 2023 10:48:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686505725; cv=none; d=google.com; s=arc-20160816; b=NLC66Hk1ZwPz0sxmSSVTixQDhDbS1YdGlI96pJfDySzru28MLs43f4P1stFlks4rG3 R5t2ErYSfd7GsshKAp3f86A4zY3HiiOPeBtvNIWjyJV8TvEmGdYb4xkEQ6RvD6r7kTiW wZSQryxpOZocqjXfcqUaMSrMQfww/y9mP/So4UPL/YbpS/dG2IyOswUPJjJfkORXCrys qHwv564hLpyaYF3QbqrrzDIv434452Is3WbzrHfHkjJZOdAR/SyyoDBuN2KSsOA4FYJm 4WFR/7bW66kUXGHQhUKY+RWRiWtb9nCeUFTTU93fewZMbT2yLIKGO7v8KsBR1wA9O9k0 SubQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:user-agent:references:message-id :in-reply-to:subject:cc:to:from:date; bh=jXtpYvkKk9QWqpFGXcqUbxFOs23Zj1fwmi1JYXP7DEM=; b=AmPTj5KhjIjYLAAQm8w+G6OX5xswvLaqwCn4cx69eIBKhFxntdu8Ob/O7H4xMCteoR wgnTUD84wQOyLzA3jqQVFxp4DvxLmZ0sS+T9NpInyYz8YuIKJSfSifai/7dWy51FJisc jUjjJIjYDXUpdhDewT7bZySKszmrCaZfWHmvQQHv6tisSSQFimcsuwgxXNNCADKOmomx +qrsREo0IgCkAmlHshg/mFF3bHAit/YKW6FPe5Ae7vBjiGoT/UOcA3Z8PDSE/ALd1jfQ ClAD978zNYIPkUlqEntZ75K9qPgIPSmFh1DAKquRBvShM0y/D9nvuPx1PazKsK1TB3AS ONVg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s1-20020a056402164100b005149aee5218si4854537edx.51.2023.06.11.10.48.21; Sun, 11 Jun 2023 10:48:45 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233729AbjFKRTY (ORCPT + 99 others); Sun, 11 Jun 2023 13:19:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43428 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233665AbjFKRTV (ORCPT ); Sun, 11 Jun 2023 13:19:21 -0400 Received: from angie.orcam.me.uk (angie.orcam.me.uk [IPv6:2001:4190:8020::34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 57FD6E68; Sun, 11 Jun 2023 10:19:20 -0700 (PDT) Received: by angie.orcam.me.uk (Postfix, from userid 500) id B62CE9200BC; Sun, 11 Jun 2023 19:19:19 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id AEEC09200B3; Sun, 11 Jun 2023 18:19:19 +0100 (BST) Date: Sun, 11 Jun 2023 18:19:19 +0100 (BST) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?utf-8?q?Pali_Roh?= =?utf-8?q?=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v9 02/14] PCI: Export PCIe link retrain timeout In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768429427684522678?= X-GMAIL-MSGID: =?utf-8?q?1768429427684522678?= Convert LINK_RETRAIN_TIMEOUT from jiffies to milliseconds, accordingly rename to PCIE_LINK_RETRAIN_TIMEOUT_MS, and make available via "pci.h" for PCI drivers to use. Use in `pcie_wait_for_link_delay'. Signed-off-by: Maciej W. Rozycki --- Changes from v8: - Convert LINK_RETRAIN_TIMEOUT from jiffies to milliseconds, rename it to PCIE_LINK_RETRAIN_TIMEOUT_MS rather than PCIE_LINK_RETRAIN_TIMEOUT, and adjust its use accordingly. - Also replace hardcoded 1000 in `pcie_wait_for_link_delay'. - Correct the change heading, s/PCI/PCIe/ for the link reference. Changes from v7: - Reorder from 1/7. No change from v6. No change from v5. New change in v5. --- drivers/pci/pci.c | 2 +- drivers/pci/pci.h | 2 ++ drivers/pci/pcie/aspm.c | 4 +--- 3 files changed, 4 insertions(+), 4 deletions(-) linux-pcie-link-retrain-timeout.diff Index: linux-macro/drivers/pci/pci.c =================================================================== --- linux-macro.orig/drivers/pci/pci.c +++ linux-macro/drivers/pci/pci.c @@ -4860,7 +4860,7 @@ static int pci_pm_reset(struct pci_dev * static bool pcie_wait_for_link_delay(struct pci_dev *pdev, bool active, int delay) { - int timeout = 1000; + int timeout = PCIE_LINK_RETRAIN_TIMEOUT_MS; bool ret; u16 lnk_status; Index: linux-macro/drivers/pci/pci.h =================================================================== --- linux-macro.orig/drivers/pci/pci.h +++ linux-macro/drivers/pci/pci.h @@ -11,6 +11,8 @@ #define PCI_VSEC_ID_INTEL_TBT 0x1234 /* Thunderbolt */ +#define PCIE_LINK_RETRAIN_TIMEOUT_MS 1000 + extern const unsigned char pcie_link_speed[]; extern bool pci_early_dump; Index: linux-macro/drivers/pci/pcie/aspm.c =================================================================== --- linux-macro.orig/drivers/pci/pcie/aspm.c +++ linux-macro/drivers/pci/pcie/aspm.c @@ -90,8 +90,6 @@ static const char *policy_str[] = { [POLICY_POWER_SUPERSAVE] = "powersupersave" }; -#define LINK_RETRAIN_TIMEOUT HZ - /* * The L1 PM substate capability is only implemented in function 0 in a * multi function device. @@ -213,7 +211,7 @@ static bool pcie_retrain_link(struct pci } /* Wait for link training end. Break out after waiting for timeout */ - end_jiffies = jiffies + LINK_RETRAIN_TIMEOUT; + end_jiffies = jiffies + msecs_to_jiffies(PCIE_LINK_RETRAIN_TIMEOUT_MS); do { pcie_capability_read_word(parent, PCI_EXP_LNKSTA, ®16); if (!(reg16 & PCI_EXP_LNKSTA_LT)) From patchwork Sun Jun 11 17:19:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Maciej W. Rozycki" X-Patchwork-Id: 106132 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2142006vqr; Sun, 11 Jun 2023 10:21:45 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4JySDFCRZu6A1EA5c+Vu9WKPbCQif7XPA+no+WIZAMqEfGw6k3GDggj/drsa8emcWbf/FP X-Received: by 2002:a17:907:169f:b0:96f:2b3f:61 with SMTP id hc31-20020a170907169f00b0096f2b3f0061mr8201385ejc.7.1686504105047; Sun, 11 Jun 2023 10:21:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686504105; cv=none; d=google.com; s=arc-20160816; b=hO8RWy/dBRs1BeI6y+9ToEvDQx550UkM5fVtTY3nBiA6tFPsnkMh6TBdbu+ZaOC2s7 mE2phCuGkyOoDy8NLURiUU5edfVXQz7+dMcJjpZBr9pQzw63Fy5GQZGbQzIxspt8rjon zoT2kcDhng1AYrqREz925YfF5TsSDAVO9EYcnP61g5oyAS7UyEeJ0wX9dDeZVtNpO0dB uNeqZiS5b3ikrCQE74NZe7PprrlA31xtffDnIrsD9hvgEAaThmg7m7CkdaKnLRAYstcy bHVCxvZDv6Mc5LMACSgyI2WJxrJy8nBBWDunfg0blolcVQYPTWJ2v3ZQiSXxhBFqa/M6 6DHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:user-agent:references:message-id :in-reply-to:subject:cc:to:from:date; bh=3nfGjyo9MsFFkROd0Ab6OTM7zxwm7wRQl3dlmOm3Sls=; b=HGAvQ91WXVytRxQomTYiO+IMlu1yYp+Ia3ZlNBubKkWxQRIdYgA4eudkrKvFtnMLus 9kASHG2TSiJ35x1S6B7ajRKhxfNYaL4HkWxE6vdcsXPapEnJx0HXtal2MjZKr3MVNUZR oVV16lM85o56yqlTuNnP7pDJsrE46uwo9B0iVCfZ7SHu0gbqh0pZvYlL3JfbGVXu8RrS 4i06rEh0ajHGblGlmdkB6ETtxFFHwJ/RLZlSQxEA+tgblNlTMc34VCzq2KR5cd9h22Fa dp+3bLUqkh2qJkyIoX+Vp+l0ScZxGOLugbWeST5ObgRIRzMXZPOjcYFxK/Iwv87X6fFe CN+w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id qh12-20020a170906ecac00b0097885374cf3si4257243ejb.392.2023.06.11.10.21.21; Sun, 11 Jun 2023 10:21:45 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233120AbjFKRTb (ORCPT + 99 others); Sun, 11 Jun 2023 13:19:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43428 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233879AbjFKRT1 (ORCPT ); Sun, 11 Jun 2023 13:19:27 -0400 Received: from angie.orcam.me.uk (angie.orcam.me.uk [IPv6:2001:4190:8020::34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 1564A10CC; Sun, 11 Jun 2023 10:19:24 -0700 (PDT) Received: by angie.orcam.me.uk (Postfix, from userid 500) id 69A929200BF; Sun, 11 Jun 2023 19:19:23 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id 637809200B3; Sun, 11 Jun 2023 18:19:23 +0100 (BST) Date: Sun, 11 Jun 2023 18:19:23 +0100 (BST) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?utf-8?q?Pali_Roh?= =?utf-8?q?=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v9 03/14] PCI: Execute `quirk_enable_clear_retrain_link' earlier In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768427728482278528?= X-GMAIL-MSGID: =?utf-8?q?1768427728482278528?= Make `quirk_enable_clear_retrain_link' `pci_fixup_early' so that any later fixups can rely on `clear_retrain_link' to have been already initialised. Signed-off-by: Maciej W. Rozycki --- No change from v8. Changes from v7: - Reorder from 2/7. No change from v6. No change from v5. New change in v5. --- drivers/pci/quirks.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) linux-pcie-clear-retrain-link-early.diff Index: linux-macro/drivers/pci/quirks.c =================================================================== --- linux-macro.orig/drivers/pci/quirks.c +++ linux-macro/drivers/pci/quirks.c @@ -2407,9 +2407,9 @@ static void quirk_enable_clear_retrain_l dev->clear_retrain_link = 1; pci_info(dev, "Enable PCIe Retrain Link quirk\n"); } -DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_PERICOM, 0xe110, quirk_enable_clear_retrain_link); -DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_PERICOM, 0xe111, quirk_enable_clear_retrain_link); -DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_PERICOM, 0xe130, quirk_enable_clear_retrain_link); +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_PERICOM, 0xe110, quirk_enable_clear_retrain_link); +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_PERICOM, 0xe111, quirk_enable_clear_retrain_link); +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_PERICOM, 0xe130, quirk_enable_clear_retrain_link); static void fixup_rev1_53c810(struct pci_dev *dev) { From patchwork Sun Jun 11 17:19:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Maciej W. Rozycki" X-Patchwork-Id: 106138 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2149717vqr; Sun, 11 Jun 2023 10:44:43 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6I+UIXPIz4Lo6j5+Esb/AY+1eZbGAsqs1+0stHDDBMkUkcjoEAy+TWV9gafmxeDH6MJ6cE X-Received: by 2002:aa7:c68f:0:b0:50d:8991:d1b9 with SMTP id n15-20020aa7c68f000000b0050d8991d1b9mr3481284edq.8.1686505483325; Sun, 11 Jun 2023 10:44:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686505483; cv=none; d=google.com; s=arc-20160816; b=ef9uXTSAa9ZW+kVQ2PketwbgzKv2bYpV30KzJPSR+C3CaKD9snBs3ZKCS/MgMP3Q9w 0SgrCxWfg7/LJVsRH7JqKdsatKjBCC/38I050pX6AzmWJ3/s3Qg6Y1FFV10U1FYw0jnT 6JqDttL1WaB7w/x7F++pHYPUgS7R+G2Na6FR+0Z0QkCt3daQVOJEJE070KzNotydqihX 1zbtIilVRt7+5ih89tKl+Eac4qbO3tqACZ3DHu3dJzRbzbKiDdk9diGr/B/xJ/TWHc0H nFqRrBmBEbUOXTXnNsWtZuhdzqcP/kCq/AxxDaQXcyr+vT51J3txbUnSQYfD5g3f2WL6 b0Og== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:user-agent:references:message-id :in-reply-to:subject:cc:to:from:date; bh=GTQgYOq1NCjYALleIk2DxLTpwSUi5p+ZwBiUC1Rpc4U=; b=Xa+A17d+GRtYzJ+abxJMfQao1MWpr7ULUl99rU4AQloK1MR6Ah5eF1otya4qJWx9R1 w9ArDBdtlNZn6711X43Qo0ZPleWlT72RH9t9LUS1b8EtPKwb7occhQDGIXluxrGFxO0b HxCOfQRSHzB0bqTz1ur5Em5qIZ8QV5uDuCHNmLFEe8s/blub35+5+I+zhkgRZPm0+eBJ EdhQXIzw/1/7w6qgwBDMlpNt3S316ZVFpBDDG2Mso1gek1HZPc00VALBERJflONLRa7B 8UQKWA8xHSDel8ELRGGgYdkAhw+Acippr4ep626KUTU8P2IT5Ab0UaeKSnePdyH0Ql/9 vnkQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id v12-20020aa7d64c000000b005164084884csi712568edr.647.2023.06.11.10.44.19; Sun, 11 Jun 2023 10:44:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232674AbjFKRTl (ORCPT + 99 others); Sun, 11 Jun 2023 13:19:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43782 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231451AbjFKRTi (ORCPT ); Sun, 11 Jun 2023 13:19:38 -0400 Received: from angie.orcam.me.uk (angie.orcam.me.uk [78.133.224.34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id A93B5E7D for ; Sun, 11 Jun 2023 10:19:28 -0700 (PDT) Received: by angie.orcam.me.uk (Postfix, from userid 500) id C29129200BF; Sun, 11 Jun 2023 19:19:27 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id C025E9200BB; Sun, 11 Jun 2023 18:19:27 +0100 (BST) Date: Sun, 11 Jun 2023 18:19:27 +0100 (BST) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?utf-8?q?Pali_Roh?= =?utf-8?q?=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v9 04/14] PCI: Initialize `link_active_reporting' earlier In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768429173949044085?= X-GMAIL-MSGID: =?utf-8?q?1768429173949044085?= Determine whether Data Link Layer Link Active Reporting is available ahead of calling any fixups so that the cached value can be used there and later on. Signed-off-by: Maciej W. Rozycki --- No change from v8. Changes from v7: - Reorder from 3/7. Changes from v6: - Regenerate against 6.3-rc5. New change in v6. --- drivers/pci/probe.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) linux-pcie-link-active-reporting-early.diff Index: linux-macro/drivers/pci/probe.c =================================================================== --- linux-macro.orig/drivers/pci/probe.c +++ linux-macro/drivers/pci/probe.c @@ -820,7 +820,6 @@ static void pci_set_bus_speed(struct pci pcie_capability_read_dword(bridge, PCI_EXP_LNKCAP, &linkcap); bus->max_bus_speed = pcie_link_speed[linkcap & PCI_EXP_LNKCAP_SLS]; - bridge->link_active_reporting = !!(linkcap & PCI_EXP_LNKCAP_DLLLARC); pcie_capability_read_word(bridge, PCI_EXP_LNKSTA, &linksta); pcie_update_link_speed(bus, linksta); @@ -1829,6 +1828,7 @@ int pci_setup_device(struct pci_dev *dev int err, pos = 0; struct pci_bus_region region; struct resource *res; + u32 linkcap; hdr_type = pci_hdr_type(dev); @@ -1876,6 +1876,10 @@ int pci_setup_device(struct pci_dev *dev /* "Unknown power state" */ dev->current_state = PCI_UNKNOWN; + /* Set it early to make it available to fixups, etc. */ + pcie_capability_read_dword(dev, PCI_EXP_LNKCAP, &linkcap); + dev->link_active_reporting = !!(linkcap & PCI_EXP_LNKCAP_DLLLARC); + /* Early fixups, before probing the BARs */ pci_fixup_device(pci_fixup_early, dev); From patchwork Sun Jun 11 17:19:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Maciej W. Rozycki" X-Patchwork-Id: 106133 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2142071vqr; Sun, 11 Jun 2023 10:21:55 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ681IhIqVstovfzlbdbrArzgM+UgeL6sJ1HwYoGmLoEY8k9YguDM8ff5EE77UUPMfLGi6fH X-Received: by 2002:a17:907:31ca:b0:94e:e6b9:fef2 with SMTP id xf10-20020a17090731ca00b0094ee6b9fef2mr7124375ejb.67.1686504114795; Sun, 11 Jun 2023 10:21:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686504114; cv=none; d=google.com; s=arc-20160816; b=N3QwrnyWPmpI/l5DkAF2FV5LQZIAS5y6vDHeiS249RhoGcOIXsD49pOj7tqdi5UTTN 5Gq96YSktC9y2H8xHsayc15t8EMcqesPJihaF6/fUYTI+nK2axkPwtA8oZfH8diXj6gr sR2inZvtVo6Tv8sAPOKJihwRXNvAMofdo6VyG0mi+3zLtBK7k8dYnfLdM/k2UY/rZn+Q mp6X/pvSfTzJfoMRgwH5r9h14riy7x+xe2C5D6CrMXuUs1ecezKrMytOFCv5O9Gtq+sw y1SmXlVGJBh1JFbvWDRalZuhqlTUiZPTFT2vgwIzw2S9uBwwyHgwmYzYAaHugRD23B4P zngA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:user-agent:references:message-id :in-reply-to:subject:cc:to:from:date; bh=NvNCnb3jQVKfCQT77/03s8ZBBMoSywcs7ILwTapEDN4=; b=qERUwcszv+aVJYTatEldiujTjLiDxxysXcD9ClolR1lMmVmKiiyIddIL1DPHFY45N+ irW6yUMeLGrMnZ4+0cf/3o4P5FOkAnS7g0Pdw4bk1MWy0jyhGMkazCkyOwAXnZQVFzck gP3FoxUgBJsn9uSIizJ3dyI5RapxsT4Tqx9XhA7xAUr6KiXBkfZ3v5a52GdyxfM8++kF Sj12nvZRfVx2uFRhrsN6cnG3uCox9T8jMhEsYpY7XnOuZoi1/VoNKKIAI9ory7jt9bOP J8RS9XBAqMEb3CVUKqD1JSIsCMxYSehv/k1VeYK03Rrw7Je1IWHj8zd4E6f8Kb9eNP34 Ihcw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ju21-20020a17090798b500b0097887762a89si4128244ejc.610.2023.06.11.10.21.30; Sun, 11 Jun 2023 10:21:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231949AbjFKRTx (ORCPT + 99 others); Sun, 11 Jun 2023 13:19:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43988 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233938AbjFKRTt (ORCPT ); Sun, 11 Jun 2023 13:19:49 -0400 Received: from angie.orcam.me.uk (angie.orcam.me.uk [78.133.224.34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 36BBEE7A for ; Sun, 11 Jun 2023 10:19:33 -0700 (PDT) Received: by angie.orcam.me.uk (Postfix, from userid 500) id 67DE69200C3; Sun, 11 Jun 2023 19:19:32 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id 640209200BB; Sun, 11 Jun 2023 18:19:32 +0100 (BST) Date: Sun, 11 Jun 2023 18:19:32 +0100 (BST) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?utf-8?q?Pali_Roh?= =?utf-8?q?=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v9 05/14] powerpc/eeh: Rely on `link_active_reporting' In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768427738590649498?= X-GMAIL-MSGID: =?utf-8?q?1768427738590649498?= Use `link_active_reporting' to determine whether Data Link Layer Link Active Reporting is available rather than re-retrieving the capability. Signed-off-by: Maciej W. Rozycki --- NB this has been compile-tested only with a PPC64LE configuration. No change from v8. Changes from v7: - Reorder from 4/7. No change from v6. New change in v6. --- arch/powerpc/kernel/eeh_pe.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) linux-pcie-link-active-reporting-eeh.diff Index: linux-macro/arch/powerpc/kernel/eeh_pe.c =================================================================== --- linux-macro.orig/arch/powerpc/kernel/eeh_pe.c +++ linux-macro/arch/powerpc/kernel/eeh_pe.c @@ -671,9 +671,8 @@ static void eeh_bridge_check_link(struct eeh_ops->write_config(edev, cap + PCI_EXP_LNKCTL, 2, val); /* Check link */ - eeh_ops->read_config(edev, cap + PCI_EXP_LNKCAP, 4, &val); - if (!(val & PCI_EXP_LNKCAP_DLLLARC)) { - eeh_edev_dbg(edev, "No link reporting capability (0x%08x) \n", val); + if (!edev->pdev->link_active_reporting) { + eeh_edev_dbg(edev, "No link reporting capability\n"); msleep(1000); return; } From patchwork Sun Jun 11 17:19:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Maciej W. Rozycki" X-Patchwork-Id: 106143 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2151317vqr; Sun, 11 Jun 2023 10:49:19 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7BtROQ6Wo3G9ABHKfCTfnvpPycyHmTP2mqFgyWzD0zg5Ipk6wm8v+E1g2wP5GxnkdmFHHe X-Received: by 2002:a17:907:2d26:b0:94b:4a4:2836 with SMTP id gs38-20020a1709072d2600b0094b04a42836mr8010542ejc.69.1686505759369; Sun, 11 Jun 2023 10:49:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686505759; cv=none; d=google.com; s=arc-20160816; b=dcKj5ODRKw1cs+bhpLqGRAqp8t3W/E4+d1dOFCS4VRo25EIjTFXXpE8Y0wVBo36plM sLgS6old1ZTzlDRmgYxioWByaUnl1eL9MEYD8blyvtxAy0aaQSz2du70QneKr0KfAi66 k4eKrwjnKmIwsMrhPu3QYP7HOMm5S+X3udNgKA8vCYkMrPkkeE9ZVJy1qPT2mch2FCbZ kLVPeIR3bps9W+yoIVjNa1pCZ+3/rrx//HWsZDnljMEVxQa7Xp69aYI8/sCiOFSY9//t jexuJcURyI7TV8n2bdAUYdoKDKiQpyW0aGhwgWW74cBSM7/LrjyAC46tkjBm0QYO+pSA epTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:user-agent:references:message-id :in-reply-to:subject:cc:to:from:date; bh=dP7IJu7EVYI3ipczc08Dg8dBO1T7j1G381JN6RJ5mVY=; b=M0QV9sf+jDV2j05zDkFy4GjcYyBdeB/GYMAdAK9YmsFxMsGOKgRT3IcTGh7vFpyBnB j32YrujRUdb14mPx3StohN5ztAhDKkUxNojNSedPQ1Geb9TpagrBYy9SEZSKlJxrJEc8 TxXz/o0k983gbL4Jw7+xuVsgYaHCqV6HCSm0Z0CZMtc8uVKVYIeKXoyHbzDg4z8zA218 v0OgruczjhIrjpc+8IRe+jAcO2KrEwItCRumb/OX9UDv7ZOtYpdigEN1umuP+clq3ay6 hj/8UR140qg9nEIlzcFyac2bnQAqti1N/Q+TzVTAAcUzINZMXLznGiCLjHIxodnUouxr Xcyg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j4-20020aa7ca44000000b005149dd97237si4932671edt.219.2023.06.11.10.48.55; Sun, 11 Jun 2023 10:49:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229604AbjFKRUW (ORCPT + 99 others); Sun, 11 Jun 2023 13:20:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230515AbjFKRUU (ORCPT ); Sun, 11 Jun 2023 13:20:20 -0400 Received: from angie.orcam.me.uk (angie.orcam.me.uk [IPv6:2001:4190:8020::34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 210661BDF; Sun, 11 Jun 2023 10:19:57 -0700 (PDT) Received: by angie.orcam.me.uk (Postfix, from userid 500) id 060279200C4; Sun, 11 Jun 2023 19:19:36 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id F3F579200C0; Sun, 11 Jun 2023 18:19:36 +0100 (BST) Date: Sun, 11 Jun 2023 18:19:36 +0100 (BST) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?utf-8?q?Pali_Roh?= =?utf-8?q?=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v9 06/14] net/mlx5: Rely on `link_active_reporting' In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768429463129707314?= X-GMAIL-MSGID: =?utf-8?q?1768429463129707314?= Use `link_active_reporting' to determine whether Data Link Layer Link Active Reporting is available rather than re-retrieving the capability. Signed-off-by: Maciej W. Rozycki --- NB this has been compile-tested only with PPC64LE and x86-64 configurations. No change from v8. Changes from v7: - Reorder from 5/7. Changes from v6: - Regenerate against 6.3-rc5. New change in v6. --- drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) linux-pcie-link-active-reporting-mlx5.diff Index: linux-macro/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c =================================================================== --- linux-macro.orig/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c +++ linux-macro/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c @@ -307,7 +307,6 @@ static int mlx5_pci_link_toggle(struct m unsigned long timeout; struct pci_dev *sdev; int cap, err; - u32 reg32; /* Check that all functions under the pci bridge are PFs of * this device otherwise fail this function. @@ -346,11 +345,8 @@ static int mlx5_pci_link_toggle(struct m return err; /* Check link */ - err = pci_read_config_dword(bridge, cap + PCI_EXP_LNKCAP, ®32); - if (err) - return err; - if (!(reg32 & PCI_EXP_LNKCAP_DLLLARC)) { - mlx5_core_warn(dev, "No PCI link reporting capability (0x%08x)\n", reg32); + if (!bridge->link_active_reporting) { + mlx5_core_warn(dev, "No PCI link reporting capability\n"); msleep(1000); goto restore; } From patchwork Sun Jun 11 17:19:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Maciej W. Rozycki" X-Patchwork-Id: 106137 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2149500vqr; Sun, 11 Jun 2023 10:44:00 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6XcdV7FxkV1JDEt5XP7IRJ5onoyUq8StYSSw3rSRKIAoykV25Z3vOnP75N5X1+roxPjwAs X-Received: by 2002:a50:ed89:0:b0:514:93f0:e15a with SMTP id h9-20020a50ed89000000b0051493f0e15amr3381736edr.16.1686505440280; Sun, 11 Jun 2023 10:44:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686505440; cv=none; d=google.com; s=arc-20160816; b=zsGcGWAvhn0yIm8zUa3KjDi+OpG9haqKv/wvbTCxmtQe9m5MUpDwFfEEiIMDSa7/1z 8qXEOMuEf6QvGHMvgofDqTSZTrhVfxP6DwGBYsg1es43XI2x6NEKo3R1iKCGu4sHdCJ9 kULU7SorC3CV0je/eL5oYjDZDhbf1b8meqsenLkWALanTZvBikKJkNjKeAEtcRkiPs7S txGsUNBGv3QFvXqC+bgn9r05FfAXs/9GvWHH+5p+R/Q0pImSiLHdcvJw3GBSUYbQFL1s TAx55M1KzGPMOnwvKhh0ldVfIi313zGAkQc+KuppLMQFTRdFIRgftF97FgFxQTZ4ZwQR SSvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:user-agent:references:message-id :in-reply-to:subject:cc:to:from:date; bh=ZHhVgI9K0KfDQ7JZtWt/40aDE7Q15DxvNlmxyl5Gcu4=; b=yamBlLhPt/A3dnXo0qXgLvv9djEmX2JDj0IxJ7zcdUi64TFY6et44N/f4imNkfO0t7 xDxCuFby6S1q4x9sgUfp6NwKi9M+L9cUSqaA7HQQcKSxICdDLFPhIxnf78//0h4Djq/e fUNu6PaqUl1lOEjHOwWZiFz69yzlEUY5ngE6b+uc1Xfz5f4/mgWdQ6neGXBapeMz8ehD yrOgA534iGNensZ6c2R1qmdozQsuMxx0KXv9fEl42v76fygvtEQUH8R54yak9HleO5ez 2Jv82ZvU3FZfsXEMg0UUYA9fvF5rOfw0jkS1k6Suv5hAa4w05hQSdra31qE6wajN0wti FGAA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j4-20020aa7ca44000000b005149dd97237si4932671edt.219.2023.06.11.10.43.30; Sun, 11 Jun 2023 10:44:00 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232418AbjFKRUf (ORCPT + 99 others); Sun, 11 Jun 2023 13:20:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44862 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233505AbjFKRU3 (ORCPT ); Sun, 11 Jun 2023 13:20:29 -0400 Received: from angie.orcam.me.uk (angie.orcam.me.uk [IPv6:2001:4190:8020::34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 9E47E10C1; Sun, 11 Jun 2023 10:20:06 -0700 (PDT) Received: by angie.orcam.me.uk (Postfix, from userid 500) id 267D59200BC; Sun, 11 Jun 2023 19:19:41 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id 21FBD9200BB; Sun, 11 Jun 2023 18:19:41 +0100 (BST) Date: Sun, 11 Jun 2023 18:19:41 +0100 (BST) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?utf-8?q?Pali_Roh?= =?utf-8?q?=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v9 07/14] PCI: Export `pcie_retrain_link' for use outside ASPM In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768429129008752288?= X-GMAIL-MSGID: =?utf-8?q?1768429129008752288?= Export `pcie_retrain_link' for link retrain needs outside ASPM. There is no functional change at this point, but `struct pcie_link_state' is local to ASPM and not used within `pcie_retrain_link' other than to get at the associated PCI device, so change the operand and adjust the lone call site accordingly. Document the interface. Signed-off-by: Maciej W. Rozycki --- There's a missing full stop added in a comment in the course of the move, not worth mentioning in the change description IMHO and not worth its own change either. This comment will go away in a subsequent change anyway. New change in v9. --- drivers/pci/pci.c | 36 ++++++++++++++++++++++++++++++++++++ drivers/pci/pci.h | 1 + drivers/pci/pcie/aspm.c | 32 +------------------------------- 3 files changed, 38 insertions(+), 31 deletions(-) linux-pcie-retrain-link-export.diff Index: linux-macro/drivers/pci/pci.c =================================================================== --- linux-macro.orig/drivers/pci/pci.c +++ linux-macro/drivers/pci/pci.c @@ -4912,6 +4912,42 @@ bool pcie_wait_for_link(struct pci_dev * return pcie_wait_for_link_delay(pdev, active, 100); } +/** + * pcie_retrain_link - Request a link retrain and wait for it to complete + * @pdev: Device whose link to retrain. + * + * Return TRUE if successful, or FALSE if training has not completed + * within PCIE_LINK_RETRAIN_TIMEOUT_MS milliseconds. + */ +bool pcie_retrain_link(struct pci_dev *pdev) +{ + unsigned long end_jiffies; + u16 reg16; + + pcie_capability_read_word(pdev, PCI_EXP_LNKCTL, ®16); + reg16 |= PCI_EXP_LNKCTL_RL; + pcie_capability_write_word(pdev, PCI_EXP_LNKCTL, reg16); + if (pdev->clear_retrain_link) { + /* + * Due to an erratum in some devices the Retrain Link bit + * needs to be cleared again manually to allow the link + * training to succeed. + */ + reg16 &= ~PCI_EXP_LNKCTL_RL; + pcie_capability_write_word(pdev, PCI_EXP_LNKCTL, reg16); + } + + /* Wait for link training end. Break out after waiting for timeout. */ + end_jiffies = jiffies + msecs_to_jiffies(PCIE_LINK_RETRAIN_TIMEOUT_MS); + do { + pcie_capability_read_word(pdev, PCI_EXP_LNKSTA, ®16); + if (!(reg16 & PCI_EXP_LNKSTA_LT)) + break; + msleep(1); + } while (time_before(jiffies, end_jiffies)); + return !(reg16 & PCI_EXP_LNKSTA_LT); +} + /* * Find maximum D3cold delay required by all the devices on the bus. The * spec says 100 ms, but firmware can lower it and we allow drivers to Index: linux-macro/drivers/pci/pci.h =================================================================== --- linux-macro.orig/drivers/pci/pci.h +++ linux-macro/drivers/pci/pci.h @@ -561,6 +561,7 @@ pci_ers_result_t pcie_do_recovery(struct pci_ers_result_t (*reset_subordinates)(struct pci_dev *pdev)); bool pcie_wait_for_link(struct pci_dev *pdev, bool active); +bool pcie_retrain_link(struct pci_dev *pdev); #ifdef CONFIG_PCIEASPM void pcie_aspm_init_link_state(struct pci_dev *pdev); void pcie_aspm_exit_link_state(struct pci_dev *pdev); Index: linux-macro/drivers/pci/pcie/aspm.c =================================================================== --- linux-macro.orig/drivers/pci/pcie/aspm.c +++ linux-macro/drivers/pci/pcie/aspm.c @@ -191,36 +191,6 @@ static void pcie_clkpm_cap_init(struct p link->clkpm_disable = blacklist ? 1 : 0; } -static bool pcie_retrain_link(struct pcie_link_state *link) -{ - struct pci_dev *parent = link->pdev; - unsigned long end_jiffies; - u16 reg16; - - pcie_capability_read_word(parent, PCI_EXP_LNKCTL, ®16); - reg16 |= PCI_EXP_LNKCTL_RL; - pcie_capability_write_word(parent, PCI_EXP_LNKCTL, reg16); - if (parent->clear_retrain_link) { - /* - * Due to an erratum in some devices the Retrain Link bit - * needs to be cleared again manually to allow the link - * training to succeed. - */ - reg16 &= ~PCI_EXP_LNKCTL_RL; - pcie_capability_write_word(parent, PCI_EXP_LNKCTL, reg16); - } - - /* Wait for link training end. Break out after waiting for timeout */ - end_jiffies = jiffies + msecs_to_jiffies(PCIE_LINK_RETRAIN_TIMEOUT_MS); - do { - pcie_capability_read_word(parent, PCI_EXP_LNKSTA, ®16); - if (!(reg16 & PCI_EXP_LNKSTA_LT)) - break; - msleep(1); - } while (time_before(jiffies, end_jiffies)); - return !(reg16 & PCI_EXP_LNKSTA_LT); -} - /* * pcie_aspm_configure_common_clock: check if the 2 ends of a link * could use common clock. If they are, configure them to use the @@ -287,7 +257,7 @@ static void pcie_aspm_configure_common_c reg16 &= ~PCI_EXP_LNKCTL_CCC; pcie_capability_write_word(parent, PCI_EXP_LNKCTL, reg16); - if (pcie_retrain_link(link)) + if (pcie_retrain_link(link->pdev)) return; /* Training failed. Restore common clock configurations */ From patchwork Sun Jun 11 17:19:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Maciej W. Rozycki" X-Patchwork-Id: 106144 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2151428vqr; Sun, 11 Jun 2023 10:49:46 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7P7tGBRqCVx5shPCFcyb0Av1owMLp8AnTLksEs5Ov79JRmKY8K4mGmfh7vf4wrbSMPAOq6 X-Received: by 2002:a17:907:6e8c:b0:973:daaa:f3e1 with SMTP id sh12-20020a1709076e8c00b00973daaaf3e1mr8845385ejc.39.1686505786037; Sun, 11 Jun 2023 10:49:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686505786; cv=none; d=google.com; s=arc-20160816; b=W4FXXnncmpM5/n7pZinxQYezWcatNGvjhEgUxz4PWYwoZT/GkoXWS6sHf3s+lIN6Nu QfuwHIp4n6zkfdY5AWsrBxOA8q8RDiwqZvQ6zgofkLwAn0xuypKQhZOfTqdIbzLTD8x0 gMcMijrxNFK2xfnIiyhzEbc0jwwVIUGOiSFPio8ST3RJPsP4YZT7p7GLtr9WeLNIUXF9 qoCyl6NDsdtr4zS5JsQOtuHGbkyQUlsfhEcMHoqjiI78cJ+gSpbOr2s14X3vnSCyWvY6 YNSWEMSOkYGO/UxbPJAC507uCsB2Mq9JvUoGJDEA0yWrHp9ybmS8B1RM1ykD1L+Ua+x3 o2aA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:user-agent:references:message-id :in-reply-to:subject:cc:to:from:date; bh=OFECUJ1shgLQhTBUDB/EGo49nH7RntEO2gKbly8vnE4=; b=UVlLwIiX6F80JmBduGM8TbvyJA2k3+Da+L1OuztMeV8ceFn7a5hjHXP05JlrDvfAqP JA3rEc2CvoCj80mdEZxDfnjby8fOXrwQEeQYUXnPwer6XsfpFeKQRDNT0jdWtx8ZwW8t oSritZbRKwxUQw40paRenAcnmbWhor958ClJUApD3/oxOvQFOUVx6CuS2MzRafr/A2pQ 8cTHpcRkk1URe6Or4AbmY/UjOUUmG6/9Yt4fooqcwc2NDDrNfPMGoHuPWUcUbe/EtRkt vJssyZTiMzHKQLj3/ePY/wGH2s9zZR36Jy4TSiKVjnEaG87LPl0ERxpiAz7C1poGo5fX xykg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s1-20020a056402164100b005149aee5218si4854537edx.51.2023.06.11.10.49.22; Sun, 11 Jun 2023 10:49:46 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233664AbjFKRUa (ORCPT + 99 others); Sun, 11 Jun 2023 13:20:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233466AbjFKRU2 (ORCPT ); Sun, 11 Jun 2023 13:20:28 -0400 Received: from angie.orcam.me.uk (angie.orcam.me.uk [IPv6:2001:4190:8020::34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id EF8F810C4; Sun, 11 Jun 2023 10:20:04 -0700 (PDT) Received: by angie.orcam.me.uk (Postfix, from userid 500) id 53DBD9200C5; Sun, 11 Jun 2023 19:19:45 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id 4D8169200C1; Sun, 11 Jun 2023 18:19:45 +0100 (BST) Date: Sun, 11 Jun 2023 18:19:45 +0100 (BST) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?utf-8?q?Pali_Roh?= =?utf-8?q?=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v9 08/14] PCI: Use distinct local vars in `pcie_retrain_link' In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768429491061556010?= X-GMAIL-MSGID: =?utf-8?q?1768429491061556010?= Use separate local variables to hold the respective values retrieved from the Link Control Register and the Link Status Register. Not only it improves readability, but it makes it possible for the compiler to detect actual uninitialised use should this code change in the future. Signed-off-by: Maciej W. Rozycki --- New change in v9. --- drivers/pci/pci.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) linux-pcie-retrain-link-lnkctl-lnksta.diff Index: linux-macro/drivers/pci/pci.c =================================================================== --- linux-macro.orig/drivers/pci/pci.c +++ linux-macro/drivers/pci/pci.c @@ -4922,30 +4922,31 @@ bool pcie_wait_for_link(struct pci_dev * bool pcie_retrain_link(struct pci_dev *pdev) { unsigned long end_jiffies; - u16 reg16; + u16 lnkctl; + u16 lnksta; - pcie_capability_read_word(pdev, PCI_EXP_LNKCTL, ®16); - reg16 |= PCI_EXP_LNKCTL_RL; - pcie_capability_write_word(pdev, PCI_EXP_LNKCTL, reg16); + pcie_capability_read_word(pdev, PCI_EXP_LNKCTL, &lnkctl); + lnkctl |= PCI_EXP_LNKCTL_RL; + pcie_capability_write_word(pdev, PCI_EXP_LNKCTL, lnkctl); if (pdev->clear_retrain_link) { /* * Due to an erratum in some devices the Retrain Link bit * needs to be cleared again manually to allow the link * training to succeed. */ - reg16 &= ~PCI_EXP_LNKCTL_RL; - pcie_capability_write_word(pdev, PCI_EXP_LNKCTL, reg16); + lnkctl &= ~PCI_EXP_LNKCTL_RL; + pcie_capability_write_word(pdev, PCI_EXP_LNKCTL, lnkctl); } /* Wait for link training end. Break out after waiting for timeout. */ end_jiffies = jiffies + msecs_to_jiffies(PCIE_LINK_RETRAIN_TIMEOUT_MS); do { - pcie_capability_read_word(pdev, PCI_EXP_LNKSTA, ®16); - if (!(reg16 & PCI_EXP_LNKSTA_LT)) + pcie_capability_read_word(pdev, PCI_EXP_LNKSTA, &lnksta); + if (!(lnksta & PCI_EXP_LNKSTA_LT)) break; msleep(1); } while (time_before(jiffies, end_jiffies)); - return !(reg16 & PCI_EXP_LNKSTA_LT); + return !(lnksta & PCI_EXP_LNKSTA_LT); } /* From patchwork Sun Jun 11 17:19:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Maciej W. Rozycki" X-Patchwork-Id: 106134 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2142812vqr; Sun, 11 Jun 2023 10:24:13 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4tCGEMxAsG0lI8UkzxudHNIvWuYVpLmADdyei9ohohJfmfQnp8PGGORYW5KTlYVSgFivNY X-Received: by 2002:a05:6402:74c:b0:514:a5f3:be61 with SMTP id p12-20020a056402074c00b00514a5f3be61mr3697278edy.31.1686504253533; Sun, 11 Jun 2023 10:24:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686504253; cv=none; d=google.com; s=arc-20160816; b=gJiCionOwcSLhNwO5QAFBNsZEK9Z0E8DbmI2Vdc3hpoPR3iIjkA02pwGozFY9btWdC wsFbPFC2n359GhygqEze4a2xE8FjxRKlCkUJiVjQKWSseWCBEdvtRcmMx0jQaWW2J1yz 7njD66Fi6x8XNT1rtdfEAqhaNFkF8eXW8aaYSl5naspU26Ifj6/KnRKbSUUPlNsD7kBj skaX8vEWIurWPxELXHwawvt20NUhiwG/BTSnVeCvOZxRinP5JYkqp9r4SoShASCJmQha IA58PrD/9K2WFWu+mmAn336iUKPj3iJNw0okbUbzM9bNbVb9kEUxCkG5D9GAQ3q0V3nB WgZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:user-agent:references:message-id :in-reply-to:subject:cc:to:from:date; bh=QhepCUpNLnlqRgEnGy56jFx9bcSEiD1T348n+Bh6hqw=; b=AIVYRtiamhrzNu5ckH9Fz0GsHeFBXbk9yV2R1VNAocoCYJn7JNh6+TV81omWAMoM76 A2a3WNbJINJLRLN0SjyqrGen8bK3iuOAZhj98sfsk9IuEC/i3927W+RspAs0SlZdNcjK q5NM0fY92lhUIE3JEIT66QWwnY87t7O74MJQDfuN/Epmj9p3Fx3eQPMwh8buaWoIbQA+ zvWCV5ptb1ElywJ72zWJnoOtZVdv4iTh685WfcSd0+meolwMgSPbaL9dEprOtXXzAh5B l6HPfOD0BZ7Bs5EL8Q34XwcWWXfovPTog6Wcq5AQ7vjBJe8pRqIOdhV5awAHE2F8JtFT fFgw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id by1-20020a0564021b0100b0051632dc69b3si4798811edb.263.2023.06.11.10.23.49; Sun, 11 Jun 2023 10:24:13 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233932AbjFKRU6 (ORCPT + 99 others); Sun, 11 Jun 2023 13:20:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45278 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230233AbjFKRUz (ORCPT ); Sun, 11 Jun 2023 13:20:55 -0400 Received: from angie.orcam.me.uk (angie.orcam.me.uk [IPv6:2001:4190:8020::34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 7BCA110D0; Sun, 11 Jun 2023 10:20:29 -0700 (PDT) Received: by angie.orcam.me.uk (Postfix, from userid 500) id 90AD99200C1; Sun, 11 Jun 2023 19:19:49 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id 8B5119200B3; Sun, 11 Jun 2023 18:19:49 +0100 (BST) Date: Sun, 11 Jun 2023 18:19:49 +0100 (BST) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?utf-8?q?Pali_Roh?= =?utf-8?q?=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v9 09/14] PCI: Factor our waiting for link training end In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768427884349064214?= X-GMAIL-MSGID: =?utf-8?q?1768427884349064214?= Move code polling for the Link Training bit to clear into a function of its own. Signed-off-by: Maciej W. Rozycki --- New change in v9. --- drivers/pci/pci.c | 37 ++++++++++++++++++++++++------------- 1 file changed, 24 insertions(+), 13 deletions(-) linux-pcie-wait-for-link-status.diff Index: linux-macro/drivers/pci/pci.c =================================================================== --- linux-macro.orig/drivers/pci/pci.c +++ linux-macro/drivers/pci/pci.c @@ -4850,6 +4850,28 @@ static int pci_pm_reset(struct pci_dev * } /** + * pcie_wait_for_link_status - Wait for link training end + * @pdev: Device whose link to wait for. + * + * Return TRUE if successful, or FALSE if training has not completed + * within PCIE_LINK_RETRAIN_TIMEOUT_MS milliseconds. + */ +static bool pcie_wait_for_link_status(struct pci_dev *pdev) +{ + unsigned long end_jiffies; + u16 lnksta; + + end_jiffies = jiffies + msecs_to_jiffies(PCIE_LINK_RETRAIN_TIMEOUT_MS); + do { + pcie_capability_read_word(pdev, PCI_EXP_LNKSTA, &lnksta); + if (!(lnksta & PCI_EXP_LNKSTA_LT)) + break; + msleep(1); + } while (time_before(jiffies, end_jiffies)); + return !(lnksta & PCI_EXP_LNKSTA_LT); +} + +/** * pcie_wait_for_link_delay - Wait until link is active or inactive * @pdev: Bridge device * @active: waiting for active or inactive? @@ -4916,14 +4938,11 @@ bool pcie_wait_for_link(struct pci_dev * * pcie_retrain_link - Request a link retrain and wait for it to complete * @pdev: Device whose link to retrain. * - * Return TRUE if successful, or FALSE if training has not completed - * within PCIE_LINK_RETRAIN_TIMEOUT_MS milliseconds. + * Return TRUE if successful, or FALSE if training has not completed. */ bool pcie_retrain_link(struct pci_dev *pdev) { - unsigned long end_jiffies; u16 lnkctl; - u16 lnksta; pcie_capability_read_word(pdev, PCI_EXP_LNKCTL, &lnkctl); lnkctl |= PCI_EXP_LNKCTL_RL; @@ -4938,15 +4957,7 @@ bool pcie_retrain_link(struct pci_dev *p pcie_capability_write_word(pdev, PCI_EXP_LNKCTL, lnkctl); } - /* Wait for link training end. Break out after waiting for timeout. */ - end_jiffies = jiffies + msecs_to_jiffies(PCIE_LINK_RETRAIN_TIMEOUT_MS); - do { - pcie_capability_read_word(pdev, PCI_EXP_LNKSTA, &lnksta); - if (!(lnksta & PCI_EXP_LNKSTA_LT)) - break; - msleep(1); - } while (time_before(jiffies, end_jiffies)); - return !(lnksta & PCI_EXP_LNKSTA_LT); + return pcie_wait_for_link_status(pdev); } /* From patchwork Sun Jun 11 17:19:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Maciej W. Rozycki" X-Patchwork-Id: 106135 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2143011vqr; Sun, 11 Jun 2023 10:24:47 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4gzVr/Jg50o7vIx++4gPvoL407OXoJ8NVpNMuXQvJ30YaZmvBKKM4KTF3t/PGdiP9/2zo2 X-Received: by 2002:a17:907:2d22:b0:978:b94e:83dd with SMTP id gs34-20020a1709072d2200b00978b94e83ddmr7943050ejc.75.1686504286911; Sun, 11 Jun 2023 10:24:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686504286; cv=none; d=google.com; s=arc-20160816; b=cj4pnlb9p63Z9VB7QhrIo9Mdp3gqJSUfK5Hm1jFm+XLmbyR3Lr+EL0MaPlz9bkhXq7 vqWMekBCFeTijWXbFSmAwWY6CwqiN9JlKxDG+YTn894/w4JU401rqZRerZJUkRWFs2NG kIp9Tdz0Dh5yH/5QaiqDsxco2+tA3Z76nQtkJl37Bbk8vxzn+ixbiH7dOC39OFr+XH3H PYUwDxCFgzOZZ6MpWULxxsRMUuDX5myHzDj+AIqiQgHz7tQ0i7OFL/1Ns6/PkqLWorwS AgMX8+D2uB03JWyjPfydfbKLOtcbhJGKYnL9lDEywvWCShMUMom//S1fIHGh+JKLamzS aOow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:user-agent:references:message-id :in-reply-to:subject:cc:to:from:date; bh=wxFbgfcf6cePRRo7BQ18Bwqcp+ZIA/mUf2+kMM7tUaM=; b=c+8L1+yFSUenAOy8+j6pjxLuOjhn2pA+Tnaz+m5AKSjnUUiIaJK6wKtUSAuXFgIHaM o1kb2ih7Cj8g3HGPiBiq1ftaPPI1qVZjSeUu3x99CigxvnPpNNMAYUUEyojHs6tWIRzw 9AxrtQdxSdjao4MisY10J2eUxchxWinBkMlsCRgSl/E+vcrH9vBX9F0p7tznR43GTIPp s+wuB6mbs6XB4iWq+ztsPIz2DquWoDvKzF2zW1xqC4VUkjVCo5c7tP+432JLP+89eUD/ M2YhPAs2jaJoB9T4Ya7FGWEjepa8R1z1XmSzLO8GnL/BRSh8w8mIfsj+cSJULi/MXDfs 4qCA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ju25-20020a17090798b900b00977ca868696si4470157ejc.243.2023.06.11.10.24.22; Sun, 11 Jun 2023 10:24:46 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234015AbjFKRVR (ORCPT + 99 others); Sun, 11 Jun 2023 13:21:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45510 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233888AbjFKRVN (ORCPT ); Sun, 11 Jun 2023 13:21:13 -0400 X-Greylist: delayed 92 seconds by postgrey-1.37 at lindbergh.monkeyblade.net; Sun, 11 Jun 2023 10:20:49 PDT Received: from angie.orcam.me.uk (angie.orcam.me.uk [78.133.224.34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id E45F71700; Sun, 11 Jun 2023 10:20:48 -0700 (PDT) Received: by angie.orcam.me.uk (Postfix, from userid 500) id 4E8B29200C6; Sun, 11 Jun 2023 19:19:53 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id 4858E9200B3; Sun, 11 Jun 2023 18:19:53 +0100 (BST) Date: Sun, 11 Jun 2023 18:19:53 +0100 (BST) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?utf-8?q?Pali_Roh?= =?utf-8?q?=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v9 10/14] PCI: Add support for polling DLLLA to `pcie_retrain_link' In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768427919278140709?= X-GMAIL-MSGID: =?utf-8?q?1768427919278140709?= Let the caller of `pcie_retrain_link' specify whether they want to use the LT bit or the DLLLA bit of the Link Status Register to determine if link training has completed. It is up to the caller to verify whether the use of the DLLLA bit, the implementation of which is optional, is valid for the device requested. Signed-off-by: Maciej W. Rozycki --- New change in v9. --- drivers/pci/pci.c | 28 ++++++++++++++++++++-------- drivers/pci/pci.h | 2 +- drivers/pci/pcie/aspm.c | 2 +- 3 files changed, 22 insertions(+), 10 deletions(-) linux-pcie-retrain-link-use-lt.diff Index: linux-macro/drivers/pci/pci.c =================================================================== --- linux-macro.orig/drivers/pci/pci.c +++ linux-macro/drivers/pci/pci.c @@ -4850,25 +4850,32 @@ static int pci_pm_reset(struct pci_dev * } /** - * pcie_wait_for_link_status - Wait for link training end + * pcie_wait_for_link_status - Wait for link status change * @pdev: Device whose link to wait for. + * @use_lt: Use the LT bit if TRUE, or the DLLLA bit if FALSE. + * @active: Waiting for active or inactive? * - * Return TRUE if successful, or FALSE if training has not completed - * within PCIE_LINK_RETRAIN_TIMEOUT_MS milliseconds. + * Return TRUE if successful, or FALSE if status has not changed within + * PCIE_LINK_RETRAIN_TIMEOUT_MS milliseconds. */ -static bool pcie_wait_for_link_status(struct pci_dev *pdev) +static bool pcie_wait_for_link_status(struct pci_dev *pdev, + bool use_lt, bool active) { + u16 lnksta_mask, lnksta_match; unsigned long end_jiffies; u16 lnksta; + lnksta_mask = use_lt ? PCI_EXP_LNKSTA_LT : PCI_EXP_LNKSTA_DLLLA; + lnksta_match = active ? lnksta_mask : 0; + end_jiffies = jiffies + msecs_to_jiffies(PCIE_LINK_RETRAIN_TIMEOUT_MS); do { pcie_capability_read_word(pdev, PCI_EXP_LNKSTA, &lnksta); - if (!(lnksta & PCI_EXP_LNKSTA_LT)) + if ((lnksta & lnksta_mask) == lnksta_match) break; msleep(1); } while (time_before(jiffies, end_jiffies)); - return !(lnksta & PCI_EXP_LNKSTA_LT); + return (lnksta & lnksta_mask) == lnksta_match; } /** @@ -4937,10 +4944,15 @@ bool pcie_wait_for_link(struct pci_dev * /** * pcie_retrain_link - Request a link retrain and wait for it to complete * @pdev: Device whose link to retrain. + * @use_lt: Use the LT bit if TRUE, or the DLLLA bit if FALSE, for status. + * + * Retrain completion status is retrieved from the Link Status Register + * according to @use_lt. It is not verified whether the use of the DLLLA + * bit is valid. * * Return TRUE if successful, or FALSE if training has not completed. */ -bool pcie_retrain_link(struct pci_dev *pdev) +bool pcie_retrain_link(struct pci_dev *pdev, bool use_lt) { u16 lnkctl; @@ -4957,7 +4969,7 @@ bool pcie_retrain_link(struct pci_dev *p pcie_capability_write_word(pdev, PCI_EXP_LNKCTL, lnkctl); } - return pcie_wait_for_link_status(pdev); + return pcie_wait_for_link_status(pdev, use_lt, !use_lt); } /* Index: linux-macro/drivers/pci/pci.h =================================================================== --- linux-macro.orig/drivers/pci/pci.h +++ linux-macro/drivers/pci/pci.h @@ -561,7 +561,7 @@ pci_ers_result_t pcie_do_recovery(struct pci_ers_result_t (*reset_subordinates)(struct pci_dev *pdev)); bool pcie_wait_for_link(struct pci_dev *pdev, bool active); -bool pcie_retrain_link(struct pci_dev *pdev); +bool pcie_retrain_link(struct pci_dev *pdev, bool use_lt); #ifdef CONFIG_PCIEASPM void pcie_aspm_init_link_state(struct pci_dev *pdev); void pcie_aspm_exit_link_state(struct pci_dev *pdev); Index: linux-macro/drivers/pci/pcie/aspm.c =================================================================== --- linux-macro.orig/drivers/pci/pcie/aspm.c +++ linux-macro/drivers/pci/pcie/aspm.c @@ -257,7 +257,7 @@ static void pcie_aspm_configure_common_c reg16 &= ~PCI_EXP_LNKCTL_CCC; pcie_capability_write_word(parent, PCI_EXP_LNKCTL, reg16); - if (pcie_retrain_link(link->pdev)) + if (pcie_retrain_link(link->pdev, true)) return; /* Training failed. Restore common clock configurations */ From patchwork Sun Jun 11 17:19:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Maciej W. Rozycki" X-Patchwork-Id: 106139 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2150213vqr; Sun, 11 Jun 2023 10:46:03 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4kjAit2hu8fBg5/unN73uAaGkToQ3Rw4jhsIoRIivMbcMFij56I6vW7bXn/OGPuaRIqqnz X-Received: by 2002:a17:907:3e90:b0:94a:7b2c:205e with SMTP id hs16-20020a1709073e9000b0094a7b2c205emr8307366ejc.72.1686505562834; Sun, 11 Jun 2023 10:46:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686505562; cv=none; d=google.com; s=arc-20160816; b=WlLD4cpkcLlKRSXrTnrnT17xNvQsjLoDA5Mwk8WOazRSDskSF9nG6huZFnUma9FwJw MxRBLoi6LS9JOsAeXC78JAdH7PklOBIOhpDtiFYzdJzMrjurpTJvcFs9DmN0jClm/THw AdkIxxUTGslVNARdLf1btuvJejsDOlHZDs/g7D4Boxexq4lSxkCBiNQ2lE3ihvDyYYhN T9rbs7wqPzKrtaeTp9OUv5ZXg6v6b9KEnuCpjP/1qYIVeAt0qneZ6Jo/+U4QKrQjXfv+ sQmtF5aeh4XAngZHee9p9nOdvtgr1P3Nqyl7EyH5iKkbcVRH/VSBcgOwyRQL4F+OKiGB CGJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:user-agent:references:message-id :in-reply-to:subject:cc:to:from:date; bh=y+2Iv8iqD/YoX2tQg597P+BNOZ5EFD2yj/px1/Lej4c=; b=hyKAHzVNIm4AHztqBenDyy49j6jmner1DxjUfcbHf7DIpXfN7CMul3AtwHMAsUzhjG udQqVyzwKyPKCva8AJEIiEbfbmOGsNvyusqAcT3Wnz88taEY3mLANUwv8KERp4TQOa8k emEJegjEcimODbhtq0hh47cTwG/JmGz9muvsukNOmKqXA4czPBm1dOHwuwlr862Rb3rm UA002nk646+3FFLKI+UKm0+736rujrwj2UuFFgp+CMA9Xbd5HCuVC98OKzxg1tybpuys Q0aATsaXUXSq1zWFXrTg58AG+IdTuRsRI9a8qVZQbzG+gEg8AvYxN5ukzZL7JVy5Ne3b fk/g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p26-20020aa7cc9a000000b005148f8c24d5si4692372edt.382.2023.06.11.10.45.38; Sun, 11 Jun 2023 10:46:02 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232433AbjFKRUu (ORCPT + 99 others); Sun, 11 Jun 2023 13:20:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45078 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230233AbjFKRUs (ORCPT ); Sun, 11 Jun 2023 13:20:48 -0400 Received: from angie.orcam.me.uk (angie.orcam.me.uk [IPv6:2001:4190:8020::34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 9B5ED199E; Sun, 11 Jun 2023 10:20:21 -0700 (PDT) Received: by angie.orcam.me.uk (Postfix, from userid 500) id E73559200C7; Sun, 11 Jun 2023 19:19:57 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id E0F309200C0; Sun, 11 Jun 2023 18:19:57 +0100 (BST) Date: Sun, 11 Jun 2023 18:19:57 +0100 (BST) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?utf-8?q?Pali_Roh?= =?utf-8?q?=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v9 11/14] PCI: Use `pcie_wait_for_link_status' in `pcie_wait_for_link_delay' In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768429257045105658?= X-GMAIL-MSGID: =?utf-8?q?1768429257045105658?= Remove a DLLLA status bit polling loop from `pcie_wait_for_link_delay' and call almost identical code in `pcie_wait_for_link_status' instead. This reduces the lower bound on the polling interval from 10ms to 1ms, possibly increasing the CPU load on the system in favour to reducing the wait time. Signed-off-by: Maciej W. Rozycki --- New change in v9. --- drivers/pci/pci.c | 17 +++-------------- 1 file changed, 3 insertions(+), 14 deletions(-) linux-pcie-wait-for-link-delay-status.diff Index: linux-macro/drivers/pci/pci.c =================================================================== --- linux-macro.orig/drivers/pci/pci.c +++ linux-macro/drivers/pci/pci.c @@ -4889,16 +4889,14 @@ static bool pcie_wait_for_link_status(st static bool pcie_wait_for_link_delay(struct pci_dev *pdev, bool active, int delay) { - int timeout = PCIE_LINK_RETRAIN_TIMEOUT_MS; bool ret; - u16 lnk_status; /* * Some controllers might not implement link active reporting. In this * case, we wait for 1000 ms + any delay requested by the caller. */ if (!pdev->link_active_reporting) { - msleep(timeout + delay); + msleep(PCIE_LINK_RETRAIN_TIMEOUT_MS + delay); return true; } @@ -4913,20 +4911,11 @@ static bool pcie_wait_for_link_delay(str */ if (active) msleep(20); - for (;;) { - pcie_capability_read_word(pdev, PCI_EXP_LNKSTA, &lnk_status); - ret = !!(lnk_status & PCI_EXP_LNKSTA_DLLLA); - if (ret == active) - break; - if (timeout <= 0) - break; - msleep(10); - timeout -= 10; - } + ret = pcie_wait_for_link_status(pdev, false, active); if (active && ret) msleep(delay); - return ret == active; + return ret; } /** From patchwork Sun Jun 11 17:20:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Maciej W. Rozycki" X-Patchwork-Id: 106140 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2150563vqr; Sun, 11 Jun 2023 10:47:02 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4RZh6x3gbzXLAPbCEIAmlHPcpFfs/Quo4saAJKJU4wxalNljpdingvbSRQzXWteEggYoCW X-Received: by 2002:a17:907:9285:b0:96f:678:d2e2 with SMTP id bw5-20020a170907928500b0096f0678d2e2mr6625384ejc.11.1686505622020; Sun, 11 Jun 2023 10:47:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686505622; cv=none; d=google.com; s=arc-20160816; b=r8r0zn1jDOJvg8RkeJwIuUzC+9iYJO2vUyPlXYbm9XfPD76BfVeCK3PG0dsKhDxCrO PT4xQhv/GD09zzf7GCHY1QAJQgvAb4x1rD5fGAF6omo2Qaobg1ZHo3sOMsgav+C5D9id xrhdPxLlPqv/t23npvVTsmGeip4bRdk8EnIyIE8QdXOPkEhcy5KkZt6sXnM8Xw7brpco bV0/gxTDssuMYBPrhjiHrOVO5ixdN2j6CBAORMADt6QyVLaJFrAqIv5XwXsy7kaa57Se yQRXFAxO+56KGDToegF89mjf98pu1nOYLGrarwwInSYbrjAf6L/tCiqf4RSfXGHoZrbh Wf6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:user-agent:references:message-id :in-reply-to:subject:cc:to:from:date; bh=TtoebyDSQynbZwaTjWw9DshX614kVJzhpuRqO3AvTMs=; b=s2eRiogfD/EfUghimVjyJFnJ64tsqzY28ggjxccEen49q/BCS8YLozZgmy8Ev8tyR0 jyoWodh/yAO3nIM+JZaE4yZn1yJjejPmqP5qXpgNL6XLKw7CxdYx/FHhzCr8YPgVsBBE 9SU70TD4E33M5Cw0sYItQE6eJIrPCZVVwLvKSetu3EHooPnyfAHpB9f227O+DcHl8hZE Qsorz2SNCnNG0yr652XOYHJzXfoWeBhFCw6lp0evLE+Rg8K3dknP4jNkedaiA+XcMKDK 0qxQ5uuqljfAycw1dCNp6RTYbVM4wozvMeW4oy9PFC6GPG0Wk+WHezgahvgUdRMsOETQ QZ9g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ka22-20020a170907991600b00977d0db2c6asi3914374ejc.63.2023.06.11.10.46.37; Sun, 11 Jun 2023 10:47:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233822AbjFKRVC (ORCPT + 99 others); Sun, 11 Jun 2023 13:21:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45280 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230156AbjFKRUz (ORCPT ); Sun, 11 Jun 2023 13:20:55 -0400 Received: from angie.orcam.me.uk (angie.orcam.me.uk [IPv6:2001:4190:8020::34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 21BFFE6F; Sun, 11 Jun 2023 10:20:30 -0700 (PDT) Received: by angie.orcam.me.uk (Postfix, from userid 500) id 529979200C9; Sun, 11 Jun 2023 19:20:02 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id 4E1479200C0; Sun, 11 Jun 2023 18:20:02 +0100 (BST) Date: Sun, 11 Jun 2023 18:20:02 +0100 (BST) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?utf-8?q?Pali_Roh?= =?utf-8?q?=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v9 12/14] PCI: Provide stub failed link recovery for device probing and hot plug In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768429319100429927?= X-GMAIL-MSGID: =?utf-8?q?1768429319100429927?= This now fails unconditionally and will be always optimised away, but provides for quirks to implement recovery for failed links detected in device probing and device hot plug events. Signed-off-by: Maciej W. Rozycki --- New change in v9, factored out from 7/7: - Rename `pcie_downstream_link_retrain' to `pcie_failed_link_retrain'. - Add stub implementation in "pci.h". --- drivers/pci/pci.c | 2 ++ drivers/pci/pci.h | 4 ++++ drivers/pci/probe.c | 2 ++ 3 files changed, 8 insertions(+) linux-pcie-failed-link-retrain.diff Index: linux-macro/drivers/pci/pci.c =================================================================== --- linux-macro.orig/drivers/pci/pci.c +++ linux-macro/drivers/pci/pci.c @@ -4912,6 +4912,8 @@ static bool pcie_wait_for_link_delay(str if (active) msleep(20); ret = pcie_wait_for_link_status(pdev, false, active); + if (active && !ret) + ret = pcie_failed_link_retrain(pdev); if (active && ret) msleep(delay); Index: linux-macro/drivers/pci/pci.h =================================================================== --- linux-macro.orig/drivers/pci/pci.h +++ linux-macro/drivers/pci/pci.h @@ -554,6 +554,10 @@ static inline int pci_dev_specific_disab return -ENOTTY; } #endif +static inline bool pcie_failed_link_retrain(struct pci_dev *dev) +{ + return false; +} /* PCI error reporting and recovery */ pci_ers_result_t pcie_do_recovery(struct pci_dev *dev, Index: linux-macro/drivers/pci/probe.c =================================================================== --- linux-macro.orig/drivers/pci/probe.c +++ linux-macro/drivers/pci/probe.c @@ -2549,6 +2549,8 @@ void pci_device_add(struct pci_dev *dev, dma_set_max_seg_size(&dev->dev, 65536); dma_set_seg_boundary(&dev->dev, 0xffffffff); + pcie_failed_link_retrain(dev); + /* Fix up broken headers */ pci_fixup_device(pci_fixup_header, dev); From patchwork Sun Jun 11 17:20:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Maciej W. Rozycki" X-Patchwork-Id: 106136 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2144381vqr; Sun, 11 Jun 2023 10:29:19 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4VoG/ho5XZSc6xqC7TqxgjjqVdTLZaEJKOHVkkVBU+yABsoMUndL66DCLzT9BL6jlnEZEl X-Received: by 2002:a05:6402:5114:b0:514:387c:930 with SMTP id m20-20020a056402511400b00514387c0930mr4250525edd.5.1686504559066; Sun, 11 Jun 2023 10:29:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686504559; cv=none; d=google.com; s=arc-20160816; b=OCP15CTR4nh2HqoFwXVkIKA+gCi6Glw6lqxGvBZP5Cffov4pqyFvYSmcOth2uM2mt+ oFPM/ooLNCRBTPFeIONWFy/x32eiojfjD621GWyz3390jApCtm042uoRErHqC9Qp0o1G 0ZGb0nteN8DRY1lBmz+mNmHdD+EMtjpJsX1LS63xalhJ1FXkyjqsXRYOyq0jtqh+AGlD PIVjimqjk/5It7cKbi4Of4pvk+09cfvxBZ2w6HHFPf5YMLgHtVRy7aK3E65z2wIRm8Gv 3hh/0UmU0VMuQwsHVg36CEEaC9C1XObU9CHjfzWFPMlpZs98/5QXLM33kp3vh8vPcl0J rU3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:user-agent:references:message-id :in-reply-to:subject:cc:to:from:date; bh=4moH3bZ6fQEGY92Cg/cgHpWL81RAv3+VKZWLn1XfRq4=; b=cTlYOoE6L2+bPyM5c0OT74eOH0C7SrmrnY1f2uDweUydQb0Z+aIIasrFndMk9lXM7v zhB61IaIOMQw6Jq00NZahv1xqoeh3i1G3DC3WVED6XKbWsVxgAeq3PRiVefdyyyY4W4c 8GBnUotELtAanA4iZU1Yyic+1wkEhTZRcBG5IBLqGM1qSbJVMuCitc65FJn7qrEL3bSE aDGKqb3EsvR01hQwoaflmoMgHgnAe8rWZDyN2SVdkZwyBiLHoe4M977U4yFyY3QUWSEs yNFSj8H6ETRYFVgOyCJsO/TuFOIXMeeKSyUYJ7clOXvsT/tNRetpAWcJNWHEhdDfjguN ApUw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id by1-20020a0564021b0100b0051632dc69b3si4798811edb.263.2023.06.11.10.28.54; Sun, 11 Jun 2023 10:29:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234045AbjFKRVX (ORCPT + 99 others); Sun, 11 Jun 2023 13:21:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45508 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233992AbjFKRVT (ORCPT ); Sun, 11 Jun 2023 13:21:19 -0400 Received: from angie.orcam.me.uk (angie.orcam.me.uk [78.133.224.34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id CCE1719AA; Sun, 11 Jun 2023 10:20:54 -0700 (PDT) Received: by angie.orcam.me.uk (Postfix, from userid 500) id 6DEA69200CB; Sun, 11 Jun 2023 19:20:06 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id 66A4F9200CA; Sun, 11 Jun 2023 18:20:06 +0100 (BST) Date: Sun, 11 Jun 2023 18:20:06 +0100 (BST) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?utf-8?q?Pali_Roh?= =?utf-8?q?=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v9 13/14] PCI: Add failed link recovery for device reset events In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768428204383020828?= X-GMAIL-MSGID: =?utf-8?q?1768428204383020828?= Request failed link recovery with any upstream bridge where a device has not come back after reset within PCI_RESET_WAIT time. Reset the polling interval if recovery succeeded, otherwise continue as usual. Signed-off-by: Maciej W. Rozycki --- New change in v9, factored out from 7/7: - Remove duplicate succesful completion report previously added (not sure where it came from, possibly an unnoticed leftover from experiments). - Make the type of `retrain' variable `bool' rather than `int' and invert the logic used. - Rename `pcie_downstream_link_retrain' to `pcie_failed_link_retrain'. - Rename `pcie_upstream_link_retrain' to `pcie_parent_link_retrain'. Add documentation. --- drivers/pci/pci.c | 39 ++++++++++++++++++++++++++++++++++----- 1 file changed, 34 insertions(+), 5 deletions(-) linux-pcie-dev-wait-link-retrain.diff Index: linux-macro/drivers/pci/pci.c =================================================================== --- linux-macro.orig/drivers/pci/pci.c +++ linux-macro/drivers/pci/pci.c @@ -1146,10 +1146,27 @@ void pci_resume_bus(struct pci_bus *bus) pci_walk_bus(bus, pci_resume_one, NULL); } +/** + * pcie_parent_link_retrain - Check and retrain link we are downstream from + * @dev: PCI device to handle. + * + * Return TRUE if the link was retrained, FALSE otherwise. + */ +static bool pcie_parent_link_retrain(struct pci_dev *dev) +{ + struct pci_dev *bridge; + + bridge = pci_upstream_bridge(dev); + if (bridge) + return pcie_failed_link_retrain(bridge); + else + return false; +} + static int pci_dev_wait(struct pci_dev *dev, char *reset_type, int timeout) { + bool retrain = true; int delay = 1; - u32 id; /* * After reset, the device should not silently discard config @@ -1163,21 +1180,33 @@ static int pci_dev_wait(struct pci_dev * * Command register instead of Vendor ID so we don't have to * contend with the CRS SV value. */ - pci_read_config_dword(dev, PCI_COMMAND, &id); - while (PCI_POSSIBLE_ERROR(id)) { + for (;;) { + u32 id; + + pci_read_config_dword(dev, PCI_COMMAND, &id); + if (!PCI_POSSIBLE_ERROR(id)) + break; + if (delay > timeout) { pci_warn(dev, "not ready %dms after %s; giving up\n", delay - 1, reset_type); return -ENOTTY; } - if (delay > PCI_RESET_WAIT) + if (delay > PCI_RESET_WAIT) { + if (retrain) { + retrain = false; + if (pcie_parent_link_retrain(dev)) { + delay = 1; + continue; + } + } pci_info(dev, "not ready %dms after %s; waiting\n", delay - 1, reset_type); + } msleep(delay); delay *= 2; - pci_read_config_dword(dev, PCI_COMMAND, &id); } if (delay > PCI_RESET_WAIT) From patchwork Sun Jun 11 17:20:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Maciej W. Rozycki" X-Patchwork-Id: 106141 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2150946vqr; Sun, 11 Jun 2023 10:48:06 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ78O8ZHcwGowLQfyJJeDmntlSmBlE9SYKx8i1jnssvRltTwldl+q/OQd4UVMLjnlHk1qVN4 X-Received: by 2002:a17:907:7244:b0:97e:a9ed:16bb with SMTP id ds4-20020a170907724400b0097ea9ed16bbmr3438203ejc.18.1686505685812; Sun, 11 Jun 2023 10:48:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1686505685; cv=none; d=google.com; s=arc-20160816; b=CSWPFPmLJRPzKr2Cj69SMwQp4IzvsI7emrh68PLf2BkGrPTUdCg0Ao9P+wItTqFAbJ KuHO2Fkf/m7wJLL44wgCWDjgCGWrGR+6w9TkwiL2Vh7xb03rtKOeYk6PeGxaDcPdI+hw 5a6N3wKx6qjfZKICtO7Ray4m64ue22LcaSKm577aVHs/IgTP7XOJpqsAocON7KaQAZ6e c3aV7gebHu20iu35YtXBuNE92l4XW07Wii5scn6rXJmwFyt2D+Nmkc7lIW9ixzI9nfgF ApFTWVsUn/5Ysp1Lq2fGTroDKVmzY1vy1TxeaaCRQXOsBrzPlEel8e03Pdr8JcWXLJmR nMpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:user-agent:references:message-id :in-reply-to:subject:cc:to:from:date; bh=6gyX1QqLDhCgzjmVw1V04sELFFA5bp/oPy4VO95FBxM=; b=No/tjz1ZgEij2JSab7OVH0wcpvRcllPcrO49fJX4eEf6sKYQlwvBlaOCrs2jhSTt2B CeE4Wd4uO+ZYLeP+Zkq5xMsQKYwIxeRbY49DVKsyGdGN8ljHQWTAwwO5Axv/IgkRtYk4 GFINn9w9gQilAaQl/FCY1Cwg95//NO8/MWnDkqyo6Qsztq5xLWQDuHBWETAlMWH2fPvw HpwxwUch2ESqSuxu/2aDmBvn0XeRgMnxLNof3XY+Kobn2XKsjnUp/E8okmkSa/OOxADJ 6rh7O68JYvggcYDpa1EwFJYnFCbl7C0sbtzsEJSCPSoTBxji4ituLrtaNKedCdxCRuKW 4NMA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id qw7-20020a170906fca700b00978970d5573si4124006ejb.156.2023.06.11.10.47.42; Sun, 11 Jun 2023 10:48:05 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233098AbjFKRV1 (ORCPT + 99 others); Sun, 11 Jun 2023 13:21:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45414 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233952AbjFKRVU (ORCPT ); Sun, 11 Jun 2023 13:21:20 -0400 Received: from angie.orcam.me.uk (angie.orcam.me.uk [78.133.224.34]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 458D7E75; Sun, 11 Jun 2023 10:20:55 -0700 (PDT) Received: by angie.orcam.me.uk (Postfix, from userid 500) id 974869200CC; Sun, 11 Jun 2023 19:20:10 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by angie.orcam.me.uk (Postfix) with ESMTP id 91D4E9200CA; Sun, 11 Jun 2023 18:20:10 +0100 (BST) Date: Sun, 11 Jun 2023 18:20:10 +0100 (BST) From: "Maciej W. Rozycki" To: Bjorn Helgaas , Mahesh J Salgaonkar , Oliver O'Halloran , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Saeed Mahameed , Leon Romanovsky , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni cc: Alex Williamson , Lukas Wunner , Mika Westerberg , Stefan Roese , Jim Wilson , David Abdurachmanov , =?utf-8?q?Pali_Roh?= =?utf-8?q?=C3=A1r?= , linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v9 14/14] PCI: Work around PCIe link training failures In-Reply-To: Message-ID: References: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1768429385817904980?= X-GMAIL-MSGID: =?utf-8?q?1768429385817904980?= Attempt to handle cases such as with a downstream port of the ASMedia ASM2824 PCIe switch where link training never completes and the link continues switching between speeds indefinitely with the data link layer never reaching the active state. It has been observed with a downstream port of the ASMedia ASM2824 Gen 3 switch wired to the upstream port of the Pericom PI7C9X2G304 Gen 2 switch, using a Delock Riser Card PCI Express x1 > 2 x PCIe x1 device, P/N 41433, wired to a SiFive HiFive Unmatched board. In this setup the switches are supposed to negotiate the link speed of preferably 5.0GT/s, falling back to 2.5GT/s. Instead the link continues oscillating between the two speeds, at the rate of 34-35 times per second, with link training reported repeatedly active ~84% of the time. Forcibly limiting the target link speed to 2.5GT/s with the upstream ASM2824 device however makes the two switches communicate correctly. Removing the speed restriction afterwards makes the two devices switch to 5.0GT/s then. Make use of these observations then and detect the inability to train the link, by checking for the Data Link Layer Link Active status bit being off while the Link Bandwidth Management Status indicating that hardware has changed the link speed or width in an attempt to correct unreliable link operation. Restrict the speed to 2.5GT/s then with the Target Link Speed field, request a retrain and wait 200ms for the data link to go up. If this turns out successful, then lift the restriction, letting the devices negotiate a higher speed. Also check for a 2.5GT/s speed restriction the firmware may have already arranged and lift it too with ports of devices known to continue working afterwards, currently the ASM2824 only, that already report their data link being up. Signed-off-by: Maciej W. Rozycki Link: https://lore.kernel.org/r/alpine.DEB.2.21.2203022037020.56670@angie.orcam.me.uk/ Link: https://source.denx.de/u-boot/u-boot/-/commit/a398a51ccc68 --- Changes from v8: - Rename `pcie_downstream_link_retrain' to `pcie_failed_link_retrain', add a prototype in "pci.h", moving the stub implementation under !PCI_QUIRKS umbrella. - Move back to quirks.c, though as an internal API call rather than a regular quirk. - Adjust for PCIE_LINK_RETRAIN_TIMEOUT_MS expressed in milliseconds rather than jiffies. - Use a `pcie_retrain_link' call rather than retraining inline, and also use it in the restriction lift path, making it another possible failure point. No changes from v7. Changes from v6: - Regenerate against 6.3-rc5. - Shorten the lore.kernel.org archive link in the change description. Changes from v5: - Move from a quirk into PCI core and call at device probing, hot-plug, reset and resume. Keep the ASMedia part under CONFIG_PCI_QUIRKS. - Rely on `dev->link_active_reporting' rather than re-retrieving the capability. Changes from v4: - Remove inclusion no longer needed. - Make the quirk generic based on probing device features rather than specific to the ASM2824 part only; take the Retrain Link bit erratum into account. - Still lift the 2.5GT/s speed restriction with the ASM2824 only. - Increase retrain timeout from 200ms to 1s (PCIE_LINK_RETRAIN_TIMEOUT). - Remove retrain success notification. - Use PCIe helpers rather than generic PCI functions throughout. - Trim down and update the wording of the change description for the switch from an ASM2824-specific to a generic fixup. Changes from v3: - Remove the entry for the ASM2824. Changes from v2: - Regenerate for 5.17-rc2 for a merge conflict. - Replace BUG_ON for a missing PCI Express capability with WARN_ON and an early return. Changes from v1: - Regenerate for a merge conflict. --- drivers/pci/pci.h | 3 + drivers/pci/quirks.c | 93 +++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 95 insertions(+), 1 deletion(-) linux-pcie-asm2824-manual-retrain.diff Index: linux-macro/drivers/pci/pci.h =================================================================== --- linux-macro.orig/drivers/pci/pci.h +++ linux-macro/drivers/pci/pci.h @@ -539,6 +539,7 @@ void pci_acs_init(struct pci_dev *dev); int pci_dev_specific_acs_enabled(struct pci_dev *dev, u16 acs_flags); int pci_dev_specific_enable_acs(struct pci_dev *dev); int pci_dev_specific_disable_acs_redir(struct pci_dev *dev); +bool pcie_failed_link_retrain(struct pci_dev *dev); #else static inline int pci_dev_specific_acs_enabled(struct pci_dev *dev, u16 acs_flags) @@ -553,11 +554,11 @@ static inline int pci_dev_specific_disab { return -ENOTTY; } -#endif static inline bool pcie_failed_link_retrain(struct pci_dev *dev) { return false; } +#endif /* PCI error reporting and recovery */ pci_ers_result_t pcie_do_recovery(struct pci_dev *dev, Index: linux-macro/drivers/pci/quirks.c =================================================================== --- linux-macro.orig/drivers/pci/quirks.c +++ linux-macro/drivers/pci/quirks.c @@ -33,6 +33,99 @@ #include #include "pci.h" +/* + * Retrain the link of a downstream PCIe port by hand if necessary. + * + * This is needed at least where a downstream port of the ASMedia ASM2824 + * Gen 3 switch is wired to the upstream port of the Pericom PI7C9X2G304 + * Gen 2 switch, and observed with the Delock Riser Card PCI Express x1 > + * 2 x PCIe x1 device, P/N 41433, plugged into the SiFive HiFive Unmatched + * board. + * + * In such a configuration the switches are supposed to negotiate the link + * speed of preferably 5.0GT/s, falling back to 2.5GT/s. However the link + * continues switching between the two speeds indefinitely and the data + * link layer never reaches the active state, with link training reported + * repeatedly active ~84% of the time. Forcing the target link speed to + * 2.5GT/s with the upstream ASM2824 device makes the two switches talk to + * each other correctly however. And more interestingly retraining with a + * higher target link speed afterwards lets the two successfully negotiate + * 5.0GT/s. + * + * With the ASM2824 we can rely on the otherwise optional Data Link Layer + * Link Active status bit and in the failed link training scenario it will + * be off along with the Link Bandwidth Management Status indicating that + * hardware has changed the link speed or width in an attempt to correct + * unreliable link operation. For a port that has been left unconnected + * both bits will be clear. So use this information to detect the problem + * rather than polling the Link Training bit and watching out for flips or + * at least the active status. + * + * Since the exact nature of the problem isn't known and in principle this + * could trigger where an ASM2824 device is downstream rather upstream, + * apply this erratum workaround to any downstream ports as long as they + * support Link Active reporting and have the Link Control 2 register. + * Restrict the speed to 2.5GT/s then with the Target Link Speed field, + * request a retrain and wait 200ms for the data link to go up. + * + * If this turns out successful and we know by the Vendor:Device ID it is + * safe to do so, then lift the restriction, letting the devices negotiate + * a higher speed. Also check for a similar 2.5GT/s speed restriction the + * firmware may have already arranged and lift it with ports that already + * report their data link being up. + * + * Return TRUE if the link has been successfully retrained, otherwise FALSE. + */ +bool pcie_failed_link_retrain(struct pci_dev *dev) +{ + static const struct pci_device_id ids[] = { + { PCI_VDEVICE(ASMEDIA, 0x2824) }, /* ASMedia ASM2824 */ + {} + }; + u16 lnksta, lnkctl2; + + if (!pci_is_pcie(dev) || !pcie_downstream_port(dev) || + !pcie_cap_has_lnkctl2(dev) || !dev->link_active_reporting) + return false; + + pcie_capability_read_word(dev, PCI_EXP_LNKCTL2, &lnkctl2); + pcie_capability_read_word(dev, PCI_EXP_LNKSTA, &lnksta); + if ((lnksta & (PCI_EXP_LNKSTA_LBMS | PCI_EXP_LNKSTA_DLLLA)) == + PCI_EXP_LNKSTA_LBMS) { + pci_info(dev, "broken device, retraining non-functional downstream link at 2.5GT/s\n"); + + lnkctl2 &= ~PCI_EXP_LNKCTL2_TLS; + lnkctl2 |= PCI_EXP_LNKCTL2_TLS_2_5GT; + pcie_capability_write_word(dev, PCI_EXP_LNKCTL2, lnkctl2); + + if (!pcie_retrain_link(dev, false)) { + pci_info(dev, "retraining failed\n"); + return false; + } + + pcie_capability_read_word(dev, PCI_EXP_LNKSTA, &lnksta); + } + + if ((lnksta & PCI_EXP_LNKSTA_DLLLA) && + (lnkctl2 & PCI_EXP_LNKCTL2_TLS) == PCI_EXP_LNKCTL2_TLS_2_5GT && + pci_match_id(ids, dev)) { + u32 lnkcap; + + pci_info(dev, "removing 2.5GT/s downstream link speed restriction\n"); + pcie_capability_read_dword(dev, PCI_EXP_LNKCAP, &lnkcap); + lnkctl2 &= ~PCI_EXP_LNKCTL2_TLS; + lnkctl2 |= lnkcap & PCI_EXP_LNKCAP_SLS; + pcie_capability_write_word(dev, PCI_EXP_LNKCTL2, lnkctl2); + + if (!pcie_retrain_link(dev, false)) { + pci_info(dev, "retraining failed\n"); + return false; + } + } + + return true; +} + static ktime_t fixup_debug_start(struct pci_dev *dev, void (*fn)(struct pci_dev *dev)) {