From patchwork Fri Feb 23 20:58:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 205674 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:a81b:b0:108:e6aa:91d0 with SMTP id bq27csp840485dyb; Fri, 23 Feb 2024 12:59:35 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCU7aSXoG5HNltPfZFnXBGvLVlTXi+S2fdaMCnhMXlFHPE7oAE8rkrS30h9iheUwWuLYEyKEbDJ32X7br9RzQ9O7BLIVvg== X-Google-Smtp-Source: AGHT+IGLGs1ZbJHw24r8pAdBEJzo7v5S967QYrG937yhTOQ8Ip+sYVhitP8ROkuxtkmIfIirXCn6 X-Received: by 2002:aa7:cccd:0:b0:565:8130:5b91 with SMTP id y13-20020aa7cccd000000b0056581305b91mr660017edt.34.1708721974799; Fri, 23 Feb 2024 12:59:34 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708721974; cv=pass; d=google.com; s=arc-20160816; b=YQcZOiYfpDcE7DPN6E4gYN5G9Vw5dekQDQ5ZfCFzVY35Gt9cRCoO2ljhe8+cvNgkRn oI9tc2nOWMezymI+RSVDPNoWqReU4C0WzuZu46YxE/lLDD2fgGgIqOtq/NsFO76RTzLQ aWqvpFKi3kV8RoRb5YPnQLfcJYJeyYnwpjSZgveh1xhUhwoTTB3XSE1amMKjisry+sqa NYUNDz2tqjMXrgU0VLvLMvdiUTbkFfNinDN15K5gOG+3mPaneaNpZ39ZudTOl0BCxV7F VHgRQ9+TX045whs6uRh19AleywjvPvJvrDc0SXWn8IQq7pJMJTkfaePsv529viYAu7Sd JNPg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=npg9cldg+GDrr7VZqhBOfLu8BV5y8H+Vds9Pq1Zlo5Q=; fh=UOtCQvdtLRjYu4lpt5Tw8oOZkSAAXsqpAt775wBcPRQ=; b=XsCfNcwQzkbNoZQ2FRiVUAFZJvHBx+OZfwJxdlpMZW77q1VTzBk+pw169/8gK/hMXE UlcmGI97DW9+4nX5QfpqT9aPKhU7uxOaHqAefxGE/84WHPgdIbg3wS9py3wIIV0Zp4sb 0lkrdXM7ZRNUoUvoDgZcqH52iqJbJ0zcOQCWLPasroYGj8VTLmVwInSU1NyyhLHGBHqh L/qJV5R1IS88VkVS2TH1dm3RUcSwg/d5eEJMkN/uXHOeje0HMjFLIJhNjMPEjbUZHbVr LMlesgWCMaZfyMDsEesrbMZ4+uTqLKJZb+SUNJ7TMcoa0EvTcDvFdCS4yfgrY5y7eZBB ZKSA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=WWR+aED1; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-79153-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-79153-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id s8-20020a056402520800b00564901a92a8si4648151edd.39.2024.02.23.12.59.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Feb 2024 12:59:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-79153-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=WWR+aED1; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-79153-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-79153-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 419C61F2426D for ; Fri, 23 Feb 2024 20:59:34 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5151F1493B7; Fri, 23 Feb 2024 20:59:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="WWR+aED1" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 110DA1482F2; Fri, 23 Feb 2024 20:58:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708721937; cv=none; b=soW36vjrVbVZDfDOpzsULQDVydvOMWDXII41rHRMOlpYZDHKZqTCw3GhHP2xNRxWGva6herb21T8zew0WMZV2qkIJMo6CaVtYGaoBpGaBXmpYdgHwfQvp3tzjB/6Wevj8BSbaSRwrph4m7nFqyRMf9RSvlAJYlQdxeWmbsSs2ts= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708721937; c=relaxed/simple; bh=sCYWCUutUwF8t97LLETKaSh/tWIJHSaDRVQHK3LTBl0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=F6rwURJEu3m5m921Tpfo/5B34SB5tZefp6Y0R4kO1I3exGPeXFFDSjyrB88ZRqQPOBk9deDr+YOb5SuPoHI4KoxSzdu/2PP8t4T5hoiBoNaTi18v8C8S0r9UUxkHv1g927ej16G9Tvl3PtnaMOJVJnt0Ok7ZChaocbywtMnylWY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=WWR+aED1; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6FBEFC4166B; Fri, 23 Feb 2024 20:58:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708721936; bh=sCYWCUutUwF8t97LLETKaSh/tWIJHSaDRVQHK3LTBl0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WWR+aED1oIMzpkVtL+tpBdAgf2q8MZ2OrZgsyavI4HZcn0morX0ROVSa7nIBv2JAT 4c4wg+qUsNNrojwieSiW5q4SdsGIgxdOJVOgyhIsCn5IeDPcuih1EYt+P58pZEx4JC Up4YpkxjOuooCtyVow/dmNvu+Uofg2Lhwusy98DPvtf2EyJCNHc+c/FB9wjVpo2P8W vzRuNasQZZl5rZv010ucp8UjVa60Ajfmle86JfgXR/LDbWMNLldiHjoyQtqBs3oGRQ gH04CPBGSKX55JlWE2eGDIS3EcR0XWrGwZJBcXoGX7fHu7JCDsyX9ngRy2oj3KiH2b YLCvl4A+7obLw== From: Bjorn Helgaas To: linux-pci@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Mika Westerberg , "David E . Box" , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , "Rafael J . Wysocki" , Tasev Nikola , Mark Enriquez , Thomas Witt , Werner Sembach , Vidya Sagar , Kai-Heng Feng , Kuppuswamy Sathyanarayanan , Ricky Wu , Mario Limonciello , Bjorn Helgaas Subject: [PATCH v7 1/5] PCI/ASPM: Move pci_configure_ltr() to aspm.c Date: Fri, 23 Feb 2024 14:58:47 -0600 Message-Id: <20240223205851.114931-2-helgaas@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240223205851.114931-1-helgaas@kernel.org> References: <20240223205851.114931-1-helgaas@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791724853016805923 X-GMAIL-MSGID: 1791724853016805923 From: "David E. Box" The Latency Tolerance Reporting (LTR) mechanism supports the ASPM L1.2 state and is only configured when CONFIG_PCIEASPM is set. Move pci_configure_ltr() and pci_bridge_reconfigure_ltr() into aspm.c since they only build when CONFIG_PCIEASPM is set. No functional change intended. Suggested-by: Bjorn Helgaas Link: https://lore.kernel.org/r/20240128233212.1139663-2-david.e.box@linux.intel.com Signed-off-by: David E. Box [bhelgaas: commit log, split build change from function moves] Signed-off-by: Bjorn Helgaas --- drivers/pci/pci.c | 18 ---------- drivers/pci/pci.h | 5 ++- drivers/pci/pcie/aspm.c | 75 +++++++++++++++++++++++++++++++++++++++++ drivers/pci/probe.c | 61 --------------------------------- 4 files changed, 79 insertions(+), 80 deletions(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index d8f11a078924..c783e0f1f2a9 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -1626,24 +1626,6 @@ static int pci_save_pcie_state(struct pci_dev *dev) return 0; } -void pci_bridge_reconfigure_ltr(struct pci_dev *dev) -{ -#ifdef CONFIG_PCIEASPM - struct pci_dev *bridge; - u32 ctl; - - bridge = pci_upstream_bridge(dev); - if (bridge && bridge->ltr_path) { - pcie_capability_read_dword(bridge, PCI_EXP_DEVCTL2, &ctl); - if (!(ctl & PCI_EXP_DEVCTL2_LTR_EN)) { - pci_dbg(bridge, "re-enabling LTR\n"); - pcie_capability_set_word(bridge, PCI_EXP_DEVCTL2, - PCI_EXP_DEVCTL2_LTR_EN); - } - } -#endif -} - static void pci_restore_pcie_state(struct pci_dev *dev) { int i = 0; diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 2336a8d1edab..9aeba82facc4 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -97,7 +97,6 @@ void pci_msi_init(struct pci_dev *dev); void pci_msix_init(struct pci_dev *dev); bool pci_bridge_d3_possible(struct pci_dev *dev); void pci_bridge_d3_update(struct pci_dev *dev); -void pci_bridge_reconfigure_ltr(struct pci_dev *dev); int pci_bridge_wait_for_secondary_bus(struct pci_dev *dev, char *reset_type); static inline void pci_wakeup_event(struct pci_dev *dev) @@ -573,11 +572,15 @@ void pcie_aspm_init_link_state(struct pci_dev *pdev); void pcie_aspm_exit_link_state(struct pci_dev *pdev); void pcie_aspm_pm_state_change(struct pci_dev *pdev); void pcie_aspm_powersave_config_link(struct pci_dev *pdev); +void pci_configure_ltr(struct pci_dev *pdev); +void pci_bridge_reconfigure_ltr(struct pci_dev *pdev); #else static inline void pcie_aspm_init_link_state(struct pci_dev *pdev) { } static inline void pcie_aspm_exit_link_state(struct pci_dev *pdev) { } static inline void pcie_aspm_pm_state_change(struct pci_dev *pdev) { } static inline void pcie_aspm_powersave_config_link(struct pci_dev *pdev) { } +static inline void pci_configure_ltr(struct pci_dev *pdev) { } +static inline void pci_bridge_reconfigure_ltr(struct pci_dev *pdev) { } #endif #ifdef CONFIG_PCIE_ECRC diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c index 5a0066ecc3c5..d1538f73f2f9 100644 --- a/drivers/pci/pcie/aspm.c +++ b/drivers/pci/pcie/aspm.c @@ -938,6 +938,81 @@ void pcie_aspm_init_link_state(struct pci_dev *pdev) up_read(&pci_bus_sem); } +void pci_bridge_reconfigure_ltr(struct pci_dev *pdev) +{ + struct pci_dev *bridge; + u32 ctl; + + bridge = pci_upstream_bridge(pdev); + if (bridge && bridge->ltr_path) { + pcie_capability_read_dword(bridge, PCI_EXP_DEVCTL2, &ctl); + if (!(ctl & PCI_EXP_DEVCTL2_LTR_EN)) { + pci_dbg(bridge, "re-enabling LTR\n"); + pcie_capability_set_word(bridge, PCI_EXP_DEVCTL2, + PCI_EXP_DEVCTL2_LTR_EN); + } + } +} + +void pci_configure_ltr(struct pci_dev *pdev) +{ + struct pci_host_bridge *host = pci_find_host_bridge(pdev->bus); + struct pci_dev *bridge; + u32 cap, ctl; + + if (!pci_is_pcie(pdev)) + return; + + /* Read L1 PM substate capabilities */ + pdev->l1ss = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_L1SS); + + pcie_capability_read_dword(pdev, PCI_EXP_DEVCAP2, &cap); + if (!(cap & PCI_EXP_DEVCAP2_LTR)) + return; + + pcie_capability_read_dword(pdev, PCI_EXP_DEVCTL2, &ctl); + if (ctl & PCI_EXP_DEVCTL2_LTR_EN) { + if (pci_pcie_type(pdev) == PCI_EXP_TYPE_ROOT_PORT) { + pdev->ltr_path = 1; + return; + } + + bridge = pci_upstream_bridge(pdev); + if (bridge && bridge->ltr_path) + pdev->ltr_path = 1; + + return; + } + + if (!host->native_ltr) + return; + + /* + * Software must not enable LTR in an Endpoint unless the Root + * Complex and all intermediate Switches indicate support for LTR. + * PCIe r4.0, sec 6.18. + */ + if (pci_pcie_type(pdev) == PCI_EXP_TYPE_ROOT_PORT) { + pcie_capability_set_word(pdev, PCI_EXP_DEVCTL2, + PCI_EXP_DEVCTL2_LTR_EN); + pdev->ltr_path = 1; + return; + } + + /* + * If we're configuring a hot-added device, LTR was likely + * disabled in the upstream bridge, so re-enable it before enabling + * it in the new device. + */ + bridge = pci_upstream_bridge(pdev); + if (bridge && bridge->ltr_path) { + pci_bridge_reconfigure_ltr(pdev); + pcie_capability_set_word(pdev, PCI_EXP_DEVCTL2, + PCI_EXP_DEVCTL2_LTR_EN); + pdev->ltr_path = 1; + } +} + /* Recheck latencies and update aspm_capable for links under the root */ static void pcie_update_aspm_capable(struct pcie_link_state *root) { diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index b7335be56008..b809c0b0e0e5 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -2209,67 +2209,6 @@ static void pci_configure_relaxed_ordering(struct pci_dev *dev) } } -static void pci_configure_ltr(struct pci_dev *dev) -{ -#ifdef CONFIG_PCIEASPM - struct pci_host_bridge *host = pci_find_host_bridge(dev->bus); - struct pci_dev *bridge; - u32 cap, ctl; - - if (!pci_is_pcie(dev)) - return; - - /* Read L1 PM substate capabilities */ - dev->l1ss = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_L1SS); - - pcie_capability_read_dword(dev, PCI_EXP_DEVCAP2, &cap); - if (!(cap & PCI_EXP_DEVCAP2_LTR)) - return; - - pcie_capability_read_dword(dev, PCI_EXP_DEVCTL2, &ctl); - if (ctl & PCI_EXP_DEVCTL2_LTR_EN) { - if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) { - dev->ltr_path = 1; - return; - } - - bridge = pci_upstream_bridge(dev); - if (bridge && bridge->ltr_path) - dev->ltr_path = 1; - - return; - } - - if (!host->native_ltr) - return; - - /* - * Software must not enable LTR in an Endpoint unless the Root - * Complex and all intermediate Switches indicate support for LTR. - * PCIe r4.0, sec 6.18. - */ - if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) { - pcie_capability_set_word(dev, PCI_EXP_DEVCTL2, - PCI_EXP_DEVCTL2_LTR_EN); - dev->ltr_path = 1; - return; - } - - /* - * If we're configuring a hot-added device, LTR was likely - * disabled in the upstream bridge, so re-enable it before enabling - * it in the new device. - */ - bridge = pci_upstream_bridge(dev); - if (bridge && bridge->ltr_path) { - pci_bridge_reconfigure_ltr(dev); - pcie_capability_set_word(dev, PCI_EXP_DEVCTL2, - PCI_EXP_DEVCTL2_LTR_EN); - dev->ltr_path = 1; - } -#endif -} - static void pci_configure_eetlp_prefix(struct pci_dev *dev) { #ifdef CONFIG_PCI_PASID From patchwork Fri Feb 23 20:58:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 205678 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:a81b:b0:108:e6aa:91d0 with SMTP id bq27csp841980dyb; Fri, 23 Feb 2024 13:01:57 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXoIegk+bjUos0B45OXdQXVw10NaFUyOKkew1HDrsRLLAtJOT95GhR9mBvyTjtkThuZFYjgrrXBpDWo+3iuA44eiea1nw== X-Google-Smtp-Source: AGHT+IGGluzN6bm5uKYCMBhL1GF6J55nfdRACWgSWmgiXXeHhH9i5zJpRZB/94QpH0JyMdFclqBO X-Received: by 2002:aa7:9d88:0:b0:6e4:62d9:6490 with SMTP id f8-20020aa79d88000000b006e462d96490mr783409pfq.6.1708722117101; Fri, 23 Feb 2024 13:01:57 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708722117; cv=pass; d=google.com; s=arc-20160816; b=oxlxjVtTVcyQ79mV3TOm3aQAjd7zQb9toKJ9+x1eJxWIl+EsRNWokQKTVpvHpBxKDM fGFAiY8odFHUKGgJCnRM85+TwyNRhpUssxkGkc93DcnxkN0+zlDqYYg48ibE5m/tYlnZ GFxgIVDUxQvrUOErvh+gz9vbatCABn8y1U0rufNXq9eSgcop0LQGY4UhHCt/OV++0DkS GEXCRAnpIJOXt3ukAM81gycwceYIkReDZoCo2Ezu0axBZ5+6vyrxp65I9YvfsQJdElym t7OF8Ohi4lnkelCQrn7qnDotaiAuhQAvWWRd5Ug7jbXE6GaRtAa0xhQJHIvBpMMgKIvM rpXA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=lAxKNWrb6EPk72Uh69l5cejT5/EZembBXbOOAIZAOVc=; fh=UOtCQvdtLRjYu4lpt5Tw8oOZkSAAXsqpAt775wBcPRQ=; b=ABiUjfzZugdTWnw+czyyhUIuRkRAzaZy/lYEO41CW92DEXgJcyubxBpTY/1tRAhfUr UNLDycWMvOF+0ahapODiKKrsW3P+dJKpC8OOnPW5bVv0O2mVLvNdfUXdArh7WHTR7vmq ++d0Z+kzHFwaYOMH0MvCIdtzg27UxKsSDMWh5VFytrIs0MoQQWcr3cg6YM5DlxgBmBEp lxWMdePZg7o3mdc0pYpxgKi2f8jtTeGt19aLF/mc2+XcfcExkE5nW/QMKe5SPzLhBayW FPTUIQoE5VZQWBddXB2zNA0f3XvN1R+ANPS8WRr8peZsBmPmCFzoIOPArPuQp7AXO9N6 UQCA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=IJDP2nx7; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-79154-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-79154-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id h15-20020a056a00170f00b006e465f8059bsi9728644pfc.278.2024.02.23.13.01.56 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Feb 2024 13:01:57 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-79154-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=IJDP2nx7; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-79154-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-79154-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id D8EA5B228B3 for ; Fri, 23 Feb 2024 20:59:45 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7CFD614CAA5; Fri, 23 Feb 2024 20:59:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="IJDP2nx7" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8023149397; Fri, 23 Feb 2024 20:58:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708721939; cv=none; b=CIquBULENHipGU+g7Ap15ycufAriKg1y58Q04Qn0sBy8ZhcRYBKzG8Qx+G82ZjG9t37TxWtDuVfyC7ygxT+3MNfAW9irG60g6hfpxVgNxF1FXWYTGpx8Y3gKNv6cCUjKEI7Y1q20TwbzYJ+QkDWJ3SZ2Pv4qvDvMx2TCeQwuHw0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708721939; c=relaxed/simple; bh=x65rMjAkCbuMIIOI1rAb9CSZyHu2Knp8Gr3cSU2oInE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=fcymUZM2vH5eX22LlcpVymG2t4Qwd2B6eDn3sezNUnznICEGODCpv1WCwOwM0aKBUttnkEmWHocviTflx115vitL9KQsxREmTsIZYjBGaXBnHyBg9zGg+dQlZpwuJ/r2UyfH+F3Qq1dBJry9hQDBVvtSnldryYsUHjNGWnLqX7I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=IJDP2nx7; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 37E6EC43390; Fri, 23 Feb 2024 20:58:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708721938; bh=x65rMjAkCbuMIIOI1rAb9CSZyHu2Knp8Gr3cSU2oInE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IJDP2nx70+e8AzyQeZipolaoUklfvNpG55x5LxdePBmRDfB60VHt0l1Q7tj82p2vT aa9gXGE7m11cd7Y22Xwe4u/i6kSG4qRI6obFvMJ7lb8teTVubeWRximRxoOurNqMRw D8IfqM2BRrPBp5PgL/Qy8Jr7vW0K25H47oDSt1HD0zIx405bj0q0AZC2x8bPX7JTCZ K8EVXO+E+KXbCSnalTa/K3yle4RkfUgFAjQEdamNW/lxJz5yBBBp6odAyEN5xpTNUN 3PBwim9CtmIgHCSZMM9tJuG5JegzeHxGtLzSLTkKVh8d/JmE614eIPHo6WH3o55/Pa HBJIbKZffYZzQ== From: Bjorn Helgaas To: linux-pci@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Mika Westerberg , "David E . Box" , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , "Rafael J . Wysocki" , Tasev Nikola , Mark Enriquez , Thomas Witt , Werner Sembach , Vidya Sagar , Kai-Heng Feng , Kuppuswamy Sathyanarayanan , Ricky Wu , Mario Limonciello , Bjorn Helgaas Subject: [PATCH v7 2/5] PCI/ASPM: Always build aspm.c Date: Fri, 23 Feb 2024 14:58:48 -0600 Message-Id: <20240223205851.114931-3-helgaas@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240223205851.114931-1-helgaas@kernel.org> References: <20240223205851.114931-1-helgaas@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791725002374495939 X-GMAIL-MSGID: 1791725002374495939 From: "David E. Box" Some ASPM-related tasks, such as save and restore of LTR and L1SS capabilities, still need to be performed when CONFIG_PCIEASPM is not enabled. To prepare for these changes, wrap the current code in aspm.c with an #ifdef and always build the file. Link: https://lore.kernel.org/r/20240128233212.1139663-2-david.e.box@linux.intel.com Signed-off-by: David E. Box [bhelgaas: split build change from function moves] Signed-off-by: Bjorn Helgaas --- drivers/pci/pcie/Makefile | 2 +- drivers/pci/pcie/aspm.c | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/pci/pcie/Makefile b/drivers/pci/pcie/Makefile index 8de4ed5f98f1..6461aa93fe76 100644 --- a/drivers/pci/pcie/Makefile +++ b/drivers/pci/pcie/Makefile @@ -6,7 +6,7 @@ pcieportdrv-y := portdrv.o rcec.o obj-$(CONFIG_PCIEPORTBUS) += pcieportdrv.o -obj-$(CONFIG_PCIEASPM) += aspm.o +obj-y += aspm.o obj-$(CONFIG_PCIEAER) += aer.o err.o obj-$(CONFIG_PCIEAER_INJECT) += aer_inject.o obj-$(CONFIG_PCIE_PME) += pme.o diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c index d1538f73f2f9..d50c0f83430f 100644 --- a/drivers/pci/pcie/aspm.c +++ b/drivers/pci/pcie/aspm.c @@ -24,6 +24,8 @@ #include "../pci.h" +#ifdef CONFIG_PCIEASPM + #ifdef MODULE_PARAM_PREFIX #undef MODULE_PARAM_PREFIX #endif @@ -1517,3 +1519,5 @@ bool pcie_aspm_support_enabled(void) { return aspm_support_enabled; } + +#endif /* CONFIG_PCIEASPM */ From patchwork Fri Feb 23 20:58:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 205675 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:a81b:b0:108:e6aa:91d0 with SMTP id bq27csp840682dyb; Fri, 23 Feb 2024 13:00:00 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVtraJyaC8JVNyzt1B6I/JA+fVmaoFYR3zxgBkVl4prsSthhW6/56x1lkQ45TjHRV51NrvSHoEN7OOAYi6C1muhxeCVzA== X-Google-Smtp-Source: AGHT+IFv3Ny3U/fEZgJwx1GeoyTwo0jCslbO1LqMY0CiP52NCw3O4eOuOUFbQmMrs/jvwlKxsiQd X-Received: by 2002:a0c:f385:0:b0:68f:2c18:3b8a with SMTP id i5-20020a0cf385000000b0068f2c183b8amr769923qvk.61.1708721999805; Fri, 23 Feb 2024 12:59:59 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708721999; cv=pass; d=google.com; s=arc-20160816; b=ki6+SvBlr03Kb0Eh/VMrhPmQceIU5APBX9celwTdTOJnZ1IJ1qLH/6VVcRVnwfWty0 BEeJXW66pJlPoUonhd0zKpHU13+qrGmEIh3APMlLU8WFr6ylPXdh/f+mrcjEqqGhYZau w1DGpzwFjp1flcMDXYSxfpMvqJR/3XwtrMWFbvOvV87SYTu/Ow26ENEeEcJfsMoEs7pj Cd6N69hARXy9MAmN7Sp4henK2cdnXmquDbL4v6o3NSkH8e4ej5MhX4H8zKLhUxV1XLkz N9W5XD+BIxWkmNMVJNSEYmAH1fYnaEMH5eVcUxvVahPYlmtBavUsWZz+KY1d31nhEn8E 6kog== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=CQGLaKHuqUPcl8Ymr2X1Ohukoz1S8oJb6G1fLKJiBN4=; fh=UOtCQvdtLRjYu4lpt5Tw8oOZkSAAXsqpAt775wBcPRQ=; b=SC9jIQUpVQ2aE+NPGCC0+BAhEdTpsSlxV4LRijAxhXqpgB1kj3f+BR27bJP34hi16x xLpNntf5d1mcTGPIkuoWUzRKQ38+aMoI4tVLKBzAG9b7C12u7aRuoQVuIn1vWr1YETbY 5CAcB6EBI0kKaBRbPY1yBZExjkLFApYyoIR9uLDeDxBO0OZXdPJRikbVqjcEXbZIiNo8 LSCzD7s4a3vL0u3kFIbT2CGfW6VGQCovNAuBeRuF0d9j6TgwfUc8zJJT0Xfv0MgG4fuR Fj5JXngHa5y/F9AmVKSRPDtsTd9EBrljv+++o5o574G0Qexhs2PBn8gZvZ8pNmbTu8Lc qvkg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=m8dcpd8e; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-79155-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-79155-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id iw12-20020a0562140f2c00b0068c7d55794bsi16181820qvb.135.2024.02.23.12.59.59 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Feb 2024 12:59:59 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-79155-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=m8dcpd8e; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-79155-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-79155-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 88F191C25262 for ; Fri, 23 Feb 2024 20:59:59 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 83D2B146E76; Fri, 23 Feb 2024 20:59:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="m8dcpd8e" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9C89D14A0BE; Fri, 23 Feb 2024 20:59:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708721940; cv=none; b=joiKDMURLHeePyMtX19Dd5xj4SqLJjyo8kmOGhxKEml9Bwl+saw4CV8T25LLqpxBpj5pxIsg10T0HgrGL8vPwkSg4BAsXdLIsyE7A15cuX2Xb61TPnGbZTyQr97jard4xcHBK3lsX3QcEv+buqw9btZ2Lbd0rSf8gu7ZkCo13wE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708721940; c=relaxed/simple; bh=ZfUmjBzp8E1Yoa6olg2QBswrQUhVxgYSSrq94pwX0ZE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=mliQN6AzFUs5LCgothA+VHU2kevyhcMvcBi/ETcFnOGVzDfYTaOUshjkMBWM8pDfbjVqgBJ4j6+biZlLlPUhRwj0TQazdWsuQoMoKHhMGZb882uzunuBCJ9mBGYXI3xSoH27Yuzif1y4Mf7hwXPtOCuPJeTHQEhcQT8j78xgo9I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=m8dcpd8e; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 002EDC433C7; Fri, 23 Feb 2024 20:58:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708721940; bh=ZfUmjBzp8E1Yoa6olg2QBswrQUhVxgYSSrq94pwX0ZE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=m8dcpd8ekx0tudIxaOtd1E0+tGQ3N/GZOaojbBM9YLpuO4y010iIocN6wsCnHl8iK hg/ddlPIXk6Om4WNwjFet4ezwdO1RO45dwbv7IR1S7ncp51vLmxStOIcH+IzwrulXV 2ciE+A25izcNBL3CS119zqxasvtD0fQpNo/5E4AqzfML8VflRhuJ6xJgEMo+m6Kd02 mMBQEzsyC6Isar8WduVLZ5DqIZuecQMFGSJ+MCj712pxeFTbGOT+9V1h5TpZfdRNjT WSoFkXzT/77dT9kVr/4Qvl/i3+BT1EnVRiPXulHKHDRoqZLFQYl0oNBC65AT9JKIHi UNifBM8RXCK6w== From: Bjorn Helgaas To: linux-pci@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Mika Westerberg , "David E . Box" , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , "Rafael J . Wysocki" , Tasev Nikola , Mark Enriquez , Thomas Witt , Werner Sembach , Vidya Sagar , Kai-Heng Feng , Kuppuswamy Sathyanarayanan , Ricky Wu , Mario Limonciello , Bjorn Helgaas Subject: [PATCH v7 3/5] PCI/ASPM: Move pci_save_ltr_state() to aspm.c Date: Fri, 23 Feb 2024 14:58:49 -0600 Message-Id: <20240223205851.114931-4-helgaas@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240223205851.114931-1-helgaas@kernel.org> References: <20240223205851.114931-1-helgaas@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791724879416888003 X-GMAIL-MSGID: 1791724879416888003 From: "David E. Box" Even when CONFIG_PCIEASPM is not set, we save and restore the LTR Capability so that if ASPM L1.2 and LTR were configured by the platform, ASPM L1.2 will still work after suspend/resume, when that platform configuration may be lost. See dbbfadf23190 ("PCI/ASPM: Save LTR Capability for suspend/resume"). Since ASPM L1.2 depends on the LTR Capability, move the save/restore code to the part of aspm.c that is always compiled regardless of CONFIG_PCIEASPM. No functional change intended. Suggested-by: Bjorn Helgaas Link: https://lore.kernel.org/r/20240128233212.1139663-5-david.e.box@linux.intel.com Signed-off-by: David E. Box [bhelgaas: commit log, reorder to make this a pure move] Signed-off-by: Bjorn Helgaas --- drivers/pci/pci.c | 40 ---------------------------------------- drivers/pci/pci.h | 5 +++++ drivers/pci/pcie/aspm.c | 40 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 45 insertions(+), 40 deletions(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index c783e0f1f2a9..564e2cf2dde5 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -1689,46 +1689,6 @@ static void pci_restore_pcix_state(struct pci_dev *dev) pci_write_config_word(dev, pos + PCI_X_CMD, cap[i++]); } -static void pci_save_ltr_state(struct pci_dev *dev) -{ - int ltr; - struct pci_cap_saved_state *save_state; - u32 *cap; - - if (!pci_is_pcie(dev)) - return; - - ltr = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_LTR); - if (!ltr) - return; - - save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_LTR); - if (!save_state) { - pci_err(dev, "no suspend buffer for LTR; ASPM issues possible after resume\n"); - return; - } - - /* Some broken devices only support dword access to LTR */ - cap = &save_state->cap.data[0]; - pci_read_config_dword(dev, ltr + PCI_LTR_MAX_SNOOP_LAT, cap); -} - -static void pci_restore_ltr_state(struct pci_dev *dev) -{ - struct pci_cap_saved_state *save_state; - int ltr; - u32 *cap; - - save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_LTR); - ltr = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_LTR); - if (!save_state || !ltr) - return; - - /* Some broken devices only support dword access to LTR */ - cap = &save_state->cap.data[0]; - pci_write_config_dword(dev, ltr + PCI_LTR_MAX_SNOOP_LAT, *cap); -} - /** * pci_save_state - save the PCI configuration space of a device before * suspending diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 9aeba82facc4..ad3add45345c 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -567,6 +567,11 @@ pci_ers_result_t pcie_do_recovery(struct pci_dev *dev, bool pcie_wait_for_link(struct pci_dev *pdev, bool active); int pcie_retrain_link(struct pci_dev *pdev, bool use_lt); + +/* ASPM-related functionality we need even without CONFIG_PCIEASPM */ +void pci_save_ltr_state(struct pci_dev *dev); +void pci_restore_ltr_state(struct pci_dev *dev); + #ifdef CONFIG_PCIEASPM void pcie_aspm_init_link_state(struct pci_dev *pdev); void pcie_aspm_exit_link_state(struct pci_dev *pdev); diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c index d50c0f83430f..21731b232fb8 100644 --- a/drivers/pci/pcie/aspm.c +++ b/drivers/pci/pcie/aspm.c @@ -24,6 +24,46 @@ #include "../pci.h" +void pci_save_ltr_state(struct pci_dev *dev) +{ + int ltr; + struct pci_cap_saved_state *save_state; + u32 *cap; + + if (!pci_is_pcie(dev)) + return; + + ltr = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_LTR); + if (!ltr) + return; + + save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_LTR); + if (!save_state) { + pci_err(dev, "no suspend buffer for LTR; ASPM issues possible after resume\n"); + return; + } + + /* Some broken devices only support dword access to LTR */ + cap = &save_state->cap.data[0]; + pci_read_config_dword(dev, ltr + PCI_LTR_MAX_SNOOP_LAT, cap); +} + +void pci_restore_ltr_state(struct pci_dev *dev) +{ + struct pci_cap_saved_state *save_state; + int ltr; + u32 *cap; + + save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_LTR); + ltr = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_LTR); + if (!save_state || !ltr) + return; + + /* Some broken devices only support dword access to LTR */ + cap = &save_state->cap.data[0]; + pci_write_config_dword(dev, ltr + PCI_LTR_MAX_SNOOP_LAT, *cap); +} + #ifdef CONFIG_PCIEASPM #ifdef MODULE_PARAM_PREFIX From patchwork Fri Feb 23 20:58:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 205676 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:a81b:b0:108:e6aa:91d0 with SMTP id bq27csp841196dyb; Fri, 23 Feb 2024 13:00:47 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCW229eIx4hh+Jav9knUj8F3m8b++WtXodnAQu5ybyC4sDYA3UMYSn2l9AEcdp8lxWYl67CdMf304UCdvzkJQYisVXJ8wA== X-Google-Smtp-Source: AGHT+IGPqc0ID0CSSUIhtBmEN/FyWtdhELgtnbv28KrKrApRllSZ5kFZF2NwRJyUf8oZIMPy3+HD X-Received: by 2002:a05:6358:620e:b0:17b:6e2c:524a with SMTP id a14-20020a056358620e00b0017b6e2c524amr1312583rwh.27.1708722046660; Fri, 23 Feb 2024 13:00:46 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708722046; cv=pass; d=google.com; s=arc-20160816; b=ZerhMu3tX5zOsfBDzULjnF9l/lbrHK9wMJ+Wda1rgdS74nGQZY8fSEOWu8GSAjDtR7 fmfrLAAHKzXo4C7SX530uc4GMhsQHhwh6MVBplH3RnUK77PKLWBBL5I5eCiP/RQAlB2k zAb+aatXZbTNLN7sEODgnHhiwTn7GhU7IKQ6E7ZOt61rYpYe5cQo19JvYF7DaVaFWaFK zh+Tc3OrIbewrTI9sXHUrE/RgsQPgJZh0hBqig1DZvKuPANFX9XWd0Ip4F1ExNNCRQxs CT866JqSVnGdJd56Uq/ylN0nNZGMcTu8OCqALE4yao8oa8unFffk9BgiVQ7MYYZafWm6 NiEw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=ePRSef9HWhLf1WVvHLgT1uoyKQwKrUFdaLDM1AThHG0=; fh=1vci0BZlSGXf5Eqe4/GNV6+Gcvw1xyQibkcOu+I6FAw=; b=r7WOvRuLSl2vZGMyuD7yUss1KaiCKQPbgiNfLtRAjgI4A0ss8AJOqTkzOZRhvuMA2N 6B2yCZByuhLyuXrZpKyn372wEGub8w2yz9HoYRYRh9w0pV+BbfeaiKRbEtSjBnu1JpXh ZBtAyTxUkx52WaNSxiIqGdOWiD6QOlgpi07g3cBAo8OnnQ2wBKXu8aa/Nvg6ElDK/KDw UUJhcvk+Ks+p2lcJsgrGl0pcJyNMVEi2l0AaO9aL//3x7ph3oTEKX282fL7GzftubX+B A6WIaqOPjpkYvdFpQZusDn1u+DhoAXyk6x5vUP6XqNdA2WRBI9JFFDSHOfKt//SqcG3q lZKw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=j5aQf8va; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-79156-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-79156-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id cw11-20020a056a00450b00b006e47dac6b71si7591154pfb.299.2024.02.23.13.00.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Feb 2024 13:00:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-79156-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=j5aQf8va; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-79156-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-79156-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 4B9B52878DB for ; Fri, 23 Feb 2024 21:00:16 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DFCF814DFC5; Fri, 23 Feb 2024 20:59:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="j5aQf8va" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3138014CAB1; Fri, 23 Feb 2024 20:59:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708721942; cv=none; b=uikbhXimspTFSqMC3ZZXH1kSICM1oW1srr6z6yIa4kwl+jj62YOmp6n8JumVKGvx5DsGxI2EfJ4hIjJcdS3envpGwVIppUBAAUBY79LAx2ZVpUG8eAbE9iiMPfd4/dqZBq28tj0AoNho2JWxIluscvbICE9VRbGnqBYCos4ZXQI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708721942; c=relaxed/simple; bh=3kOdjLWSBzyCR8MnX5A46CuygIj+MZWFIthe0qZ9HaM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=hAaDlASmHetT/kiX+3EAKoD7DE58iSC34A/FvLX90oSfDV7d7LcD4x9WqQn6GrHYaKL0E/cH3f9tANNuCdJEK7aBIbbY0WSJ5+TESaMfKv/hDHq5SrKmmvbgI6kLLXZw3y0MMsDZWFuNZ5jvlZBa0ZS1JKcIu+ejxJ5tD6y86Pk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=j5aQf8va; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id C8856C43394; Fri, 23 Feb 2024 20:59:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708721942; bh=3kOdjLWSBzyCR8MnX5A46CuygIj+MZWFIthe0qZ9HaM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=j5aQf8vaFkyJd6khdYNq7kyhkQZRX4+APSt07igYODyQiN2NQ6uDjbfgJIPC+l+DG 4Eyo2zvCedHcant2K+ylNM2GDqJNb1fTG5IR8S99mP2/P41zw6aeer4ulujiaPv3SY RYqYtNqLHw/DFuTkwq6ZBe/c+dQp8PJ/bP1uzo8fDPn4XlwbY/f/kxHL4XKP5g9w9p TG04Yk2fogISZWjmKGdp4T2OUn1EHilckNcfl2ZP1/+O0gPMbY0vfMzTVFdumDBQIe jK5A/gCr/45t3byzD6f9B5T9Z/GrLfa5WqR3OLMEZ1phSrcdetxUr7MhMF+UqLl9Ta SByIQZDizsMdw== From: Bjorn Helgaas To: linux-pci@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Mika Westerberg , "David E . Box" , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , "Rafael J . Wysocki" , Tasev Nikola , Mark Enriquez , Thomas Witt , Werner Sembach , Vidya Sagar , Kai-Heng Feng , Kuppuswamy Sathyanarayanan , Ricky Wu , Mario Limonciello , Koba Ko , Bjorn Helgaas Subject: [PATCH v7 4/5] PCI/ASPM: Save L1 PM Substates Capability for suspend/resume Date: Fri, 23 Feb 2024 14:58:50 -0600 Message-Id: <20240223205851.114931-5-helgaas@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240223205851.114931-1-helgaas@kernel.org> References: <20240223205851.114931-1-helgaas@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791724929094856525 X-GMAIL-MSGID: 1791724929094856525 From: "David E. Box" 4ff116d0d5fd ("PCI/ASPM: Save L1 PM Substates Capability for suspend/resume") restored the L1 PM Substates Capability after resume, which reduced power consumption by making the ASPM L1.x states work after resume. a7152be79b62 ("Revert "PCI/ASPM: Save L1 PM Substates Capability for suspend/resume"") reverted 4ff116d0d5fd because resume failed on some systems, so power consumption after resume increased again. a7152be79b62 mentioned that we restore L1 PM substate configuration even though ASPM L1 may already be enabled. This is due the fact that the pci_restore_aspm_l1ss_state() was called before pci_restore_pcie_state(). Save and restore the L1 PM Substates Capability, following PCIe r6.1, sec 5.5.4 more closely by: 1) Do not restore ASPM configuration in pci_restore_pcie_state() but do that after PCIe capability is restored in pci_restore_aspm_state() following PCIe r6.1, sec 5.5.4. 2) If BIOS reenables L1SS, particularly L1.2, we need to clear the enables in the right order, downstream before upstream. Defer restoring the L1SS config until we are at the downstream component. Then update the config for both ends of the link in the prescribed order. 3) Program ASPM L1 PM substate configuration before L1 enables. 4) Program ASPM L1 PM substate enables last, after rest of the fields in the capability are programmed. Link: https://lore.kernel.org/r/20240128233212.1139663-3-david.e.box@linux.intel.com Link: https://lore.kernel.org/r/20240128233212.1139663-4-david.e.box@linux.intel.com Reported-by: Koba Ko Closes: https://bugzilla.kernel.org/show_bug.cgi?id=217321 Link: https://bugzilla.kernel.org/show_bug.cgi?id=216782 Link: https://bugzilla.kernel.org/show_bug.cgi?id=216877 Co-developed-by: Mika Westerberg Signed-off-by: Mika Westerberg Co-developed-by: David E. Box Signed-off-by: David E. Box [bhelgaas: commit log, squash L1SS-related patches, do both LNKCTL restores in pci_restore_pcie_state()] Signed-off-by: Bjorn Helgaas Cc: Tasev Nikola Cc: Mark Enriquez Cc: Thomas Witt Cc: Werner Sembach Cc: Vidya Sagar --- drivers/pci/pci.c | 17 ++++++- drivers/pci/pci.h | 3 ++ drivers/pci/pcie/aspm.c | 102 ++++++++++++++++++++++++++++++++++++++-- drivers/pci/probe.c | 1 + include/linux/pci.h | 2 +- 5 files changed, 119 insertions(+), 6 deletions(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 564e2cf2dde5..ca6673588bc0 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -1623,6 +1623,8 @@ static int pci_save_pcie_state(struct pci_dev *dev) pcie_capability_read_word(dev, PCI_EXP_LNKCTL2, &cap[i++]); pcie_capability_read_word(dev, PCI_EXP_SLTCTL2, &cap[i++]); + pci_save_aspm_l1ss_state(dev); + return 0; } @@ -1630,7 +1632,7 @@ static void pci_restore_pcie_state(struct pci_dev *dev) { int i = 0; struct pci_cap_saved_state *save_state; - u16 *cap; + u16 *cap, lnkctl; save_state = pci_find_saved_cap(dev, PCI_CAP_ID_EXP); if (!save_state) @@ -1645,12 +1647,23 @@ static void pci_restore_pcie_state(struct pci_dev *dev) cap = (u16 *)&save_state->cap.data[0]; pcie_capability_write_word(dev, PCI_EXP_DEVCTL, cap[i++]); - pcie_capability_write_word(dev, PCI_EXP_LNKCTL, cap[i++]); + + /* Restore LNKCTL register with ASPM control field clear */ + lnkctl = cap[i++]; + pcie_capability_write_word(dev, PCI_EXP_LNKCTL, + lnkctl & ~PCI_EXP_LNKCTL_ASPMC); + pcie_capability_write_word(dev, PCI_EXP_SLTCTL, cap[i++]); pcie_capability_write_word(dev, PCI_EXP_RTCTL, cap[i++]); pcie_capability_write_word(dev, PCI_EXP_DEVCTL2, cap[i++]); pcie_capability_write_word(dev, PCI_EXP_LNKCTL2, cap[i++]); pcie_capability_write_word(dev, PCI_EXP_SLTCTL2, cap[i++]); + + pci_restore_aspm_l1ss_state(dev); + + /* Restore ASPM control after restoring L1SS state */ + pcie_capability_set_word(dev, PCI_EXP_LNKCTL, + lnkctl & PCI_EXP_LNKCTL_ASPMC); } static int pci_save_pcix_state(struct pci_dev *dev) diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index ad3add45345c..eca5938deb07 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -571,6 +571,9 @@ int pcie_retrain_link(struct pci_dev *pdev, bool use_lt); /* ASPM-related functionality we need even without CONFIG_PCIEASPM */ void pci_save_ltr_state(struct pci_dev *dev); void pci_restore_ltr_state(struct pci_dev *dev); +void pci_configure_aspm_l1ss(struct pci_dev *dev); +void pci_save_aspm_l1ss_state(struct pci_dev *dev); +void pci_restore_aspm_l1ss_state(struct pci_dev *dev); #ifdef CONFIG_PCIEASPM void pcie_aspm_init_link_state(struct pci_dev *pdev); diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c index 21731b232fb8..977eca893b2a 100644 --- a/drivers/pci/pcie/aspm.c +++ b/drivers/pci/pcie/aspm.c @@ -64,6 +64,105 @@ void pci_restore_ltr_state(struct pci_dev *dev) pci_write_config_dword(dev, ltr + PCI_LTR_MAX_SNOOP_LAT, *cap); } +void pci_configure_aspm_l1ss(struct pci_dev *pdev) +{ + int rc; + + pdev->l1ss = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_L1SS); + + rc = pci_add_ext_cap_save_buffer(pdev, PCI_EXT_CAP_ID_L1SS, + 2 * sizeof(u32)); + if (rc) + pci_err(pdev, "unable to allocate ASPM L1SS save buffer (%pe)\n", + ERR_PTR(rc)); +} + +void pci_save_aspm_l1ss_state(struct pci_dev *pdev) +{ + struct pci_cap_saved_state *save_state; + u16 l1ss = pdev->l1ss; + u32 *cap; + + /* + * Save L1 substate configuration. The ASPM L0s/L1 configuration + * in PCI_EXP_LNKCTL_ASPMC is saved by pci_save_pcie_state(). + */ + if (!l1ss) + return; + + save_state = pci_find_saved_ext_cap(pdev, PCI_EXT_CAP_ID_L1SS); + if (!save_state) + return; + + cap = &save_state->cap.data[0]; + pci_read_config_dword(pdev, l1ss + PCI_L1SS_CTL2, cap++); + pci_read_config_dword(pdev, l1ss + PCI_L1SS_CTL1, cap++); +} + +void pci_restore_aspm_l1ss_state(struct pci_dev *pdev) +{ + struct pci_cap_saved_state *pl_save_state, *cl_save_state; + struct pci_dev *parent = pdev->bus->self; + u32 *cap, pl_ctl1, pl_ctl2, pl_l1_2_enable; + u32 cl_ctl1, cl_ctl2, cl_l1_2_enable; + + /* + * In case BIOS enabled L1.2 when resuming, we need to disable it first + * on the downstream component before the upstream. So, don't attempt to + * restore either until we are at the downstream component. + */ + if (pcie_downstream_port(pdev) || !parent) + return; + + if (!pdev->l1ss || !parent->l1ss) + return; + + cl_save_state = pci_find_saved_ext_cap(pdev, PCI_EXT_CAP_ID_L1SS); + pl_save_state = pci_find_saved_ext_cap(parent, PCI_EXT_CAP_ID_L1SS); + if (!cl_save_state || !pl_save_state) + return; + + cap = &cl_save_state->cap.data[0]; + cl_ctl2 = *cap++; + cl_ctl1 = *cap; + cap = &pl_save_state->cap.data[0]; + pl_ctl2 = *cap++; + pl_ctl1 = *cap; + + /* + * Disable L1.2 on this downstream endpoint device first, followed + * by the upstream + */ + pci_clear_and_set_config_dword(pdev, pdev->l1ss + PCI_L1SS_CTL1, + PCI_L1SS_CTL1_L1_2_MASK, 0); + pci_clear_and_set_config_dword(parent, parent->l1ss + PCI_L1SS_CTL1, + PCI_L1SS_CTL1_L1_2_MASK, 0); + + /* + * In addition, Common_Mode_Restore_Time and LTR_L1.2_THRESHOLD + * in PCI_L1SS_CTL1 must be programmed *before* setting the L1.2 + * enable bits, even though they're all in PCI_L1SS_CTL1. + */ + pl_l1_2_enable = pl_ctl1 & PCI_L1SS_CTL1_L1_2_MASK; + pl_ctl1 &= ~PCI_L1SS_CTL1_L1_2_MASK; + cl_l1_2_enable = cl_ctl1 & PCI_L1SS_CTL1_L1_2_MASK; + cl_ctl1 &= ~PCI_L1SS_CTL1_L1_2_MASK; + + /* Write back without enables first (above we cleared them in ctl1) */ + pci_write_config_dword(parent, parent->l1ss + PCI_L1SS_CTL2, pl_ctl2); + pci_write_config_dword(pdev, pdev->l1ss + PCI_L1SS_CTL2, cl_ctl2); + pci_write_config_dword(parent, parent->l1ss + PCI_L1SS_CTL1, pl_ctl1); + pci_write_config_dword(pdev, pdev->l1ss + PCI_L1SS_CTL1, cl_ctl1); + + /* Then write back the enables */ + if (pl_l1_2_enable || cl_l1_2_enable) { + pci_write_config_dword(parent, parent->l1ss + PCI_L1SS_CTL1, + pl_ctl1 | pl_l1_2_enable); + pci_write_config_dword(pdev, pdev->l1ss + PCI_L1SS_CTL1, + cl_ctl1 | cl_l1_2_enable); + } +} + #ifdef CONFIG_PCIEASPM #ifdef MODULE_PARAM_PREFIX @@ -1005,9 +1104,6 @@ void pci_configure_ltr(struct pci_dev *pdev) if (!pci_is_pcie(pdev)) return; - /* Read L1 PM substate capabilities */ - pdev->l1ss = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_L1SS); - pcie_capability_read_dword(pdev, PCI_EXP_DEVCAP2, &cap); if (!(cap & PCI_EXP_DEVCAP2_LTR)) return; diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index b809c0b0e0e5..1434bf495db3 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -2259,6 +2259,7 @@ static void pci_configure_device(struct pci_dev *dev) pci_configure_extended_tags(dev, NULL); pci_configure_relaxed_ordering(dev); pci_configure_ltr(dev); + pci_configure_aspm_l1ss(dev); pci_configure_eetlp_prefix(dev); pci_configure_serr(dev); diff --git a/include/linux/pci.h b/include/linux/pci.h index add9368e6314..6967ae7b4115 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -390,9 +390,9 @@ struct pci_dev { unsigned int d3hot_delay; /* D3hot->D0 transition time in ms */ unsigned int d3cold_delay; /* D3cold->D0 transition time in ms */ + u16 l1ss; /* L1SS Capability pointer */ #ifdef CONFIG_PCIEASPM struct pcie_link_state *link_state; /* ASPM link state */ - u16 l1ss; /* L1SS Capability pointer */ unsigned int ltr_path:1; /* Latency Tolerance Reporting supported from root to here */ #endif From patchwork Fri Feb 23 20:58:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 205677 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:a81b:b0:108:e6aa:91d0 with SMTP id bq27csp841216dyb; Fri, 23 Feb 2024 13:00:48 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVCvStvlgfGGqd05tJfNHVpVSOTLc7RpXggqbs83Oo1V6+DN78EVdTU6FizMYTfKL0zvYc4kX0NwJdAzK6cVHry4BVaVA== X-Google-Smtp-Source: AGHT+IHszyDY6ujEIHVou5y/g2dyQFuOGX5HHGderjlotaBBXZA3MXT8zUY6Lmm/NgcmLsaYRzm3 X-Received: by 2002:a05:6871:5220:b0:21e:b695:dff7 with SMTP id ht32-20020a056871522000b0021eb695dff7mr1037118oac.32.1708722047836; Fri, 23 Feb 2024 13:00:47 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708722047; cv=pass; d=google.com; s=arc-20160816; b=GqtwQAGIVGgKIu09OEPGdqKcmCv//md9j7aMQgEsDcHkR69d14tvdW4SHVOCG0pE8u Tc1oWLj32l39HM/vYALAYFZ00vD+Btndcg45MrsjJIx2J1HOdQ/u3ZPW62OG7ZZg8fx1 5t0OGY9rIrnJOuWZ4lK5d/g05zsX+izM6ngLDf+GcR43X7/81XFgL7XnrUW1J5StHkTF xwS62IGy3ELGN3v/hne962m4E3pAm1BkACsTc8mwG3sLdDBbvbePW0sKNxjyWhLjTcBJ uDMOmNX2CHuuX1HOWc1duP92MdsCXGR2v70y+j40Z20sNXLZVU7sACcxAE09LY3g/LvD /urA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=JLs2FUfoKUOY8qleSDWIlenwNmJhC61jwzTRF/9aL7A=; fh=UOtCQvdtLRjYu4lpt5Tw8oOZkSAAXsqpAt775wBcPRQ=; b=mIHZ7Msy1L1RALuMrN/sGV7qtxLAyZUGaMFUf3BNwhiflnW7Saiyg/c6/4yi8dEfVT nTvXzpeGe5L8P4uTbgmrC1oaNUFwRIhtBZ2YblYC9N91CYejn9h7glD6wJSlO1PfnZYH WuoUbgBpOYk3xZHzu6cmARkCRvYAXFcnmfntKEBNF5FrK/ceEJNDGtxcd8Xw/5eYSV0B gTmgcrORK7rUrDeq+MJ3vph8FbF8em/9dBbpl2gI9ChA5EfZtcpmP0L/vfRsfa1uE4wE tJHLAIlJUTFroWij4mWwvxUvwmWsN87rfC9BK58tbt3ltVAVuUHWSE6IaDKOEAKNLmM9 YQVQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=H9Bwwffo; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-79157-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-79157-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id w14-20020a05620a444e00b00787bed7b3dfsi246719qkp.266.2024.02.23.13.00.47 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Feb 2024 13:00:47 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-79157-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=H9Bwwffo; arc=pass (i=1 dkim=pass dkdomain=kernel.org); spf=pass (google.com: domain of linux-kernel+bounces-79157-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-79157-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 164901C2085B for ; Fri, 23 Feb 2024 21:00:22 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D493314DFD1; Fri, 23 Feb 2024 20:59:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="H9Bwwffo" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 505D9146E93; Fri, 23 Feb 2024 20:59:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708721944; cv=none; b=ZxkPK1xRMa08eBfCGLpj9B3Y/fKZsuvfaXvG+0NnE6GLxWkqIGDJDIagRJ1TKehIaMNGQz/Hjv3yUFRiLKS+12bPZS7t1O+3oyBMcm6jKdFwxlQIsJYRS4VVvLh9fNLF15Fu1koEmLKt2v+KvPjnKbojjxAC0jId9fsZYwfQQwQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708721944; c=relaxed/simple; bh=edfl9Qyi1j5BIY4+eq7G/coY2cxhe7prrDOpLepU/Gc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=b4FFaGST6nowmUcelnFLewqAqr9FRytW5lQhm1Vxpo6Pz9Vrq5y3kQXUwo4gVUD4E8hBNxoZJjED79joxrMcZdwQ/W2f/6CpR8vL4ovw6anh8EmR0iO+OocnGVzL1o+jjiqGOx65maqDBEsGdlyDlWqabAEevbEglOYkJCzAuL4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=H9Bwwffo; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8DD5AC433F1; Fri, 23 Feb 2024 20:59:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708721943; bh=edfl9Qyi1j5BIY4+eq7G/coY2cxhe7prrDOpLepU/Gc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=H9BwwffoTSjhsPYDQ27tETDYUNRORa092Hi0lrS0GK2AQ4fPwZc4yjjpLDR0h5pxj 1HEhjHR676WDULoG70M4YdPrupwn0L+u7f0cCDfKk3gfHfGovrlQ/tH0FfPDu5v19s JFDIb3HTNJVSew7BBpI8ax7IhEzoD8m+FEMyxCV+tQk/smk2HUHXD/WIJq2oLW/wGA 8QwYBQIdUClTMwt+3qUjGSMmjfkx8kS/Tja9/obW9fDtXWeUhQQ5bMNNpDNX0gS3DK qfaLOmrQIPnCeX60xWnafWeu9zLg5Zuf9AhHzYbl8k8PWIeFua85zpfedahqUHR7h4 wJk6tczGAoX5A== From: Bjorn Helgaas To: linux-pci@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Mika Westerberg , "David E . Box" , =?utf-8?q?Ilpo_J=C3=A4rvinen?= , "Rafael J . Wysocki" , Tasev Nikola , Mark Enriquez , Thomas Witt , Werner Sembach , Vidya Sagar , Kai-Heng Feng , Kuppuswamy Sathyanarayanan , Ricky Wu , Mario Limonciello , Bjorn Helgaas Subject: [PATCH v7 5/5] PCI/ASPM: Call pci_save_ltr_state() from pci_save_pcie_state() Date: Fri, 23 Feb 2024 14:58:51 -0600 Message-Id: <20240223205851.114931-6-helgaas@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240223205851.114931-1-helgaas@kernel.org> References: <20240223205851.114931-1-helgaas@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791724929789401428 X-GMAIL-MSGID: 1791724929789401428 From: "David E. Box" ASPM state is saved and restored from pci_save/restore_pcie_state(). Since the LTR Capability is linked with ASPM, move the LTR save and restore calls there as well. No functional change intended. Suggested-by: Bjorn Helgaas Link: https://lore.kernel.org/r/20240128233212.1139663-6-david.e.box@linux.intel.com Signed-off-by: David E. Box Signed-off-by: Bjorn Helgaas --- drivers/pci/pci.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index ca6673588bc0..4ea98665172d 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -1624,6 +1624,7 @@ static int pci_save_pcie_state(struct pci_dev *dev) pcie_capability_read_word(dev, PCI_EXP_SLTCTL2, &cap[i++]); pci_save_aspm_l1ss_state(dev); + pci_save_ltr_state(dev); return 0; } @@ -1634,6 +1635,12 @@ static void pci_restore_pcie_state(struct pci_dev *dev) struct pci_cap_saved_state *save_state; u16 *cap, lnkctl; + /* + * Restore max latencies (in the LTR capability) before enabling + * LTR itself in PCI_EXP_DEVCTL2. + */ + pci_restore_ltr_state(dev); + save_state = pci_find_saved_cap(dev, PCI_CAP_ID_EXP); if (!save_state) return; @@ -1726,7 +1733,6 @@ int pci_save_state(struct pci_dev *dev) if (i != 0) return i; - pci_save_ltr_state(dev); pci_save_dpc_state(dev); pci_save_aer_state(dev); pci_save_ptm_state(dev); @@ -1827,12 +1833,6 @@ void pci_restore_state(struct pci_dev *dev) if (!dev->state_saved) return; - /* - * Restore max latencies (in the LTR capability) before enabling - * LTR itself (in the PCIe capability). - */ - pci_restore_ltr_state(dev); - pci_restore_pcie_state(dev); pci_restore_pasid_state(dev); pci_restore_pri_state(dev);