Message ID | 20231114135553.32301-2-johan+linaro@kernel.org |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b909:0:b0:403:3b70:6f57 with SMTP id t9csp1874317vqg; Tue, 14 Nov 2023 05:57:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IEAQ2CPcphq3X/RSvqdZmec1p5wGSoCXEgJJKIuOfq2mtOMInnrNCUL1u0ASKP8B14Xnjg2 X-Received: by 2002:a17:903:11d0:b0:1c9:faef:5765 with SMTP id q16-20020a17090311d000b001c9faef5765mr2510926plh.5.1699970254286; Tue, 14 Nov 2023 05:57:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1699970254; cv=none; d=google.com; s=arc-20160816; b=fNVvzVJ7wc5L8sH3eIcKVHoawCIH+aHzX4SiYL0QgEC0mTZZCnbjGF+a0v4sxcH4V1 kMaDkTC7g5FUkJACB2jIq2NM8444MBishxlx2goS0Ae26IDqJVpOgdhBh/EhM48Mf7XH dFeVrLdEZ/NZ1ZzKW/AOhU+i1ae4ZDhOffBNezNeQ7HuvUQzGLmYfqdjv4bbHFpbcqMP AtF5yAQPes33lzJeVxlMwnO34j9GTiA/xCh7o3gTDeJ4SUBNZa1TMyO0yOc160XDpjok E6pFA7HFMEVBw3D/VuKGvsCaMO9zHUzZdlzCGd5E6e7BokIE6tmgJiJvM07xGbtgrUL8 IP3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=YueWTewMGbnrFXUzrhpqJHbTrRgDyShyMiheMzeYBiw=; fh=f41aoNjmAiOEyLMnqyIyK2t7KtKRMELjVBNz1UXAmhs=; b=cnsz3gV+9f1RcRcQvLY9NfnOvCZv14UitVAgP5uE58tn8Y08cZPdGnY1xkXPekEMrv w7RF5nNN2JngBW8tRvjRO3/E0LTyWdvGa3YKg56sUVXNhNVOqcDGP4SIjaAVtjUZe3SZ FJuGMQZowhssYbRfOYYGyHf5D8k3CcHDlPuQzge3Gt+QQ6xgreYxO6r/xIqvHnYApnrb uiEwvRXUvra26IsEsqYby8LfbQjqUJBupEOfVLcm6dAiJtrCyUwxyoRWUF+yNMolhcUo DJJ+n8sZ0Nb0tiCbvNpai81AgGzoH0J0ONPCUywXIUiOaZ4X3OER5Hk1WIcz9RqOefPW d+4A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=n1pE7bt2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id jb20-20020a170903259400b001c9fb3b55f0si7659119plb.652.2023.11.14.05.57.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Nov 2023 05:57:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=n1pE7bt2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id BEC5580B8E68; Tue, 14 Nov 2023 05:56:58 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233319AbjKNN4w (ORCPT <rfc822;lhua1029@gmail.com> + 29 others); Tue, 14 Nov 2023 08:56:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49412 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232040AbjKNN4j (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Tue, 14 Nov 2023 08:56:39 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E195D4B; Tue, 14 Nov 2023 05:56:34 -0800 (PST) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A7492C433CB; Tue, 14 Nov 2023 13:56:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1699970193; bh=FQzhhHnHVLuWZguWtCeMQCQce7lbuXpiz5kGYt0gjTI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=n1pE7bt2zjsr5fCQayJMtmeTKpYw9ejeIOJCj/heI2mhqOc4YBsxFuhsLK+Z0XG+t 0QKJTmjGJ4PaHrBdPzL0hZcCIk7JFunIlKryWjb7YTLRiCvwqoswcUDcuU7YBv0L+H sTlTAotjjcfJksXzZkXrnv/X3RdErT7VeA9i59wGYjUzn3jWk+D9dpdpGXuJLNdTFJ 3O4FlL+vzkOq/eNVeLSwpLf7Of4OtRj4G3mIr2e3K0Ka6SdSmihQHkZvLiLChsBsh0 1FzXuFEbjsvyJnvt7aOiWo6qfWrq/w1BDy1RM1BGa9gSZDy1WHd9SUKWL1mF1iz12e XoivxJP+BzVfg== Received: from johan by xi.lan with local (Exim 4.96.2) (envelope-from <johan+linaro@kernel.org>) id 1r2tu2-0008PP-24; Tue, 14 Nov 2023 14:56:30 +0100 From: Johan Hovold <johan+linaro@kernel.org> To: Lorenzo Pieralisi <lpieralisi@kernel.org>, =?utf-8?q?Krzysztof_Wilczy?= =?utf-8?q?=C5=84ski?= <kw@linux.com>, Bjorn Helgaas <bhelgaas@google.com> Cc: Andy Gross <agross@kernel.org>, Bjorn Andersson <andersson@kernel.org>, Konrad Dybcio <konrad.dybcio@linaro.org>, Manivannan Sadhasivam <mani@kernel.org>, Rob Herring <robh@kernel.org>, Nirmal Patel <nirmal.patel@linux.intel.com>, Jonathan Derrick <jonathan.derrick@linux.dev>, linux-arm-msm@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Johan Hovold <johan+linaro@kernel.org>, stable@vger.kernel.org, Michael Bottini <michael.a.bottini@linux.intel.com>, "David E . Box" <david.e.box@linux.intel.com> Subject: [PATCH 1/6] PCI/ASPM: Add locked helper for enabling link state Date: Tue, 14 Nov 2023 14:55:48 +0100 Message-ID: <20231114135553.32301-2-johan+linaro@kernel.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231114135553.32301-1-johan+linaro@kernel.org> References: <20231114135553.32301-1-johan+linaro@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 14 Nov 2023 05:56:58 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1782548009203709558 X-GMAIL-MSGID: 1782548009203709558 |
Series |
PCI: Fix deadlocks when enabling ASPM
|
|
Commit Message
Johan Hovold
Nov. 14, 2023, 1:55 p.m. UTC
Add a helper for enabling link states that can be used in contexts where
a pci_bus_sem read lock is already held (e.g. from pci_walk_bus()).
This helper will be used to fix a couple of potential deadlocks where
the current helper is called with the lock already held, hence the CC
stable tag.
Fixes: f492edb40b54 ("PCI: vmd: Add quirk to configure PCIe ASPM and LTR")
Cc: stable@vger.kernel.org # 6.3
Cc: Michael Bottini <michael.a.bottini@linux.intel.com>
Cc: David E. Box <david.e.box@linux.intel.com>
Signed-off-by: Johan Hovold <johan+linaro@kernel.org>
---
drivers/pci/pcie/aspm.c | 53 +++++++++++++++++++++++++++++++----------
include/linux/pci.h | 3 +++
2 files changed, 43 insertions(+), 13 deletions(-)
Comments
Hi Johan, kernel test robot noticed the following build warnings: [auto build test WARNING on pci/next] [also build test WARNING on pci/for-linus linus/master v6.7-rc1 next-20231115] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch#_base_tree_information] url: https://github.com/intel-lab-lkp/linux/commits/Johan-Hovold/PCI-ASPM-Add-locked-helper-for-enabling-link-state/20231114-220117 base: https://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git next patch link: https://lore.kernel.org/r/20231114135553.32301-2-johan%2Blinaro%40kernel.org patch subject: [PATCH 1/6] PCI/ASPM: Add locked helper for enabling link state config: x86_64-randconfig-002-20231115 (https://download.01.org/0day-ci/archive/20231115/202311152225.ZdVv0ufT-lkp@intel.com/config) compiler: gcc-12 (Debian 12.2.0-14) 12.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231115/202311152225.ZdVv0ufT-lkp@intel.com/reproduce) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp@intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202311152225.ZdVv0ufT-lkp@intel.com/ All warnings (new ones prefixed by >>): >> drivers/pci/pcie/aspm.c:1186: warning: expecting prototype for pci_enable_link_state(). Prototype was for pci_enable_link_state_locked() instead vim +1186 drivers/pci/pcie/aspm.c 1172 1173 /** 1174 * pci_enable_link_state - Clear and set the default device link state so that 1175 * the link may be allowed to enter the specified states. Note that if the 1176 * BIOS didn't grant ASPM control to the OS, this does nothing because we can't 1177 * touch the LNKCTL register. Also note that this does not enable states 1178 * disabled by pci_disable_link_state(). Return 0 or a negative errno. 1179 * 1180 * @pdev: PCI device 1181 * @state: Mask of ASPM link states to enable 1182 * 1183 * Context: Caller holds pci_bus_sem read lock. 1184 */ 1185 int pci_enable_link_state_locked(struct pci_dev *pdev, int state) > 1186 { 1187 lockdep_assert_held_read(&pci_bus_sem); 1188 1189 return __pci_enable_link_state(pdev, state, true); 1190 } 1191 EXPORT_SYMBOL(pci_enable_link_state_locked); 1192
On Wed, Nov 15, 2023 at 10:26:40PM +0800, kernel test robot wrote: > >> drivers/pci/pcie/aspm.c:1186: warning: expecting prototype for pci_enable_link_state(). Prototype was for pci_enable_link_state_locked() instead > > > vim +1186 drivers/pci/pcie/aspm.c > > 1172 > 1173 /** > 1174 * pci_enable_link_state - Clear and set the default device link state so that I apparently forgot to update the name here to pci_enable_link_state_locked [ ... ] > 1184 */ > 1185 int pci_enable_link_state_locked(struct pci_dev *pdev, int state) Perhaps this can be fixed up when applying, or otherwise I'll respin. Johan
On Wed, Nov 15, 2023 at 05:06:23PM +0100, Johan Hovold wrote: > On Wed, Nov 15, 2023 at 10:26:40PM +0800, kernel test robot wrote: > > > >> drivers/pci/pcie/aspm.c:1186: warning: expecting prototype for pci_enable_link_state(). Prototype was for pci_enable_link_state_locked() instead > > > > > > vim +1186 drivers/pci/pcie/aspm.c > > > > 1172 > > 1173 /** > > 1174 * pci_enable_link_state - Clear and set the default device link state so that > > I apparently forgot to update the name here to > > pci_enable_link_state_locked > While fixing the name, please add locking information in description as well. - Mani > [ ... ] > > > 1184 */ > > 1185 int pci_enable_link_state_locked(struct pci_dev *pdev, int state) > > Perhaps this can be fixed up when applying, or otherwise I'll respin. > > Johan
On Tue, Nov 14, 2023 at 02:55:48PM +0100, Johan Hovold wrote: > Add a helper for enabling link states that can be used in contexts where > a pci_bus_sem read lock is already held (e.g. from pci_walk_bus()). > > This helper will be used to fix a couple of potential deadlocks where > the current helper is called with the lock already held, hence the CC > stable tag. > > Fixes: f492edb40b54 ("PCI: vmd: Add quirk to configure PCIe ASPM and LTR") > Cc: stable@vger.kernel.org # 6.3 > Cc: Michael Bottini <michael.a.bottini@linux.intel.com> > Cc: David E. Box <david.e.box@linux.intel.com> > Signed-off-by: Johan Hovold <johan+linaro@kernel.org> With the Kdoc comment fixed, Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> - Mani > --- > drivers/pci/pcie/aspm.c | 53 +++++++++++++++++++++++++++++++---------- > include/linux/pci.h | 3 +++ > 2 files changed, 43 insertions(+), 13 deletions(-) > > diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c > index 50b04ae5c394..8cf8cc2d6bba 100644 > --- a/drivers/pci/pcie/aspm.c > +++ b/drivers/pci/pcie/aspm.c > @@ -1109,17 +1109,7 @@ int pci_disable_link_state(struct pci_dev *pdev, int state) > } > EXPORT_SYMBOL(pci_disable_link_state); > > -/** > - * pci_enable_link_state - Clear and set the default device link state so that > - * the link may be allowed to enter the specified states. Note that if the > - * BIOS didn't grant ASPM control to the OS, this does nothing because we can't > - * touch the LNKCTL register. Also note that this does not enable states > - * disabled by pci_disable_link_state(). Return 0 or a negative errno. > - * > - * @pdev: PCI device > - * @state: Mask of ASPM link states to enable > - */ > -int pci_enable_link_state(struct pci_dev *pdev, int state) > +static int __pci_enable_link_state(struct pci_dev *pdev, int state, bool locked) > { > struct pcie_link_state *link = pcie_aspm_get_link(pdev); > > @@ -1136,7 +1126,8 @@ int pci_enable_link_state(struct pci_dev *pdev, int state) > return -EPERM; > } > > - down_read(&pci_bus_sem); > + if (!locked) > + down_read(&pci_bus_sem); > mutex_lock(&aspm_lock); > link->aspm_default = 0; > if (state & PCIE_LINK_STATE_L0S) > @@ -1157,12 +1148,48 @@ int pci_enable_link_state(struct pci_dev *pdev, int state) > link->clkpm_default = (state & PCIE_LINK_STATE_CLKPM) ? 1 : 0; > pcie_set_clkpm(link, policy_to_clkpm_state(link)); > mutex_unlock(&aspm_lock); > - up_read(&pci_bus_sem); > + if (!locked) > + up_read(&pci_bus_sem); > > return 0; > } > + > +/** > + * pci_enable_link_state - Clear and set the default device link state so that > + * the link may be allowed to enter the specified states. Note that if the > + * BIOS didn't grant ASPM control to the OS, this does nothing because we can't > + * touch the LNKCTL register. Also note that this does not enable states > + * disabled by pci_disable_link_state(). Return 0 or a negative errno. > + * > + * @pdev: PCI device > + * @state: Mask of ASPM link states to enable > + */ > +int pci_enable_link_state(struct pci_dev *pdev, int state) > +{ > + return __pci_enable_link_state(pdev, state, false); > +} > EXPORT_SYMBOL(pci_enable_link_state); > > +/** > + * pci_enable_link_state - Clear and set the default device link state so that > + * the link may be allowed to enter the specified states. Note that if the > + * BIOS didn't grant ASPM control to the OS, this does nothing because we can't > + * touch the LNKCTL register. Also note that this does not enable states > + * disabled by pci_disable_link_state(). Return 0 or a negative errno. > + * > + * @pdev: PCI device > + * @state: Mask of ASPM link states to enable > + * > + * Context: Caller holds pci_bus_sem read lock. > + */ > +int pci_enable_link_state_locked(struct pci_dev *pdev, int state) > +{ > + lockdep_assert_held_read(&pci_bus_sem); > + > + return __pci_enable_link_state(pdev, state, true); > +} > +EXPORT_SYMBOL(pci_enable_link_state_locked); > + > static int pcie_aspm_set_policy(const char *val, > const struct kernel_param *kp) > { > diff --git a/include/linux/pci.h b/include/linux/pci.h > index 60ca768bc867..dea043bc1e38 100644 > --- a/include/linux/pci.h > +++ b/include/linux/pci.h > @@ -1829,6 +1829,7 @@ extern bool pcie_ports_native; > int pci_disable_link_state(struct pci_dev *pdev, int state); > int pci_disable_link_state_locked(struct pci_dev *pdev, int state); > int pci_enable_link_state(struct pci_dev *pdev, int state); > +int pci_enable_link_state_locked(struct pci_dev *pdev, int state); > void pcie_no_aspm(void); > bool pcie_aspm_support_enabled(void); > bool pcie_aspm_enabled(struct pci_dev *pdev); > @@ -1839,6 +1840,8 @@ static inline int pci_disable_link_state_locked(struct pci_dev *pdev, int state) > { return 0; } > static inline int pci_enable_link_state(struct pci_dev *pdev, int state) > { return 0; } > +static inline int pci_enable_link_state_locked(struct pci_dev *pdev, int state) > +{ return 0; } > static inline void pcie_no_aspm(void) { } > static inline bool pcie_aspm_support_enabled(void) { return false; } > static inline bool pcie_aspm_enabled(struct pci_dev *pdev) { return false; } > -- > 2.41.0 >
On Fri, Nov 17, 2023 at 03:57:24PM +0530, Manivannan Sadhasivam wrote: > On Wed, Nov 15, 2023 at 05:06:23PM +0100, Johan Hovold wrote: > > On Wed, Nov 15, 2023 at 10:26:40PM +0800, kernel test robot wrote: > > > > > >> drivers/pci/pcie/aspm.c:1186: warning: expecting prototype for pci_enable_link_state(). Prototype was for pci_enable_link_state_locked() instead > > > > > > > > > vim +1186 drivers/pci/pcie/aspm.c > > > > > > 1172 > > > 1173 /** > > > 1174 * pci_enable_link_state - Clear and set the default device link state so that > > > > I apparently forgot to update the name here to > > > > pci_enable_link_state_locked > > > > While fixing the name, please add locking information in description as well. It's already there: Context: Caller holds pci_bus_sem read lock. Johan
On Fri, Nov 17, 2023 at 11:30:39AM +0100, Johan Hovold wrote: > On Fri, Nov 17, 2023 at 03:57:24PM +0530, Manivannan Sadhasivam wrote: > > On Wed, Nov 15, 2023 at 05:06:23PM +0100, Johan Hovold wrote: > > > On Wed, Nov 15, 2023 at 10:26:40PM +0800, kernel test robot wrote: > > > > > > > >> drivers/pci/pcie/aspm.c:1186: warning: expecting prototype for pci_enable_link_state(). Prototype was for pci_enable_link_state_locked() instead > > > > > > > > > > > > vim +1186 drivers/pci/pcie/aspm.c > > > > > > > > 1172 > > > > 1173 /** > > > > 1174 * pci_enable_link_state - Clear and set the default device link state so that > > > > > > I apparently forgot to update the name here to > > > > > > pci_enable_link_state_locked > > > > > > > While fixing the name, please add locking information in description as well. > > It's already there: > > Context: Caller holds pci_bus_sem read lock. Ah, I was looking for it in the description. Fine then! - Mani > > Johan
On Tue, 2023-11-14 at 14:55 +0100, Johan Hovold wrote: > Add a helper for enabling link states that can be used in contexts where > a pci_bus_sem read lock is already held (e.g. from pci_walk_bus()). > > This helper will be used to fix a couple of potential deadlocks where > the current helper is called with the lock already held, hence the CC > stable tag. Thanks for sending a patch to address this. It had fallen off my radar. > > Fixes: f492edb40b54 ("PCI: vmd: Add quirk to configure PCIe ASPM and LTR") > Cc: stable@vger.kernel.org # 6.3 > Cc: Michael Bottini <michael.a.bottini@linux.intel.com> > Cc: David E. Box <david.e.box@linux.intel.com> > Signed-off-by: Johan Hovold <johan+linaro@kernel.org> > --- > drivers/pci/pcie/aspm.c | 53 +++++++++++++++++++++++++++++++---------- > include/linux/pci.h | 3 +++ > 2 files changed, 43 insertions(+), 13 deletions(-) > > diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c > index 50b04ae5c394..8cf8cc2d6bba 100644 > --- a/drivers/pci/pcie/aspm.c > +++ b/drivers/pci/pcie/aspm.c > @@ -1109,17 +1109,7 @@ int pci_disable_link_state(struct pci_dev *pdev, int > state) > } > EXPORT_SYMBOL(pci_disable_link_state); > > -/** > - * pci_enable_link_state - Clear and set the default device link state so > that > - * the link may be allowed to enter the specified states. Note that if the > - * BIOS didn't grant ASPM control to the OS, this does nothing because we > can't > - * touch the LNKCTL register. Also note that this does not enable states > - * disabled by pci_disable_link_state(). Return 0 or a negative errno. > - * > - * @pdev: PCI device > - * @state: Mask of ASPM link states to enable > - */ > -int pci_enable_link_state(struct pci_dev *pdev, int state) > +static int __pci_enable_link_state(struct pci_dev *pdev, int state, bool > locked) > { > struct pcie_link_state *link = pcie_aspm_get_link(pdev); > > @@ -1136,7 +1126,8 @@ int pci_enable_link_state(struct pci_dev *pdev, int > state) > return -EPERM; > } > > - down_read(&pci_bus_sem); > + if (!locked) > + down_read(&pci_bus_sem); This solution is similar to the original one I proposed [1]. It just creates a separate locked function rather than using a flag. While the API is consistent with pci_disable_link_state_locked(), its usage is not. The vmd driver calls pci_enable_link_state() from pci_walk_bus() which was problematic for Bjorn [2]. I owed an attempt to implement this as a fixup. However, now qcom_pcie_enabled_aspm() is also using it from pci_walk_bus(). The fixup could make sense for VMD since there it's fixing missing BIOS settings, but I'm not sure about qcom. Bjorn, do you still see issues with the use in pci_bus_walk()? [1] https://lore.kernel.org/lkml/20230321233849.3408339-1-david.e.box@linux.intel.com/ [2] https://lore.kernel.org/lkml/20230322205702.GA2493123@bhelgaas/ David > mutex_lock(&aspm_lock); > link->aspm_default = 0; > if (state & PCIE_LINK_STATE_L0S) > @@ -1157,12 +1148,48 @@ int pci_enable_link_state(struct pci_dev *pdev, int > state) > link->clkpm_default = (state & PCIE_LINK_STATE_CLKPM) ? 1 : 0; > pcie_set_clkpm(link, policy_to_clkpm_state(link)); > mutex_unlock(&aspm_lock); > - up_read(&pci_bus_sem); > + if (!locked) > + up_read(&pci_bus_sem); > > return 0; > } > + > +/** > + * pci_enable_link_state - Clear and set the default device link state so > that > + * the link may be allowed to enter the specified states. Note that if the > + * BIOS didn't grant ASPM control to the OS, this does nothing because we > can't > + * touch the LNKCTL register. Also note that this does not enable states > + * disabled by pci_disable_link_state(). Return 0 or a negative errno. > + * > + * @pdev: PCI device > + * @state: Mask of ASPM link states to enable > + */ > +int pci_enable_link_state(struct pci_dev *pdev, int state) > +{ > + return __pci_enable_link_state(pdev, state, false); > +} > EXPORT_SYMBOL(pci_enable_link_state); > > +/** > + * pci_enable_link_state - Clear and set the default device link state so > that > + * the link may be allowed to enter the specified states. Note that if the > + * BIOS didn't grant ASPM control to the OS, this does nothing because we > can't > + * touch the LNKCTL register. Also note that this does not enable states > + * disabled by pci_disable_link_state(). Return 0 or a negative errno. > + * > + * @pdev: PCI device > + * @state: Mask of ASPM link states to enable > + * > + * Context: Caller holds pci_bus_sem read lock. > + */ > +int pci_enable_link_state_locked(struct pci_dev *pdev, int state) > +{ > + lockdep_assert_held_read(&pci_bus_sem); > + > + return __pci_enable_link_state(pdev, state, true); > +} > +EXPORT_SYMBOL(pci_enable_link_state_locked); > + > static int pcie_aspm_set_policy(const char *val, > const struct kernel_param *kp) > { > diff --git a/include/linux/pci.h b/include/linux/pci.h > index 60ca768bc867..dea043bc1e38 100644 > --- a/include/linux/pci.h > +++ b/include/linux/pci.h > @@ -1829,6 +1829,7 @@ extern bool pcie_ports_native; > int pci_disable_link_state(struct pci_dev *pdev, int state); > int pci_disable_link_state_locked(struct pci_dev *pdev, int state); > int pci_enable_link_state(struct pci_dev *pdev, int state); > +int pci_enable_link_state_locked(struct pci_dev *pdev, int state); > void pcie_no_aspm(void); > bool pcie_aspm_support_enabled(void); > bool pcie_aspm_enabled(struct pci_dev *pdev); > @@ -1839,6 +1840,8 @@ static inline int pci_disable_link_state_locked(struct > pci_dev *pdev, int state) > { return 0; } > static inline int pci_enable_link_state(struct pci_dev *pdev, int state) > { return 0; } > +static inline int pci_enable_link_state_locked(struct pci_dev *pdev, int > state) > +{ return 0; } > static inline void pcie_no_aspm(void) { } > static inline bool pcie_aspm_support_enabled(void) { return false; } > static inline bool pcie_aspm_enabled(struct pci_dev *pdev) { return false; }
On Fri, Nov 17, 2023 at 04:00:46PM -0800, David E. Box wrote: > On Tue, 2023-11-14 at 14:55 +0100, Johan Hovold wrote: > > Add a helper for enabling link states that can be used in contexts where > > a pci_bus_sem read lock is already held (e.g. from pci_walk_bus()). > > > > This helper will be used to fix a couple of potential deadlocks where > > the current helper is called with the lock already held, hence the CC > > stable tag. > This solution is similar to the original one I proposed [1]. It just creates a > separate locked function rather than using a flag. While the API is consistent > with pci_disable_link_state_locked(), its usage is not. The vmd driver calls > pci_enable_link_state() from pci_walk_bus() which was problematic for Bjorn [2]. > I owed an attempt to implement this as a fixup. However, now > qcom_pcie_enabled_aspm() is also using it from pci_walk_bus(). The fixup could > make sense for VMD since there it's fixing missing BIOS settings, but I'm not > sure about qcom. Bjorn, do you still see issues with the use in pci_bus_walk()? We need this for Qualcomm platforms where the boot firmware does not enable ASPM, and the hotplug inconsistency was also discussed with Bjorn here: https://lore.kernel.org/linux-pci/20231018164731.GA1365588@bhelgaas/ > [1] > https://lore.kernel.org/lkml/20230321233849.3408339-1-david.e.box@linux.intel.com/ > > [2] https://lore.kernel.org/lkml/20230322205702.GA2493123@bhelgaas/ Johan
diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c index 50b04ae5c394..8cf8cc2d6bba 100644 --- a/drivers/pci/pcie/aspm.c +++ b/drivers/pci/pcie/aspm.c @@ -1109,17 +1109,7 @@ int pci_disable_link_state(struct pci_dev *pdev, int state) } EXPORT_SYMBOL(pci_disable_link_state); -/** - * pci_enable_link_state - Clear and set the default device link state so that - * the link may be allowed to enter the specified states. Note that if the - * BIOS didn't grant ASPM control to the OS, this does nothing because we can't - * touch the LNKCTL register. Also note that this does not enable states - * disabled by pci_disable_link_state(). Return 0 or a negative errno. - * - * @pdev: PCI device - * @state: Mask of ASPM link states to enable - */ -int pci_enable_link_state(struct pci_dev *pdev, int state) +static int __pci_enable_link_state(struct pci_dev *pdev, int state, bool locked) { struct pcie_link_state *link = pcie_aspm_get_link(pdev); @@ -1136,7 +1126,8 @@ int pci_enable_link_state(struct pci_dev *pdev, int state) return -EPERM; } - down_read(&pci_bus_sem); + if (!locked) + down_read(&pci_bus_sem); mutex_lock(&aspm_lock); link->aspm_default = 0; if (state & PCIE_LINK_STATE_L0S) @@ -1157,12 +1148,48 @@ int pci_enable_link_state(struct pci_dev *pdev, int state) link->clkpm_default = (state & PCIE_LINK_STATE_CLKPM) ? 1 : 0; pcie_set_clkpm(link, policy_to_clkpm_state(link)); mutex_unlock(&aspm_lock); - up_read(&pci_bus_sem); + if (!locked) + up_read(&pci_bus_sem); return 0; } + +/** + * pci_enable_link_state - Clear and set the default device link state so that + * the link may be allowed to enter the specified states. Note that if the + * BIOS didn't grant ASPM control to the OS, this does nothing because we can't + * touch the LNKCTL register. Also note that this does not enable states + * disabled by pci_disable_link_state(). Return 0 or a negative errno. + * + * @pdev: PCI device + * @state: Mask of ASPM link states to enable + */ +int pci_enable_link_state(struct pci_dev *pdev, int state) +{ + return __pci_enable_link_state(pdev, state, false); +} EXPORT_SYMBOL(pci_enable_link_state); +/** + * pci_enable_link_state - Clear and set the default device link state so that + * the link may be allowed to enter the specified states. Note that if the + * BIOS didn't grant ASPM control to the OS, this does nothing because we can't + * touch the LNKCTL register. Also note that this does not enable states + * disabled by pci_disable_link_state(). Return 0 or a negative errno. + * + * @pdev: PCI device + * @state: Mask of ASPM link states to enable + * + * Context: Caller holds pci_bus_sem read lock. + */ +int pci_enable_link_state_locked(struct pci_dev *pdev, int state) +{ + lockdep_assert_held_read(&pci_bus_sem); + + return __pci_enable_link_state(pdev, state, true); +} +EXPORT_SYMBOL(pci_enable_link_state_locked); + static int pcie_aspm_set_policy(const char *val, const struct kernel_param *kp) { diff --git a/include/linux/pci.h b/include/linux/pci.h index 60ca768bc867..dea043bc1e38 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1829,6 +1829,7 @@ extern bool pcie_ports_native; int pci_disable_link_state(struct pci_dev *pdev, int state); int pci_disable_link_state_locked(struct pci_dev *pdev, int state); int pci_enable_link_state(struct pci_dev *pdev, int state); +int pci_enable_link_state_locked(struct pci_dev *pdev, int state); void pcie_no_aspm(void); bool pcie_aspm_support_enabled(void); bool pcie_aspm_enabled(struct pci_dev *pdev); @@ -1839,6 +1840,8 @@ static inline int pci_disable_link_state_locked(struct pci_dev *pdev, int state) { return 0; } static inline int pci_enable_link_state(struct pci_dev *pdev, int state) { return 0; } +static inline int pci_enable_link_state_locked(struct pci_dev *pdev, int state) +{ return 0; } static inline void pcie_no_aspm(void) { } static inline bool pcie_aspm_support_enabled(void) { return false; } static inline bool pcie_aspm_enabled(struct pci_dev *pdev) { return false; }