From patchwork Fri Oct 20 06:43:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Manivannan Sadhasivam X-Patchwork-Id: 155845 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2010:b0:403:3b70:6f57 with SMTP id fe16csp866518vqb; Thu, 19 Oct 2023 23:44:40 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHoSc6Dx54zR+IUGL/22ah21ZVS2iwiIri6oMpfHLwn1zseopVBE4o+MCyjAoPXehwACzmE X-Received: by 2002:a17:902:e746:b0:1c9:fb8e:2c33 with SMTP id p6-20020a170902e74600b001c9fb8e2c33mr1252043plf.7.1697784279928; Thu, 19 Oct 2023 23:44:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697784279; cv=none; d=google.com; s=arc-20160816; b=lguNTTkCUTKNvsN9KuDEPriUQi8ebNeDpb6QxuMj0pLeiLFnwWvVG6WBMqKUpuZrek Ubi3AJGg+gKJQg82ygvAwddpL5VSdHxHPnGyXdGvV5lSpLP3Rvn+5eYmO8DP4a0aLXGv 7kOa6oeaDehYSy38h0xf5BPi+ZrKE2U8WkjJXwIq/IAnoHKVPIHljU1bdSjH0YZE5uGg srND2nmew14AgCqXJlMw9vtHFk7QkfQTa4YnzBy/9Hm4tNTv3Qb70AogleVj50PVQ9jM dyjZAZ2IPAdjJ0suorbFcTdV5Rd6gJHUV4nXWFtgw5evpcig36xun/jTYE4O5geNDZlI rRVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=LfdpGLwQeRlYw+Dni+EVnY74+wxL6+4rYWTqswH92yQ=; fh=zZlCCqN7aWEy9ELbOXxcc/bLOxxSSA1zzKLmboWMk+0=; b=jo74VwfRDZy/QvwEr5Pg6w0u1ROKR9eZxdoYjQ1FMp9V2txOO8y6MCnxqqA/9qqrtj BxgS9UdxOJ7Sxk/8k2G7u2VnxP87iuph94g/zRR9yjIEGwVIcUwKqRIdPgPZ0tlSXlrD Pw4qZu2m35iXxFh51qmAA5+Z7vx9bcUYndxva2Tetz8GXoeWTawDOA79sxarsOlmjwWj Ejt/hHaGYlNU+iH27hRkhYs7M1DZmKtpr0wTx0CvsekOoe2NhOcUTdwuv5xNi1jSlKGD TUGBcUsR5po3BRrI9YSVWoJhazAOQUkOidmogz+JOiFgLCIQugVHvqgqrrau2Vbhuy4T erwQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kuI21qWE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id e5-20020a170902b78500b001b9c1821f6bsi1137108pls.98.2023.10.19.23.44.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Oct 2023 23:44:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kuI21qWE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 0831382CC459; Thu, 19 Oct 2023 23:44:37 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376385AbjJTGoL (ORCPT + 25 others); Fri, 20 Oct 2023 02:44:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58424 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1376401AbjJTGn4 (ORCPT ); Fri, 20 Oct 2023 02:43:56 -0400 Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 483E4D76 for ; Thu, 19 Oct 2023 23:43:44 -0700 (PDT) Received: by mail-pl1-x636.google.com with SMTP id d9443c01a7336-1c9bf22fe05so3378705ad.2 for ; Thu, 19 Oct 2023 23:43:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1697784223; x=1698389023; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LfdpGLwQeRlYw+Dni+EVnY74+wxL6+4rYWTqswH92yQ=; b=kuI21qWEsLMlKnLNECi0DClcZpS7gpVV2ZK1gUGwoE80xiGZ5Pmnjn0qzjwJybSOy8 l1nMxnmnC8yLjlTDpyJsEPrmv6BLUurnH8dTCAK2bl77cg5MGRPx5yK9RX6jYddvrr6c jzhAjrAlwZiuU0ADtoByzFXAUF9YAlHv73b8GwB0kDfUh2+LQieAISykxQ/V54ZDLvia +sUAp0HE5UA669VvNN9iUoGY3k7XRQRzUZLAqxQGOEycc4qzvAfmWB0GnVBMG0pKvjne ak2SK39X9I6XlPLQ5ebbhB5DrY61eQr4NZUvD9VONTdBiWIFgvcmDFM53HoVRiyqrNly UvFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697784223; x=1698389023; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LfdpGLwQeRlYw+Dni+EVnY74+wxL6+4rYWTqswH92yQ=; b=bfwRQutj/5OIDTsa4zXpp8pKL3lEUGxgrUAaXf0fRPq23sYvSIl0xdZQXxZxpvn8c+ lYhgocOFgCLGrprnMRET9sfgRjbzMFQ/LZLv0v6xfptNJSnduZL9D2MzlwIrsCJAmaNT YwsxmpsDrxzEpiUkOLDkcO8QTbYy8B77yK6opuzpZZs+9BgJNH+RWM/W9Cb6SQgBaYpj Ohj/ciutylC5yAB/J+paWlFiRhe2gQW5K0kUVoMb1X9sG5fZm0bnqd5xBem1f5RtE2dj GEyfuky3K6crdG/QvLG6RbdEcmkrlay2EOznpg/JyUJ0CsYhUjE2q3DeIDQq8vPUZlXo D9Fg== X-Gm-Message-State: AOJu0Yz3ls7c4sUx4Y0OP00vZ5WidwCZkRwgsfBl15Fv6Ycu19/9K1q+ PBhxWhtPRTpUiaDXcCMOq9JH X-Received: by 2002:a17:902:e551:b0:1bc:3944:9391 with SMTP id n17-20020a170902e55100b001bc39449391mr1000696plf.25.1697784222874; Thu, 19 Oct 2023 23:43:42 -0700 (PDT) Received: from localhost.localdomain ([117.202.186.40]) by smtp.gmail.com with ESMTPSA id t3-20020a170902bc4300b001c60ba709b7sm760951plz.125.2023.10.19.23.43.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Oct 2023 23:43:42 -0700 (PDT) From: Manivannan Sadhasivam To: jingoohan1@gmail.com, gustavo.pimentel@synopsys.com, lpieralisi@kernel.org, robh@kernel.org, kw@linux.com, bhelgaas@google.com Cc: linux-pci@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-kernel@vger.kernel.org, vidyas@nvidia.com, Manivannan Sadhasivam Subject: [PATCH v6 1/3] PCI: designware-ep: Fix DBI access before core init Date: Fri, 20 Oct 2023 12:13:18 +0530 Message-Id: <20231020064320.5302-2-manivannan.sadhasivam@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231020064320.5302-1-manivannan.sadhasivam@linaro.org> References: <20231020064320.5302-1-manivannan.sadhasivam@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Thu, 19 Oct 2023 23:44:37 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780255814284589570 X-GMAIL-MSGID: 1780255848770494671 From: Vidya Sagar Platforms that cannot support their core initialization without the reference clock from the host, implement the feature 'core_init_notifier' to indicate the DesignWare sub-system about when their core is getting initialized. Any accesses to the core (Ex:- DBI) would the core being ready result in system hang in such systems (Ex:- tegra194). This patch moves any access to the core to dw_pcie_ep_init_complete() API which is effectively called only after the core initialization. It also introduces .ep_init_late() ops hook to be used for any post init work that platform drivers may have to do. Acked-by: Jingoo Han Signed-off-by: Vidya Sagar [mani: rebased and removed ep_init_late() callback] Signed-off-by: Manivannan Sadhasivam --- .../pci/controller/dwc/pcie-designware-ep.c | 142 +++++++++++------- drivers/pci/controller/dwc/pcie-designware.h | 9 +- 2 files changed, 87 insertions(+), 64 deletions(-) diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c index f9182f8d552f..f37c4481e7d0 100644 --- a/drivers/pci/controller/dwc/pcie-designware-ep.c +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c @@ -21,14 +21,6 @@ void dw_pcie_ep_linkup(struct dw_pcie_ep *ep) } EXPORT_SYMBOL_GPL(dw_pcie_ep_linkup); -void dw_pcie_ep_init_notify(struct dw_pcie_ep *ep) -{ - struct pci_epc *epc = ep->epc; - - pci_epc_init_notify(epc); -} -EXPORT_SYMBOL_GPL(dw_pcie_ep_init_notify); - struct dw_pcie_ep_func * dw_pcie_ep_get_func_from_ep(struct dw_pcie_ep *ep, u8 func_no) { @@ -641,14 +633,19 @@ static unsigned int dw_pcie_ep_find_ext_capability(struct dw_pcie *pci, int cap) return 0; } -int dw_pcie_ep_init_complete(struct dw_pcie_ep *ep) +static int dw_pcie_ep_init_late(struct dw_pcie_ep *ep) { struct dw_pcie *pci = to_dw_pcie_from_ep(ep); + struct dw_pcie_ep_func *ep_func; + struct device *dev = pci->dev; + struct pci_epc *epc = ep->epc; unsigned int offset, ptm_cap_base; unsigned int nbars; u8 hdr_type; + u8 func_no; + int i, ret; + void *addr; u32 reg; - int i; hdr_type = dw_pcie_readb_dbi(pci, PCI_HEADER_TYPE) & PCI_HEADER_TYPE_MASK; @@ -659,6 +656,55 @@ int dw_pcie_ep_init_complete(struct dw_pcie_ep *ep) return -EIO; } + dw_pcie_version_detect(pci); + + dw_pcie_iatu_detect(pci); + + ret = dw_pcie_edma_detect(pci); + if (ret) + return ret; + + if (!ep->ib_window_map) { + ep->ib_window_map = devm_bitmap_zalloc(dev, pci->num_ib_windows, + GFP_KERNEL); + if (!ep->ib_window_map) + goto err_remove_edma; + } + + if (!ep->ob_window_map) { + ep->ob_window_map = devm_bitmap_zalloc(dev, pci->num_ob_windows, + GFP_KERNEL); + if (!ep->ob_window_map) + goto err_remove_edma; + } + + if (!ep->outbound_addr) { + addr = devm_kcalloc(dev, pci->num_ob_windows, sizeof(phys_addr_t), + GFP_KERNEL); + if (!addr) + goto err_remove_edma; + ep->outbound_addr = addr; + } + + for (func_no = 0; func_no < epc->max_functions; func_no++) { + + ep_func = dw_pcie_ep_get_func_from_ep(ep, func_no); + if (ep_func) + continue; + + ep_func = devm_kzalloc(dev, sizeof(*ep_func), GFP_KERNEL); + if (!ep_func) + goto err_remove_edma; + + ep_func->func_no = func_no; + ep_func->msi_cap = dw_pcie_ep_find_capability(ep, func_no, + PCI_CAP_ID_MSI); + ep_func->msix_cap = dw_pcie_ep_find_capability(ep, func_no, + PCI_CAP_ID_MSIX); + + list_add_tail(&ep_func->list, &ep->func_list); + } + offset = dw_pcie_ep_find_ext_capability(pci, PCI_EXT_CAP_ID_REBAR); ptm_cap_base = dw_pcie_ep_find_ext_capability(pci, PCI_EXT_CAP_ID_PTM); @@ -693,14 +739,31 @@ int dw_pcie_ep_init_complete(struct dw_pcie_ep *ep) dw_pcie_dbi_ro_wr_dis(pci); return 0; + +err_remove_edma: + dw_pcie_edma_remove(pci); + + return ret; } -EXPORT_SYMBOL_GPL(dw_pcie_ep_init_complete); + +int dw_pcie_ep_init_notify(struct dw_pcie_ep *ep) +{ + struct pci_epc *epc = ep->epc; + int ret; + + ret = dw_pcie_ep_init_late(ep); + if (ret) + return ret; + + pci_epc_init_notify(epc); + + return 0; +} +EXPORT_SYMBOL_GPL(dw_pcie_ep_init_notify); int dw_pcie_ep_init(struct dw_pcie_ep *ep) { int ret; - void *addr; - u8 func_no; struct resource *res; struct pci_epc *epc; struct dw_pcie *pci = to_dw_pcie_from_ep(ep); @@ -708,7 +771,6 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) struct platform_device *pdev = to_platform_device(dev); struct device_node *np = dev->of_node; const struct pci_epc_features *epc_features; - struct dw_pcie_ep_func *ep_func; INIT_LIST_HEAD(&ep->func_list); @@ -723,26 +785,6 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) ep->phys_base = res->start; ep->addr_size = resource_size(res); - dw_pcie_version_detect(pci); - - dw_pcie_iatu_detect(pci); - - ep->ib_window_map = devm_bitmap_zalloc(dev, pci->num_ib_windows, - GFP_KERNEL); - if (!ep->ib_window_map) - return -ENOMEM; - - ep->ob_window_map = devm_bitmap_zalloc(dev, pci->num_ob_windows, - GFP_KERNEL); - if (!ep->ob_window_map) - return -ENOMEM; - - addr = devm_kcalloc(dev, pci->num_ob_windows, sizeof(phys_addr_t), - GFP_KERNEL); - if (!addr) - return -ENOMEM; - ep->outbound_addr = addr; - epc = devm_pci_epc_create(dev, &epc_ops); if (IS_ERR(epc)) { dev_err(dev, "Failed to create epc device\n"); @@ -756,20 +798,6 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) if (ret < 0) epc->max_functions = 1; - for (func_no = 0; func_no < epc->max_functions; func_no++) { - ep_func = devm_kzalloc(dev, sizeof(*ep_func), GFP_KERNEL); - if (!ep_func) - return -ENOMEM; - - ep_func->func_no = func_no; - ep_func->msi_cap = dw_pcie_ep_find_capability(ep, func_no, - PCI_CAP_ID_MSI); - ep_func->msix_cap = dw_pcie_ep_find_capability(ep, func_no, - PCI_CAP_ID_MSIX); - - list_add_tail(&ep_func->list, &ep->func_list); - } - if (ep->ops->ep_init) ep->ops->ep_init(ep); @@ -788,25 +816,25 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) goto err_exit_epc_mem; } - ret = dw_pcie_edma_detect(pci); - if (ret) - goto err_free_epc_mem; - if (ep->ops->get_features) { epc_features = ep->ops->get_features(ep); if (epc_features->core_init_notifier) return 0; } - ret = dw_pcie_ep_init_complete(ep); + /* + * NOTE:- Avoid accessing the hardware (Ex:- DBI space) before this + * step as platforms that implement 'core_init_notifier' feature may + * not have the hardware ready (i.e. core initialized) for access + * (Ex: tegra194). Any hardware access on such platforms result + * in system hard hang. + */ + ret = dw_pcie_ep_init_late(ep); if (ret) - goto err_remove_edma; + goto err_free_epc_mem; return 0; -err_remove_edma: - dw_pcie_edma_remove(pci); - err_free_epc_mem: pci_epc_mem_free_addr(epc, ep->msi_mem_phys, ep->msi_mem, epc->mem->window.page_size); diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h index efb4d4754fc8..9e8e95df320c 100644 --- a/drivers/pci/controller/dwc/pcie-designware.h +++ b/drivers/pci/controller/dwc/pcie-designware.h @@ -573,8 +573,7 @@ static inline void __iomem *dw_pcie_own_conf_map_bus(struct pci_bus *bus, #ifdef CONFIG_PCIE_DW_EP void dw_pcie_ep_linkup(struct dw_pcie_ep *ep); int dw_pcie_ep_init(struct dw_pcie_ep *ep); -int dw_pcie_ep_init_complete(struct dw_pcie_ep *ep); -void dw_pcie_ep_init_notify(struct dw_pcie_ep *ep); +int dw_pcie_ep_init_notify(struct dw_pcie_ep *ep); void dw_pcie_ep_exit(struct dw_pcie_ep *ep); int dw_pcie_ep_raise_legacy_irq(struct dw_pcie_ep *ep, u8 func_no); int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no, @@ -596,15 +595,11 @@ static inline int dw_pcie_ep_init(struct dw_pcie_ep *ep) return 0; } -static inline int dw_pcie_ep_init_complete(struct dw_pcie_ep *ep) +static inline int dw_pcie_ep_init_notify(struct dw_pcie_ep *ep) { return 0; } -static inline void dw_pcie_ep_init_notify(struct dw_pcie_ep *ep) -{ -} - static inline void dw_pcie_ep_exit(struct dw_pcie_ep *ep) { }