From patchwork Sat Feb 24 06:54:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Manivannan Sadhasivam X-Patchwork-Id: 205777 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:a81b:b0:108:e6aa:91d0 with SMTP id bq27csp1020735dyb; Fri, 23 Feb 2024 22:56:01 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVjDEgjJaSuf32E2Di7mgT4TZ7VTsFm88IWAgaGpvx1tgifTQhmJ9F1jkSUQfGnh2w4YaPMAAxlDj8VYct63C04Y6D0vw== X-Google-Smtp-Source: AGHT+IFVyXrXkBs5aV/ZFjuvKa2MrQ9bQhnKqWD40P5j0582OhyjfRPQtNIybxy1wt2ahQ2UM3z4 X-Received: by 2002:a05:6214:2422:b0:68f:e766:5ee5 with SMTP id gy2-20020a056214242200b0068fe7665ee5mr1822015qvb.34.1708757761191; Fri, 23 Feb 2024 22:56:01 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708757761; cv=pass; d=google.com; s=arc-20160816; b=JUYR/2Wd6Brmx0yH5TmqLQbim/5tSWNrXjcr/xf5ZlJms1GyDgzsr4jcVmG6O4bO+Y QK0/rm+uJ4WmW9lyY9NvCP61AOanTNOmCQkMNE1JBGGI6PgNV/9SyLDFLUnBc5FFDLw6 9TUBotnd4TD3nuWwUr+Ec1AUSsUPougUYv89NCKLCpWbLLvMV4DM1xmajdf6RS+gN064 6/a42A6MlYSkGUtx/PPnafzxIwlVu7xtrhQ/WHACeHArd2cj1bxks3tdt0zdGwyov31M bZckbBhyO49t53TUqCP/losfs0+axn60Y01AIIC3AkVrz+X6DaZQW2UAQv9YaIfr+u6l T1bQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :subject:date:from:dkim-signature; bh=joeXMwEVA7mDgrHHJmbcOh+YUIfmjaNkgN+gOStshl4=; fh=1s2RkR0ilUbG/IkXaoJ9NKatLuMrvassrvgF8wiqlA4=; b=0r/GcAISjtnEUvUTwQRrL0FkqG9P7FYS57jiDbEsmxWm2AymzsbWskS0RjO0Up7hhk lMGTHNdtRHorVL3/bZS53rWKgJ4ioA+rHanEvEeiAElgGv1halyxN+CuF9urDf428A7H vuAj7Pr5LnuE+exaPfmxDxqvCOf/o3Wb33Q39DoRbiZ6X8tu3jddCcWEfaXxjAIjPHxG 5P+A1sPuuAWMwN9eqHEAw8YsMmhyPs/k1XCkqC2lDMA3zxH0vDgEc5mGSI1AxA5Yo31q Wu4lFrY12S0ZaWmKhKVMG36bGsfS6mq33A2PdcxjcW81qUd9YWr0kBU3KIzUm1alAq7B e9Ug==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="dvw/IPH2"; arc=pass (i=1 spf=pass spfdomain=linaro.org dkim=pass dkdomain=linaro.org dmarc=pass fromdomain=linaro.org); spf=pass (google.com: domain of linux-kernel+bounces-79488-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-79488-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id c12-20020a056214224c00b0068f8eee6161si542902qvc.404.2024.02.23.22.56.01 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Feb 2024 22:56:01 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-79488-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="dvw/IPH2"; arc=pass (i=1 spf=pass spfdomain=linaro.org dkim=pass dkdomain=linaro.org dmarc=pass fromdomain=linaro.org); spf=pass (google.com: domain of linux-kernel+bounces-79488-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-79488-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id E0E8C1C209A3 for ; Sat, 24 Feb 2024 06:56:00 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1C10318622; Sat, 24 Feb 2024 06:55:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="dvw/IPH2" Received: from mail-oa1-f54.google.com (mail-oa1-f54.google.com [209.85.160.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CDED617592 for ; Sat, 24 Feb 2024 06:55:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708757703; cv=none; b=bzHDn/P129H399cccsANB8xX6BGx6chk8CTEQfAAkYmBA1g1WrnHIenyZpeBj0wt4nY72eFOjREWoEFawHPAroaiOVTD1iEoR1cQn9PVLakStKncqyB27HeH5SozM2F/3oLDZE4lLzo6ikethp1Jlx65YK63Vi8rk2iuZmUr15U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708757703; c=relaxed/simple; bh=qldMTtVGUo5lDMZM4xG3tzrJt0x+Ru12VAwPkqahGjw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=a8RgxfQMWQu0gCmFxnNwpy69moBg1whQjV7RiPkOWB36Dq7e41CETgFX0DpfN9/umvvLlF4jZE5abgpXYHyxMb1KDMizzT3MfepXK7qBOvDLbjD7+4Yi1tw1gRCGpDHHZHHjzlG5jDOo9yb1Fa7VKCxbjCAXfMvNgQxOS2NhrbY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=dvw/IPH2; arc=none smtp.client-ip=209.85.160.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Received: by mail-oa1-f54.google.com with SMTP id 586e51a60fabf-21e6be74db4so598463fac.2 for ; Fri, 23 Feb 2024 22:55:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1708757700; x=1709362500; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=joeXMwEVA7mDgrHHJmbcOh+YUIfmjaNkgN+gOStshl4=; b=dvw/IPH22Mn5PdULt+ca8ifWhzwoHzL4jqNWCUYNgc8nbmz+8dOIsltmUpkPd2K5J9 RaN6vD+fu6xdi7/TUwIyhnT0ISH1imeyHdfzcwaSHTP/LSl7C7fIVJV7ipmltiUFnn89 CXCJbDRNNTV88QGMF1wwsO3toRGySo5k8ba1tDsHCVsYDP+RYg+SmlZgutLmrdtlLKIQ w+YF33wR1Z1ZWiYEl862S2WFF2zM+rTlJuaJ2/wfAUFzJ42NLeMuddtoUevBoKXeimaB gQKAhYW7WLuvKwIqjIuX3GjwV0RrRoQVFBRI5F0uiciNxG6PIrwcrVpoHP4GZAg7s5wS 64SA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708757700; x=1709362500; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=joeXMwEVA7mDgrHHJmbcOh+YUIfmjaNkgN+gOStshl4=; b=DzC9Op8hiLVgObHj7AR2yeGiW44B5HS4yGpF51AGy94NxgjYT1kMQ+7crN6IGVUZOM W/VKCena7muqd5tFks55JpQCjllSpIrCl9qdr5kSYLqCxb4Kfudm68p9VhHLR52Ih6l0 4wMcW1ZNciumT0Klrbgf91SHBnPOyYT9tHLxaWZOt2cKv1owDt5Pl9S2XW8RhkQD34Yq 8u6G/bR46LeftdnjMvpvNKe99zsHuSbDZmZcAzkv2q6jPu0ihEVLnbAmFD1r0ajJ+MrO SZP/VPf20ySPk5HFS2YAQja3DHc8djytojvHPd4rVsWMUD+mCOnoE88VSb9AlysaXaEc UH9Q== X-Forwarded-Encrypted: i=1; AJvYcCW0tftBIMbEeIWR+d7i7KXsqk1oGnO7YZpT45TdgFiPX7VrM2n5qLweNJl8f5o6o5jRYgmuc4QBPlF8aomR5QRUiI8IFoytxj6qM919 X-Gm-Message-State: AOJu0YxI/9mfHO8sgcPDKDlshUzA+lqdRvWKgljKrdyDL/U0QmKruLhR v2PYWpaVfsX/kugGo9y/qy73a0OUqaZpRGp+bSXgCE4MkuKbNB3Wmr/oXlqP4Q== X-Received: by 2002:a05:6358:5623:b0:17b:5dd1:a38d with SMTP id b35-20020a056358562300b0017b5dd1a38dmr2266837rwf.1.1708757699831; Fri, 23 Feb 2024 22:54:59 -0800 (PST) Received: from [127.0.1.1] ([120.138.12.46]) by smtp.gmail.com with ESMTPSA id r5-20020aa78b85000000b006e4cb7f4393sm502932pfd.165.2024.02.23.22.54.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Feb 2024 22:54:59 -0800 (PST) From: Manivannan Sadhasivam Date: Sat, 24 Feb 2024 12:24:10 +0530 Subject: [PATCH v8 04/10] PCI: dwc: ep: Fix DBI access failure for drivers requiring refclk from host Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240224-pci-dbi-rework-v8-4-64c7fd0cfe64@linaro.org> References: <20240224-pci-dbi-rework-v8-0-64c7fd0cfe64@linaro.org> In-Reply-To: <20240224-pci-dbi-rework-v8-0-64c7fd0cfe64@linaro.org> To: Jingoo Han , Gustavo Pimentel , Lorenzo Pieralisi , =?utf-8?q?Krzysztof_Wilczy=C5=84?= =?utf-8?q?ski?= , Rob Herring , Bjorn Helgaas , Marek Vasut , Yoshihiro Shimoda , Thierry Reding , Jonathan Hunter , Kishon Vijay Abraham I , Vidya Sagar , Vignesh Raghavendra , Richard Zhu , Lucas Stach , Shawn Guo , Sascha Hauer , Pengutronix Kernel Team , Fabio Estevam , NXP Linux Team , Minghuan Lian , Mingkai Hu , Roy Zang , Kunihiko Hayashi , Masami Hiramatsu , Kishon Vijay Abraham I Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-renesas-soc@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-tegra@vger.kernel.org, linux-omap@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Niklas Cassel , Manivannan Sadhasivam X-Mailer: b4 0.12.4 X-Developer-Signature: v=1; a=openpgp-sha256; l=7390; i=manivannan.sadhasivam@linaro.org; h=from:subject:message-id; bh=qldMTtVGUo5lDMZM4xG3tzrJt0x+Ru12VAwPkqahGjw=; b=owEBbQGS/pANAwAKAVWfEeb+kc71AcsmYgBl2ZKWRKBz+u4mIJBImyyENYXdGJCigRqJ+VZ6I xNQ03ZNJ2iJATMEAAEKAB0WIQRnpUMqgUjL2KRYJ5dVnxHm/pHO9QUCZdmSlgAKCRBVnxHm/pHO 9X+rB/0XGKWgslByHRknPL5uaQJs6G4qN4NnmaUNEbfXjxQWEnSZPe68Kr3YrdqU8nlLQ7VxVZK ctAdojob7nujJOeAVFVX1/OY2M4OgL8duskQBdbWNBCMhkruB+zp+62BOF1eDMqMntXQhdWnr2M viK7WuMrPa4p0Kj1nGrdQfQ50v1TAt9JmhxMgNB2u+lZXmA8DwKyCUsQtcTeYiUN4L22yz8QOg1 LO4Zbo5HoGmIa6xlWtKQkc2AcnfEdPVICI9rW9llHPtbVUld/O/j/TgTcIVqKWUCSSYGaNLZKTx 6ZUL2f0FOtdWTxdwnL65bUnvHWQvKmJbidJyOLAcE9oUfJBf X-Developer-Key: i=manivannan.sadhasivam@linaro.org; a=openpgp; fpr=C668AEC3C3188E4C611465E7488550E901166008 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791762300450584739 X-GMAIL-MSGID: 1791762378043680499 The DWC glue drivers requiring an active reference clock from the PCIe host for initializing their PCIe EP core, set a flag called 'core_init_notifier' to let DWC driver know that these drivers need a special attention during initialization. In these drivers, access to the hw registers (like DBI) before receiving the active refclk from host will result in access failure and also could cause a whole system hang. But the current DWC EP driver doesn't honor the requirements of the drivers setting 'core_init_notifier' flag and tries to access the DBI registers during dw_pcie_ep_init(). This causes the system hang for glue drivers such as Tegra194 and Qcom EP as they depend on refclk from host and have set the above mentioned flag. To workaround this issue, users of the affected platforms have to maintain the dependency with the PCIe host by booting the PCIe EP after host boot. But this won't provide a good user experience, since PCIe EP is _one_ of the features of those platforms and it doesn't make sense to delay the whole platform booting due to PCIe requiring active refclk. So to fix this issue, let's move all the DBI access from dw_pcie_ep_init() in the DWC EP driver to the dw_pcie_ep_init_complete() API. This API will only be called by the drivers setting 'core_init_notifier' flag once refclk is received from host. For the rest of the drivers that gets the refclk locally, this API will be called within dw_pcie_ep_init(). Fixes: e966f7390da9 ("PCI: dwc: Refactor core initialization code for EP mode") Co-developed-by: Vidya Sagar Signed-off-by: Vidya Sagar Signed-off-by: Manivannan Sadhasivam Reviewed-by: Frank Li --- drivers/pci/controller/dwc/pcie-designware-ep.c | 120 ++++++++++++++---------- 1 file changed, 71 insertions(+), 49 deletions(-) diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c index 1205bfba8310..99d66b0fa59b 100644 --- a/drivers/pci/controller/dwc/pcie-designware-ep.c +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c @@ -606,11 +606,16 @@ static unsigned int dw_pcie_ep_find_ext_capability(struct dw_pcie *pci, int cap) int dw_pcie_ep_init_complete(struct dw_pcie_ep *ep) { struct dw_pcie *pci = to_dw_pcie_from_ep(ep); + struct dw_pcie_ep_func *ep_func; + struct device *dev = pci->dev; + struct pci_epc *epc = ep->epc; unsigned int offset, ptm_cap_base; unsigned int nbars; u8 hdr_type; + u8 func_no; + int i, ret; + void *addr; u32 reg; - int i; hdr_type = dw_pcie_readb_dbi(pci, PCI_HEADER_TYPE) & PCI_HEADER_TYPE_MASK; @@ -621,6 +626,58 @@ int dw_pcie_ep_init_complete(struct dw_pcie_ep *ep) return -EIO; } + dw_pcie_version_detect(pci); + + dw_pcie_iatu_detect(pci); + + ret = dw_pcie_edma_detect(pci); + if (ret) + return ret; + + if (!ep->ib_window_map) { + ep->ib_window_map = devm_bitmap_zalloc(dev, pci->num_ib_windows, + GFP_KERNEL); + if (!ep->ib_window_map) + goto err_remove_edma; + } + + if (!ep->ob_window_map) { + ep->ob_window_map = devm_bitmap_zalloc(dev, pci->num_ob_windows, + GFP_KERNEL); + if (!ep->ob_window_map) + goto err_remove_edma; + } + + if (!ep->outbound_addr) { + addr = devm_kcalloc(dev, pci->num_ob_windows, sizeof(phys_addr_t), + GFP_KERNEL); + if (!addr) + goto err_remove_edma; + ep->outbound_addr = addr; + } + + for (func_no = 0; func_no < epc->max_functions; func_no++) { + + ep_func = dw_pcie_ep_get_func_from_ep(ep, func_no); + if (ep_func) + continue; + + ep_func = devm_kzalloc(dev, sizeof(*ep_func), GFP_KERNEL); + if (!ep_func) + goto err_remove_edma; + + ep_func->func_no = func_no; + ep_func->msi_cap = dw_pcie_ep_find_capability(ep, func_no, + PCI_CAP_ID_MSI); + ep_func->msix_cap = dw_pcie_ep_find_capability(ep, func_no, + PCI_CAP_ID_MSIX); + + list_add_tail(&ep_func->list, &ep->func_list); + } + + if (ep->ops->init) + ep->ops->init(ep); + offset = dw_pcie_ep_find_ext_capability(pci, PCI_EXT_CAP_ID_REBAR); ptm_cap_base = dw_pcie_ep_find_ext_capability(pci, PCI_EXT_CAP_ID_PTM); @@ -655,14 +712,17 @@ int dw_pcie_ep_init_complete(struct dw_pcie_ep *ep) dw_pcie_dbi_ro_wr_dis(pci); return 0; + +err_remove_edma: + dw_pcie_edma_remove(pci); + + return ret; } EXPORT_SYMBOL_GPL(dw_pcie_ep_init_complete); int dw_pcie_ep_init(struct dw_pcie_ep *ep) { int ret; - void *addr; - u8 func_no; struct resource *res; struct pci_epc *epc; struct dw_pcie *pci = to_dw_pcie_from_ep(ep); @@ -670,7 +730,6 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) struct platform_device *pdev = to_platform_device(dev); struct device_node *np = dev->of_node; const struct pci_epc_features *epc_features; - struct dw_pcie_ep_func *ep_func; INIT_LIST_HEAD(&ep->func_list); @@ -688,26 +747,6 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) if (ep->ops->pre_init) ep->ops->pre_init(ep); - dw_pcie_version_detect(pci); - - dw_pcie_iatu_detect(pci); - - ep->ib_window_map = devm_bitmap_zalloc(dev, pci->num_ib_windows, - GFP_KERNEL); - if (!ep->ib_window_map) - return -ENOMEM; - - ep->ob_window_map = devm_bitmap_zalloc(dev, pci->num_ob_windows, - GFP_KERNEL); - if (!ep->ob_window_map) - return -ENOMEM; - - addr = devm_kcalloc(dev, pci->num_ob_windows, sizeof(phys_addr_t), - GFP_KERNEL); - if (!addr) - return -ENOMEM; - ep->outbound_addr = addr; - epc = devm_pci_epc_create(dev, &epc_ops); if (IS_ERR(epc)) { dev_err(dev, "Failed to create epc device\n"); @@ -721,23 +760,6 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) if (ret < 0) epc->max_functions = 1; - for (func_no = 0; func_no < epc->max_functions; func_no++) { - ep_func = devm_kzalloc(dev, sizeof(*ep_func), GFP_KERNEL); - if (!ep_func) - return -ENOMEM; - - ep_func->func_no = func_no; - ep_func->msi_cap = dw_pcie_ep_find_capability(ep, func_no, - PCI_CAP_ID_MSI); - ep_func->msix_cap = dw_pcie_ep_find_capability(ep, func_no, - PCI_CAP_ID_MSIX); - - list_add_tail(&ep_func->list, &ep->func_list); - } - - if (ep->ops->init) - ep->ops->init(ep); - ret = pci_epc_mem_init(epc, ep->phys_base, ep->addr_size, ep->page_size); if (ret < 0) { @@ -753,25 +775,25 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) goto err_exit_epc_mem; } - ret = dw_pcie_edma_detect(pci); - if (ret) - goto err_free_epc_mem; - if (ep->ops->get_features) { epc_features = ep->ops->get_features(ep); if (epc_features->core_init_notifier) return 0; } + /* + * NOTE:- Avoid accessing the hardware (Ex:- DBI space) before this + * step as platforms that implement 'core_init_notifier' feature may + * not have the hardware ready (i.e. core initialized) for access + * (Ex: tegra194). Any hardware access on such platforms result + * in system hang. + */ ret = dw_pcie_ep_init_complete(ep); if (ret) - goto err_remove_edma; + goto err_free_epc_mem; return 0; -err_remove_edma: - dw_pcie_edma_remove(pci); - err_free_epc_mem: pci_epc_mem_free_addr(epc, ep->msi_mem_phys, ep->msi_mem, epc->mem->window.page_size);