From patchwork Fri Sep 15 12:01:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ilpo_J=C3=A4rvinen?= X-Patchwork-Id: 140512 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp1138448vqi; Fri, 15 Sep 2023 08:42:08 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGHQDHSS4dLBDRCnt62ZTgjqVVbW16E93bceAUhWBOMFnxNOhdKtC9GZwk/6/UbNaakPEBB X-Received: by 2002:a17:902:e744:b0:1c1:f0b4:f68f with SMTP id p4-20020a170902e74400b001c1f0b4f68fmr2963921plf.10.1694792528275; Fri, 15 Sep 2023 08:42:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694792528; cv=none; d=google.com; s=arc-20160816; b=ZI42y5p7aBrhTDQhIe88zMIWhMT78Yaeu9yUiVQ5DjAhE76Sfv4CM9y4QjHAxaHnu4 GVQSQUH7l9QRvHhC5aRHjasFST1diH9CqCL8Nyz981yeQTuqGnWO4nxVaEwixq1Ev5O+ 8xRSR6F6vUzBxTPU9vDvMR6rrKK298FC+SbX6njcLiqtpuQMlHjM98vaYzl1XpTTesrv 0sxDpMVgHFQjpnEOKzPOAEdcPBICr0pvP339TfLoxWT0WcKFi5fg8VdMTiYxqPbEPmwy a8PL9TIUjM9s49fjsYQJS79fg154m1GfY4rrxg/QExGUAzl0FP3gKfdfzzq8lqqF1vUX D83w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=0eKrtUxMvlCT3PBH4+tv8enhebEQRDGJ/jTDIw77Izc=; fh=Iyg1UUFYXJ+2/bQakf80bkIvWfltZneEx3PkUb9G/eo=; b=0geE251xlJYFDQMzftCzmXlc7QVN+GIVmKWCxSHakaSweH51RzdvlG60CUm4IbnmKi MyIU20klekqo6zU2lO21rFDCTolohFGG0aB+CfVVDX0eJVWlehnMfghsOkBhhihgk3pA mpS3N63slVULKqKLutJLvTqpyEgDFoZERs2vSGjWtBNnauyzempqIhmhTPH34e3R5HoA caaAu0XsDX8SK0CvcSC2TaR0cKp5tuKcbfzBVuR8VMvWaWZt7251OdOQgdb+uxEr8FWv Sh3RQARB3SrPf42N0EEcHhD0D6pGFNU7CJU6x4/F+x3XF2TvYwyP/qp9MHrrQa0qNo0s tETQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=gDvMv4yS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id i2-20020a170902c94200b001b9e9ef7e1fsi3844902pla.346.2023.09.15.08.42.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 08:42:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=gDvMv4yS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id DA7B68352EF9; Fri, 15 Sep 2023 05:04:11 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234596AbjIOMED (ORCPT + 32 others); Fri, 15 Sep 2023 08:04:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54598 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234546AbjIOMEB (ORCPT ); Fri, 15 Sep 2023 08:04:01 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 508C630D5; Fri, 15 Sep 2023 05:03:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1694779390; x=1726315390; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Lpc/55qfjPc+bstEFURf/p/RCtJBjsTaLxy0OsxpmoE=; b=gDvMv4ySZ/9IaYLpP0uo3V5zX1Vqy4X06G1qCUpuxwzzDyPvd68KLPQc yXtm5wTT22SSLEv0uiXQ+j8cfyZaKAuS/mMlqjQgTXuctvomomjE9oe6k zyvgj81jxz9YWlDlcWXNEQwRE+aFMqHHnLv77IPgS2RJvW7Ni6YgdJE+9 aaNr401UJKV5fWn+1x8BxgYzjW//YpLonq53huO/z2+dNbaM1P3iFoRjx tANHq04WMqanYsEEKmIUfF2iwOL2uLskK7LECKjGn0Vq30Jhkmaf5JCqL olkxz4H4SLTKH4O1M7IuNd1FJBqkr7x3MzeisZWlche2ypBxdCmadvjg3 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10833"; a="378146116" X-IronPort-AV: E=Sophos;i="6.02,149,1688454000"; d="scan'208";a="378146116" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Sep 2023 05:02:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10833"; a="774293033" X-IronPort-AV: E=Sophos;i="6.02,149,1688454000"; d="scan'208";a="774293033" Received: from srdoo-mobl1.ger.corp.intel.com (HELO ijarvine-mobl2.ger.corp.intel.com) ([10.252.38.99]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Sep 2023 05:02:45 -0700 From: =?utf-8?q?Ilpo_J=C3=A4rvinen?= To: linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Rob Herring , =?utf-8?q?Krzysztof_Wilczy=C5=84ski?= , Lukas Wunner , Alexandru Gagniuc , Krishna chaitanya chundru , Srinivas Pandruvada , "Rafael J . Wysocki" , linux-pm@vger.kernel.org, =?utf-8?q?Ilpo_J=C3=A4rvinen?= , Bjorn Helgaas , Daniel Lezcano , Amit Kucheria , Zhang Rui , linux-kernel@vger.kernel.org Cc: Alex Deucher Subject: [PATCH v2 09/10] thermal: Add PCIe cooling driver Date: Fri, 15 Sep 2023 15:01:41 +0300 Message-Id: <20230915120142.32987-10-ilpo.jarvinen@linux.intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230915120142.32987-1-ilpo.jarvinen@linux.intel.com> References: <20230915120142.32987-1-ilpo.jarvinen@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Fri, 15 Sep 2023 05:04:11 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777118770233293516 X-GMAIL-MSGID: 1777118770233293516 Add a thermal cooling driver to provide path to access PCIe bandwidth controller using the usual thermal interfaces. A cooling device is instantiated for controllable PCIe ports from the bwctrl service driver. The thermal side state 0 means no throttling, i.e., maximum supported PCIe speed. Signed-off-by: Ilpo Järvinen Acked-by: Rafael J. Wysocki # From the cooling device interface perspective --- MAINTAINERS | 1 + drivers/pci/pcie/bwctrl.c | 11 ++++ drivers/thermal/Kconfig | 10 +++ drivers/thermal/Makefile | 2 + drivers/thermal/pcie_cooling.c | 107 +++++++++++++++++++++++++++++++++ include/linux/pci-bwctrl.h | 16 +++++ 6 files changed, 147 insertions(+) create mode 100644 drivers/thermal/pcie_cooling.c diff --git a/MAINTAINERS b/MAINTAINERS index cd5c9b9ad32b..32974417ad52 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -16573,6 +16573,7 @@ PCIE BANDWIDTH CONTROLLER M: Ilpo Järvinen S: Supported F: drivers/pci/pcie/bwctrl.c +F: drivers/thermal/pcie_cooling.c F: include/linux/pci-bwctrl.h PCIE DRIVER FOR AMAZON ANNAPURNA LABS diff --git a/drivers/pci/pcie/bwctrl.c b/drivers/pci/pcie/bwctrl.c index e3172d69476f..13c73546244e 100644 --- a/drivers/pci/pcie/bwctrl.c +++ b/drivers/pci/pcie/bwctrl.c @@ -34,9 +34,11 @@ /** * struct bwctrl_service_data - PCIe Port Bandwidth Controller * @set_speed_mutex: serializes link speed changes + * @cdev: thermal cooling device associated with the port */ struct bwctrl_service_data { struct mutex set_speed_mutex; + struct thermal_cooling_device *cdev; }; static bool bwctrl_valid_pcie_speed(enum pci_bus_speed speed) @@ -253,8 +255,16 @@ static int pcie_bandwidth_notification_probe(struct pcie_device *srv) pcie_enable_link_bandwidth_notification(port); pci_info(port, "enabled with IRQ %d\n", srv->irq); + data->cdev = pcie_cooling_device_register(port, srv); + if (IS_ERR(data->cdev)) { + ret = PTR_ERR(data->cdev); + goto disable_notifications; + } return 0; +disable_notifications: + pcie_disable_link_bandwidth_notification(srv->port); + kfree(data); free_irq: free_irq(srv->irq, srv); return ret; @@ -264,6 +274,7 @@ static void pcie_bandwidth_notification_remove(struct pcie_device *srv) { struct bwctrl_service_data *data = get_service_data(srv); + pcie_cooling_device_unregister(data->cdev); pcie_disable_link_bandwidth_notification(srv->port); free_irq(srv->irq, srv); mutex_destroy(&data->set_speed_mutex); diff --git a/drivers/thermal/Kconfig b/drivers/thermal/Kconfig index c81a00fbca7d..3a071396f1c6 100644 --- a/drivers/thermal/Kconfig +++ b/drivers/thermal/Kconfig @@ -219,6 +219,16 @@ config DEVFREQ_THERMAL If you want this support, you should say Y here. +config PCIE_THERMAL + bool "PCIe cooling support" + depends on PCIEPORTBUS + select PCIE_BW + help + This implements PCIe cooling mechanism through bandwidth reduction + for PCIe devices. + + If you want this support, you should say Y here. + config THERMAL_EMULATION bool "Thermal emulation mode support" help diff --git a/drivers/thermal/Makefile b/drivers/thermal/Makefile index c934cab309ae..a0b25a2742b7 100644 --- a/drivers/thermal/Makefile +++ b/drivers/thermal/Makefile @@ -30,6 +30,8 @@ thermal_sys-$(CONFIG_CPU_IDLE_THERMAL) += cpuidle_cooling.o # devfreq cooling thermal_sys-$(CONFIG_DEVFREQ_THERMAL) += devfreq_cooling.o +thermal_sys-$(CONFIG_PCIE_THERMAL) += pcie_cooling.o + obj-$(CONFIG_K3_THERMAL) += k3_bandgap.o k3_j72xx_bandgap.o # platform thermal drivers obj-y += broadcom/ diff --git a/drivers/thermal/pcie_cooling.c b/drivers/thermal/pcie_cooling.c new file mode 100644 index 000000000000..c23b59dd0331 --- /dev/null +++ b/drivers/thermal/pcie_cooling.c @@ -0,0 +1,107 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * PCIe cooling device + * + * Copyright (C) 2023 Intel Corporation. + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#define COOLING_DEV_TYPE_PREFIX "PCIe_Port_Link_Speed_" + +struct pcie_cooling_device { + struct pci_dev *port; + struct pcie_device *pdev; +}; + +static int pcie_cooling_get_max_level(struct thermal_cooling_device *cdev, unsigned long *state) +{ + struct pcie_cooling_device *pcie_cdev = cdev->devdata; + + /* cooling state 0 is same as the maximum PCIe speed */ + *state = pcie_cdev->port->subordinate->max_bus_speed - PCIE_SPEED_2_5GT; + + return 0; +} + +static int pcie_cooling_get_cur_level(struct thermal_cooling_device *cdev, unsigned long *state) +{ + struct pcie_cooling_device *pcie_cdev = cdev->devdata; + + /* cooling state 0 is same as the maximum PCIe speed */ + *state = cdev->max_state - + (pcie_cdev->port->subordinate->cur_bus_speed - PCIE_SPEED_2_5GT); + + return 0; +} + +static int pcie_cooling_set_cur_level(struct thermal_cooling_device *cdev, unsigned long state) +{ + struct pcie_cooling_device *pcie_cdev = cdev->devdata; + enum pci_bus_speed speed; + + /* cooling state 0 is same as the maximum PCIe speed */ + speed = (cdev->max_state - state) + PCIE_SPEED_2_5GT; + + return bwctrl_set_current_speed(pcie_cdev->pdev, speed); +} + +static struct thermal_cooling_device_ops pcie_cooling_ops = { + .get_max_state = pcie_cooling_get_max_level, + .get_cur_state = pcie_cooling_get_cur_level, + .set_cur_state = pcie_cooling_set_cur_level, +}; + +struct thermal_cooling_device *pcie_cooling_device_register(struct pci_dev *port, + struct pcie_device *pdev) +{ + struct pcie_cooling_device *pcie_cdev; + struct thermal_cooling_device *cdev; + size_t name_len; + char *name; + + pcie_cdev = kzalloc(sizeof(*pcie_cdev), GFP_KERNEL); + if (!pcie_cdev) + return ERR_PTR(-ENOMEM); + + pcie_cdev->port = port; + pcie_cdev->pdev = pdev; + + name_len = strlen(COOLING_DEV_TYPE_PREFIX) + strlen(pci_name(port)) + 1; + name = kzalloc(name_len, GFP_KERNEL); + if (!name) { + kfree(pcie_cdev); + return ERR_PTR(-ENOMEM); + } + + snprintf(name, name_len, COOLING_DEV_TYPE_PREFIX "%s", pci_name(port)); + cdev = thermal_cooling_device_register(name, pcie_cdev, &pcie_cooling_ops); + kfree(name); + + return cdev; +} + +void pcie_cooling_device_unregister(struct thermal_cooling_device *cdev) +{ + struct pcie_cooling_device *pcie_cdev = cdev->devdata; + + thermal_cooling_device_unregister(cdev); + kfree(pcie_cdev); +} + +/* For bus_speed <-> state arithmetic */ +static_assert(PCIE_SPEED_2_5GT + 1 == PCIE_SPEED_5_0GT); +static_assert(PCIE_SPEED_5_0GT + 1 == PCIE_SPEED_8_0GT); +static_assert(PCIE_SPEED_8_0GT + 1 == PCIE_SPEED_16_0GT); +static_assert(PCIE_SPEED_16_0GT + 1 == PCIE_SPEED_32_0GT); +static_assert(PCIE_SPEED_32_0GT + 1 == PCIE_SPEED_64_0GT); + +MODULE_AUTHOR("Ilpo Järvinen "); +MODULE_DESCRIPTION("PCIe cooling driver"); diff --git a/include/linux/pci-bwctrl.h b/include/linux/pci-bwctrl.h index 8eae09bd03b5..366445517b72 100644 --- a/include/linux/pci-bwctrl.h +++ b/include/linux/pci-bwctrl.h @@ -11,7 +11,23 @@ #include struct pcie_device; +struct thermal_cooling_device; int bwctrl_set_current_speed(struct pcie_device *srv, enum pci_bus_speed speed); +#ifdef CONFIG_PCIE_THERMAL +struct thermal_cooling_device *pcie_cooling_device_register(struct pci_dev *port, + struct pcie_device *pdev); +void pcie_cooling_device_unregister(struct thermal_cooling_device *cdev); +#else +static inline struct thermal_cooling_device *pcie_cooling_device_register(struct pci_dev *port, + struct pcie_device *pdev) +{ + return NULL; +} +static inline void pcie_cooling_device_unregister(struct thermal_cooling_device *cdev) +{ +} +#endif + #endif