From patchwork Fri Sep 29 11:57:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ilpo_J=C3=A4rvinen?= X-Patchwork-Id: 14725 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp4137419vqu; Fri, 29 Sep 2023 09:00:21 -0700 (PDT) X-Google-Smtp-Source: AGHT+IG/NoD+uThLnjGrg2vCMzJEj/Esw7E1vFXkcyf7gA2M2CyHFS1CqZr0lyA/QTVZiVd9Y85t X-Received: by 2002:a17:902:c409:b0:1c5:ea60:85c1 with SMTP id k9-20020a170902c40900b001c5ea6085c1mr5637309plk.12.1696003220382; Fri, 29 Sep 2023 09:00:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696003220; cv=none; d=google.com; s=arc-20160816; b=BU16J+c9Gw2hXu3p+jqBMs63HmxTO/pq5APZQTca+9XFMBDCQNiNfA7AafQRe6Dqd9 CkeiQfFznon9gzhiY/sze1/t8pK59X0oEVdsiR3FiyePW/tSS1HsdRSKzoNG4m2ND1IO 3OSzyr4rZ8DSIU/CRyV5oyiBoUkcVpFC/YwvIJlNTWU/YBinyu1K+qJiy2/KGjy2y4Cg sYpIT7VZeUPa+ln+Q4pNzyX9DIB+xTpyeCmbr4SOiGyaC7aHgeXx+K1UTDPrrXvZwJOh vJgL+9A7XTAI3SFCOqyhW5OQIHwuW0EqTuVuSMtogbvW0tea/KgbsWgm6Ylb1u+Y0Dli lHdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=PT/gxRoshkKTmcPZcWfZDHHm18qFEqlVOXhm8WI1FfY=; fh=TrSvJtkCL2r+xwmHkjLtKiVT6lX2BXt9y5z9/DQVde4=; b=Rh4I0Kd5d6rzqqj4t+YkxFtpdNfOJtbMQa+E3sDFsoPyVvXdSHaIIvueYmNI8A7NW1 VaTZbB/IApWKp1We0e9DyptTw0aOYRlyS43oizVtUU71BD0sZUdFG657d+NMYaLBSzLM FXdT02G+GzWKtt2r7HmwYPqqI0OvwksURdP8CWOw4sC8mvnIHTi3Ucbh3uTiOWSq4ReQ HPXmO1AShHVgqdrb86hXLLE4u6Sc8/5UAvOlWhEvLDUcYBNhiX6HFyuzRZOGPvPshig4 N1vVd7x7xsW0hw9Gf2DT+v4wTQbQujElQYOn+6LQ7LDRVFGBFBrWcrWX/cGieQlT6tQP 7HOg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=mAaTDsqR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id u12-20020a170902e80c00b001bb9e2c38ecsi22669673plg.264.2023.09.29.09.00.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Sep 2023 09:00:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=mAaTDsqR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 1F69E82A88BB; Fri, 29 Sep 2023 04:58:03 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233104AbjI2L5p (ORCPT + 20 others); Fri, 29 Sep 2023 07:57:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55326 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229508AbjI2L5o (ORCPT ); Fri, 29 Sep 2023 07:57:44 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1A830193; Fri, 29 Sep 2023 04:57:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1695988661; x=1727524661; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=O9aEWMt/4svyxRYZ/LYDGVClZj5FenXgVAAtjploJC8=; b=mAaTDsqR9Vd1YgUXYrqv8GzYHjxJnn3Ft89F4ebzvLbAV/1vf4kPVIhz e9Lbh3dVnhsdn8HXoi3LhXpyy4vN95YsyxAbEnJNJrlHJmCQihZMdijTu 7afZrCavWuqh3fwQiPW6kr82k4LwdbfZUgT8rGkbK/Z2uX0WmB55kxREI awj66U9N/rkpr7eDHU+SdHQQDkqjkoSeXP3M8Owj+zT2fE0RttwhWoYR/ REYsQ8dA0tbMN4KK1n7tl7tmGmfqIpsLOn+MGSA5SoseVFdPiAYpWBB/3 1svPJxGK1MDJS4q3rhPt9dk/xLK+HhbvUyAreiTAMNwDBGOSv6N4L2L5S A==; X-IronPort-AV: E=McAfee;i="6600,9927,10847"; a="361658664" X-IronPort-AV: E=Sophos;i="6.03,187,1694761200"; d="scan'208";a="361658664" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Sep 2023 04:57:39 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10847"; a="873668379" X-IronPort-AV: E=Sophos;i="6.03,187,1694761200"; d="scan'208";a="873668379" Received: from valeks2x-mobl.ger.corp.intel.com (HELO localhost) ([10.252.53.242]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Sep 2023 04:57:32 -0700 From: =?utf-8?q?Ilpo_J=C3=A4rvinen?= To: linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Rob Herring , =?utf-8?q?Krzysztof_Wilczy=C5=84ski?= , Lukas Wunner , Alexandru Gagniuc , Krishna chaitanya chundru , Srinivas Pandruvada , "Rafael J . Wysocki" , linux-pm@vger.kernel.org Cc: LKML , Alex Deucher , Daniel Lezcano , Amit Kucheria , Zhang Rui , =?utf-8?q?Ilpo_J=C3=A4rvinen?= Subject: [PATCH v3 00/10] Add PCIe Bandwidth Controller Date: Fri, 29 Sep 2023 14:57:13 +0300 Message-Id: <20230929115723.7864-1-ilpo.jarvinen@linux.intel.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Fri, 29 Sep 2023 04:58:03 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1778388272099503239 X-GMAIL-MSGID: 1778388272099503239 Hi all, This series adds PCIe bandwidth controller (bwctrl) and associated PCIe cooling driver to the thermal core side for limiting PCIe Link Speed due to thermal reasons. PCIe bandwidth controller is a PCI express bus port service driver. A cooling device is created for each port the service driver finds if they support changing speeds. This series only adds support for controlling PCIe Link Speed. Controlling PCIe Link Width might also be useful but AFAIK, there is no mechanism for that until PCIe 6.0 (L0p). Based on feedback for v1, the thermal/cooling device side prefers Link Speed and Link Width to be separate cooling devices [1] which is taken into account in naming the cooling device for Link Speed but the Link Width one is not added yet as it would not be able to control anything at the moment. bwctrl is built on top of BW notifications revert. I'm just not sure what is the best practice when re-adding some old functionality in a modified form so please let me know if I need to somehow alter that patch. [1] https://lore.kernel.org/linux-pci/f35db90cd67adf4b0f48cd6f2a6ad8fbd0c1a679.camel@linux.intel.com/ v3: - Correct hfi1 shortlog prefix - Improve error prints in hfi1 - Add L: linux-pci to the MAINTAINERS entry v2: - Adds LNKCTL2 to RMW safe list in Documentation/PCI/pciebus-howto.rst - Renamed cooling devices from PCIe_Port_* to PCIe_Port_Link_Speed_* in order to plan for possibility of adding Link Width cooling devices later on - Moved struct thermal_cooling_device declaration to the correct patch - Small tweaks to Kconfig texts - Series rebased to resolve conflict (in the selftest list) Ilpo Järvinen (10): PCI: Protect Link Control 2 Register with RMW locking drm/radeon: Use RMW accessors for changing LNKCTL2 drm/amdgpu: Use RMW accessors for changing LNKCTL2 RDMA/hfi1: Use RMW accessors for changing LNKCTL2 PCI: Store all PCIe Supported Link Speeds PCI: Cache PCIe device's Supported Speed Vector PCI/LINK: Re-add BW notification portdrv as PCIe BW controller PCI/bwctrl: Add "controller" part into PCIe bwctrl thermal: Add PCIe cooling driver selftests/pcie_bwctrl: Create selftests Documentation/PCI/pciebus-howto.rst | 8 +- MAINTAINERS | 9 + drivers/gpu/drm/amd/amdgpu/cik.c | 41 +-- drivers/gpu/drm/amd/amdgpu/si.c | 41 +-- drivers/gpu/drm/radeon/cik.c | 40 +-- drivers/gpu/drm/radeon/si.c | 40 +-- drivers/infiniband/hw/hfi1/pcie.c | 30 +- drivers/pci/pcie/Kconfig | 9 + drivers/pci/pcie/Makefile | 1 + drivers/pci/pcie/bwctrl.c | 309 ++++++++++++++++++ drivers/pci/pcie/portdrv.c | 9 +- drivers/pci/pcie/portdrv.h | 10 +- drivers/pci/probe.c | 38 ++- drivers/pci/remove.c | 2 + drivers/thermal/Kconfig | 10 + drivers/thermal/Makefile | 2 + drivers/thermal/pcie_cooling.c | 107 ++++++ include/linux/pci-bwctrl.h | 33 ++ include/linux/pci.h | 3 + include/uapi/linux/pci_regs.h | 1 + tools/testing/selftests/Makefile | 1 + tools/testing/selftests/pcie_bwctrl/Makefile | 2 + .../pcie_bwctrl/set_pcie_cooling_state.sh | 122 +++++++ .../selftests/pcie_bwctrl/set_pcie_speed.sh | 67 ++++ 24 files changed, 800 insertions(+), 135 deletions(-) create mode 100644 drivers/pci/pcie/bwctrl.c create mode 100644 drivers/thermal/pcie_cooling.c create mode 100644 include/linux/pci-bwctrl.h create mode 100644 tools/testing/selftests/pcie_bwctrl/Makefile create mode 100755 tools/testing/selftests/pcie_bwctrl/set_pcie_cooling_state.sh create mode 100755 tools/testing/selftests/pcie_bwctrl/set_pcie_speed.sh