From patchwork Fri Jan 5 11:25:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ilpo_J=C3=A4rvinen?= X-Patchwork-Id: 18759 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:6f82:b0:100:9c79:88ff with SMTP id tb2csp6157687dyb; Fri, 5 Jan 2024 03:26:48 -0800 (PST) X-Google-Smtp-Source: AGHT+IGLzLkwMH2njOCSKGK+GkywSg/phq6TOQhNspsbyNDjadnE51N2CEpb/nMs8N5dr+2+AYCz X-Received: by 2002:a17:902:7ed0:b0:1d0:84f1:6fc5 with SMTP id p16-20020a1709027ed000b001d084f16fc5mr1525889plb.88.1704454008462; Fri, 05 Jan 2024 03:26:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704454008; cv=none; d=google.com; s=arc-20160816; b=c/RX79u8+lda4CM4N3f5nWLiqbMOZxuUjTZFVUvatRw6iK64BCYBe1xDFazKzJ6CEh APnF5fiTs9ci9uYrZSZBhdevpR48rcxPwtgpzY7jiFM9c/eF1XU6uGmFe7QJx54Y936H vDHvBDbTdyYcC0THHc+KzkNQtCjHzdfvRkiwEN7/Zsg0CQWxufrCjKuFYe/Sv0zZXDf8 57XgO8t2LSSi9TLM1ZuSPjAAFNbgE0zOyz27w6mcgmTYr5wWTr/E9J1FHxQi4zvU9CKH s0i0j9QCf/GyVIzmS3ZJzkI3IU/GxzwDLgD/3Zl/z1Yy3/HTFtYMuVQZ9TDArxben9KB B1kQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:message-id:date:subject:cc:to :from:dkim-signature; bh=uu7aELA3/i3jvsvwPTGWHVFbCyE8hx/2BnoPOycUumY=; fh=ynYDlS2bmtvLWUnIMcmfx3ZG6PNSDFY3DSQaU8v6LEk=; b=elJKhX9NpOhd7XsdNjxnfyaLnjr/Sl8mKhrDx7lZilGHwYNcoHavy9aDuzAlWPPjj7 6j5rP78kS2B00gn3IrC1Q0MvZKPnoPzOQd1HzZkRL8qTHDmVl6S4cuWiu2l/UhX1mTSM wGzM7XwI5MocBvhokMvLwRHq1pSwsNi9LbJ0Cm/7V+q1aAe0w4a4Yqg/7smLVi6esI+N Fkhyt5nmoR4NIaMVePr1tvvWq3hLEQO+/shT9o9nag3NMJcPcy4iXRfA7OQa0kadWc8m vvCLocNL1zE4ZIy7vdc2HamPUKiap7W3/KgrPv3qfV1bScvGM/7W1LipPskiU2cgYKzi UGjA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Kvt8mVVW; spf=pass (google.com: domain of linux-kernel+bounces-17787-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-17787-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id jh3-20020a170903328300b001d3ea5fccb5si1043935plb.121.2024.01.05.03.26.48 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Jan 2024 03:26:48 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-17787-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Kvt8mVVW; spf=pass (google.com: domain of linux-kernel+bounces-17787-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-17787-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 67C93B233C3 for ; Fri, 5 Jan 2024 11:26:22 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8A5842CCB2; Fri, 5 Jan 2024 11:26:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Kvt8mVVW" X-Original-To: linux-kernel@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B9CE728E17; Fri, 5 Jan 2024 11:26:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1704453962; x=1735989962; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=o6uJDkcI/k+9njrwFa0eJ0S2DhlFTVliRLHGjulV7UQ=; b=Kvt8mVVW8G5YbXJh00++7jiLs6bkLjT6d6vdK9fFhDsBzbZE4unJl4v7 BFgtwf0YW9n6ceLTmuYnREmLiBar5GnxbxZvZxLm2pH6guZs90Vd/SRMY tF5IdmAjRLMOX2q5IM+0fzRGp444rakfNFn0+c2O4x2E3Svl7oIi2Bneg kOx9aB1npvrZ5SDcli3lIHSLHDT7+yHZ/nI6tJojNq+CKZfffrXuU67Rt hHs41af4oxjg8WNFacwjlZq/4BV/QjqzPAtBmaP1UssZywjl98sEta2Dv RJ06sxXGJNOeSHL01jdEgPn3GsIGz0DSpdk42h7Goscr5ZnaD1wm6w18S A==; X-IronPort-AV: E=McAfee;i="6600,9927,10943"; a="483662296" X-IronPort-AV: E=Sophos;i="6.04,333,1695711600"; d="scan'208";a="483662296" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jan 2024 03:26:01 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10943"; a="730447440" X-IronPort-AV: E=Sophos;i="6.04,333,1695711600"; d="scan'208";a="730447440" Received: from ijarvine-desk1.ger.corp.intel.com (HELO localhost) ([10.246.32.38]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jan 2024 03:25:55 -0800 From: =?utf-8?q?Ilpo_J=C3=A4rvinen?= To: linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Rob Herring , =?utf-8?q?Krzysztof_Wilczy=C5=84ski?= , Lukas Wunner , Alexandru Gagniuc , Krishna chaitanya chundru , Srinivas Pandruvada , "Rafael J. Wysocki" , linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Alex Deucher , Daniel Lezcano , Amit Kucheria , Zhang Rui , =?utf-8?q?Ilpo_J=C3=A4rvinen?= Subject: [PATCH v4 0/8] Add PCIe bandwidth controller Date: Fri, 5 Jan 2024 13:25:39 +0200 Message-Id: <20240105112547.7301-1-ilpo.jarvinen@linux.intel.com> X-Mailer: git-send-email 2.39.2 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1787249566705075064 X-GMAIL-MSGID: 1787249566705075064 Hi all, This series adds PCIe bandwidth controller (bwctrl) and associated PCIe cooling driver to the thermal core side for limiting PCIe Link Speed due to thermal reasons. PCIe bandwidth controller is a PCI express bus port service driver. A cooling device is created for each port the service driver finds if they support changing speeds. This series only adds support for controlling PCIe Link Speed. Controlling PCIe Link Width might also be useful but AFAIK, there is no mechanism for that until PCIe 6.0 (L0p) so Link Width throttling is not added by this series. bwctrl is built on top of BW notifications revert. It tried to look into using cached link speed values more in code unrelated to what bwctrl needs but every case I looked non-trivial so I left such attempts as further work. v4: - Merge Port's and Endpoint's Supported Link Speeds Vectors into supported_speeds in the struct pci_bus - Reuse pcie_get_speed_cap()'s code for pcie_get_supported_speeds() - Setup supported_speeds with PCI_EXP_LNKCAP2_SLS_2_5GB when no Endpoint exists - Squash revert + add bwctrl patches into one - Change to use threaded IRQ + IRQF_ONESHOT - Enable also LABIE / LABS - Convert Link Speed selection to use bit logic instead of loop - Allocate before requesting IRQ during probe - Use devm_*() - Use u8 for speed_conv array instead of u16 - Removed READ_ONCE() - Improve changelogs, comments, and Kconfig - Name functions slightly more consistently - Use bullet list for RMW protected registers in docs v3: - Correct hfi1 shortlog prefix - Improve error prints in hfi1 - Add L: linux-pci to the MAINTAINERS entry v2: - Adds LNKCTL2 to RMW safe list in Documentation/PCI/pciebus-howto.rst - Renamed cooling devices from PCIe_Port_* to PCIe_Port_Link_Speed_* in order to plan for possibility of adding Link Width cooling devices later on - Moved struct thermal_cooling_device declaration to the correct patch - Small tweaks to Kconfig texts - Series rebased to resolve conflict (in the selftest list) Ilpo Järvinen (8): PCI: Protect Link Control 2 Register with RMW locking drm/radeon: Use RMW accessors for changing LNKCTL2 drm/amdgpu: Use RMW accessors for changing LNKCTL2 RDMA/hfi1: Use RMW accessors for changing LNKCTL2 PCI: Store all PCIe Supported Link Speeds PCI/link: Re-add BW notification portdrv as PCIe BW controller thermal: Add PCIe cooling driver selftests/pcie_bwctrl: Create selftests Documentation/PCI/pciebus-howto.rst | 14 +- MAINTAINERS | 9 + drivers/gpu/drm/amd/amdgpu/cik.c | 41 +-- drivers/gpu/drm/amd/amdgpu/si.c | 41 +-- drivers/gpu/drm/radeon/cik.c | 40 +-- drivers/gpu/drm/radeon/si.c | 40 +-- drivers/infiniband/hw/hfi1/pcie.c | 30 +- drivers/pci/pci.c | 59 ++-- drivers/pci/pcie/Kconfig | 12 + drivers/pci/pcie/Makefile | 1 + drivers/pci/pcie/bwctrl.c | 269 ++++++++++++++++++ drivers/pci/pcie/portdrv.c | 9 +- drivers/pci/pcie/portdrv.h | 10 +- drivers/pci/probe.c | 8 + drivers/pci/remove.c | 3 + drivers/thermal/Kconfig | 10 + drivers/thermal/Makefile | 2 + drivers/thermal/pcie_cooling.c | 107 +++++++ include/linux/pci-bwctrl.h | 33 +++ include/linux/pci.h | 11 + include/uapi/linux/pci_regs.h | 1 + tools/testing/selftests/Makefile | 1 + tools/testing/selftests/pcie_bwctrl/Makefile | 2 + .../pcie_bwctrl/set_pcie_cooling_state.sh | 122 ++++++++ .../selftests/pcie_bwctrl/set_pcie_speed.sh | 67 +++++ 25 files changed, 789 insertions(+), 153 deletions(-) create mode 100644 drivers/pci/pcie/bwctrl.c create mode 100644 drivers/thermal/pcie_cooling.c create mode 100644 include/linux/pci-bwctrl.h create mode 100644 tools/testing/selftests/pcie_bwctrl/Makefile create mode 100755 tools/testing/selftests/pcie_bwctrl/set_pcie_cooling_state.sh create mode 100755 tools/testing/selftests/pcie_bwctrl/set_pcie_speed.sh