From patchwork Fri Sep 15 12:01:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ilpo_J=C3=A4rvinen?= X-Patchwork-Id: 14049 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp1037102vqi; Fri, 15 Sep 2023 06:14:53 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEnAE54VkhdiQRyl+oyzXJaqQvuMz5ZjDWI6Tyn4nSvYV2Dv3uYWUn0rg1loORAyYQjfZfp X-Received: by 2002:a05:6a20:748e:b0:14e:315b:d9c with SMTP id p14-20020a056a20748e00b0014e315b0d9cmr1897331pzd.22.1694783692655; Fri, 15 Sep 2023 06:14:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694783692; cv=none; d=google.com; s=arc-20160816; b=J2nA+5reoH5ktpdjeU43Y+sL5bKv34SEri5rljkS4s8FLaKr3zuf2VJ83WCnyf6Ulq 4f5JfDDo1CnihhHdVvytJZrZ7g3qN6C5t22InytF2c5QBNLLh1C3z467NL5vrLRS/dXI sIU+82Bl8zQ/p5Dk18TnGqL7zBrqZyVYUsM4YNHb9JEA1fFRl9TnU2y5DLfxZVuJ2ipW 2jr9UjU2GFmlROrCgEJVmG0ZW4ZOtvrUpYKRBKfcYNrzWsIvxcjTFD4Fw0Io7eAv2TmW gfUcFPLNa25VHCasi9+b4CRJ/7wfoPLvGGcajofr7grkB5v8AhDm+ORnJ+0gKOE2GfJb VehA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=UTcL0hlwtHMCZVYef5dyz1ndVoU72BjOo/qOG/eYTm4=; fh=TrSvJtkCL2r+xwmHkjLtKiVT6lX2BXt9y5z9/DQVde4=; b=IKjCHwwsnxiyxeaOaBPHN39mn4vtLIjXlV0Hy4eXpNh4xfKf9pe6W8sBQBKVjsde4X /NhlZhOxpd5vgZKI/NwldlOEVEy2nkjKi2NOIk2E5I44KvvIj7erLnIQldrfLsY5tcJC UqVfsMMhMitKu5RkbCtowscEsLFS/Ra9B+7VaxPZCYp9xuna7/BfPpGvkzuFqhxXDUkg 0ktHzVql9vJTgiw6zd7iBgCr52IFeTjBs5MDYqLNhX/W/IBCG/NONtQYvndGABApr4mr 7PVZITKz4iYqbvqqMHTVlSdYmZ1IhBs2m+If/Ra4PmWMengj1GXJdgHW+YHCWXYnj82l /GqQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=cKo8F8Wf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id k24-20020a63ff18000000b00565f65e16bfsi3151815pgi.762.2023.09.15.06.14.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 06:14:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=cKo8F8Wf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id EBF2B8325E8F; Fri, 15 Sep 2023 05:02:11 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234185AbjIOMCB (ORCPT + 32 others); Fri, 15 Sep 2023 08:02:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232836AbjIOMCA (ORCPT ); Fri, 15 Sep 2023 08:02:00 -0400 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2164E1FF1; Fri, 15 Sep 2023 05:01:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1694779315; x=1726315315; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=Ge0rc2WPYKSx8MYcdUbDDS1gOtJoE56w9sC9a+IVtOI=; b=cKo8F8WfnP++cJzvCn4A6YMhijkQjv9zpog32B/S5QzXAkaPi40yNfsY N8i9Be0MSdKrXQ8MZN6ntXIhXJmGmEYM3k4sUnXL7soqgE+dTdkkeoajk bZ+cf7JG83Tz6MHA/6bhOb6ps9clzfQeocYs7AlSEyIk90rH39t5ZgPcj ZrLpRVEDtlFPf44Qt6VhnYv/j6o+WMqjvWNCuoKZYfotaWRx4eUO8eHJD 2mJNMHdlxp4sh8FlW+i6iQetZxhSfmrdls1pLRGNTzgdmaCELXBkN19Kq jv4/7fr/OPm2sPma6XYAgvVwQ7J0Tdh053Vigr+zm7+5gjb/zj7NYV6Q2 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10833"; a="378145904" X-IronPort-AV: E=Sophos;i="6.02,149,1688454000"; d="scan'208";a="378145904" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Sep 2023 05:01:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10833"; a="774292728" X-IronPort-AV: E=Sophos;i="6.02,149,1688454000"; d="scan'208";a="774292728" Received: from srdoo-mobl1.ger.corp.intel.com (HELO ijarvine-mobl2.ger.corp.intel.com) ([10.252.38.99]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Sep 2023 05:01:50 -0700 From: =?utf-8?q?Ilpo_J=C3=A4rvinen?= To: linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Rob Herring , =?utf-8?q?Krzysztof_Wilczy=C5=84ski?= , Lukas Wunner , Alexandru Gagniuc , Krishna chaitanya chundru , Srinivas Pandruvada , "Rafael J . Wysocki" , linux-pm@vger.kernel.org Cc: LKML , Alex Deucher , Daniel Lezcano , Amit Kucheria , Zhang Rui , =?utf-8?q?Ilpo_J=C3=A4rvinen?= Subject: [PATCH v2 00/10] Add PCIe Bandwidth Controller Date: Fri, 15 Sep 2023 15:01:32 +0300 Message-Id: <20230915120142.32987-1-ilpo.jarvinen@linux.intel.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Fri, 15 Sep 2023 05:02:12 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777109505006754792 X-GMAIL-MSGID: 1777109505006754792 Hi all, This series adds PCIe bandwidth controller (bwctrl) and associated PCIe cooling driver to the thermal core side for limiting PCIe Link Speed due to thermal reasons. PCIe bandwidth controller is a PCI express bus port service driver. A cooling device is created for each port the service driver finds if they support changing speeds. This series only adds support for controlling PCIe Link Speed. Controlling PCIe Link Width might also be useful but AFAIK, there is no mechanism for that until PCIe 6.0 (L0p). Based on feedback for v1, the thermal/cooling device side prefers Link Speed and Link Width to be separate cooling devices [1] which is taken into account in naming the cooling device for Link Speed but the Link Width one is not added yet as it would not be able to control anything at the moment. bwctrl is built on top of BW notifications revert. I'm just not sure what is the best practice when re-adding some old functionality in a modified form so please let me know if I need to somehow alter that patch. [1] https://lore.kernel.org/linux-pci/f35db90cd67adf4b0f48cd6f2a6ad8fbd0c1a679.camel@linux.intel.com/ v2: - Adds LNKCTL2 to RMW safe list in Documentation/PCI/pciebus-howto.rst - Renamed cooling devices from PCIe_Port_* to PCIe_Port_Link_Speed_* in order to plan for possibility of adding Link Width cooling devices later on - Moved struct thermal_cooling_device declaration to the correct patch - Small tweaks to Kconfig texts - Series rebased to resolve conflict (in the selftest list) Ilpo Järvinen (10): PCI: Protect Link Control 2 Register with RMW locking drm/radeon: Use RMW accessors for changing LNKCTL2 drm/amdgpu: Use RMW accessors for changing LNKCTL2 drm/IB/hfi1: Use RMW accessors for changing LNKCTL2 PCI: Store all PCIe Supported Link Speeds PCI: Cache PCIe device's Supported Speed Vector PCI/LINK: Re-add BW notification portdrv as PCIe BW controller PCI/bwctrl: Add "controller" part into PCIe bwctrl thermal: Add PCIe cooling driver selftests/pcie_bwctrl: Create selftests Documentation/PCI/pciebus-howto.rst | 8 +- MAINTAINERS | 8 + drivers/gpu/drm/amd/amdgpu/cik.c | 41 +-- drivers/gpu/drm/amd/amdgpu/si.c | 41 +-- drivers/gpu/drm/radeon/cik.c | 40 +-- drivers/gpu/drm/radeon/si.c | 40 +-- drivers/infiniband/hw/hfi1/pcie.c | 30 +- drivers/pci/pcie/Kconfig | 9 + drivers/pci/pcie/Makefile | 1 + drivers/pci/pcie/bwctrl.c | 309 ++++++++++++++++++ drivers/pci/pcie/portdrv.c | 9 +- drivers/pci/pcie/portdrv.h | 10 +- drivers/pci/probe.c | 38 ++- drivers/pci/remove.c | 2 + drivers/thermal/Kconfig | 10 + drivers/thermal/Makefile | 2 + drivers/thermal/pcie_cooling.c | 107 ++++++ include/linux/pci-bwctrl.h | 33 ++ include/linux/pci.h | 3 + include/uapi/linux/pci_regs.h | 1 + tools/testing/selftests/Makefile | 1 + tools/testing/selftests/pcie_bwctrl/Makefile | 2 + .../pcie_bwctrl/set_pcie_cooling_state.sh | 122 +++++++ .../selftests/pcie_bwctrl/set_pcie_speed.sh | 67 ++++ 24 files changed, 799 insertions(+), 135 deletions(-) create mode 100644 drivers/pci/pcie/bwctrl.c create mode 100644 drivers/thermal/pcie_cooling.c create mode 100644 include/linux/pci-bwctrl.h create mode 100644 tools/testing/selftests/pcie_bwctrl/Makefile create mode 100755 tools/testing/selftests/pcie_bwctrl/set_pcie_cooling_state.sh create mode 100755 tools/testing/selftests/pcie_bwctrl/set_pcie_speed.sh