From patchwork Wed Oct 19 17:32:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 5782 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp451621wrs; Wed, 19 Oct 2022 10:37:18 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5PnHskey/f/rr+uojKrYUHakeUQ61bpVNyDCoY0ysnoJtyVDx9o8QiqUfaqPOjtyAapDFz X-Received: by 2002:a17:907:1b1e:b0:783:8e33:2d1c with SMTP id mp30-20020a1709071b1e00b007838e332d1cmr7693452ejc.304.1666201038309; Wed, 19 Oct 2022 10:37:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666201038; cv=none; d=google.com; s=arc-20160816; b=YlPduUN+wSI0kOnsE1dAgraEbJUyAEbIPJ0JWl2za0jkhLgI2XDw9GaQiszJ0eY7D2 5k57ht/e1ni/8rg0MJ1PdVQcrGvD3PN9w4y5gafE3Ln5oNNxKSunN/eSH21rvygqWwzN zSq/CxJCdN8DWCOdkWkATsM2DzX5WL5VGkHNyroaVrqYcwYLV8Pem85r4uu0HXqIcxYv fg20FWlIkuRHznag9umYBKRtzZFFIbhMr7HAyI6xTr60MS2TUwhfQmGit1EpJCRxtwkp Du/mKvcfy3lA03FjHNr/PAJl6torMZ0y8HCUHRP7IlNcIPW+zIOGUEFhFyLIYeSTC6zJ vwLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=xSZMHmF+625r5KWwqWitIJVKDEtziD+dgPsRudbOhqk=; b=wTw/xqY6+Jte34icmU2q5jDr7IK9Kzp+S5qfbtysfuJpy3T4jLWlZW/K2vSVSG7GWT z2isJ/eYC63QK70BdEm6c6yVbnluP0a1LTZn/FpqHsTIt+ouqBxD20KUWRsSClBXmDIG b58xF6tFuBwDEr7asooMxFOnP5lnbJoP3i6cyWsBEM5DsfnmEQKOKV9wykl2nRubnr+t JreIL45Tzk3VJdTd/8kZTP6fgIP1npfMvHGX7a11F+iTvNaGYJi/Q5ASfAkLx+5POXri DJcK9MwTkFME+dObjc3EI0yixmAHIdakFdGdA7zjdYlSFvopwLVfKL4CUbRuZzP2bxcg j5tg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Nfa62ouY; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y3-20020a170906524300b0078d148dafa3si12845981ejm.329.2022.10.19.10.36.50; Wed, 19 Oct 2022 10:37:18 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Nfa62ouY; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230420AbiJSRd1 (ORCPT + 99 others); Wed, 19 Oct 2022 13:33:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60468 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229862AbiJSRdY (ORCPT ); Wed, 19 Oct 2022 13:33:24 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F2451BFB8A; Wed, 19 Oct 2022 10:33:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666200803; x=1697736803; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QHrhf+AzAmghP60cmLV+8/AQtnNJCeogCK+jfUMlLc8=; b=Nfa62ouY6u2xXa2nrXHT5p5uTjcNv9prYRy5TVnMGykGlgNvePoRi3Ls /cwMoDbfHElzScVpQVclDkTPgpGhDfoBn9rxXTPsRqcg32uVr5PJfKNCl xI2R7GpiX3dqtIHrG/kZetZk53YfT7r/hszNJdSN0vYc7wuCb8mrNE3Wi wtj8BRy0B0YQhdR7ENfqfmVBQrhgUYdMeHp1RXrOwGSLDbwlN+ID1c3Tv 89QddjMG6dM/f6FXN3tYjdjeW+dKzaU5b7VOgMRk20mGus7wCv04Sow8G Px7Zl0Ip5FEmuVuOkC2qvBBstwFudVKKrfNMOVQeWwyZ2rNc4pW0kf/M9 Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="306474288" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="306474288" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:22 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="607204686" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="607204686" Received: from mjmcener-mobl1.amr.corp.intel.com (HELO localhost.localdomain) ([10.213.233.40]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:19 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 01/17] cgroup: Add the DRM cgroup controller Date: Wed, 19 Oct 2022 18:32:38 +0100 Message-Id: <20221019173254.3361334-2-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> References: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747138419672735275?= X-GMAIL-MSGID: =?utf-8?q?1747138419672735275?= From: Tvrtko Ursulin Skeleton controller without any functionality. Signed-off-by: Tvrtko Ursulin --- include/linux/cgroup_drm.h | 9 ++++++ include/linux/cgroup_subsys.h | 4 +++ init/Kconfig | 7 +++++ kernel/cgroup/Makefile | 1 + kernel/cgroup/drm.c | 54 +++++++++++++++++++++++++++++++++++ 5 files changed, 75 insertions(+) create mode 100644 include/linux/cgroup_drm.h create mode 100644 kernel/cgroup/drm.c diff --git a/include/linux/cgroup_drm.h b/include/linux/cgroup_drm.h new file mode 100644 index 000000000000..bf8abc6b8ebf --- /dev/null +++ b/include/linux/cgroup_drm.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright © 2022 Intel Corporation + */ + +#ifndef _CGROUP_DRM_H +#define _CGROUP_DRM_H + +#endif /* _CGROUP_DRM_H */ diff --git a/include/linux/cgroup_subsys.h b/include/linux/cgroup_subsys.h index 445235487230..49460494a010 100644 --- a/include/linux/cgroup_subsys.h +++ b/include/linux/cgroup_subsys.h @@ -65,6 +65,10 @@ SUBSYS(rdma) SUBSYS(misc) #endif +#if IS_ENABLED(CONFIG_CGROUP_DRM) +SUBSYS(drm) +#endif + /* * The following subsystems are not supported on the default hierarchy. */ diff --git a/init/Kconfig b/init/Kconfig index 694f7c160c9c..6dd7faca7749 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1087,6 +1087,13 @@ config CGROUP_RDMA Attaching processes with active RDMA resources to the cgroup hierarchy is allowed even if can cross the hierarchy's limit. +config CGROUP_DRM + bool "DRM controller" + help + Provides the DRM subsystem controller. + + ... + config CGROUP_FREEZER bool "Freezer controller" help diff --git a/kernel/cgroup/Makefile b/kernel/cgroup/Makefile index 12f8457ad1f9..849bd2917477 100644 --- a/kernel/cgroup/Makefile +++ b/kernel/cgroup/Makefile @@ -6,4 +6,5 @@ obj-$(CONFIG_CGROUP_PIDS) += pids.o obj-$(CONFIG_CGROUP_RDMA) += rdma.o obj-$(CONFIG_CPUSETS) += cpuset.o obj-$(CONFIG_CGROUP_MISC) += misc.o +obj-$(CONFIG_CGROUP_DRM) += drm.o obj-$(CONFIG_CGROUP_DEBUG) += debug.o diff --git a/kernel/cgroup/drm.c b/kernel/cgroup/drm.c new file mode 100644 index 000000000000..b88c93661df3 --- /dev/null +++ b/kernel/cgroup/drm.c @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright © 2022 Intel Corporation + */ + +#include +#include +#include +#include + +struct drm_cgroup_state { + struct cgroup_subsys_state css; +}; + +static inline struct drm_cgroup_state * +css_to_drmcs(struct cgroup_subsys_state *css) +{ + return container_of(css, struct drm_cgroup_state, css); +} + +static void drmcs_free(struct cgroup_subsys_state *css) +{ + kfree(css_to_drmcs(css)); +} + +static struct drm_cgroup_state root_drmcs = { +}; + +static struct cgroup_subsys_state * +drmcs_alloc(struct cgroup_subsys_state *parent_css) +{ + struct drm_cgroup_state *drmcs; + + if (!parent_css) + return &root_drmcs.css; + + drmcs = kzalloc(sizeof(*drmcs), GFP_KERNEL); + if (!drmcs) + return ERR_PTR(-ENOMEM); + + return &drmcs->css; +} + +struct cftype files[] = { + { } /* Zero entry terminates. */ +}; + +struct cgroup_subsys drm_cgrp_subsys = { + .css_alloc = drmcs_alloc, + .css_free = drmcs_free, + .early_init = false, + .legacy_cftypes = files, + .dfl_cftypes = files, +}; From patchwork Wed Oct 19 17:32:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 5781 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp451340wrs; Wed, 19 Oct 2022 10:36:52 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4IUSm2jU37KKsf4x8oox1XitZqt4z5THvb9WVZeQEN+bNwx/jQcGc6GV9RUvj9iDWKJXQy X-Received: by 2002:a05:6402:170f:b0:458:9653:6466 with SMTP id y15-20020a056402170f00b0045896536466mr8605432edu.181.1666201011871; Wed, 19 Oct 2022 10:36:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666201011; cv=none; d=google.com; s=arc-20160816; b=j2b8TQ8r8SDfpWpWAhsAVbGEPzIcRUguGeADHxmlJ3rMFgXiZ8MN1hAC1kI2rVUGtK /90CLhcVk4WSREXdDrdKxNH79SqWTlYslE1abRW9CPPxSGBtiuw47CI+rOIAt1rSvmnU B1xYYx9P/5gjcqvBhKvcYOt+M1sPAJPXVRS+s6eyfXP9S8wAskfHBuDFkmgoPEF/pjK3 T+iapfCltaLzwpDqQeR6MGlPMJ1pC4rzsMVDfRj/szZhhyd41P1bqfkvy5ndSlPAbCGh dwChyA69ghTk97Hpi5DF1zHerQsQCDjPqYJS/JoBIUEZ+wMAc5SpPQZBNXXExdb7z37h 6ehw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=WadaNEmSLG49Eme1He5P3yIfjQs3fNME429HI1S5wKU=; b=j14WfKzke9brOZo/8PPELSlkmc1eZ4D5S2zRHndq9HcVsIKNErC15MJgpVQvfGs8KK yfRcw6RpgO02MwururYY/yw4xWqidRSiDZ/Qc75yw38mU++M0ElpujBCtG96KMXeQqQy zxNuoUDhtqdBNCryyKbUOCeRsXhk+OUpB/9XvsVmN09JaaMR6dqMVzHmjOpX5Efuf5Gx OAtIPo48yguCbYBZ5W3QSbdb+ck15gvoNQhXA4utAEBW6S4TE0g65Bn+PncUKqnAJSMN TWvODwx22jKyKRhvJ8a2idLZmmr9/i9uPnH7oEINCKIeuoseQhuR34xju6+GY7X1c6Is CHoQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=SxQl+4FR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j20-20020a1709066dd400b0078d9b2facaesi13418575ejt.339.2022.10.19.10.36.23; Wed, 19 Oct 2022 10:36:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=SxQl+4FR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230501AbiJSRdd (ORCPT + 99 others); Wed, 19 Oct 2022 13:33:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230136AbiJSRd1 (ORCPT ); Wed, 19 Oct 2022 13:33:27 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C9B391C20BF; Wed, 19 Oct 2022 10:33:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666200806; x=1697736806; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=34jcEyaboM7Cpl5hmtqGUgBiY4OSzKr0bOyVxNTPRuA=; b=SxQl+4FRxYkORNOe0E29yxbuq18rp/vkb10ImMboVEMmhd5MSqrwwD5s pjefJ4UB4COb2dMzKcAwHXB55DqqpjDKwjCrvGaH6wXldzVng4z/4CGGT ZvZYTYbPskHJhLakqLVJsZeqhP5bHNqikUfnr9kHr096R2hDMih5CGQYP NA6eOPh8Xo4JlLDgy7rjqaVisHisBnnlkqg747G7nWK0ujN2xMJ3t2XEM LmuSdb2ubbUC84Stmt5F84WEdJdeBdE870AH6gy10ksRb32HqJZHsSmi3 NVbd8eb/U4VhsnC+Un2XLGRShFwcwrfY1reyokCg6YLz2BBAjjNgTRFbu g==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="306474309" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="306474309" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:26 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="607204700" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="607204700" Received: from mjmcener-mobl1.amr.corp.intel.com (HELO localhost.localdomain) ([10.213.233.40]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:22 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 02/17] drm: Track clients per owning process Date: Wed, 19 Oct 2022 18:32:39 +0100 Message-Id: <20221019173254.3361334-3-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> References: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747138392276466543?= X-GMAIL-MSGID: =?utf-8?q?1747138392276466543?= From: Tvrtko Ursulin To enable propagation of settings from the cgroup drm controller to drm we need to start tracking which processes own which drm clients. Implement that by tracking the struct pid pointer of the owning process in a new XArray, pointing to a structure containing a list of associated struct drm_file pointers. Clients are added and removed under the filelist mutex and RCU list operations are used below it to allow for lockless lookup. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/Makefile | 1 + drivers/gpu/drm/drm_cgroup.c | 60 ++++++++++++++++++++++++++++++++++++ drivers/gpu/drm/drm_file.c | 18 ++++++++--- include/drm/drm_clients.h | 31 +++++++++++++++++++ include/drm/drm_file.h | 4 +++ 5 files changed, 110 insertions(+), 4 deletions(-) create mode 100644 drivers/gpu/drm/drm_cgroup.c create mode 100644 include/drm/drm_clients.h diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile index 6e55c47288e4..0719970d17ee 100644 --- a/drivers/gpu/drm/Makefile +++ b/drivers/gpu/drm/Makefile @@ -59,6 +59,7 @@ drm-$(CONFIG_DRM_LEGACY) += \ drm_scatter.o \ drm_vm.o drm-$(CONFIG_DRM_LIB_RANDOM) += lib/drm_random.o +drm-$(CONFIG_CGROUP_DRM) += drm_cgroup.o drm-$(CONFIG_COMPAT) += drm_ioc32.o drm-$(CONFIG_DRM_PANEL) += drm_panel.o drm-$(CONFIG_OF) += drm_of.o diff --git a/drivers/gpu/drm/drm_cgroup.c b/drivers/gpu/drm/drm_cgroup.c new file mode 100644 index 000000000000..a31ff1d593ab --- /dev/null +++ b/drivers/gpu/drm/drm_cgroup.c @@ -0,0 +1,60 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright © 2022 Intel Corporation + */ + +#include +#include + +static DEFINE_XARRAY(drm_pid_clients); + +void drm_clients_close(struct drm_file *file_priv) +{ + unsigned long pid = (unsigned long)file_priv->pid; + struct drm_device *dev = file_priv->minor->dev; + struct drm_pid_clients *clients; + + lockdep_assert_held(&dev->filelist_mutex); + + clients = xa_load(&drm_pid_clients, pid); + list_del_rcu(&file_priv->clink); + if (atomic_dec_and_test(&clients->num)) { + xa_erase(&drm_pid_clients, pid); + kfree_rcu(clients, rcu); + } +} + +int drm_clients_open(struct drm_file *file_priv) +{ + unsigned long pid = (unsigned long)file_priv->pid; + struct drm_device *dev = file_priv->minor->dev; + struct drm_pid_clients *clients; + bool new_client = false; + + lockdep_assert_held(&dev->filelist_mutex); + + clients = xa_load(&drm_pid_clients, pid); + if (!clients) { + clients = kmalloc(sizeof(*clients), GFP_KERNEL); + if (!clients) + return -ENOMEM; + atomic_set(&clients->num, 0); + INIT_LIST_HEAD(&clients->file_list); + init_rcu_head(&clients->rcu); + new_client = true; + } + atomic_inc(&clients->num); + list_add_tail_rcu(&file_priv->clink, &clients->file_list); + if (new_client) { + void *xret; + + xret = xa_store(&drm_pid_clients, pid, clients, GFP_KERNEL); + if (xa_err(xret)) { + list_del_init(&file_priv->clink); + kfree(clients); + return PTR_ERR(clients); + } + } + + return 0; +} diff --git a/drivers/gpu/drm/drm_file.c b/drivers/gpu/drm/drm_file.c index a8b4d918e9a3..ce58d5c513db 100644 --- a/drivers/gpu/drm/drm_file.c +++ b/drivers/gpu/drm/drm_file.c @@ -40,6 +40,7 @@ #include #include +#include #include #include #include @@ -298,6 +299,7 @@ static void drm_close_helper(struct file *filp) mutex_lock(&dev->filelist_mutex); list_del(&file_priv->lhead); + drm_clients_close(file_priv); mutex_unlock(&dev->filelist_mutex); drm_file_free(file_priv); @@ -349,10 +351,8 @@ static int drm_open_helper(struct file *filp, struct drm_minor *minor) if (drm_is_primary_client(priv)) { ret = drm_master_open(priv); - if (ret) { - drm_file_free(priv); - return ret; - } + if (ret) + goto err_free; } filp->private_data = priv; @@ -360,6 +360,9 @@ static int drm_open_helper(struct file *filp, struct drm_minor *minor) priv->filp = filp; mutex_lock(&dev->filelist_mutex); + ret = drm_clients_open(priv); + if (ret) + goto err_unlock; list_add(&priv->lhead, &dev->filelist); mutex_unlock(&dev->filelist_mutex); @@ -387,6 +390,13 @@ static int drm_open_helper(struct file *filp, struct drm_minor *minor) #endif return 0; + +err_unlock: + mutex_unlock(&dev->filelist_mutex); +err_free: + drm_file_free(priv); + + return ret; } /** diff --git a/include/drm/drm_clients.h b/include/drm/drm_clients.h new file mode 100644 index 000000000000..4ae553a03d1e --- /dev/null +++ b/include/drm/drm_clients.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright © 2022 Intel Corporation + */ + +#ifndef _DRM_CLIENTS_H_ +#define _DRM_CLIENTS_H_ + +#include + +struct drm_pid_clients { + atomic_t num; + struct list_head file_list; + struct rcu_head rcu; +}; + +#if IS_ENABLED(CONFIG_CGROUP_DRM) +void drm_clients_close(struct drm_file *file_priv); +int drm_clients_open(struct drm_file *file_priv); +#else +static inline void drm_clients_close(struct drm_file *file_priv) +{ +} + +static inline int drm_clients_open(struct drm_file *file_priv) +{ + return 0; +} +#endif + +#endif diff --git a/include/drm/drm_file.h b/include/drm/drm_file.h index d780fd151789..0965eb111f24 100644 --- a/include/drm/drm_file.h +++ b/include/drm/drm_file.h @@ -268,6 +268,10 @@ struct drm_file { /** @minor: &struct drm_minor for this file. */ struct drm_minor *minor; +#if IS_ENABLED(CONFIG_CGROUP_DRM) + struct list_head clink; +#endif + /** * @object_idr: * From patchwork Wed Oct 19 17:32:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 5783 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp451632wrs; Wed, 19 Oct 2022 10:37:19 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6yq7E1mnYjY9ysYIy0R1kPCtMaVGq78t7ymFURBl8ewkXxA/wAU5y43faUIKovuc882JAp X-Received: by 2002:aa7:cc13:0:b0:453:52dc:1bbf with SMTP id q19-20020aa7cc13000000b0045352dc1bbfmr8628651edt.30.1666201039747; Wed, 19 Oct 2022 10:37:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666201039; cv=none; d=google.com; s=arc-20160816; b=uhPGWn1Yu2S0Lc4Rde0ru5jP2LPeIPgwiJ9YJ4TPHfgPB4bFNhW5Fuuvk1DLTkvomK cGCrHb8APD2U3SBaqlVGrCb6fnpssABkOusBxrKEd+BBrS9d2fZe9k6cWKSl46Vo+uc0 gjRtn33H8Kp0jh/iBlT7mBWQxi136Y9uHFMz80DPWJ/TMLdZ0Ls2lKVNR6oIGRCyyuDO mdTPdC+jO9IncSq3TTFZm3hgWMEijkrgeMajUiiQ83E8SJx+VyTor2L6++5zbGVI7dfy Pu1x0SF2X9jm9MkZYfASaHl4z8G2IzSYRO6deEuIDISnbBirK+mf72qaXGA0QhLpGD3+ w0XA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Z9FmI03lC8vHDYAC28fMTPf9XVVHPd0QzbyD4AyVuTU=; b=e990nSsD1tUeKdwFIBJ2YUgY+Jm9DVDtUSl/c4vsfQYWSPu1M9fVEZ9edHy0Kzi8Mj 8wy0qd5nCa0khfBsK/0qLAXlnwp4M5RILRtC3mEQeRzh9kQW2rZwjHueeAqdvqMesf8Q 4by/Ftrswcak8D4KeMNKu+vKWViP77lhBNt4uaJ2Brz2RHXRHSD1kS3dR8sIDOG7wAvT /TAo/cXt+rerWcPVQtWZAsEANDjPVKCoPhmEkbZ0AYOb/TK7/W+12fj+HkDfkASrYsGP D3+jb58TIvr4mfApfQz63NjdY/CFEjktGNCsVyzUoAg2fNblDnDLsWSZyiPCD1/yX3Eg 2ODw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=klzOA7UX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bo17-20020a0564020b3100b0045c234508e2si12338100edb.627.2022.10.19.10.36.54; Wed, 19 Oct 2022 10:37:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=klzOA7UX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230455AbiJSRdf (ORCPT + 99 others); Wed, 19 Oct 2022 13:33:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60598 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230432AbiJSRdb (ORCPT ); Wed, 19 Oct 2022 13:33:31 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 453131C7107; Wed, 19 Oct 2022 10:33:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666200810; x=1697736810; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=so+cW5QlleQSFpTNdeeHNSHQtjYir/C5XqB3E0TiChk=; b=klzOA7UXLuQcEsHxOQ0Mrdmd6tPhvmVE7pnYdSny5X6LmZqumltixeyX 1Dd6I5PV5KSHcIzRpnowgo3wnbRjoZ9+S9qVwawWxOdWBDHZZKM1XLMJZ LKI6rzI7lGKdjc2HOMD41Z+JtgVfroS1b4QnxULI6VLh7HmK7H2nHeap/ 6CGUbcFCKJX0+PouJV2Qsq2SUyU8U9pRNU5NdvS5BNYV5OB4RnvQVPult qGj7abm/VaQdkHsvT2jRtv9oBhBdBtvpK2nmAEgfitGdZRndMg9f9RxdZ hXlecNxo60d1AwbdonW1vAazt5H5FguX4C/zHr8oiQEbQtDxoeCCX/HwX g==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="306474341" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="306474341" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:29 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="607204710" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="607204710" Received: from mjmcener-mobl1.amr.corp.intel.com (HELO localhost.localdomain) ([10.213.233.40]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:26 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 03/17] cgroup/drm: Support cgroup priority control Date: Wed, 19 Oct 2022 18:32:40 +0100 Message-Id: <20221019173254.3361334-4-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> References: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747138421544936020?= X-GMAIL-MSGID: =?utf-8?q?1747138421544936020?= From: Tvrtko Ursulin A lot of the drm drivers support a concept of a scheduling priority. Add support for controlling it via the drm cgroup controller. Abstract priority control range of [DRM_CGROUP_PRIORITY_MIN, DRM_CGROUP_PRIORITY_MAX] is used and each group hierarchy adjusts it's base level based on a priority of its parent. In terms of an example that looks like this: P=-1000 /\ / \ / \ A=0 B=100 This results in the effective priority of a group A of -1000 and B of -900. In other words the fact B is configured for elevated priority is relative to the parent being a low priority and hence is only elevated in the context of its siblings. Implementation does not impose any further policy and leaves sensible configuration to the system administrator. Individual drm drivers are expected to transparently convert the drm cgroup priority into values suitable for their capabilities. No guarantees on effectiveness or granularity are provided by the controller, apart the available range being chosen to be an integer and hence allowing a generic concept of normal (zero), lower (negative values) and higher (positive values). Every cgroup starts with a default priority of zero. Signed-off-by: Tvrtko Ursulin --- Documentation/admin-guide/cgroup-v2.rst | 58 +++++++++++++ include/linux/cgroup_drm.h | 4 + kernel/cgroup/drm.c | 110 ++++++++++++++++++++++++ 3 files changed, 172 insertions(+) diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index dc254a3cb956..0a6d97c83ea4 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -2398,6 +2398,64 @@ HugeTLB Interface Files hugetlb pages of in this cgroup. Only active in use hugetlb pages are included. The per-node values are in bytes. +DRM +--- + +The DRM controller allows configuring static hierarchical scheduling priority. + +DRM static priority control +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Static priority control exposes a hierarchical control interface for the +scheduling priority support present in many DRM device drivers. + +Hierarchical meaning that the child group priorities are relative to their +parent. As an example: + + A=-1000 + /\ + / \ + / \ + B=0 C=100 + +This results in the effective priority of a group B of -1000 and C of -900. In +other words the fact C is configured for elevated priority is relative to its +parent being a low priority and hence is only elevated in the context of its +siblings. + +The scope of individual DRM scheduling priority may be per device or per device +driver, or a combination of both, depending on the implementation. The +controller does not ensure any priority ordering across multiple DRM drivers nor +does it impose any further policy and leaves desired configuration to the system +administrator. + +Individual DRM drivers are required to transparently convert the cgroup priority +into values suitable for their capabilities. + +No guarantees on effectiveness or granularity are provided by the controller, +apart the available range being chosen to be an integer and hence allowing a +generic concept of normal (zero), lower (negative values) and higher (positive +values) priority. + +DRM static priority interface files +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + drm.priority_levels + One of: + 1) And integer representing the minimum number of discrete priority + levels for the whole group. + 2) '0'- indicating one or more DRM clients in the group has no support + for static priority control. + 3) 'n/a' - when there are no DRM clients in the configured group. + + drm.priority + A read-write integer between -10000 and 10000 (inclusive) representing + an abstract static priority level. + + drm.effective_priority + Read only integer showing the current effective priority level for the + group. Effective meaning taking into account the chain of inherited + Misc ---- diff --git a/include/linux/cgroup_drm.h b/include/linux/cgroup_drm.h index bf8abc6b8ebf..a59792ccb550 100644 --- a/include/linux/cgroup_drm.h +++ b/include/linux/cgroup_drm.h @@ -6,4 +6,8 @@ #ifndef _CGROUP_DRM_H #define _CGROUP_DRM_H +#define DRM_CGROUP_PRIORITY_MIN (-10000) +#define DRM_CGROUP_PRIORITY_DEF (0) +#define DRM_CGROUP_PRIORITY_MAX (10000) + #endif /* _CGROUP_DRM_H */ diff --git a/kernel/cgroup/drm.c b/kernel/cgroup/drm.c index b88c93661df3..2350e1f8a48a 100644 --- a/kernel/cgroup/drm.c +++ b/kernel/cgroup/drm.c @@ -6,24 +6,117 @@ #include #include #include +#include +#include #include struct drm_cgroup_state { struct cgroup_subsys_state css; + + int priority; + int effective_priority; }; +static DEFINE_MUTEX(drmcg_mutex); + static inline struct drm_cgroup_state * css_to_drmcs(struct cgroup_subsys_state *css) { return container_of(css, struct drm_cgroup_state, css); } +static int drmcs_show_priority_levels(struct seq_file *sf, void *v) +{ + seq_printf(sf, "%u\n", 0); + + return 0; +} + +static s64 +drmcs_read_effective_priority(struct cgroup_subsys_state *css, + struct cftype *cft) +{ + struct drm_cgroup_state *drmcs = css_to_drmcs(css); + + return drmcs->effective_priority; +} + +static s64 +drmcs_read_priority(struct cgroup_subsys_state *css, struct cftype *cft) +{ + struct drm_cgroup_state *drmcs = css_to_drmcs(css); + + return drmcs->priority; +} + +static void update_priority(struct drm_cgroup_state *drmcs, int priority) +{ + struct cgroup_subsys_state *node; + + lockdep_assert_held(&drmcg_mutex); + + if (priority == drmcs->priority) + return; + + drmcs->priority = priority; + + rcu_read_lock(); + css_for_each_descendant_pre(node, &drmcs->css) { + struct drm_cgroup_state *dnode = css_to_drmcs(node); + int pprio; + + if (!node->parent) + pprio = DRM_CGROUP_PRIORITY_DEF; + else + pprio = css_to_drmcs(node->parent)->effective_priority; + + dnode->effective_priority = + clamp(pprio + dnode->priority, + DRM_CGROUP_PRIORITY_MIN, + DRM_CGROUP_PRIORITY_MAX); + } + rcu_read_unlock(); +} + +static int +drmcs_write_priority(struct cgroup_subsys_state *css, struct cftype *cftype, + s64 priority) +{ + struct drm_cgroup_state *drmcs = css_to_drmcs(css); + int ret; + + if (priority < (s64)DRM_CGROUP_PRIORITY_MIN || + priority > (s64)DRM_CGROUP_PRIORITY_MAX) + return -ERANGE; + + ret = mutex_lock_interruptible(&drmcg_mutex); + if (ret) + return ret; + update_priority(drmcs, (int)priority); + mutex_unlock(&drmcg_mutex); + + return 0; +} + +static int drmcs_online(struct cgroup_subsys_state *css) +{ + struct drm_cgroup_state *drmcs = css_to_drmcs(css); + + mutex_lock(&drmcg_mutex); + update_priority(drmcs, DRM_CGROUP_PRIORITY_DEF); + mutex_unlock(&drmcg_mutex); + + return 0; +} + static void drmcs_free(struct cgroup_subsys_state *css) { kfree(css_to_drmcs(css)); } static struct drm_cgroup_state root_drmcs = { + .priority = DRM_CGROUP_PRIORITY_DEF, + .effective_priority = DRM_CGROUP_PRIORITY_DEF, }; static struct cgroup_subsys_state * @@ -42,12 +135,29 @@ drmcs_alloc(struct cgroup_subsys_state *parent_css) } struct cftype files[] = { + { + .name = "priority_levels", + .flags = CFTYPE_NOT_ON_ROOT, + .seq_show = drmcs_show_priority_levels, + }, + { + .name = "priority", + .flags = CFTYPE_NOT_ON_ROOT, + .read_s64 = drmcs_read_priority, + .write_s64 = drmcs_write_priority, + }, + { + .name = "effective_priority", + .flags = CFTYPE_NOT_ON_ROOT, + .read_s64 = drmcs_read_effective_priority, + }, { } /* Zero entry terminates. */ }; struct cgroup_subsys drm_cgrp_subsys = { .css_alloc = drmcs_alloc, .css_free = drmcs_free, + .css_online = drmcs_online, .early_init = false, .legacy_cftypes = files, .dfl_cftypes = files, From patchwork Wed Oct 19 17:32:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 5778 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp451237wrs; Wed, 19 Oct 2022 10:36:36 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5jWprsjg9TgGpNIPBQr3AtJ9ixIf2DaycMBFsy2IDqflTAWneseCICkCGG4bwc1ZSRFVoB X-Received: by 2002:a17:907:9707:b0:78d:45d1:487e with SMTP id jg7-20020a170907970700b0078d45d1487emr7677256ejc.566.1666200996716; Wed, 19 Oct 2022 10:36:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666200996; cv=none; d=google.com; s=arc-20160816; b=hSCpc0f0dGTfCDNwo8YWy0OHLxQ2gV/jENC8Ags0SmgXtFyt5y6nu0yy9bmH/LvmYD 2GzQJxszgP4MFnfyHTMJHt6E+FgnbWRPaLNNq5pvnfevggWlmFHH09wMPzT8hbSgsGHN Ws3x3Fkpqp5oCkcQsvoa7WUyGEFt3b16ns5jmBX0dV5RBt12iTX5TxDuIMeE5HQSyZIA isrOOwJM9GVQXIBxIYvvpjw6d8PzKJHKhj7J4cIcmW+FE5olp6C+Xv2EfiO4u0YKFpeh YlBf2Bi+lPO4RA+M2pwgaTcorrw7fTS3aAZMMLL+sapwxVimDkgnfyjCHDGSGl+0c14c FIZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=LGJxsPyOGB0inDOsSMSGOFXoDdODZwIg6VKoaqplcPo=; b=TGbSL/CySsnTeuDWPk7DOBR+EicEN6n9Yc7vpIVrMJo+pfFfxgwP3D4O5MYAvoy4Ue vE1Z4j9RGOniVm0l2XsMHCQj+izn6GWhtwrEdF2s6tH8Wjn5vgergWUbrLgv6h8Fq8Vd Zdzd5Sn+qg3Dn6zFczglfBl2ky7oJActMH5FmXWeCG1QHbCl/8JqxWplX/GowVfjNV6f EjuFBRRjMx2oLe2gSmGTXH8ArVmzfRvjrX1l1i2ULe4PgNLh0lRYd6jLRNJNbe6B8DKb Jy8uAUUnVfdb5tSLacWf9k1Z0O534OHyZJybGEOL4Mf7lGLWovoJg+ESHCSXIsl/a6zn 4Gzg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=grXApSql; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bx20-20020a170906a1d400b0078d40f7ef1fsi13225338ejb.330.2022.10.19.10.36.11; Wed, 19 Oct 2022 10:36:36 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=grXApSql; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230084AbiJSReB (ORCPT + 99 others); Wed, 19 Oct 2022 13:34:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60722 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231258AbiJSRdq (ORCPT ); Wed, 19 Oct 2022 13:33:46 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 357331C39FC; Wed, 19 Oct 2022 10:33:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666200816; x=1697736816; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=VJyRakDrcYw5JZqsmPYIUKEzZIT9xwrJ50NQxku3Mso=; b=grXApSqlyW6KchmC8fKANx4CHVlVZaPDtPTGy16CNyKcWOFFMxK/6JUG NHBzLcu1cycwqGfS72u/typshMAyz9ZNtKrr7kSa7k5T0+MATw8gujLGR 2OBpqQbz+pIqfN7oSLl28kOVB09XMjrTzVFHQ5i3N7bGwlPYv+xWyWqOp k+kUw+0xp8iHtDHhQ8FMzlb09s92/LCDO5JQqDPe80pA672flM95YYkdn E5J0JJdLqh4y90bVx+rlpBt3ixLliH7xzjd+2PEzXBcAUxQ6lD+kkAFUH epY0vW4AGxnP3V4gZDsmzm5mvHNHbEz9ZRAvDZ3g4Nii05EmW9HBXK2Ub A==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="306474362" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="306474362" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:34 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="607204723" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="607204723" Received: from mjmcener-mobl1.amr.corp.intel.com (HELO localhost.localdomain) ([10.213.233.40]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:29 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 04/17] drm/cgroup: Allow safe external access to file_priv Date: Wed, 19 Oct 2022 18:32:41 +0100 Message-Id: <20221019173254.3361334-5-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> References: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747138375978500786?= X-GMAIL-MSGID: =?utf-8?q?1747138375978500786?= From: Tvrtko Ursulin Entry points from the cgroup subsystem into the drm cgroup controller will need to walk the file_priv structures associated with registered clients and since those are not RCU protected lets add a hack for now to make this safe. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/drm_cgroup.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/gpu/drm/drm_cgroup.c b/drivers/gpu/drm/drm_cgroup.c index a31ff1d593ab..9e9caeb0aa87 100644 --- a/drivers/gpu/drm/drm_cgroup.c +++ b/drivers/gpu/drm/drm_cgroup.c @@ -21,6 +21,13 @@ void drm_clients_close(struct drm_file *file_priv) if (atomic_dec_and_test(&clients->num)) { xa_erase(&drm_pid_clients, pid); kfree_rcu(clients, rcu); + + /* + * FIXME: file_priv is not RCU protected so we add this hack + * to avoid any races with code which walks clients->file_list + * and accesses file_priv. + */ + synchronize_rcu(); } } From patchwork Wed Oct 19 17:32:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 5780 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp451265wrs; Wed, 19 Oct 2022 10:36:40 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4R69202f4bJNJ4kPnmOD1/k6Sz954QZpFsz9gKhPRaE0QoRH7C0OpIG/MQIsfjOF8nTj2n X-Received: by 2002:a17:907:7d8d:b0:78d:d467:dd3 with SMTP id oz13-20020a1709077d8d00b0078dd4670dd3mr7759470ejc.547.1666201000335; Wed, 19 Oct 2022 10:36:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666201000; cv=none; d=google.com; s=arc-20160816; b=mWK0Zthy4vZd4PmZWAzO198OLFqU279TZP6otVRIVLHcuXtF7LgSFSk9oQ0nsgnM7i xhAOba0DwvWePkL7jfq27zGTF9Ty7cZwmtskkMrp8eoSg0ML4SkDRz/h0L7vQ3lhr1eh kaehF0GCFCACVhkFrXaz/QkypeDboZ9O4ku54r6nCbPTlOi6Hjaj9hAA1Xw21CiRfQ2m Qmq2iwbBTpOJbygOdATLN7kQoTOKOBIP/tQYwSQALpwN7/UNP49+svDkU/nGYkzZY42J sG/MQW4HSiqxNUrvA2XVxuCKgwrr2CVk7oV2oJY7sBmyOKcw9Klig0EaBHRJybNoPZ8w hrZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Xxro16Ely5LkKUowSGf44lUyhY1XbzCBKy9c1H/fcSY=; b=X9Uzfuf6j+CdsS8OywacL/dFYzPEHEXHj+n/PXyt0ex7e8JknOw5bGwUNdxI3pFZtE uk9gBzCJFZI8S/7YsQ0JkHZZ4SMfthvIUKsRNtHj61XUN7E7XydGY2AsKXeSIki2QKGy tHi9lSg5KZ8MdsVQyMarwBsYvEQhsbRSs2HQKOJLlnAuud/B87qh2otLbz4ibDyLCnjy y3we7NiUkSLFpPsVUtUrfBGI5SdxCpbvcRsBc9vEG2Odb1s4RizLIM3uV3fjJXDpj7FB honp4Li9Qg51br4Q1he+3zQa6cc3rtPOeMFqj0fWSp5S4x5uIldTbU9l/VlkFppEkvwf B1fw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ardjVYFg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m15-20020a1709060d8f00b0073db9474d06si13398373eji.325.2022.10.19.10.36.12; Wed, 19 Oct 2022 10:36:40 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ardjVYFg; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231258AbiJSReE (ORCPT + 99 others); Wed, 19 Oct 2022 13:34:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32888 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231292AbiJSRdq (ORCPT ); Wed, 19 Oct 2022 13:33:46 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D22321D3A5C; Wed, 19 Oct 2022 10:33:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666200818; x=1697736818; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=d4D/BlXX4NlKxFrkTxlp93219A7CroFmytaEV3UfKfI=; b=ardjVYFgNejJcnFCwUkXk2WXx/ZzzFLgnF16WncmTCwYKJDH1GkqsQE5 OsUe4puFBqER4FYvPq21yMhiIa6JYmQZb2YGNGMMw/Jl3TTGHQUzmrm4o mtKhgcAuzroqgY94qmZhoYeedxNlyBh9qVeroQk5hzxlXyP9yvucrVVdH 67V1ASBVaeCLWKLFJLzcRiiRTcyjOj0wvhEYyzyDZaz2Q4tLTomJnVat5 ZJ1D19TsTLtin9qOTX8IS/vYjAminhSoO1390xSmorlJLPidDt0v+AYHk 1PZSK4LI7/jdYYamYBQpHdS/BXHgUIlQ8UEfPtAMPNEdqoAq1S0t8FhBB A==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="306474378" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="306474378" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:36 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="607204741" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="607204741" Received: from mjmcener-mobl1.amr.corp.intel.com (HELO localhost.localdomain) ([10.213.233.40]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:33 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 05/17] drm: Connect priority updates to drm core Date: Wed, 19 Oct 2022 18:32:42 +0100 Message-Id: <20221019173254.3361334-6-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> References: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747138380243957411?= X-GMAIL-MSGID: =?utf-8?q?1747138380243957411?= From: Tvrtko Ursulin On priority updates, drm cgroup controller is made walk all the processes belonging to the group being updated, and notifies the drm core of them via a new helper. DRM core itself stores the current effective drm cgroup priority in struct drm_file, while individual drivers can also register an optional hook to be called at the same time, via struct drm_cgroup_ops which can be provided as part of struct drm_driver used at driver registration time. DRM cgroup controller on the other hand exports a new helper which the drm core uses at client registration time in order to query to current drm cgroup effective priority. This establishes a two way communication channel between the drm cgroup controller and the drm core and hence drm core module now has to be built into the kernel. Signed-off-by: Tvrtko Ursulin --- Documentation/admin-guide/cgroup-v2.rst | 2 + drivers/gpu/drm/Kconfig | 1 + drivers/gpu/drm/drm_cgroup.c | 56 +++++++++++++++++ drivers/gpu/drm/drm_file.c | 4 ++ include/drm/drm_clients.h | 3 + include/drm/drm_drv.h | 47 ++++++++++++++ include/drm/drm_file.h | 10 +++ include/linux/cgroup_drm.h | 4 ++ init/Kconfig | 1 + kernel/cgroup/drm.c | 82 ++++++++++++++++++++++++- 10 files changed, 209 insertions(+), 1 deletion(-) diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index 0a6d97c83ea4..1f3cca4e2572 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -2444,6 +2444,8 @@ DRM static priority interface files One of: 1) And integer representing the minimum number of discrete priority levels for the whole group. + Optionally followed by an asterisk ('*') indicating some DRM clients + in the group support more than the minimum number. 2) '0'- indicating one or more DRM clients in the group has no support for static priority control. 3) 'n/a' - when there are no DRM clients in the configured group. diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig index 34f5a092c99e..8f3c169ced10 100644 --- a/drivers/gpu/drm/Kconfig +++ b/drivers/gpu/drm/Kconfig @@ -7,6 +7,7 @@ # menuconfig DRM tristate "Direct Rendering Manager (XFree86 4.1.0 and higher DRI support)" + default y if CGROUP_DRM=y depends on (AGP || AGP=n) && !EMULATED_CMPXCHG && HAS_DMA select DRM_NOMODESET select DRM_PANEL_ORIENTATION_QUIRKS diff --git a/drivers/gpu/drm/drm_cgroup.c b/drivers/gpu/drm/drm_cgroup.c index 9e9caeb0aa87..0fbb88f08cef 100644 --- a/drivers/gpu/drm/drm_cgroup.c +++ b/drivers/gpu/drm/drm_cgroup.c @@ -65,3 +65,59 @@ int drm_clients_open(struct drm_file *file_priv) return 0; } + +unsigned int drm_pid_priority_levels(struct pid *pid, bool *non_uniform) +{ + unsigned int min_levels = UINT_MAX; + struct drm_pid_clients *clients; + + *non_uniform = false; + + rcu_read_lock(); + clients = xa_load(&drm_pid_clients, (unsigned long)pid); + if (clients) { + struct drm_file *fpriv; + + list_for_each_entry_rcu(fpriv, &clients->file_list, clink) { + const struct drm_cgroup_ops *cg_ops = + fpriv->minor->dev->driver->cg_ops; + unsigned int l; + + if (cg_ops && cg_ops->priority_levels) + l = cg_ops->priority_levels(fpriv); + else + l = 0; + + if (min_levels != UINT_MAX && l != min_levels) + *non_uniform = true; + if (l < min_levels) + min_levels = l; + } + } + rcu_read_unlock(); + + return min_levels; +} +EXPORT_SYMBOL_GPL(drm_pid_priority_levels); + +void drm_pid_update_priority(struct pid *pid, int priority) +{ + struct drm_pid_clients *clients; + + rcu_read_lock(); + clients = xa_load(&drm_pid_clients, (unsigned long)pid); + if (clients) { + struct drm_file *fpriv; + + list_for_each_entry_rcu(fpriv, &clients->file_list, clink) { + const struct drm_cgroup_ops *cg_ops = + fpriv->minor->dev->driver->cg_ops; + + fpriv->drm_cgroup_priority = priority; + if (cg_ops && cg_ops->update_priority) + cg_ops->update_priority(fpriv, priority); + } + } + rcu_read_unlock(); +} +EXPORT_SYMBOL_GPL(drm_pid_update_priority); diff --git a/drivers/gpu/drm/drm_file.c b/drivers/gpu/drm/drm_file.c index ce58d5c513db..38eb6003e74d 100644 --- a/drivers/gpu/drm/drm_file.c +++ b/drivers/gpu/drm/drm_file.c @@ -32,6 +32,7 @@ */ #include +#include #include #include #include @@ -359,6 +360,9 @@ static int drm_open_helper(struct file *filp, struct drm_minor *minor) filp->f_mode |= FMODE_UNSIGNED_OFFSET; priv->filp = filp; + priv->drm_cgroup_priority = + drmcgroup_lookup_effective_priority(current); + mutex_lock(&dev->filelist_mutex); ret = drm_clients_open(priv); if (ret) diff --git a/include/drm/drm_clients.h b/include/drm/drm_clients.h index 4ae553a03d1e..10d21138f7af 100644 --- a/include/drm/drm_clients.h +++ b/include/drm/drm_clients.h @@ -28,4 +28,7 @@ static inline int drm_clients_open(struct drm_file *file_priv) } #endif +unsigned int drm_pid_priority_levels(struct pid *pid, bool *non_uniform); +void drm_pid_update_priority(struct pid *pid, int priority); + #endif diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h index f6159acb8856..2371d73e12cf 100644 --- a/include/drm/drm_drv.h +++ b/include/drm/drm_drv.h @@ -148,6 +148,43 @@ enum drm_driver_feature { DRIVER_KMS_LEGACY_CONTEXT = BIT(31), }; +/** + * struct drm_cgroup_ops + * + * This structure contains a number of callbacks that drivers can provide if + * they are able to support one or more of the functionalities implemented by + * the DRM cgroup controller. + */ +struct drm_cgroup_ops { + /** + * @priority_levels: + * + * Returns the discrete number of priority levels supported by the DRM + * driver owning this client. + * + * The value is used by the DRM core when informing the DRM cgroup + * controller on the scheduling priority capability of a group of + * clients. + * + * If the callback is not implemented no support for scheduling priority + * is assumed and reported as such. + */ + unsigned int (*priority_levels) (struct drm_file *); + + /** + * @update_priority: + * + * Optional callback used by the DRM core for informing individual + * drivers of DRM cgroup priority changes. + * + * If not implemented drivers are still able to access the most recent + * priority via the drm_file->drm_cgroup_priority field. Therefore the + * main purpose of the callback is for drivers which are able to adjust + * priorities of already running workloads. + */ + void (*update_priority) (struct drm_file *, int priority); +}; + /** * struct drm_driver - DRM driver structure * @@ -459,6 +496,16 @@ struct drm_driver { */ const struct file_operations *fops; +#ifdef CONFIG_CGROUP_DRM + /** + * @cg_ops: + * + * Optional pointer to driver callbacks facilitating integration with + * the DRM cgroup controller. + */ + const struct drm_cgroup_ops *cg_ops; +#endif + #ifdef CONFIG_DRM_LEGACY /* Everything below here is for legacy driver, never use! */ /* private: */ diff --git a/include/drm/drm_file.h b/include/drm/drm_file.h index 0965eb111f24..a4360e28e2db 100644 --- a/include/drm/drm_file.h +++ b/include/drm/drm_file.h @@ -223,6 +223,16 @@ struct drm_file { */ bool is_master; +#ifdef CONFIG_CGROUP_DRM + /** + * @drm_cgroup_priority: + * + * Last known DRM cgroup priority is stored here by the DRM code when + * informed of changes by the cgroup controller. + */ + int drm_cgroup_priority; +#endif + /** * @master: * diff --git a/include/linux/cgroup_drm.h b/include/linux/cgroup_drm.h index a59792ccb550..66063b4708e8 100644 --- a/include/linux/cgroup_drm.h +++ b/include/linux/cgroup_drm.h @@ -6,8 +6,12 @@ #ifndef _CGROUP_DRM_H #define _CGROUP_DRM_H +struct task_struct; + #define DRM_CGROUP_PRIORITY_MIN (-10000) #define DRM_CGROUP_PRIORITY_DEF (0) #define DRM_CGROUP_PRIORITY_MAX (10000) +int drmcgroup_lookup_effective_priority(struct task_struct *task); + #endif /* _CGROUP_DRM_H */ diff --git a/init/Kconfig b/init/Kconfig index 6dd7faca7749..cfc7a1f2634c 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1089,6 +1089,7 @@ config CGROUP_RDMA config CGROUP_DRM bool "DRM controller" + select DRM help Provides the DRM subsystem controller. diff --git a/kernel/cgroup/drm.c b/kernel/cgroup/drm.c index 2350e1f8a48a..01954c3a2087 100644 --- a/kernel/cgroup/drm.c +++ b/kernel/cgroup/drm.c @@ -10,6 +10,8 @@ #include #include +#include + struct drm_cgroup_state { struct cgroup_subsys_state css; @@ -25,9 +27,52 @@ css_to_drmcs(struct cgroup_subsys_state *css) return container_of(css, struct drm_cgroup_state, css); } +static inline struct drm_cgroup_state *get_task_drmcs(struct task_struct *task) +{ + return css_to_drmcs(task_get_css(task, drm_cgrp_id)); +} + +int drmcgroup_lookup_effective_priority(struct task_struct *task) +{ + struct drm_cgroup_state *drmcs = get_task_drmcs(task); + int prio = drmcs->effective_priority; + + css_put(&drmcs->css); + + return prio; +} +EXPORT_SYMBOL_GPL(drmcgroup_lookup_effective_priority); + static int drmcs_show_priority_levels(struct seq_file *sf, void *v) { - seq_printf(sf, "%u\n", 0); + struct cgroup *cgrp = seq_css(sf)->cgroup; + unsigned int min_levels = UINT_MAX; + bool non_uniform = false; + struct task_struct *task; + struct css_task_iter it; + + css_task_iter_start(&cgrp->self, + CSS_TASK_ITER_PROCS | CSS_TASK_ITER_THREADED, &it); + while ((task = css_task_iter_next(&it))) { + unsigned int l; + bool nu; + + /* Ignore kernel threads here. */ + if (task->flags & PF_KTHREAD) + continue; + + l = drm_pid_priority_levels(task_pid(task), &nu); + if (nu || (min_levels != UINT_MAX && l != min_levels)) + non_uniform = true; + if (l < min_levels) + min_levels = l; + } + css_task_iter_end(&it); + + if (min_levels != UINT_MAX) + seq_printf(sf, "%u%s\n", min_levels, non_uniform ? "*" : ""); + else + seq_puts(sf, "n/a\n"); return 0; } @@ -49,6 +94,24 @@ drmcs_read_priority(struct cgroup_subsys_state *css, struct cftype *cft) return drmcs->priority; } +static void update_drm_priority(struct drm_cgroup_state *drmcs) +{ + struct cgroup *cgrp = drmcs->css.cgroup; + struct task_struct *task; + struct css_task_iter it; + + css_task_iter_start(&cgrp->self, + CSS_TASK_ITER_PROCS | CSS_TASK_ITER_THREADED, &it); + while ((task = css_task_iter_next(&it))) { + /* Ignore kernel threads here. */ + if (task->flags & PF_KTHREAD) + continue; + drm_pid_update_priority(task_pid(task), + drmcs->effective_priority); + } + css_task_iter_end(&it); +} + static void update_priority(struct drm_cgroup_state *drmcs, int priority) { struct cgroup_subsys_state *node; @@ -74,6 +137,8 @@ static void update_priority(struct drm_cgroup_state *drmcs, int priority) clamp(pprio + dnode->priority, DRM_CGROUP_PRIORITY_MIN, DRM_CGROUP_PRIORITY_MAX); + + update_drm_priority(dnode); } rcu_read_unlock(); } @@ -114,6 +179,20 @@ static void drmcs_free(struct cgroup_subsys_state *css) kfree(css_to_drmcs(css)); } +static void drmcs_attach(struct cgroup_taskset *tset) +{ + struct cgroup_subsys_state *css; + struct task_struct *task; + + /* + * As processes are assigned to groups we need to notify them of the + * current priority. + */ + cgroup_taskset_for_each(task, css, tset) + drm_pid_update_priority(task_pid(task), + css_to_drmcs(css)->effective_priority); +} + static struct drm_cgroup_state root_drmcs = { .priority = DRM_CGROUP_PRIORITY_DEF, .effective_priority = DRM_CGROUP_PRIORITY_DEF, @@ -158,6 +237,7 @@ struct cgroup_subsys drm_cgrp_subsys = { .css_alloc = drmcs_alloc, .css_free = drmcs_free, .css_online = drmcs_online, + .attach = drmcs_attach, .early_init = false, .legacy_cftypes = files, .dfl_cftypes = files, From patchwork Wed Oct 19 17:32:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 5779 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp451261wrs; Wed, 19 Oct 2022 10:36:40 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5bNsn1cCVgShEDl+sQ8dCqRfgt6G3MtYkXQIh4yF8CGWUqUWKqaG2bTtjmDYkL6iF4JXDK X-Received: by 2002:a17:907:1de1:b0:78d:9919:38cc with SMTP id og33-20020a1709071de100b0078d991938ccmr7617160ejc.311.1666201000200; Wed, 19 Oct 2022 10:36:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666201000; cv=none; d=google.com; s=arc-20160816; b=R986fefdkssiYVW7LxqmMn2J89ZpNQgWLRUOm9yBS92oe/fRZZm1H+lb9AanWmLY55 pTGvW5BFo76dX2waMv0CAjqv5dmjbJBNL37MsNdhpTj+Dl/Ue+kH5X8XhRGNt6ImZRfY xOsp6URwZPVCuIq3Yo6DqF6G/NuFOTdJmsF7zjUY2et6KQaRU3vhyGgmtmip5EQ4oy+I O5iR4z0Ecxu50JkD1pu+IyvMaWEdAQjIUNTYkYLESrcO1JV/zCTFeoWKe7FbxyNgXuly 0blHuoF7ipwR3o/MY+aUodRusnvke90GbLb7ozi4Ou52vycsCAsXcgoTp4yqltLoH/Qq 7psQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=LZlTfHGK3L0tWv+lUGOXIB8Dbuv6fyQW58oFlV4fWeQ=; b=aep2j0D+vtHVmGZCJnEUBzC4LrpAXewiK/QwVL4tPKXAGg2a+9abw6hKr8jizzkLoP B/7LFQOU5v36L73mJu/7u4H1kT86GNGYu1o0Neh3f7fYsdrNTcoH6245DcKXbMOqLTnj IzAYf48GDo8R6tPEq51LELXmCn+EpeWwRPJ668TP9LAoentJRAlxOnJjq+3dsUYBEMNz 46ZDeRlxaO1lB95ZAX05VLUou6Q88ZUIrCNRZ8bj6vGjcmJ77OKDTT08RSq8bbZAmkEf dLNhDIkv8Xcrg6Dv3R1h7NGxRt3ifUWxr8F0On1tw40wwHKQz5WhH8gIFvgGNZ3ciX+p kFHA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="jPHY5/x8"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id d22-20020a50f696000000b00459fc3fcf0bsi13738684edn.101.2022.10.19.10.36.12; Wed, 19 Oct 2022 10:36:40 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="jPHY5/x8"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231311AbiJSReH (ORCPT + 99 others); Wed, 19 Oct 2022 13:34:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32832 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231312AbiJSRdt (ORCPT ); Wed, 19 Oct 2022 13:33:49 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 52E141D3A76; Wed, 19 Oct 2022 10:33:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666200821; x=1697736821; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=lqVOHpteUQeCev/lgNedZY6IzAYCvefhlH9jg3GlMDs=; b=jPHY5/x81a0xSghusQWgWUnxMIGI7eLUovAd9IlxAWf5rqfxRx1xCiI2 RjrUvZSjLoHukW1LCkOV0gvbidoHk0kctR/TEGE+maw5hYYDWPYXTRaXN xNDGBXcv7lHbZj9cSkghu46jEnHPEG899xZCm2jQY+M17Sk5XhwEhlXI/ mYdNUl2d8JptTdxYql1l54S0NsYKBbqasHTjI+bpJyjMuOnIoXMNkQW7w gn/X0pnfX0D5YjEGcvpvTgzBC2XVwNh+Ql+aRff++nfimRWLeBq5nZFmB WnASx8D0asUFzc5Z5/gIJ4FULwnKMA0mKgtpJCG8GX15DlxhARfmxq0W+ A==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="306474406" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="306474406" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:39 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="607204754" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="607204754" Received: from mjmcener-mobl1.amr.corp.intel.com (HELO localhost.localdomain) ([10.213.233.40]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:36 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 06/17] drm: Only track clients which are providing drm_cgroup_ops Date: Wed, 19 Oct 2022 18:32:43 +0100 Message-Id: <20221019173254.3361334-7-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> References: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747138379730976849?= X-GMAIL-MSGID: =?utf-8?q?1747138379730976849?= From: Tvrtko Ursulin To reduce the number of tracking going on, especially with drivers which will not support any sort of control from the drm cgroup controller side, lets express the funcionality as opt-in and use the presence of drm_cgroup_ops as activation criteria. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/drm_cgroup.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/gpu/drm/drm_cgroup.c b/drivers/gpu/drm/drm_cgroup.c index 0fbb88f08cef..7ed9c7150cae 100644 --- a/drivers/gpu/drm/drm_cgroup.c +++ b/drivers/gpu/drm/drm_cgroup.c @@ -16,6 +16,9 @@ void drm_clients_close(struct drm_file *file_priv) lockdep_assert_held(&dev->filelist_mutex); + if (!dev->driver->cg_ops) + return; + clients = xa_load(&drm_pid_clients, pid); list_del_rcu(&file_priv->clink); if (atomic_dec_and_test(&clients->num)) { @@ -40,6 +43,9 @@ int drm_clients_open(struct drm_file *file_priv) lockdep_assert_held(&dev->filelist_mutex); + if (!dev->driver->cg_ops) + return 0; + clients = xa_load(&drm_pid_clients, pid); if (!clients) { clients = kmalloc(sizeof(*clients), GFP_KERNEL); From patchwork Wed Oct 19 17:32:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 5785 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp452118wrs; Wed, 19 Oct 2022 10:38:14 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7jsqrbNPENwe/fkDDV6iV9TSJREgwtajFBdNEaCIvB9LATy5K82klyKPVY1/yxj8FeJr0H X-Received: by 2002:a05:6402:5206:b0:45d:88f:4f00 with SMTP id s6-20020a056402520600b0045d088f4f00mr8631512edd.130.1666201094704; Wed, 19 Oct 2022 10:38:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666201094; cv=none; d=google.com; s=arc-20160816; b=vf3Gyvbto7zpPRvVo5cq5QSNLAuYWvLTqRoBX8hO07pvXn72SuLwrvYi1PC5rC2Ie4 5G+4KMH7cnn5FAQFUbzMfOnEqN1fBtq9ABzV8754paIOaqqUOeEZPkbOdZlkjYq+xYLV AlmLsEuZdHycOueuZAx9KwzWsBSYPQgdUXrmOyvs0wSLAcwOVqAd9iAhRawgbot3+cuG h6ixv9VhBIXAG9/PPa1s7It+e1eU7QPqIiV0cr12vF8aEy6EyVdniLDeaEoEroK1OCXd efvDeeUgD04ptgErsBCctqbG39VKVjhuESDtYcH4hWXDw8boloSXQeldu33xOmmGC+Gy nn4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=R1tonuT08FPKX4s4fRgTnZQ2xBAPW7CqI6dYnZrhqj8=; b=OkfmOOz03CEbvkLNCpZhMxApJe+jytmibguiwB6ZD2iwYF0RTjIv+4C1WYZyMU9dQ5 zH4PvHl8jTQvSgxZc+suubH2XO68OXCEzuEl0xIPwCCpud5pnCeHHmea4i6DOEeXW5SS tdjUG5PNcHnjPG8drHjkiAUIJkiOT1iXeRtRa0ZzBaJlbSDTYwKyHBqrRPnUzHmSBFek d1X38ZJTlPtQ17vP+pvP5U/31V7pxlDWh4VKxHMNuHpoaKVrONEXQU8XEgLqLOAbxDcL tMd0d2PUnbGSD8iQ9IPkS3C8ixdPdGKeMOc5A7bmjfraPnVXQxrB9rLyU+mng7KDNOzg qx/w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Jk2CkDAW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w10-20020a50d78a000000b004539b045326si14898672edi.417.2022.10.19.10.37.49; Wed, 19 Oct 2022 10:38:14 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Jk2CkDAW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231168AbiJSRek (ORCPT + 99 others); Wed, 19 Oct 2022 13:34:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33174 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231299AbiJSReG (ORCPT ); Wed, 19 Oct 2022 13:34:06 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BA45F1D375A; Wed, 19 Oct 2022 10:34:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666200844; x=1697736844; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=OrYOCnpePJk9ybiiJejaW8hbTOJ7aQUhUryDqIC49tk=; b=Jk2CkDAW8TKed+HTJlONtLcB6lgEUZMO8S4rneffuzGyBgo6vgtLRvzF l9mhFkYPeixEg7dgHIeYvjfddyK4s7GrHjTOBIXPYLCE1oOI6Ttjwxiay R1o7pDulir0vvfFTgAzB9qpV/ttkDpzg310PK6tjL62cipwUnBQiZHj3i EHiCwnws0KdKSPW9rz91yc3ZImQj9oU40roJI+13D6/bsRGSXO0HA2Byl vgFb+B4KZczbdke2sEb882gCGca6CTzkVC7iQCnFpt1WY96z+GL8P7bL0 lTIWwY/YQZ0CkT/r2OnUzZgEncQCB427txy0keO4PRLS4vX5ia6VJZqpg A==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="306474438" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="306474438" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:43 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="607204773" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="607204773" Received: from mjmcener-mobl1.amr.corp.intel.com (HELO localhost.localdomain) ([10.213.233.40]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:39 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 07/17] drm/i915: i915 priority Date: Wed, 19 Oct 2022 18:32:44 +0100 Message-Id: <20221019173254.3361334-8-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> References: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747138479530351247?= X-GMAIL-MSGID: =?utf-8?q?1747138479530351247?= From: Tvrtko Ursulin Register i915 as supporting the drm cgroup controller priority management and wire it up at execbuf time. GEM context configured priority then works as a relative value on top of the base level obtained from the drm cgroup controller. Signed-off-by: Tvrtko Ursulin --- .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 27 ++++++++++++++++++- drivers/gpu/drm/i915/i915_driver.c | 10 +++++++ drivers/gpu/drm/i915/i915_drm_client.c | 16 +++++++++++ drivers/gpu/drm/i915/i915_drm_client.h | 4 +++ 4 files changed, 56 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c index 1160723c9d2d..391c5b5c80be 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c @@ -4,8 +4,10 @@ * Copyright © 2008,2010 Intel Corporation */ +#include #include #include +#include #include #include @@ -3015,6 +3017,29 @@ static void retire_requests(struct intel_timeline *tl, struct i915_request *end) break; } +#ifdef CONFIG_CGROUP_DRM +static void copy_priority(struct i915_sched_attr *attr, + const struct i915_execbuffer *eb) +{ + const int scale = DIV_ROUND_CLOSEST(DRM_CGROUP_PRIORITY_MAX, + I915_CONTEXT_MAX_USER_PRIORITY); + int prio; + + *attr = eb->gem_context->sched; + prio = attr->priority * scale + eb->file->drm_cgroup_priority; + prio = DIV_ROUND_UP(prio, scale); + attr->priority = clamp(prio, + I915_CONTEXT_MIN_USER_PRIORITY, + I915_CONTEXT_MAX_USER_PRIORITY); +} +#else +static void copy_priority(struct i915_sched_attr *attr, + const struct i915_execbuffer *eb) +{ + *attr = eb->gem_context->sched; +} +#endif + static int eb_request_add(struct i915_execbuffer *eb, struct i915_request *rq, int err, bool last_parallel) { @@ -3031,7 +3056,7 @@ static int eb_request_add(struct i915_execbuffer *eb, struct i915_request *rq, /* Check that the context wasn't destroyed before submission */ if (likely(!intel_context_is_closed(eb->context))) { - attr = eb->gem_context->sched; + copy_priority(&attr, eb); } else { /* Serialise with context_close via the add_to_timeline */ i915_request_set_error_once(rq, -ENOENT); diff --git a/drivers/gpu/drm/i915/i915_driver.c b/drivers/gpu/drm/i915/i915_driver.c index ffff49868dc5..7912782b87cc 100644 --- a/drivers/gpu/drm/i915/i915_driver.c +++ b/drivers/gpu/drm/i915/i915_driver.c @@ -1893,6 +1893,12 @@ static const struct drm_ioctl_desc i915_ioctls[] = { DRM_IOCTL_DEF_DRV(I915_GEM_VM_DESTROY, i915_gem_vm_destroy_ioctl, DRM_RENDER_ALLOW), }; +#ifdef CONFIG_CGROUP_DRM +static const struct drm_cgroup_ops i915_drm_cgroup_ops = { + .priority_levels = i915_drm_priority_levels, +}; +#endif + /* * Interface history: * @@ -1921,6 +1927,10 @@ static const struct drm_driver i915_drm_driver = { .lastclose = i915_driver_lastclose, .postclose = i915_driver_postclose, +#ifdef CONFIG_CGROUP_DRM + .cg_ops = &i915_drm_cgroup_ops, +#endif + .prime_handle_to_fd = drm_gem_prime_handle_to_fd, .prime_fd_to_handle = drm_gem_prime_fd_to_handle, .gem_prime_import = i915_gem_prime_import, diff --git a/drivers/gpu/drm/i915/i915_drm_client.c b/drivers/gpu/drm/i915/i915_drm_client.c index b09d1d386574..61a3cdaa7b16 100644 --- a/drivers/gpu/drm/i915/i915_drm_client.c +++ b/drivers/gpu/drm/i915/i915_drm_client.c @@ -75,6 +75,22 @@ void i915_drm_clients_fini(struct i915_drm_clients *clients) xa_destroy(&clients->xarray); } +#ifdef CONFIG_CGROUP_DRM +unsigned int i915_drm_priority_levels(struct drm_file *file) +{ + struct drm_i915_file_private *fpriv = file->driver_priv; + struct i915_drm_client *client = fpriv->client; + struct drm_i915_private *i915 = client->clients->i915; + + if (GRAPHICS_VER(i915) < 8) + return 0; + else if (intel_uc_uses_guc_submission(&to_gt(i915)->uc)) + return 3; + else + return 2047; +} +#endif + #ifdef CONFIG_PROC_FS static const char * const uabi_class_names[] = { [I915_ENGINE_CLASS_RENDER] = "render", diff --git a/drivers/gpu/drm/i915/i915_drm_client.h b/drivers/gpu/drm/i915/i915_drm_client.h index 69496af996d9..bd5925241007 100644 --- a/drivers/gpu/drm/i915/i915_drm_client.h +++ b/drivers/gpu/drm/i915/i915_drm_client.h @@ -15,6 +15,8 @@ #define I915_LAST_UABI_ENGINE_CLASS I915_ENGINE_CLASS_COMPUTE +struct drm_file; + struct drm_i915_private; struct i915_drm_clients { @@ -65,4 +67,6 @@ void i915_drm_client_fdinfo(struct seq_file *m, struct file *f); void i915_drm_clients_fini(struct i915_drm_clients *clients); +unsigned int i915_drm_priority_levels(struct drm_file *file); + #endif /* !__I915_DRM_CLIENT_H__ */ From patchwork Wed Oct 19 17:32:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 5787 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp452205wrs; Wed, 19 Oct 2022 10:38:27 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4AtouX40ZvyRxL4tmzgCWf1DWroUkRamhz+ugLqU3c8dfUnNovgQUyaJkoD+aY9lniHCgM X-Received: by 2002:a17:907:2cce:b0:77a:6958:5aaa with SMTP id hg14-20020a1709072cce00b0077a69585aaamr7353699ejc.245.1666201107514; Wed, 19 Oct 2022 10:38:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666201107; cv=none; d=google.com; s=arc-20160816; b=wjZ/fno71CGkys00QzBThjgg34QgTSsi3XexsAQqr27axauJuusH/p0Fo2fXOPLVCF dyYVfywlPSM/o2c4DKnVJ3PgvU3I/A2+aGvc94fLSfWGpnPavJkn829Kdf6vryXdaifc 1qPxqUBxaOwZDV/YlGEqV4FNjRUmsLblsF3IUtED/mWRzl191nSTH33XLbMYeh3J4JjW luR+GxX8TnLIhN356Ylu1XymNsfgNjz3tTnheuGGVXpZd6sjaKz77IeIMW2Lq0UuXhpp f0f1VLTyIY6/3iu5EpKKT3RlMNhOulkQAMPeP95TCK6LaOrR0+IJBQ4DjYzcioyFkjMX hUdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=JVb/MsZMIZbXTnt8AbxsrbXN5rmRtMYy8AHTA4ha0A0=; b=lWo4tLVrgp8JEWxMLk9U1BA5JLHmpXKbbfahp2eD6hgOK6ug036X48W3voa9jvuM36 3RpJcP43G13G2dIv3FYJiKV15I7Sw3y1l3vnyYgUEptGXflOlqL/zRWIZDvpHcHd+CSX GJLva67A+Hp0hFC9vjP8/L+uwb1wSnLLuqjPd8xHrYPrrLQW341EvB2F3kLwZ7RPtMO5 li67ckuzAmtzbczajCj6jv9UCp9/RWv4vsFpjyBirS2nqdQ5ahN7CZmR6G4eMTTk6Nqq I9IOVt6caNLqnDGDx8mGMIg4x7oVXg3sGVtOFyhdrUyvqRQ/mpc65VLWBkBPbaQIIfe3 Gaxg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="c0C5n/3K"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s17-20020aa7c551000000b0045c9dbe290csi13307724edr.406.2022.10.19.10.38.02; Wed, 19 Oct 2022 10:38:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="c0C5n/3K"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230074AbiJSRec (ORCPT + 99 others); Wed, 19 Oct 2022 13:34:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33230 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231310AbiJSReH (ORCPT ); Wed, 19 Oct 2022 13:34:07 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 825801D3473; Wed, 19 Oct 2022 10:34:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666200844; x=1697736844; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=iblWtj3oQ+Gzcpob6q0KDZMo8KPs6N0OMQ7W8Hrwr8E=; b=c0C5n/3KLGiKF2usH6erg2PnyHShFnFTJJ1TrCRDdUHGYYK1zO6otd9u SJGwz9NtddkANEhYM1ZwKKCLljxo0mFDQRDWuYuyUSqo3MA8tn7kHM8Z/ TYwukPvK53m9diJmJx+AmiQXAmY2/RTcSJSMB/Biz/DUIn0xVBhfsFUsg 1nBWJ710UZ8O6W5IoNHes1vM8GQOYUlkIKq8LuizsDHd/28/q3FAHkxK5 ZDchScIDOy1rBu+xKlArJvvmUyC9+rg2EYcOMjiI8/L62VnW6rEuRyYad s8YKAnNymScoZ/oY4rhNxrvT5H7ILw09CFBl9Wfpd51ZsWX/WAtsCpi+v Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="306474462" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="306474462" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:46 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="607204781" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="607204781" Received: from mjmcener-mobl1.amr.corp.intel.com (HELO localhost.localdomain) ([10.213.233.40]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:43 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 08/17] drm: Allow for migration of clients Date: Wed, 19 Oct 2022 18:32:45 +0100 Message-Id: <20221019173254.3361334-9-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> References: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747138492526477932?= X-GMAIL-MSGID: =?utf-8?q?1747138492526477932?= From: Tvrtko Ursulin Add a helper which allows migrating the tracked client from one process to another. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/drm_cgroup.c | 111 ++++++++++++++++++++++++++++++----- include/drm/drm_clients.h | 7 +++ include/drm/drm_file.h | 1 + 3 files changed, 103 insertions(+), 16 deletions(-) diff --git a/drivers/gpu/drm/drm_cgroup.c b/drivers/gpu/drm/drm_cgroup.c index 7ed9c7150cae..59b730ed1334 100644 --- a/drivers/gpu/drm/drm_cgroup.c +++ b/drivers/gpu/drm/drm_cgroup.c @@ -8,9 +8,21 @@ static DEFINE_XARRAY(drm_pid_clients); +static void +__del_clients(struct drm_pid_clients *clients, struct drm_file *file_priv) +{ + list_del_rcu(&file_priv->clink); + if (atomic_dec_and_test(&clients->num)) { + xa_erase(&drm_pid_clients, (unsigned long)file_priv->cpid); + kfree_rcu(clients, rcu); + } + + put_pid(file_priv->cpid); + file_priv->cpid = NULL; +} + void drm_clients_close(struct drm_file *file_priv) { - unsigned long pid = (unsigned long)file_priv->pid; struct drm_device *dev = file_priv->minor->dev; struct drm_pid_clients *clients; @@ -19,19 +31,32 @@ void drm_clients_close(struct drm_file *file_priv) if (!dev->driver->cg_ops) return; - clients = xa_load(&drm_pid_clients, pid); - list_del_rcu(&file_priv->clink); - if (atomic_dec_and_test(&clients->num)) { - xa_erase(&drm_pid_clients, pid); - kfree_rcu(clients, rcu); + clients = xa_load(&drm_pid_clients, (unsigned long)file_priv->cpid); + if (WARN_ON_ONCE(!clients)) + return; - /* - * FIXME: file_priv is not RCU protected so we add this hack - * to avoid any races with code which walks clients->file_list - * and accesses file_priv. - */ - synchronize_rcu(); + __del_clients(clients, file_priv); + + /* + * FIXME: file_priv is not RCU protected so we add this hack + * to avoid any races with code which walks clients->file_list + * and accesses file_priv. + */ + synchronize_rcu(); +} + +static struct drm_pid_clients *__alloc_clients(void) +{ + struct drm_pid_clients *clients; + + clients = kmalloc(sizeof(*clients), GFP_KERNEL); + if (clients) { + atomic_set(&clients->num, 0); + INIT_LIST_HEAD(&clients->file_list); + init_rcu_head(&clients->rcu); } + + return clients; } int drm_clients_open(struct drm_file *file_priv) @@ -48,12 +73,9 @@ int drm_clients_open(struct drm_file *file_priv) clients = xa_load(&drm_pid_clients, pid); if (!clients) { - clients = kmalloc(sizeof(*clients), GFP_KERNEL); + clients = __alloc_clients(); if (!clients) return -ENOMEM; - atomic_set(&clients->num, 0); - INIT_LIST_HEAD(&clients->file_list); - init_rcu_head(&clients->rcu); new_client = true; } atomic_inc(&clients->num); @@ -69,9 +91,66 @@ int drm_clients_open(struct drm_file *file_priv) } } + file_priv->cpid = get_pid(file_priv->pid); + return 0; } +void drm_clients_migrate(struct drm_file *file_priv) +{ + struct drm_device *dev = file_priv->minor->dev; + struct drm_pid_clients *existing_clients; + struct drm_pid_clients *clients, *spare; + struct pid *pid = task_pid(current); + + if (!dev->driver->cg_ops) + return; + + // TODO: only do this if drmcs level property allows it? + + spare = __alloc_clients(); + if (WARN_ON(!spare)) + return; + + mutex_lock(&dev->filelist_mutex); + rcu_read_lock(); + + existing_clients = xa_load(&drm_pid_clients, (unsigned long)pid); + clients = xa_load(&drm_pid_clients, (unsigned long)file_priv->cpid); + + if (WARN_ON_ONCE(!clients)) + goto out_unlock; + else if (clients == existing_clients) + goto out_unlock; + + __del_clients(clients, file_priv); + smp_mb(); /* hmmm? del_rcu followed by add_rcu? */ + + if (!existing_clients) { + void *xret; + + xret = xa_store(&drm_pid_clients, (unsigned long)pid, spare, + GFP_KERNEL); + if (WARN_ON(xa_err(xret))) + goto out_unlock; + clients = spare; + spare = NULL; + } else { + clients = existing_clients; + } + + atomic_inc(&clients->num); + list_add_tail_rcu(&file_priv->clink, &clients->file_list); + file_priv->cpid = get_pid(pid); + +out_unlock: + rcu_read_unlock(); + mutex_unlock(&dev->filelist_mutex); + + kfree(spare); +} +EXPORT_SYMBOL_GPL(drm_clients_migrate); + unsigned int drm_pid_priority_levels(struct pid *pid, bool *non_uniform) { unsigned int min_levels = UINT_MAX; diff --git a/include/drm/drm_clients.h b/include/drm/drm_clients.h index 10d21138f7af..3a0b1cdb338f 100644 --- a/include/drm/drm_clients.h +++ b/include/drm/drm_clients.h @@ -17,6 +17,8 @@ struct drm_pid_clients { #if IS_ENABLED(CONFIG_CGROUP_DRM) void drm_clients_close(struct drm_file *file_priv); int drm_clients_open(struct drm_file *file_priv); + +void drm_clients_migrate(struct drm_file *file_priv); #else static inline void drm_clients_close(struct drm_file *file_priv) { @@ -26,6 +28,11 @@ static inline int drm_clients_open(struct drm_file *file_priv) { return 0; } + +static inline void drm_clients_migrate(struct drm_file *file_priv) +{ + +} #endif unsigned int drm_pid_priority_levels(struct pid *pid, bool *non_uniform); diff --git a/include/drm/drm_file.h b/include/drm/drm_file.h index a4360e28e2db..2c1e356d3b73 100644 --- a/include/drm/drm_file.h +++ b/include/drm/drm_file.h @@ -280,6 +280,7 @@ struct drm_file { #if IS_ENABLED(CONFIG_CGROUP_DRM) struct list_head clink; + struct pid *cpid; #endif /** From patchwork Wed Oct 19 17:32:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 5788 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp452252wrs; Wed, 19 Oct 2022 10:38:33 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7rl+sfrL5u8IJXBTzWYwUTxO4U6Wop/kndH4V4tc7cFx1wyoJEJxN4x18VzxCy6Pd12jY1 X-Received: by 2002:a17:907:d04:b0:76e:e208:27ba with SMTP id gn4-20020a1709070d0400b0076ee20827bamr7992401ejc.652.1666201113672; Wed, 19 Oct 2022 10:38:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666201113; cv=none; d=google.com; s=arc-20160816; b=lDsgiByHOoZ7pij3Mgmzrk+CaJ8ZHi5PT/tjFEF5FHULzRS1G0Wwm593872D2Gjvp7 G3wbk/9FpASCOVOsKJKeWvSzBG/5QJK717L8ZS2Sw8jjkibrtvI/s/uyun38udNz8C9F gkjALbCYAzm3EO6R+zVRgV2kgVfQntLIPw/BwPPMl/vq5+ZP13n5Qf4ayl+CeZ/5amkl V3SDWtiuQrSv5yyCnTuFbT7XiHZm7k6HMsSXXcxnqaPse+1X7OIh9Xb+fXjLgGR+eN8F VrZxtM1K/DP5F/T83i/55/Dz6dTzaSvrf76fqtLbDhfAeUAQoy+saRlmsgHCSKYjjxQ2 f6BQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=+0SYbttWEjnSNgY9JDD51A1Ms/+g73ZNZSo/fI60Yj0=; b=eiaae2lFkPTbqkagk+FcOWoORuDq+vehp176Zv+Xd6pfgVhia/em8ja7+ST3fMbvOE 0hxV+s5l4TlusRVC6GC7I5E6VjULPbqmnQr6pNzqXjnl17Cyp+dTqjoJSI3SMY2Ls+RC tciiQFSI5o+u4Ne1JmQwcG9UpHu+pms3/sKG5K3VS5QvSO0gLwTTKkZ6vhMY+ged9sWh J+FS5UWG5P09Xq7zAxQ3BjziB7wwdanIyWHgdUDk2mq2kE1ckw0rrh7LOwJcT9bF29i+ lJf8YjIVZN59ZNfLcI1Jz967WR+lu/sIf49f1IMfdKkVjx60n4hA/+WQy/UFKSruNODj gTUg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=IVcznIwZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id gs17-20020a1709072d1100b0078d27c1b499si18725166ejc.500.2022.10.19.10.38.08; Wed, 19 Oct 2022 10:38:33 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=IVcznIwZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231297AbiJSRev (ORCPT + 99 others); Wed, 19 Oct 2022 13:34:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231504AbiJSReQ (ORCPT ); Wed, 19 Oct 2022 13:34:16 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 347D21D3473; Wed, 19 Oct 2022 10:34:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666200850; x=1697736850; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1PhDcr2XGs8uIfUaec8gKHHe5Qyfm3zBRnEHtyvj3RQ=; b=IVcznIwZ5QqLvgB7+xhCORjk2JBODXlTGjC2Gv19V7X4oqGukeYnJMAz yCts2qL9vgUVaCxK7nrqHnu4wjqFR0eOWHFk8ncoeGLG+RAfmNd0Im/Rv xeFItnQyK5JiCr9BNQWYjpu9a+y3UVpQfB2nUl5RWI4SiZbZZ39rpMvlR 68D9ft49KUHYaTkL1CWV8UZ6mmkbgdwM6TIud07R6GGr89oMX9onTX9MX 2WK/ItwN56NF0Yz1AZ+wFlKmDy1psGXqbeyVwWCgWmZ+iv+WoLwv0WYIL lizBTLo8T95KuhnxtcGBBhkr75CtSI5wPUKu05mZX2hfMOTV6MAjuLxN1 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="306474481" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="306474481" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:50 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="607204795" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="607204795" Received: from mjmcener-mobl1.amr.corp.intel.com (HELO localhost.localdomain) ([10.213.233.40]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:46 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 09/17] cgroup/drm: Introduce weight based drm cgroup control Date: Wed, 19 Oct 2022 18:32:46 +0100 Message-Id: <20221019173254.3361334-10-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> References: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747138499099062547?= X-GMAIL-MSGID: =?utf-8?q?1747138499099062547?= From: Tvrtko Ursulin Similar to CPU scheduling, implement a concept of weight in the drm cgroup controller. Uses the same range and default as the CPU controller - CGROUP_WEIGHT_MIN, CGROUP_WEIGHT_DFL and CGROUP_WEIGHT_MAX. Later each cgroup is assigned a time budget proportionaly based on the relative weights of it's siblings. This time budget is in turn split by the group's children and so on. Children of the root cgroup will be exempt from split budgets and therefore compete for the GPU time independently and without weight based control. This will be used to implement a soft, or best effort signal from drm cgroup to drm core notifying about groups which are over their allotted budget. No guarantees that the limit can be enforced are provided or implied. Signed-off-by: Tvrtko Ursulin --- kernel/cgroup/drm.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/kernel/cgroup/drm.c b/kernel/cgroup/drm.c index 01954c3a2087..4b6f88d8236e 100644 --- a/kernel/cgroup/drm.c +++ b/kernel/cgroup/drm.c @@ -17,6 +17,7 @@ struct drm_cgroup_state { int priority; int effective_priority; + unsigned int weight; }; static DEFINE_MUTEX(drmcg_mutex); @@ -163,6 +164,33 @@ drmcs_write_priority(struct cgroup_subsys_state *css, struct cftype *cftype, return 0; } +static u64 +drmcs_read_weight(struct cgroup_subsys_state *css, struct cftype *cft) +{ + struct drm_cgroup_state *drmcs = css_to_drmcs(css); + + return drmcs->weight; +} + +static int +drmcs_write_weight(struct cgroup_subsys_state *css, struct cftype *cftype, + u64 weight) +{ + struct drm_cgroup_state *drmcs = css_to_drmcs(css); + int ret; + + if (weight < CGROUP_WEIGHT_MIN || weight > CGROUP_WEIGHT_MAX) + return -ERANGE; + + ret = mutex_lock_interruptible(&drmcg_mutex); + if (ret) + return ret; + drmcs->weight = weight; + mutex_unlock(&drmcg_mutex); + + return 0; +} + static int drmcs_online(struct cgroup_subsys_state *css) { struct drm_cgroup_state *drmcs = css_to_drmcs(css); @@ -210,6 +238,8 @@ drmcs_alloc(struct cgroup_subsys_state *parent_css) if (!drmcs) return ERR_PTR(-ENOMEM); + drmcs->weight = CGROUP_WEIGHT_DFL; + return &drmcs->css; } @@ -230,6 +260,12 @@ struct cftype files[] = { .flags = CFTYPE_NOT_ON_ROOT, .read_s64 = drmcs_read_effective_priority, }, + { + .name = "weight", + .flags = CFTYPE_NOT_ON_ROOT, + .read_u64 = drmcs_read_weight, + .write_u64 = drmcs_write_weight, + }, { } /* Zero entry terminates. */ }; From patchwork Wed Oct 19 17:32:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 5789 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp452262wrs; Wed, 19 Oct 2022 10:38:36 -0700 (PDT) X-Google-Smtp-Source: AMsMyM75Bgx3s/8qiWy67EqjAXNXQvBG7+oEqVrZGIYbObBTP8oRfAWoSyRDvD+IIUJbQZD/tCtZ X-Received: by 2002:a17:906:ee86:b0:741:89bc:27a1 with SMTP id wt6-20020a170906ee8600b0074189bc27a1mr7893655ejb.725.1666201116419; Wed, 19 Oct 2022 10:38:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666201116; cv=none; d=google.com; s=arc-20160816; b=xjVqR1iB7+B9O0kPNYF5oD3uepazV3wzorzGrRDTVARhKJLwjq763a7+e5tQnWvjR8 7QCWEnPYQJeJJMinG5HeR8TGApNF75Y6dRk9QGCon2xZAn5I0bucpZPQHZLGKEgc7uhC nmE8VwT53zcfxqFHWwAcIWIs5+rRRnTltCtrZB39fQZ9/IyAswKOL9c4Fq5gDcgN+tms OVZvOg++c3ozcuYEY4kCiF69UligHVXFOARUqvJJov8bQXA6T4PhHqDzhtJ2hssT0FpK BI5TkoP61/LqCbVyIINavjxhBW9/y/ElcS4U6g6y0PfawsurbTx4sRtusm56nTMB15ki bWhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Sb4uoz9KdFzr88tTKQ1xmLBEFUwdur38e5TTmfOsURo=; b=w2W0Uo7nzixqbxwrZivIqaui1LS/Z9RQgMFn8bHbTaQ+Mtz8qjslUb8euSa+sPD8JG Cg0QZT7WCSkUnfepLjRe7pSyNdBKhQt2lFAuQ7cC9hA7TsjRMOcxw8wM/O48vO6SMY+a uLPrlzLMaputQOfUaJhp/pCQ3eisd4ARnQpi5oE88QnXAdYraT0/9rFkDXbeLFGMnTe/ X3kpGtFEvLlPzGRhdD1YCJXU79wPJw1CmeFwzGdjUmvzggli/G9YjJ938QKN0L8xQ+Se 0zbPAlXk8+0UKGrJxbbkCSG375GCsS6KE+a423ju6v2rOmW5fSVMZbDm6aILnAZGDhyB 4SSw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="b403/DjI"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q22-20020a170906771600b0078da3218b54si14186131ejm.171.2022.10.19.10.38.11; Wed, 19 Oct 2022 10:38:36 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="b403/DjI"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231273AbiJSRep (ORCPT + 99 others); Wed, 19 Oct 2022 13:34:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231397AbiJSReM (ORCPT ); Wed, 19 Oct 2022 13:34:12 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5CDD41D3765; Wed, 19 Oct 2022 10:34:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666200850; x=1697736850; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=57xevMaPNuTYw9loIK3v8HjbtAVBJSLqQ4lTiTCwMqQ=; b=b403/DjIO2r0FckCx6SOM0koAAHJZ2j5vZVUjwN7GcB6jJf6KNBKeme3 opYJEwTVl6Ge+palbio+0/ishxAb71+4mJiJILJnrJHMRv2DPN70P7zP4 KW2AiNyfdnBjyxQw8eevt6AKaCztGm6N0F1BMKd68k3F4ADNmdJQ1QvTT fKsESVkZiBuoDyiUF7/iH+1c5Ozzh0ZBDtQka7YH64PPIafL3TAZt5EDu 5NaUXct4Hga8uQjwgkx5vkx7jeRV06UYkUony3vaJ8cbl61i8xwtmkg2j k1bRLEd3er4WtEKSC7WpM29VRiVeh4OJ0YEyYjX7g1jfgqb2D8qVHH2Fi w==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="306474511" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="306474511" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:53 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="607204806" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="607204806" Received: from mjmcener-mobl1.amr.corp.intel.com (HELO localhost.localdomain) ([10.213.233.40]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:50 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 10/17] drm: Add ability to query drm cgroup GPU time Date: Wed, 19 Oct 2022 18:32:47 +0100 Message-Id: <20221019173254.3361334-11-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> References: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747138502074502236?= X-GMAIL-MSGID: =?utf-8?q?1747138502074502236?= From: Tvrtko Ursulin Add a driver callback and core helper which allow querying the time spent on GPUs for processes belonging to a group. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/drm_cgroup.c | 24 ++++++++++++++++++++++++ include/drm/drm_clients.h | 1 + include/drm/drm_drv.h | 9 +++++++++ 3 files changed, 34 insertions(+) diff --git a/drivers/gpu/drm/drm_cgroup.c b/drivers/gpu/drm/drm_cgroup.c index 59b730ed1334..e0cadb5e5659 100644 --- a/drivers/gpu/drm/drm_cgroup.c +++ b/drivers/gpu/drm/drm_cgroup.c @@ -206,3 +206,27 @@ void drm_pid_update_priority(struct pid *pid, int priority) rcu_read_unlock(); } EXPORT_SYMBOL_GPL(drm_pid_update_priority); + +u64 drm_pid_get_active_time_us(struct pid *pid) +{ + struct drm_pid_clients *clients; + u64 total = 0; + + rcu_read_lock(); + clients = xa_load(&drm_pid_clients, (unsigned long)pid); + if (clients) { + struct drm_file *fpriv; + + list_for_each_entry_rcu(fpriv, &clients->file_list, clink) { + const struct drm_cgroup_ops *cg_ops = + fpriv->minor->dev->driver->cg_ops; + + if (cg_ops && cg_ops->active_time_us) + total += cg_ops->active_time_us(fpriv); + } + } + rcu_read_unlock(); + + return total; +} +EXPORT_SYMBOL_GPL(drm_pid_get_active_time_us); diff --git a/include/drm/drm_clients.h b/include/drm/drm_clients.h index 3a0b1cdb338f..f25e09ed5feb 100644 --- a/include/drm/drm_clients.h +++ b/include/drm/drm_clients.h @@ -37,5 +37,6 @@ static inline void drm_clients_migrate(struct drm_file *file_priv) unsigned int drm_pid_priority_levels(struct pid *pid, bool *non_uniform); void drm_pid_update_priority(struct pid *pid, int priority); +u64 drm_pid_get_active_time_us(struct pid *pid); #endif diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h index 2371d73e12cf..0f1802df01fe 100644 --- a/include/drm/drm_drv.h +++ b/include/drm/drm_drv.h @@ -183,6 +183,15 @@ struct drm_cgroup_ops { * priorities of already running workloads. */ void (*update_priority) (struct drm_file *, int priority); + + /** + * @active_time_us: + * + * Optional callback for reporting the GPU time consumed by this client. + * + * Used by the DRM core when queried by the DRM cgroup controller. + */ + u64 (*active_time_us) (struct drm_file *); }; /** From patchwork Wed Oct 19 17:32:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 5792 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp452499wrs; Wed, 19 Oct 2022 10:39:10 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5li46LKDWWmvZjrSs+uwCff3qjVf/QoS3TOkzx2TQ/ZhVpAQxBL3npBi7snfIkD6L6b7RZ X-Received: by 2002:a17:907:3e14:b0:78d:b14b:6fa0 with SMTP id hp20-20020a1709073e1400b0078db14b6fa0mr7550763ejc.650.1666201149823; Wed, 19 Oct 2022 10:39:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666201149; cv=none; d=google.com; s=arc-20160816; b=y2FCWJKfTojEEtm9SNrySTcwh5nAnT34AO6HYv0zk1Y5fFiEXzsBh7rUHjdYMmMcF/ OHm1PzdV1Hsw/AhHbEPKTTR301tZ93qJICMOg94844QlPHo+E0z1yyQsJXGkhXOVlqn+ vsApnSR3tn6hUG54V1Qo9c3FHueGtDAzJUeijLvALiNjzQ9kuT+5qghAhwi/aq9iMR1P 9e7ZwdGVYwH67WB515rgJMPIOtZEHChpU4tJETWoib0qCZsehhyJY1BnND9tcZD6y1Lz a+WQzIGGR3r1EvTt1d25gBSYGFgIMuGtdb95jRkL+85eBHL+WAFBtUrGZKpuCnd29FEH Ztqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=tuHN3b8i9JjwlZv9qe3ZfNCpHXqhn1shZAywGh4qoXY=; b=ced2vgjU2X1NgvumfXiMwa0y8x2CofgKM6Ucjf4J3oOKfpBRT8piRpVnkMKTbtj2ma rGfw5Vj7vM8aTJhxrmu86jOXQvDC0s9kqL13Mij6XsDcrIgQkrXONM9QPIp8Hzgbttnj aams5y6Q94TKWkSfrmPRGNIxHIf4hRecTpylM7bcvMAAqa5k15X+8PLW0bKUGf/yuQBD 6JiH7fvhsnPre4DL3DDIA7bHEohKlpAvJYUJGjJtEkCv9WucR7Th7gZrH/nUcsEM8OmQ ZIcjWGQ/67emmVswdpaZNEBNfYsxdWExmCIxpkQu5T8xRjRkafm+OzDSRonrw3j8cn9m MA8A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YT2wlRma; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ae8-20020a17090725c800b0078db794016csi13628461ejc.907.2022.10.19.10.38.45; Wed, 19 Oct 2022 10:39:09 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YT2wlRma; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231382AbiJSRe6 (ORCPT + 99 others); Wed, 19 Oct 2022 13:34:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33836 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230043AbiJSReQ (ORCPT ); Wed, 19 Oct 2022 13:34:16 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E50AB1D345D; Wed, 19 Oct 2022 10:34:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666200850; x=1697736850; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SCLxQXssqCGWpksvQ3lmggIlRVu0wTOV5RqTAwzpGxU=; b=YT2wlRmaNRcXW/JlKsOJWoWSF2xWdU1Gv+Qgjo6yjhwKvjZlBlPz/H1d FcIr8njPSKs6MoEnhsZKZmIw9sXoPM4vDcnWWwQzH7aJE2bagZyYes5d4 cpXpZl9z8GRTo3vUZfvr4uizNPe3iv7HwoMVTvTIkaKhlB+agwSMPdHT3 yyS5LfToopSOcYzFOTA1oJ85AGW0I3IZD+drSdzSSGIW1Lm2XH1+14hDQ PnLg+x3P+0fCo1qNBq9hPEK59+n04wgL3UVXw+5VsHVFf7kX1UiSARnHJ nedAlE7xk3G1WTptIunaKd+nBmfdIXhHVEhCWFp5CbUMVhtWaTHfvOVNR g==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="306474529" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="306474529" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:56 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="607204816" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="607204816" Received: from mjmcener-mobl1.amr.corp.intel.com (HELO localhost.localdomain) ([10.213.233.40]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:53 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 11/17] drm: Add over budget signalling callback Date: Wed, 19 Oct 2022 18:32:48 +0100 Message-Id: <20221019173254.3361334-12-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> References: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747138536769501657?= X-GMAIL-MSGID: =?utf-8?q?1747138536769501657?= From: Tvrtko Ursulin Add a new callback via which the drm cgroup controller is notifying the drm core that a certain process is above its allotted GPU time. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/drm_cgroup.c | 21 +++++++++++++++++++++ include/drm/drm_clients.h | 1 + include/drm/drm_drv.h | 8 ++++++++ 3 files changed, 30 insertions(+) diff --git a/drivers/gpu/drm/drm_cgroup.c b/drivers/gpu/drm/drm_cgroup.c index e0cadb5e5659..e36bc4333924 100644 --- a/drivers/gpu/drm/drm_cgroup.c +++ b/drivers/gpu/drm/drm_cgroup.c @@ -230,3 +230,24 @@ u64 drm_pid_get_active_time_us(struct pid *pid) return total; } EXPORT_SYMBOL_GPL(drm_pid_get_active_time_us); + +void drm_pid_signal_budget(struct pid *pid, u64 usage, u64 budget) +{ + struct drm_pid_clients *clients; + + rcu_read_lock(); + clients = xa_load(&drm_pid_clients, (unsigned long)pid); + if (clients) { + struct drm_file *fpriv; + + list_for_each_entry_rcu(fpriv, &clients->file_list, clink) { + const struct drm_cgroup_ops *cg_ops = + fpriv->minor->dev->driver->cg_ops; + + if (cg_ops && cg_ops->signal_budget) + cg_ops->signal_budget(fpriv, usage, budget); + } + } + rcu_read_unlock(); +} +EXPORT_SYMBOL_GPL(drm_pid_signal_budget); diff --git a/include/drm/drm_clients.h b/include/drm/drm_clients.h index f25e09ed5feb..7ad09fd0a404 100644 --- a/include/drm/drm_clients.h +++ b/include/drm/drm_clients.h @@ -38,5 +38,6 @@ static inline void drm_clients_migrate(struct drm_file *file_priv) unsigned int drm_pid_priority_levels(struct pid *pid, bool *non_uniform); void drm_pid_update_priority(struct pid *pid, int priority); u64 drm_pid_get_active_time_us(struct pid *pid); +void drm_pid_signal_budget(struct pid *pid, u64 usage, u64 budget); #endif diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h index 0f1802df01fe..07dec956ebfb 100644 --- a/include/drm/drm_drv.h +++ b/include/drm/drm_drv.h @@ -192,6 +192,14 @@ struct drm_cgroup_ops { * Used by the DRM core when queried by the DRM cgroup controller. */ u64 (*active_time_us) (struct drm_file *); + + /** + * @signal_budget: + * + * Optional callback used by the DRM core to forward over/under GPU time + * messages sent by the DRM cgroup controller. + */ + void (*signal_budget) (struct drm_file *, u64 used, u64 budget); }; /** From patchwork Wed Oct 19 17:32:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 5790 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp452329wrs; Wed, 19 Oct 2022 10:38:45 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7QhGmitQpg6Mlr6fl2Yp8rh+jgfLEDSvUJyQ2xPk34eEMNqCcpE8POhxlPIdPtED95LQOZ X-Received: by 2002:a17:906:fc6:b0:72f:d080:416 with SMTP id c6-20020a1709060fc600b0072fd0800416mr7850271ejk.1.1666201125744; Wed, 19 Oct 2022 10:38:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666201125; cv=none; d=google.com; s=arc-20160816; b=ZIG+fxDXcpVsmUPkkOnzsaCtWDWxKlIki4cq85ej1fHW+Ho3fUJ2l5hRUaLuL3hUQN b9TNmexrdkKy2w2QAybE77+1JEcwJ9g/XrnlwuxKsER8eoY0yjUfU0Jw7CRbo+q+e/To 2FwtusDK/x+ehE0tj2ggBCvC2MBw3zdCvzNjItba2NBZ20ug3ulvh4uRiT16AAJ8cDII rh2siOrtfqf8lKbz1hXB2OdnpFNyDdLRtus73IAYFbju+RP9okV7EVVjjqufL0W6K7JG 8RP3Pf1SK6pa1byo48ZihqMf5VmvCtcGf0/O+BNONem63r7WkENXs37Byuu5/DSTAkrG /sgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=NSkS7M8YosxLuSsHaOOSZY7WYyRcJmNMMsHK5yXz4RA=; b=GiL953LuU80/W/6uP7vdWfiojUFB+Ce386bs2AWtcJbxDlW+ornS1CBgsMqBZmKB+b r/ailIc3buuTGkhosrnPEZ3aS63F6hXPyKOluHHAps1fE/sgbfGbozzKfgEbr5K1x3id x7I3BZuJDAD6iK6JEbnNgFuo7IkZsM21o+DOm+sw2DQrOD+KnAxzsvQP/5QJ+Sqrn1Qg LgQizhzfn6wO5CysOUX7B3ePIvX2zUfC18jQY8A4q0znJ2OoEnejMs1BsJ3a9+JN5I1D YqYVRG9zesO/wG/5gg7dMPyL32WJOEhxgW382HTV/RzxWyUhUTPbyurl9wnqoUQ2G7+7 mzog== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=UZDN5ecu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j11-20020a05640211cb00b0045f802c0b98si2826708edw.443.2022.10.19.10.38.20; Wed, 19 Oct 2022 10:38:45 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=UZDN5ecu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231441AbiJSRfB (ORCPT + 99 others); Wed, 19 Oct 2022 13:35:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33860 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230243AbiJSReR (ORCPT ); Wed, 19 Oct 2022 13:34:17 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5A54A1C20A8; Wed, 19 Oct 2022 10:34:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666200852; x=1697736852; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2SjnzP/RjutkVE2GWCovZkYcvG7xuDDB28V5+LUt3bM=; b=UZDN5ecuQzQMwOX56Ll2FRO2VGAm7ii9b8WiAyLfnIZPtdVuqWvI+8uh xC0g7YzBZH34qUDvvR4TSwnqUcyHjEHfy0PfdSEENpUIXd2fs9RCID3Xw QUvYhXIuoMtsmQW3K+XxpR3bCgs1XcJvBnHKmL3vXaM1NFb2dslugUcU8 BWLbpmlqK6amndbYgDJx12UKGNMwM2JT7yQOjcW46mXLG3bcjSc8WiLoH 2ZPwbJdUnVXqg20iVaq9WdibSi/Cu35zt6noEmyrJ5MOyDXyx34Fq57Tc sFxInBtg5eBRkLq1DTPLvBXkoF7l8QxBEpHGWuKUC/y96bX5/NY2lBkVt w==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="306474544" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="306474544" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:34:00 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="607204833" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="607204833" Received: from mjmcener-mobl1.amr.corp.intel.com (HELO localhost.localdomain) ([10.213.233.40]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:33:56 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 12/17] cgroup/drm: Client exit hook Date: Wed, 19 Oct 2022 18:32:49 +0100 Message-Id: <20221019173254.3361334-13-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> References: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747138511557314305?= X-GMAIL-MSGID: =?utf-8?q?1747138511557314305?= From: Tvrtko Ursulin We need the ability for DRM core to inform the cgroup controller when a client has closed a DRM file descriptor. This will allow us not needing to keep state relating to GPU time usage by tasks sets in the cgroup controller itself. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/drm_cgroup.c | 8 ++++++++ include/linux/cgroup_drm.h | 1 + kernel/cgroup/drm.c | 8 ++++++++ 3 files changed, 17 insertions(+) diff --git a/drivers/gpu/drm/drm_cgroup.c b/drivers/gpu/drm/drm_cgroup.c index e36bc4333924..ff99d1f4f1d4 100644 --- a/drivers/gpu/drm/drm_cgroup.c +++ b/drivers/gpu/drm/drm_cgroup.c @@ -5,6 +5,7 @@ #include #include +#include static DEFINE_XARRAY(drm_pid_clients); @@ -25,6 +26,7 @@ void drm_clients_close(struct drm_file *file_priv) { struct drm_device *dev = file_priv->minor->dev; struct drm_pid_clients *clients; + struct task_struct *task; lockdep_assert_held(&dev->filelist_mutex); @@ -35,6 +37,12 @@ void drm_clients_close(struct drm_file *file_priv) if (WARN_ON_ONCE(!clients)) return; + task = get_pid_task(file_priv->cpid, PIDTYPE_PID); + if (task) { + drmcgroup_client_exited(task); + put_task_struct(task); + } + __del_clients(clients, file_priv); /* diff --git a/include/linux/cgroup_drm.h b/include/linux/cgroup_drm.h index 66063b4708e8..c84516d3e50a 100644 --- a/include/linux/cgroup_drm.h +++ b/include/linux/cgroup_drm.h @@ -13,5 +13,6 @@ struct task_struct; #define DRM_CGROUP_PRIORITY_MAX (10000) int drmcgroup_lookup_effective_priority(struct task_struct *task); +void drmcgroup_client_exited(struct task_struct *task); #endif /* _CGROUP_DRM_H */ diff --git a/kernel/cgroup/drm.c b/kernel/cgroup/drm.c index 4b6f88d8236e..48f1eaaa1c07 100644 --- a/kernel/cgroup/drm.c +++ b/kernel/cgroup/drm.c @@ -221,6 +221,14 @@ static void drmcs_attach(struct cgroup_taskset *tset) css_to_drmcs(css)->effective_priority); } +void drmcgroup_client_exited(struct task_struct *task) +{ + struct drm_cgroup_state *drmcs = get_task_drmcs(task); + + css_put(&drmcs->css); +} +EXPORT_SYMBOL_GPL(drmcgroup_client_exited); + static struct drm_cgroup_state root_drmcs = { .priority = DRM_CGROUP_PRIORITY_DEF, .effective_priority = DRM_CGROUP_PRIORITY_DEF, From patchwork Wed Oct 19 17:32:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 5786 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp452134wrs; Wed, 19 Oct 2022 10:38:16 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4kZFXacBe/dD2Jp0V1/sW6AjbK0ViS/RTcME3SXYylUeEEuCut2ABKkwwoq93y6znlLiuE X-Received: by 2002:a17:907:1dc4:b0:78d:a20e:fb33 with SMTP id og4-20020a1709071dc400b0078da20efb33mr8015974ejc.38.1666201096254; Wed, 19 Oct 2022 10:38:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666201096; cv=none; d=google.com; s=arc-20160816; b=hmEVmN4i918X2GX3ig1pViN+AZXDEf33s8osHg2KiO921g7lOpGSNTEAsG7njGYgp6 /uAmdchFGAQT0e5xcWlquJpTxDZhJ6pDWcAdvTumEYNNtIp5w5w8fZqDM1Pin9JWZ2De hzICKEiSZCox+FlX2PMQRmhniFSOwhj4iSKoINBkxufPQfjQju1OEayu+sopDNsODLyW ONXUpDoaI6BnQ9BwOh6F1ieizWi9JmsP2jtB+AUYcLvIJdqXYdMcCb0HO61BmGs0G5pj B2+fRy2p6lIACAYKGTiCqBbkHCht6fAtIDsLh+I2TwX6mmiqANJins1F3M80ESjrNrxW POYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=NkkgL+GSxSo0OM+L/m039IdkEgiYcrPLgppu2Fdl2Hk=; b=0PLt+2juwt3zv6RiOgN29bR2K+5BvhapfxHi8v82wx/OO/evDY3c7utXIKTgcANKqe 0PhToN+QNdtIQo7VS20kPvSKlLe46pqXBKWvh508xnEK0R9th8MzFnNVmlxMlycLHLJM PHIRxAAmiGePFZq001JKNLdJAM3054wp84WHyjUCEqpNbc2x7vGs0ROacLUpX2v1jrn0 7p6e9R9Nv8lGnT20tD+RllCiCmi9q1Xq2b3BRPNr3jn+VCvER6KQ1dykRs4kwDY/+OOm 1W4peZXZIg78blz1MvRcNUHwZeZX7xqkX3T669WLpdr5cnKUiCN1VQ8x1ApMHVmrlEGj XZXQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=PfrrEeh7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id v4-20020a056402184400b00457c123d802si12757611edy.476.2022.10.19.10.37.50; Wed, 19 Oct 2022 10:38:16 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=PfrrEeh7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231230AbiJSRfX (ORCPT + 99 others); Wed, 19 Oct 2022 13:35:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33876 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231531AbiJSReU (ORCPT ); Wed, 19 Oct 2022 13:34:20 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2ECB61C19E8; Wed, 19 Oct 2022 10:34:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666200856; x=1697736856; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vOPTUUTIulygJxZPPj0sccp0T+KD8nT50/5do9tmp3Y=; b=PfrrEeh78wqJlh7KeYiyYLR2cT9vCCkNQbzcIXIqrky7uB6ItCi9iB+j 6APOuYfmuNonqqgzHf+O6L7Z8bIbyzprntbtshTfBEAub+UhbRyf4QVqL ghcjOi61CFSm++t2txLBEzhoUTZAM3GNVwDYf5XY0I/CQ4MPwjGSLnSpv z1AZNOiyZDnEoID6Glbr4L+/kGss6lXIKCp0pe0jDS0PQyLWDcFpq72ja 9GFyicqzFf+ynRbPHvnqGLWknOBqppj+bwXv+QAY7axmmG2O+BwqV9/Ib 3SjwaYqbOjdnNV89f4J8Q77GlYMEvBGPdsQvlLGNj8bVuAuk+/7nnZ8st g==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="306474566" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="306474566" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:34:03 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="607204871" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="607204871" Received: from mjmcener-mobl1.amr.corp.intel.com (HELO localhost.localdomain) ([10.213.233.40]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:34:00 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 13/17] cgroup/drm: Ability to periodically scan cgroups for over budget GPU usage Date: Wed, 19 Oct 2022 18:32:50 +0100 Message-Id: <20221019173254.3361334-14-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> References: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747138481149950583?= X-GMAIL-MSGID: =?utf-8?q?1747138481149950583?= From: Tvrtko Ursulin Add a scanning worker, which if enabled, periodically queries the cgroup for GPU usage and if over budget (as configured by it's relative weight share) notifies the drm core about the fact. This is off by default and can be enabled by configuring a scanning period using the drm.period_us cgroup control file. Signed-off-by: Tvrtko Ursulin --- Documentation/admin-guide/cgroup-v2.rst | 35 +- kernel/cgroup/drm.c | 426 +++++++++++++++++++++++- 2 files changed, 459 insertions(+), 2 deletions(-) diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index 1f3cca4e2572..318f463a1316 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -2401,7 +2401,8 @@ HugeTLB Interface Files DRM --- -The DRM controller allows configuring static hierarchical scheduling priority. +The DRM controller allows configuring static hierarchical scheduling priority +and scheduling soft limits. DRM static priority control ~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -2458,6 +2459,38 @@ DRM static priority interface files Read only integer showing the current effective priority level for the group. Effective meaning taking into account the chain of inherited +DRM scheduling soft limits +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Because of the heterogenous hardware and driver DRM capabilities, soft limits +are implemented as a loose co-operative (bi-directional) interface between the +controller and DRM core. + +The controller configures the GPU time allowed per group and periodically scans +the belonging tasks to detect the over budget condition, at which point it +invokes a callback notifying the DRM core of the condition. + +DRM core provides an API to query per process GPU utilization and 2nd API to +receive notification from the cgroup controller when the group enters or exits +the over budget condition. + +Individual DRM drivers which implement the interface are expected to act on this +in the best-effort manner only. There are no guarantees that the soft limits +will be respected. + +DRM scheduling soft limits interface files +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + drm.weight + Standard cgroup weight based control [1, 10000] used to configure the + relative distributing of GPU time between the sibling groups. + + drm.period_us + An integer representing the period with which the controller should look + at the GPU usage by the group and potentially send the over/under budget + signal. + Value of zero (defaul) disables the soft limit checking. + Misc ---- diff --git a/kernel/cgroup/drm.c b/kernel/cgroup/drm.c index 48f1eaaa1c07..af50ead1564a 100644 --- a/kernel/cgroup/drm.c +++ b/kernel/cgroup/drm.c @@ -18,6 +18,29 @@ struct drm_cgroup_state { int priority; int effective_priority; unsigned int weight; + unsigned int period_us; + + bool scanning_suspended; + unsigned int suspended_period_us; + + struct delayed_work scan_work; + + /* + * Below fields are owned and updated by the scan worker. Either the + * worker accesses them, or worker needs to be suspended and synced + * before they can be touched from the outside. + */ + bool scanned; + + ktime_t prev_timestamp; + + u64 sum_children_weights; + u64 children_active_us; + u64 per_s_budget_ns; + u64 prev_active_us; + u64 active_us; + + bool over_budget; }; static DEFINE_MUTEX(drmcg_mutex); @@ -33,6 +56,31 @@ static inline struct drm_cgroup_state *get_task_drmcs(struct task_struct *task) return css_to_drmcs(task_get_css(task, drm_cgrp_id)); } +static u64 drmcs_get_active_time_us(struct drm_cgroup_state *drmcs) +{ + struct cgroup *cgrp = drmcs->css.cgroup; + struct task_struct *task; + struct css_task_iter it; + u64 total = 0; + + css_task_iter_start(&cgrp->self, + CSS_TASK_ITER_PROCS | CSS_TASK_ITER_THREADED, + &it); + while ((task = css_task_iter_next(&it))) { + u64 time; + + /* Ignore kernel threads here. */ + if (task->flags & PF_KTHREAD) + continue; + + time = drm_pid_get_active_time_us(task_pid(task)); + total += time; + } + css_task_iter_end(&it); + + return total; +} + int drmcgroup_lookup_effective_priority(struct task_struct *task) { struct drm_cgroup_state *drmcs = get_task_drmcs(task); @@ -202,9 +250,301 @@ static int drmcs_online(struct cgroup_subsys_state *css) return 0; } +static void +signal_drm_budget(struct drm_cgroup_state *drmcs, u64 usage, u64 budget) +{ + struct cgroup *cgrp = drmcs->css.cgroup; + struct task_struct *task; + struct css_task_iter it; + + css_task_iter_start(&cgrp->self, + CSS_TASK_ITER_PROCS | CSS_TASK_ITER_THREADED, + &it); + while ((task = css_task_iter_next(&it))) { + /* Ignore kernel threads here. */ + if (task->flags & PF_KTHREAD) + continue; + + drm_pid_signal_budget(task_pid(task), usage, budget); + } + css_task_iter_end(&it); +} + +static bool __start_scanning(struct drm_cgroup_state *root) +{ + struct cgroup_subsys_state *node; + bool ok = true; + + rcu_read_lock(); + css_for_each_descendant_pre(node, &root->css) { + struct drm_cgroup_state *drmcs = css_to_drmcs(node); + unsigned long active; + + if (!css_tryget_online(node)) { + ok = false; + continue; + } + + drmcs->scanned = false; + drmcs->sum_children_weights = 0; + drmcs->children_active_us = 0; + if (node == &root->css) + drmcs->per_s_budget_ns = NSEC_PER_SEC; + else + drmcs->per_s_budget_ns = 0; + + active = drmcs_get_active_time_us(drmcs); + if (active >= drmcs->prev_active_us) + drmcs->active_us = active - drmcs->prev_active_us; + else + drmcs->active_us = 0; + drmcs->prev_active_us = active; + + css_put(node); + } + rcu_read_unlock(); + + return ok; +} + +static void scan_worker(struct work_struct *work) +{ + struct drm_cgroup_state *root = + container_of(work, typeof(*root), scan_work.work); + struct cgroup_subsys_state *node; + unsigned int period_us; + ktime_t now; + + rcu_read_lock(); + + if (WARN_ON_ONCE(!css_tryget_online(&root->css))) + return; + + /* + * 1st pass - reset accumulated values and update group GPU activity. + */ + if (!__start_scanning(root)) + goto out_retry; /* + * Always come back later if scanner races with + * core cgroup management. (Repeated pattern.) + */ + + now = ktime_get(); + period_us = ktime_to_us(ktime_sub(now, root->prev_timestamp)); + root->prev_timestamp = now; + + /* + * 2nd pass - calculate accumulated GPU activity and relative weights + * for each parent's children. + */ + css_for_each_descendant_pre(node, &root->css) { + struct drm_cgroup_state *drmcs = css_to_drmcs(node); + + if (!css_tryget_online(node)) + goto out_retry; + + if (!drmcs->scanned) { + struct cgroup_subsys_state *css; + + css_for_each_child(css, &drmcs->css) { + struct drm_cgroup_state *sibling = + css_to_drmcs(css); + + if (!css_tryget_online(css)) { + css_put(node); + goto out_retry; + } + + drmcs->children_active_us += sibling->active_us; + drmcs->sum_children_weights += sibling->weight; + + css_put(css); + } + + drmcs->scanned = true; + } + + css_put(node); + } + + /* + * 3rd pass - calculate relative budgets for each group based on + * relative weights and parent's budget. + * + * FIXME: This is for now incomplete in more than one way. There is + * no downward propagation of unused budgets, and even no utilisation of + * the unused budgets at all. + */ + css_for_each_descendant_pre(node, &root->css) { + struct drm_cgroup_state *drmcs, *pdrmcs; + bool over, was_over; + u64 budget; + + if (!css_tryget_online(node)) + goto out_retry; + if (node->cgroup->level == 1) { + css_put(node); + continue; + } + if (!css_tryget_online(node->parent)) { + css_put(node); + goto out_retry; + } + + drmcs = css_to_drmcs(node); + pdrmcs = css_to_drmcs(node->parent); + + drmcs->per_s_budget_ns = + DIV_ROUND_UP_ULL(pdrmcs->per_s_budget_ns * + drmcs->weight, + pdrmcs->sum_children_weights); + budget = DIV_ROUND_UP_ULL(drmcs->per_s_budget_ns * period_us, + NSEC_PER_SEC); + over = drmcs->active_us > budget; + was_over = drmcs->over_budget; + drmcs->over_budget = over; + if (over || (!over && was_over)) + signal_drm_budget(drmcs, drmcs->active_us, budget); + + css_put(node); + css_put(node->parent); + } + +out_retry: + rcu_read_unlock(); + + period_us = READ_ONCE(root->period_us); + if (period_us) + schedule_delayed_work(&root->scan_work, + usecs_to_jiffies(period_us)); + + css_put(&root->css); +} + +static void start_scanning(struct drm_cgroup_state *drmcs, u64 period_us) +{ + drmcs->period_us = (unsigned int)period_us; + WARN_ON_ONCE(!__start_scanning(drmcs)); + drmcs->prev_timestamp = ktime_get(); + mod_delayed_work(system_wq, &drmcs->scan_work, + usecs_to_jiffies(period_us)); +} + +static void stop_scanning(struct drm_cgroup_state *drmcs) +{ + drmcs->period_us = 0; + cancel_delayed_work_sync(&drmcs->scan_work); + if (drmcs->over_budget) { + /* + * Signal under budget when scanning goes off so drivers + * correctly update their state. + */ + signal_drm_budget(drmcs, 0, drmcs->per_s_budget_ns); + drmcs->over_budget = false; + } +} + +static struct drm_cgroup_state *drmcs_scanner(struct drm_cgroup_state *drmcs) +{ + while (drmcs->css.cgroup->level > 1) + drmcs = css_to_drmcs(drmcs->css.parent); + + return drmcs; +} + +static void start_suspend_scanning(struct drm_cgroup_state *drmcs) +{ + drmcs = drmcs_scanner(drmcs); + + if (drmcs->scanning_suspended) + return; + + drmcs->scanning_suspended = true; + drmcs->suspended_period_us = drmcs->period_us; + drmcs->period_us = 0; +} + +static void finish_suspend_scanning(struct drm_cgroup_state *drmcs) +{ + drmcs = drmcs_scanner(drmcs); + + if (drmcs->suspended_period_us) + cancel_delayed_work_sync(&drmcs->scan_work); +} + +static void resume_scanning(struct drm_cgroup_state *drmcs) +{ + drmcs = drmcs_scanner(drmcs); + + if (!drmcs->scanning_suspended) + return; + + drmcs->scanning_suspended = false; + if (drmcs->suspended_period_us) { + start_scanning(drmcs, drmcs->suspended_period_us); + drmcs->suspended_period_us = 0; + } +} + static void drmcs_free(struct cgroup_subsys_state *css) { - kfree(css_to_drmcs(css)); + struct drm_cgroup_state *drmcs = css_to_drmcs(css); + + stop_scanning(drmcs); + + kfree(drmcs); +} + +static int drmcs_can_attach(struct cgroup_taskset *tset) +{ + struct cgroup_subsys_state *new_css; + struct task_struct *task; + int ret; + + /* + * As processes are getting moved between groups we need to ensure + * both that the old group does not see a sudden downward jump in the + * GPU utilisation, and that the new group does not see a sudden jump + * up with all the GPU time clients belonging to the migrated process + * have accumulated. + * + * To achieve that we suspend the scanner until the migration is + * completed where the resume at the end ensures both groups start + * observing GPU utilisation from a reset state. + */ + + ret = mutex_lock_interruptible(&drmcg_mutex); + if (ret) + return ret; + + cgroup_taskset_for_each(task, new_css, tset) { + start_suspend_scanning(css_to_drmcs(task_css(task, + drm_cgrp_id))); + start_suspend_scanning(css_to_drmcs(new_css)); + } + + mutex_unlock(&drmcg_mutex); + + cgroup_taskset_for_each(task, new_css, tset) { + finish_suspend_scanning(css_to_drmcs(task_css(task, + drm_cgrp_id))); + finish_suspend_scanning(css_to_drmcs(new_css)); + } + + return 0; +} + +static void tset_resume_scanning(struct cgroup_taskset *tset) +{ + struct cgroup_subsys_state *new_css; + struct task_struct *task; + + mutex_lock(&drmcg_mutex); + cgroup_taskset_for_each(task, new_css, tset) { + resume_scanning(css_to_drmcs(task_css(task, drm_cgrp_id))); + resume_scanning(css_to_drmcs(new_css)); + } + mutex_unlock(&drmcg_mutex); } static void drmcs_attach(struct cgroup_taskset *tset) @@ -219,12 +559,86 @@ static void drmcs_attach(struct cgroup_taskset *tset) cgroup_taskset_for_each(task, css, tset) drm_pid_update_priority(task_pid(task), css_to_drmcs(css)->effective_priority); + + tset_resume_scanning(tset); +} + +static void drmcs_cancel_attach(struct cgroup_taskset *tset) +{ + tset_resume_scanning(tset); +} + +static u64 +drmcs_read_period_us(struct cgroup_subsys_state *css, struct cftype *cft) +{ + struct drm_cgroup_state *drmcs = css_to_drmcs(css); + + return drmcs->period_us; +} + +static int +drmcs_write_period_us(struct cgroup_subsys_state *css, struct cftype *cftype, + u64 period_us) +{ + struct drm_cgroup_state *drmcs = css_to_drmcs(css); + int ret; + + if (WARN_ON_ONCE(!css->parent)) + return -EINVAL; + if (css->cgroup->level != 1) + return -EINVAL; + if ((period_us && period_us < 500000) || period_us > USEC_PER_SEC * 60) + return -EINVAL; + + ret = mutex_lock_interruptible(&drmcg_mutex); + if (ret) + return ret; + + if (!drmcs->scanning_suspended) { + if (period_us) + start_scanning(drmcs, period_us); + else + stop_scanning(drmcs); + } else { + /* + * If scanning is temporarily suspended just update the period + * which will apply once resumed, or simply skip resuming in + * case of disabling. + */ + drmcs->suspended_period_us = period_us; + if (!period_us) + drmcs->scanning_suspended = false; + } + + mutex_unlock(&drmcg_mutex); + + return 0; } void drmcgroup_client_exited(struct task_struct *task) { struct drm_cgroup_state *drmcs = get_task_drmcs(task); + /* + * Since we are not tracking accumulated GPU time for each cgroup, + * avoid jumps in group observed GPU usage by re-setting the scanner + * at a point when GPU usage can suddenly jump down. + * + * Downside is clients can influence the effectiveness of the over- + * budget scanning by continuosly closing DRM file descriptors but for + * now we do not worry about it. + */ + + mutex_lock(&drmcg_mutex); + start_suspend_scanning(drmcs); + mutex_unlock(&drmcg_mutex); + + finish_suspend_scanning(drmcs); + + mutex_lock(&drmcg_mutex); + resume_scanning(drmcs); + mutex_unlock(&drmcg_mutex); + css_put(&drmcs->css); } EXPORT_SYMBOL_GPL(drmcgroup_client_exited); @@ -232,6 +646,7 @@ EXPORT_SYMBOL_GPL(drmcgroup_client_exited); static struct drm_cgroup_state root_drmcs = { .priority = DRM_CGROUP_PRIORITY_DEF, .effective_priority = DRM_CGROUP_PRIORITY_DEF, + .weight = CGROUP_WEIGHT_DFL, }; static struct cgroup_subsys_state * @@ -247,6 +662,7 @@ drmcs_alloc(struct cgroup_subsys_state *parent_css) return ERR_PTR(-ENOMEM); drmcs->weight = CGROUP_WEIGHT_DFL; + INIT_DELAYED_WORK(&drmcs->scan_work, scan_worker); return &drmcs->css; } @@ -274,6 +690,12 @@ struct cftype files[] = { .read_u64 = drmcs_read_weight, .write_u64 = drmcs_write_weight, }, + { + .name = "period_us", + .flags = CFTYPE_NOT_ON_ROOT, + .read_u64 = drmcs_read_period_us, + .write_u64 = drmcs_write_period_us, + }, { } /* Zero entry terminates. */ }; @@ -281,7 +703,9 @@ struct cgroup_subsys drm_cgrp_subsys = { .css_alloc = drmcs_alloc, .css_free = drmcs_free, .css_online = drmcs_online, + .can_attach = drmcs_can_attach, .attach = drmcs_attach, + .cancel_attach = drmcs_cancel_attach, .early_init = false, .legacy_cftypes = files, .dfl_cftypes = files, From patchwork Wed Oct 19 17:32:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 5794 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp452789wrs; Wed, 19 Oct 2022 10:39:52 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7QlJZvguMHM9REynimE20xsWOv+XCX14RgygbC4El4/7BVZAHlEp6mIN8UwR60AeBs4oXt X-Received: by 2002:a05:6402:440c:b0:43a:1124:e56a with SMTP id y12-20020a056402440c00b0043a1124e56amr8731757eda.134.1666201192222; Wed, 19 Oct 2022 10:39:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666201192; cv=none; d=google.com; s=arc-20160816; b=YsNaKvol4AXV51sSDenfZn6BbcPkCroh6qsJ3JMgBBpwJ9kEnwjxv0THQw/AGjAHxp ajAdDO6fmCxWpd28F9uc99iUjImxMG2pe2yg+tcTI0YxkZ8KxmMPWsFOt/IJjUW1FW6F bD8FqspRPUi1k4dqW7SbXQ1C9ao9R6rgEF2ZOcy5lfETpl3rkAp2bSOZPjz+soMbcFUW O7hZ3wETjBGcx4Nb2EMUxgMtPpmRSZcqQqIy1sJyMNh2mosZciun21WJbuwkpDSw6gHD p93NKzQGGKLCm6RDr/l7ICyESz/BW/RHF8+28frR74BeHI5ZpiMpqHhRLSS+FDfvpPGx eKoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=LkrazSwH5fa40F2CRHzP84Lni8TwNQ4nVVDFVGMZ9z8=; b=GAuXVi+Cutx/ndcuRh3OXTfjyrtIGu4aWISE/khCHhZ9cHmd/V8oIGPdfS4I5CuAo5 vuf96SpAXwo49Krny70Ok/7BAKvfkJanh0p2qsqLkGLkFzBivPHai/vD7DTpmOaU/7Tk efV/pOsJ7iMXIUlHEb9WJVV2RjVw+wygjnYFsAQBCg2bYuC4ySra6GVIKaP6jn25Gnnt A78rj8E/VEimxRRw0naduToY19D2v+2fdQp3H2m1Q1icsPxU5k2ns5HdAI7rkjCQYhkO QRCwsFxLAr4AkkmHMCBMPj8ldCE3W9obxMOCp+yiFE1nxiWFlQXerUjQZnRXHGTrV7aK OWxg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=FDpirong; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id qb7-20020a1709077e8700b007721f88d3e0si15003474ejc.554.2022.10.19.10.39.26; Wed, 19 Oct 2022 10:39:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=FDpirong; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230235AbiJSRfI (ORCPT + 99 others); Wed, 19 Oct 2022 13:35:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231521AbiJSReS (ORCPT ); Wed, 19 Oct 2022 13:34:18 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 870101D2B55; Wed, 19 Oct 2022 10:34:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666200856; x=1697736856; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Ji8CP4xluWOS2NrGBbUVXyKLzEJJ2hTlHVJTBMnur4M=; b=FDpironghNhixps0sAEyfhSrkLnoEuNbigtNltdkXJNHGsr69VWR0ROu GavqLkZ9gAioBMNh0SI/xamthtjtU7sMhORU8O4pAP6RyiEm/E3DBN1Pd XTGRYoeX8OtFAQIgJO4VhLRefb7kpWaA3GbU3oHlPd0lXYO6+MVwfLV7C WEAuaRH8jFO30NK6G0jUUKfHyIL4ne/2nb7/+gx7FujXdg/8oehAvHLmi j2jHIZ3gtiKjQMBT3A4ZxZZgdY4b/YYLsmYd8c7sqIp0yLGYGsv4mcYim Eg5Y29yXV2JoQrqCIVOh35h1E1vxQE12FjdScOapjnCQ+LNApEhR+K0d9 Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="306474584" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="306474584" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:34:09 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="607204894" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="607204894" Received: from mjmcener-mobl1.amr.corp.intel.com (HELO localhost.localdomain) ([10.213.233.40]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:34:03 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 14/17] cgroup/drm: Show group budget signaling capability in sysfs Date: Wed, 19 Oct 2022 18:32:51 +0100 Message-Id: <20221019173254.3361334-15-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> References: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747138581528665579?= X-GMAIL-MSGID: =?utf-8?q?1747138581528665579?= From: Tvrtko Ursulin Show overall status of a task group - whether all DRM clients in a group support over budget signaling, or some do not, or if there are no DRM clients in the group to start with. Signed-off-by: Tvrtko Ursulin --- Documentation/admin-guide/cgroup-v2.rst | 7 ++++ drivers/gpu/drm/drm_cgroup.c | 33 ++++++++++++++++ include/drm/drm_clients.h | 7 ++++ include/drm/drm_drv.h | 11 +++++- kernel/cgroup/drm.c | 52 +++++++++++++++++++++++++ 5 files changed, 109 insertions(+), 1 deletion(-) diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index 318f463a1316..6ee94ee109f0 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -2491,6 +2491,13 @@ DRM scheduling soft limits interface files signal. Value of zero (defaul) disables the soft limit checking. + drm.budget_supported + One of: + 1) 'yes' - when all DRM clients in the group support the functionality. + 2) 'no' - when at least one of the DRM clients does not support the + functionality. + 3) 'n/a' - when there are no DRM clients in the group. + Misc ---- diff --git a/drivers/gpu/drm/drm_cgroup.c b/drivers/gpu/drm/drm_cgroup.c index ff99d1f4f1d4..d2d8b2cb4ab3 100644 --- a/drivers/gpu/drm/drm_cgroup.c +++ b/drivers/gpu/drm/drm_cgroup.c @@ -259,3 +259,36 @@ void drm_pid_signal_budget(struct pid *pid, u64 usage, u64 budget) rcu_read_unlock(); } EXPORT_SYMBOL_GPL(drm_pid_signal_budget); + +enum drm_cg_supported drm_pid_signal_budget_supported(struct pid *pid) +{ + enum drm_cg_supported supported = DRM_CG_NOT_APPLICABLE; + struct drm_pid_clients *clients; + + rcu_read_lock(); + clients = xa_load(&drm_pid_clients, (unsigned long)pid); + if (clients) { + struct drm_file *fpriv; + + list_for_each_entry_rcu(fpriv, &clients->file_list, clink) { + const struct drm_cgroup_ops *cg_ops = + fpriv->minor->dev->driver->cg_ops; + + if (!cg_ops || + !cg_ops->active_time_us || + !cg_ops->signal_budget || + cg_ops->signal_budget(fpriv, 0, 0) < 0) { + supported = DRM_CG_NOT_SUPPORTED; + break; + } + + if (supported == DRM_CG_NOT_APPLICABLE) + supported = DRM_CG_SUPPORTED; + + } + } + rcu_read_unlock(); + + return supported; +} +EXPORT_SYMBOL_GPL(drm_pid_signal_budget_supported); diff --git a/include/drm/drm_clients.h b/include/drm/drm_clients.h index 7ad09fd0a404..5d14ae26ece6 100644 --- a/include/drm/drm_clients.h +++ b/include/drm/drm_clients.h @@ -14,6 +14,12 @@ struct drm_pid_clients { struct rcu_head rcu; }; +enum drm_cg_supported { + DRM_CG_NOT_APPLICABLE = -1, + DRM_CG_NOT_SUPPORTED = 0, + DRM_CG_SUPPORTED +}; + #if IS_ENABLED(CONFIG_CGROUP_DRM) void drm_clients_close(struct drm_file *file_priv); int drm_clients_open(struct drm_file *file_priv); @@ -39,5 +45,6 @@ unsigned int drm_pid_priority_levels(struct pid *pid, bool *non_uniform); void drm_pid_update_priority(struct pid *pid, int priority); u64 drm_pid_get_active_time_us(struct pid *pid); void drm_pid_signal_budget(struct pid *pid, u64 usage, u64 budget); +enum drm_cg_supported drm_pid_signal_budget_supported(struct pid *pid); #endif diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h index 07dec956ebfb..7a1a20d1b8de 100644 --- a/include/drm/drm_drv.h +++ b/include/drm/drm_drv.h @@ -198,8 +198,17 @@ struct drm_cgroup_ops { * * Optional callback used by the DRM core to forward over/under GPU time * messages sent by the DRM cgroup controller. + * + * Zero used with zero budget is a special budgeting support status + * query which needs to return either zero or -EINVAL if client does not + * support budget control. + * + * Returns: + * * 1 when client has been throttled. + * * 0 when no action has been taken. + * * -EINVAL when not supported by the client. */ - void (*signal_budget) (struct drm_file *, u64 used, u64 budget); + int (*signal_budget) (struct drm_file *, u64 used, u64 budget); }; /** diff --git a/kernel/cgroup/drm.c b/kernel/cgroup/drm.c index af50ead1564a..dd7db70c2831 100644 --- a/kernel/cgroup/drm.c +++ b/kernel/cgroup/drm.c @@ -239,6 +239,53 @@ drmcs_write_weight(struct cgroup_subsys_state *css, struct cftype *cftype, return 0; } +static int drmcs_show_budget_supported(struct seq_file *sf, void *v) +{ + struct drm_cgroup_state *drmcs = css_to_drmcs(seq_css(sf)); + enum drm_cg_supported overall = DRM_CG_NOT_APPLICABLE; + struct cgroup *cgrp = drmcs->css.cgroup; + struct task_struct *task; + struct css_task_iter it; + + css_task_iter_start(&cgrp->self, + CSS_TASK_ITER_PROCS | CSS_TASK_ITER_THREADED, + &it); + while ((task = css_task_iter_next(&it))) { + enum drm_cg_supported supported; + + /* Ignore kernel threads here. */ + if (task->flags & PF_KTHREAD) + continue; + + supported = drm_pid_signal_budget_supported(task_pid(task)); + if (supported == DRM_CG_SUPPORTED && + overall == DRM_CG_NOT_APPLICABLE) { + overall = DRM_CG_SUPPORTED; + } else if (supported == DRM_CG_NOT_SUPPORTED) { + overall = DRM_CG_NOT_SUPPORTED; + break; + } + } + css_task_iter_end(&it); + + switch (overall) { + case DRM_CG_NOT_APPLICABLE: + seq_puts(sf, "n/a\n"); + break; + case DRM_CG_NOT_SUPPORTED: + seq_puts(sf, "no\n"); + break; + case DRM_CG_SUPPORTED: + seq_puts(sf, "yes\n"); + break; + default: + seq_printf(sf, "%u\n", overall); + break; + } + + return 0; +} + static int drmcs_online(struct cgroup_subsys_state *css) { struct drm_cgroup_state *drmcs = css_to_drmcs(css); @@ -690,6 +737,11 @@ struct cftype files[] = { .read_u64 = drmcs_read_weight, .write_u64 = drmcs_write_weight, }, + { + .name = "budget_supported", + .flags = CFTYPE_NOT_ON_ROOT, + .seq_show = drmcs_show_budget_supported, + }, { .name = "period_us", .flags = CFTYPE_NOT_ON_ROOT, From patchwork Wed Oct 19 17:32:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 5793 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp452787wrs; Wed, 19 Oct 2022 10:39:52 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4wYnbIwPqUCvqGPUIgmjvyxOeUcJj+dQ51fEDT9lHp/Ltx2BcjAQYMqErTThybj4RNHF8J X-Received: by 2002:a05:6402:51d0:b0:45d:5ba4:c3d2 with SMTP id r16-20020a05640251d000b0045d5ba4c3d2mr8613335edd.132.1666201191843; Wed, 19 Oct 2022 10:39:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666201191; cv=none; d=google.com; s=arc-20160816; b=STYlht8HuVqASIjuYpl8ne6dGOTJlXC13SDZdWH6FddyfBeLegEPDb+7bRIuNlyRdj SD9p7XNVZcVCEX/M97jEaRt9JrUJD5GxSudjOY/6/tq0foIiLD4KyiHHKi0wQsT4SCBq yZ0DGMyC+Ln3D21XKCD+yA0vcCIlx+LHUbc7wo9aBQVEk8R7UP+2FWITvt0PZ2X92F6Z rz3x9aRTPBbedmToY6kBAXH2wgfyfFgPaCDYq8YOOBeF29+uBuaPFVDHSMPhWJvG0Eef mhA0jqBp16yCXFPmckQij6YdiJAJ1hAoY9yHcrZwsFgOoJ43ahLY11dphEsWg1rmF3Cl MJdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=kbfACSa5mB7jyETfg2PTHiogEqnNj2Y9dHr5Ki3RBHA=; b=svP+NcPWyLdgCL2TdpP2gel7p4sN9wpsbmGxQuo7LYWYj3Z4b0IZ7koyzpkaLxDO6a NlNQ+YJ7t1aM8uwN9ScooadukTLn1DvGnLqGf02ADhnKl4J2k8kK5b9mMxvggCDDLiuN P5APzCnX+QuMQvNwzdWmqlsRbz62CCqW9fTUpTnil1ehQX7zFfAPzcLpnazXdLXIpgUj Wjsxor8/9ppRMpjOtiQO9f10MfoXqp1ibI6dzEi0m+tM9Hxr/+nPRTNACNupMvHXuSBC L3OFBSAREErqjDhzOkIW7o+k1Sn8X1A+iaMqeoCUOSelnuqgeRXta1yzlYkg/BobKjr5 JTXA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=cF7UvkPo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id hg7-20020a1709072cc700b0078e1216fb07si2625563ejc.558.2022.10.19.10.39.26; Wed, 19 Oct 2022 10:39:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=cF7UvkPo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229649AbiJSRfN (ORCPT + 99 others); Wed, 19 Oct 2022 13:35:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33876 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231526AbiJSReS (ORCPT ); Wed, 19 Oct 2022 13:34:18 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7B43F1C7107; Wed, 19 Oct 2022 10:34:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666200857; x=1697736857; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=U7LArZuNthshDvVK5CBxK7+jHExXUGsLdkjMMWrwBEw=; b=cF7UvkPoaPZs1sLj5+Z8tOqRHRkel9ouohTALHiU9QFb7AfUk3Trfd2K LPx276iMsuVEzA3HiOWz8ox2iSd8Gx4/tnh2u+JQKpL8q3Bit1ntKxibz b2rNz9seQ8ji1VVO9B1anafszowRZeciXIXUxA/czHj7xT2yM9wP4UFbj 6n2gAosJ+GAFzVf8t/hy/5LZV1YbzHBh8M+ZDU9fdR3jP+UP69Pq/hIeH ulKAQDHdou9ToBWFrXrlVgaVReQR+1CrRK627MiIZWcJDaDUb+Dmov7Qn rKxjhD9eeq2t2N1XN0WetSrYEINJSczM9wlyoQdChqAab8dSkpOiBRcom g==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="306474591" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="306474591" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:34:10 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="607204910" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="607204910" Received: from mjmcener-mobl1.amr.corp.intel.com (HELO localhost.localdomain) ([10.213.233.40]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:34:07 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 15/17] drm/i915: Migrate client to new owner on context create Date: Wed, 19 Oct 2022 18:32:52 +0100 Message-Id: <20221019173254.3361334-16-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> References: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747138581012972281?= X-GMAIL-MSGID: =?utf-8?q?1747138581012972281?= From: Tvrtko Ursulin Some usage models pass a drm file descriptor from a creating process to the client which will actually use it. Use the core drm helper on GEM context create to account for this and ensure client's resource usage is tracked in the correct cgroup. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/i915/gem/i915_gem_context.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c index 1e29b1e6d186..5f6af306e147 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c @@ -69,6 +69,7 @@ #include #include +#include #include #include "gt/gen6_ppgtt.h" @@ -2300,6 +2301,8 @@ int i915_gem_context_create_ioctl(struct drm_device *dev, void *data, args->ctx_id = id; drm_dbg(&i915->drm, "HW context %d created\n", args->ctx_id); + drm_clients_migrate(file); + return 0; err_pc: From patchwork Wed Oct 19 17:32:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 5784 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp452079wrs; Wed, 19 Oct 2022 10:38:10 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6HnLZkuu+ZQ8YWaSp6T7L1hpHbnGiiAPvw34SVImp9cfroW9mIU9K/PYV5z0kXBlWfxQ3m X-Received: by 2002:a05:6402:4310:b0:45c:c16c:5c7d with SMTP id m16-20020a056402431000b0045cc16c5c7dmr8513778edc.246.1666201090808; Wed, 19 Oct 2022 10:38:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666201090; cv=none; d=google.com; s=arc-20160816; b=SCm0j/13cpg3BiZVjSiqzwkJubsIlOfR2P8DwRVCdTrrOLd01qHlSBVgrT6ESkTSjC ffj4ZbCqgqRxjIr0dGzXAS9SfhFYiLqOGnKP6uNh5/S5KduJUS/aYv3dg3V+3zBrmpWj eb2RUlidtGdWpIYyjSUZ0QAsdsHjptPKAon255GSrgI1L9YdB6I9POhzBXuuTpI56tkJ vcgDLXOQuCvCy+vd6T0D1x8FbRJp1mJJ1cV7r08bh1z1M4QMsCMVHdcCxcE5HLhLcJdc KQFioriz2ntQlVrEZ+jiacLQTINMgvC3PAuZPIzbXeQ+Xo2U9rdrkOUwYoxzApLj5MD4 YuvA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=LpSFiWlAZPw3ChfC3mEVFADkKypFgkC0zLN+S1Pe8r8=; b=dTjgjAvdrqN3mXoW475Nd5IoNBcczwcyzQMTX6aJEicw3ne13I+3SO513b5iwRBL4d 2e4XVq+wmC2ajsjdjdFZDzAvgJz/ndwU3g1l2U+ShUqHJ2yzj5mTXhDJ7iahXeIfOTf+ eAf1leCbbNOqcL6V64D3CBbmGsDNZRduAiKPEsZoHh0x4ZAjq1NQG8biz0WvLAq1ApAL kXDwqhUvFhrci90JFBYkA7euvQoBCTx3Yv/M7ySiiK8Pyk7vrxIW7aVSBO7gJ53aZCXP q+f+28SPlTL+SYkzD1ATKMRuNiZr2MDsJhQ5M31jSZszNJh93LjsS4rvbv4gRXgRh3XY PoZA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=K8Qm5BGD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f5-20020a50a6c5000000b00459528ef81bsi13876935edc.324.2022.10.19.10.37.46; Wed, 19 Oct 2022 10:38:10 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=K8Qm5BGD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230322AbiJSRfS (ORCPT + 99 others); Wed, 19 Oct 2022 13:35:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33886 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231535AbiJSReU (ORCPT ); Wed, 19 Oct 2022 13:34:20 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D94851C25F1; Wed, 19 Oct 2022 10:34:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666200858; x=1697736858; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=oo5u0fET53D72nvRAabovnCgKyLAZjX42LHHrPrrjv8=; b=K8Qm5BGDs9mFL6w5gMvb5T/N5v+NwR+fhsZKoIWRO0XxP8a0obb9A3TU KLyt+RdvpyHHUOKBPa9oq0TLvRRWv/uq+4UrfoYLnO5nPMr4ud6dT1XTX eQflU6/b/7fRjzR7xn0g3jNVrwHUC9+V0Xxc78+FPnwaStt+Q6KPeVh5M MyRwUegJ+UxoPH5i3qhcsAQBUZxvRwxdu2aHW9D5yDt2ziuBqbjKf/AMs KGGkVjI1SxMl83hdRPuyHq4pjfDlHF6mj+vwfCZyRZbiG+iRiMITWLeBp JLJrZw8D2C97jnPzZbnanGvNUOQ8qv2yM2SpevEnoyA6IzBaLf9iBPS0p A==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="306474610" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="306474610" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:34:13 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="607204926" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="607204926" Received: from mjmcener-mobl1.amr.corp.intel.com (HELO localhost.localdomain) ([10.213.233.40]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:34:10 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 16/17] drm/i915: Wire up with drm controller GPU time query Date: Wed, 19 Oct 2022 18:32:53 +0100 Message-Id: <20221019173254.3361334-17-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> References: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747138475116041228?= X-GMAIL-MSGID: =?utf-8?q?1747138475116041228?= From: Tvrtko Ursulin Implement the drm_cgroup_ops->active_time_us callback. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/i915/i915_driver.c | 1 + drivers/gpu/drm/i915/i915_drm_client.c | 106 +++++++++++++++++++------ drivers/gpu/drm/i915/i915_drm_client.h | 2 + 3 files changed, 83 insertions(+), 26 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_driver.c b/drivers/gpu/drm/i915/i915_driver.c index 7912782b87cc..b949fd715202 100644 --- a/drivers/gpu/drm/i915/i915_driver.c +++ b/drivers/gpu/drm/i915/i915_driver.c @@ -1896,6 +1896,7 @@ static const struct drm_ioctl_desc i915_ioctls[] = { #ifdef CONFIG_CGROUP_DRM static const struct drm_cgroup_ops i915_drm_cgroup_ops = { .priority_levels = i915_drm_priority_levels, + .active_time_us = i915_drm_cgroup_get_active_time_us, }; #endif diff --git a/drivers/gpu/drm/i915/i915_drm_client.c b/drivers/gpu/drm/i915/i915_drm_client.c index 61a3cdaa7b16..8527fe80d449 100644 --- a/drivers/gpu/drm/i915/i915_drm_client.c +++ b/drivers/gpu/drm/i915/i915_drm_client.c @@ -75,23 +75,7 @@ void i915_drm_clients_fini(struct i915_drm_clients *clients) xa_destroy(&clients->xarray); } -#ifdef CONFIG_CGROUP_DRM -unsigned int i915_drm_priority_levels(struct drm_file *file) -{ - struct drm_i915_file_private *fpriv = file->driver_priv; - struct i915_drm_client *client = fpriv->client; - struct drm_i915_private *i915 = client->clients->i915; - - if (GRAPHICS_VER(i915) < 8) - return 0; - else if (intel_uc_uses_guc_submission(&to_gt(i915)->uc)) - return 3; - else - return 2047; -} -#endif - -#ifdef CONFIG_PROC_FS +#if defined(CONFIG_PROC_FS) || defined(CONFIG_CGROUP_DRM) static const char * const uabi_class_names[] = { [I915_ENGINE_CLASS_RENDER] = "render", [I915_ENGINE_CLASS_COPY] = "copy", @@ -116,22 +100,92 @@ static u64 busy_add(struct i915_gem_context *ctx, unsigned int class) return total; } -static void -show_client_class(struct seq_file *m, - struct i915_drm_client *client, - unsigned int class) +static u64 get_class_active_ns(struct i915_drm_client *client, + unsigned int class, + unsigned int *capacity) { - const struct list_head *list = &client->ctx_list; - u64 total = atomic64_read(&client->past_runtime[class]); - const unsigned int capacity = - client->clients->i915->engine_uabi_class_count[class]; struct i915_gem_context *ctx; + u64 total; + + *capacity = + client->clients->i915->engine_uabi_class_count[class]; + if (!*capacity) + return 0; + + total = atomic64_read(&client->past_runtime[class]); rcu_read_lock(); - list_for_each_entry_rcu(ctx, list, client_link) + list_for_each_entry_rcu(ctx, &client->ctx_list, client_link) total += busy_add(ctx, class); rcu_read_unlock(); + return total; +} +#endif + +#ifdef CONFIG_CGROUP_DRM +unsigned int i915_drm_priority_levels(struct drm_file *file) +{ + struct drm_i915_file_private *fpriv = file->driver_priv; + struct i915_drm_client *client = fpriv->client; + struct drm_i915_private *i915 = client->clients->i915; + + if (GRAPHICS_VER(i915) < 8) + return 0; + else if (intel_uc_uses_guc_submission(&to_gt(i915)->uc)) + return 3; + else + return 2047; +} + +static bool supports_stats(struct drm_i915_private *i915) +{ + if (GRAPHICS_VER(i915) < 8) + return false; + + /* temporary... */ + if (intel_uc_uses_guc_submission(&to_gt(i915)->uc)) + return false; + + return true; +} + +u64 i915_drm_cgroup_get_active_time_us(struct drm_file *file) +{ + struct drm_i915_file_private *fpriv = file->driver_priv; + struct i915_drm_client *client = fpriv->client; + unsigned int i; + u64 busy = 0; + + if (!supports_stats(client->clients->i915)) + return 0; + + for (i = 0; i < ARRAY_SIZE(uabi_class_names); i++) { + unsigned int capacity; + u64 b; + + b = get_class_active_ns(client, i, &capacity); + if (capacity) { + b = DIV_ROUND_UP_ULL(b, capacity * 1000); + busy += b; + } + } + + return busy; +} +#endif + +#ifdef CONFIG_PROC_FS +static void +show_client_class(struct seq_file *m, + struct i915_drm_client *client, + unsigned int class) +{ + unsigned int capacity; + u64 total; + + total = get_class_active_ns(client, class, &capacity); + if (capacity) seq_printf(m, "drm-engine-%s:\t%llu ns\n", uabi_class_names[class], total); diff --git a/drivers/gpu/drm/i915/i915_drm_client.h b/drivers/gpu/drm/i915/i915_drm_client.h index bd5925241007..99b8ae01c183 100644 --- a/drivers/gpu/drm/i915/i915_drm_client.h +++ b/drivers/gpu/drm/i915/i915_drm_client.h @@ -69,4 +69,6 @@ void i915_drm_clients_fini(struct i915_drm_clients *clients); unsigned int i915_drm_priority_levels(struct drm_file *file); +u64 i915_drm_cgroup_get_active_time_us(struct drm_file *file); + #endif /* !__I915_DRM_CLIENT_H__ */ From patchwork Wed Oct 19 17:32:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 5791 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp452396wrs; Wed, 19 Oct 2022 10:38:54 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7pshKeVVpj26512VriNiT1ArNVNMKagufK3MWVyJ1Kc1O+ij49c/mq2+MiCB7demO5xWEO X-Received: by 2002:a05:6402:33c5:b0:447:e4a3:c930 with SMTP id a5-20020a05640233c500b00447e4a3c930mr8486339edc.401.1666201134406; Wed, 19 Oct 2022 10:38:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666201134; cv=none; d=google.com; s=arc-20160816; b=LcTdtGE5TaHqJmNVkQHwUZdvOJhXc6X+ZtWc0LsIWim4fFVF5hbXRSBLO6cTNI22vt ZdU9YfHJJVzR2Yv1e4+74lMkdnQGXhNLsgWzpOnhwJZzSTWOz9CFcZUBy6wMOZQ2lgod Uae8hQhLK/sWXsnYmwyTku4jOeiL2+nD68IhiyqRdOzffoqe9aA0dQTQK38vT+nQyXJy TU7mFWqgBDJRjurSidbF0g3KGICiZW/ulSGC0b4dVeWHTRMoN3upN/q9LDoSIU1Ev6Eu yGSxFmmc1+kEJK0XMjDV68KntgIqVmcgkVRyIsIuDA3L1CmalonG/aKnfUf6UGkHa7bR 19gg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=XU+seLaDJbZ3w7VBvNkBIoCemgQDwkIbdjbkoCxxGIQ=; b=sGZkhNkRqgQ8Bix99x5hV58Xtzp0k4HX7KdyrRWDQj55E2nUtOqCy949gbzX9e8qJ0 vFA9e0b3STlKlDBXxdsvhKZ4acNZ+aqf0alSOfuRWsMvylPhwrXv76+FFGZvvZHEAnp0 Adcd0KMJZxQOb8MMQbijvt2EgATobLWCm98kvWYzAKIghcuNulTSdDc9lnXQnJzgh6yq lrX6fi4CfNdZM2zX1THbNJ5JoFCDA6vYEVo5LeLoJY2HhxQOEKreRvFcaoUcSL4zzfCV 9MHSNX8oVT5A9bkKputSEPtnRzpyWmhtJ2ek76/WarxQSqJ5hiXlyxLsP2dbxcXaEMdP 7BNw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Y7d7E7OD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q16-20020a056402519000b004584decbde1si17946515edd.369.2022.10.19.10.38.29; Wed, 19 Oct 2022 10:38:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Y7d7E7OD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231477AbiJSRfb (ORCPT + 99 others); Wed, 19 Oct 2022 13:35:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231537AbiJSReU (ORCPT ); Wed, 19 Oct 2022 13:34:20 -0400 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE11C1C2EA7; Wed, 19 Oct 2022 10:34:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666200858; x=1697736858; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7rJx5gTjnj7urPJTsmnnJY+QejvZumuwu5CnaP5NC0E=; b=Y7d7E7ODuMUzNET7qnn3bkuPgecdQqJrvkzQTET3qid15Hqa2XqVkioZ /jUljU9nYzzZYrS1HdcVV/OQ+uw5VMTk3BgRvaPjfR23g3xNibMtUzZp0 vavvrfbzA/7mcvHhXJOX2ZlEmVuR3lNbo326JiRnZ8h229VIRQZ2iPK7S IO+seC1s3rahEM1/7CMaLmS60na7QvZGtMuk9IciIQFZoPAerNtL6dP1n +2r7LtMBiRoDX8jJYNNdD0OuM3BUXkzcxfKxkxYx2Wc4l7KKmXYuBdUBI iYXRwvxjRUpD++21MyLn6i3Mq+CZEOBvhpheAgYpJww2y4Uab+idKoAUp w==; X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="306474634" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="306474634" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:34:16 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10505"; a="607204941" X-IronPort-AV: E=Sophos;i="5.95,196,1661842800"; d="scan'208";a="607204941" Received: from mjmcener-mobl1.amr.corp.intel.com (HELO localhost.localdomain) ([10.213.233.40]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Oct 2022 10:34:13 -0700 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 17/17] drm/i915: Implement cgroup controller over budget throttling Date: Wed, 19 Oct 2022 18:32:54 +0100 Message-Id: <20221019173254.3361334-18-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> References: <20221019173254.3361334-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747138520713798966?= X-GMAIL-MSGID: =?utf-8?q?1747138520713798966?= From: Tvrtko Ursulin When notified by the drm core we are over our allotted time budget, i915 instance will check if any of the GPU engines it is reponsible for is fully saturated. If it is, and the client in question is using that engine, it will throttle it. For now throttling is done simplistically by lowering the scheduling priority while client is throttled. Signed-off-by: Tvrtko Ursulin --- .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 22 ++++- drivers/gpu/drm/i915/i915_driver.c | 1 + drivers/gpu/drm/i915/i915_drm_client.c | 93 +++++++++++++++++++ drivers/gpu/drm/i915/i915_drm_client.h | 9 ++ 4 files changed, 123 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c index 391c5b5c80be..efcbd827f6a0 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include @@ -3018,15 +3019,32 @@ static void retire_requests(struct intel_timeline *tl, struct i915_request *end) } #ifdef CONFIG_CGROUP_DRM +static unsigned int +__get_class(struct drm_i915_file_private *fpriv, const struct i915_request *rq) +{ + unsigned int class; + + class = rq->context->engine->uabi_class; + + if (WARN_ON_ONCE(class >= ARRAY_SIZE(fpriv->client->throttle))) + class = 0; + + return class; +} + static void copy_priority(struct i915_sched_attr *attr, - const struct i915_execbuffer *eb) + const struct i915_execbuffer *eb, + const struct i915_request *rq) { + struct drm_i915_file_private *file_priv = eb->file->driver_priv; const int scale = DIV_ROUND_CLOSEST(DRM_CGROUP_PRIORITY_MAX, I915_CONTEXT_MAX_USER_PRIORITY); int prio; *attr = eb->gem_context->sched; prio = attr->priority * scale + eb->file->drm_cgroup_priority; + if (file_priv->client->throttle[__get_class(file_priv, rq)]) + prio -= 1 + prandom_u32_max(-DRM_CGROUP_PRIORITY_MIN / 2); prio = DIV_ROUND_UP(prio, scale); attr->priority = clamp(prio, I915_CONTEXT_MIN_USER_PRIORITY, @@ -3056,7 +3074,7 @@ static int eb_request_add(struct i915_execbuffer *eb, struct i915_request *rq, /* Check that the context wasn't destroyed before submission */ if (likely(!intel_context_is_closed(eb->context))) { - copy_priority(&attr, eb); + copy_priority(&attr, eb, rq); } else { /* Serialise with context_close via the add_to_timeline */ i915_request_set_error_once(rq, -ENOENT); diff --git a/drivers/gpu/drm/i915/i915_driver.c b/drivers/gpu/drm/i915/i915_driver.c index b949fd715202..abac9bb5bf27 100644 --- a/drivers/gpu/drm/i915/i915_driver.c +++ b/drivers/gpu/drm/i915/i915_driver.c @@ -1897,6 +1897,7 @@ static const struct drm_ioctl_desc i915_ioctls[] = { static const struct drm_cgroup_ops i915_drm_cgroup_ops = { .priority_levels = i915_drm_priority_levels, .active_time_us = i915_drm_cgroup_get_active_time_us, + .signal_budget = i915_drm_cgroup_signal_budget, }; #endif diff --git a/drivers/gpu/drm/i915/i915_drm_client.c b/drivers/gpu/drm/i915/i915_drm_client.c index 8527fe80d449..ce497055cc3f 100644 --- a/drivers/gpu/drm/i915/i915_drm_client.c +++ b/drivers/gpu/drm/i915/i915_drm_client.c @@ -4,6 +4,7 @@ */ #include +#include #include #include @@ -173,6 +174,98 @@ u64 i915_drm_cgroup_get_active_time_us(struct drm_file *file) return busy; } + +int i915_drm_cgroup_signal_budget(struct drm_file *file, u64 usage, u64 budget) +{ + struct drm_i915_file_private *fpriv = file->driver_priv; + u64 class_usage[I915_LAST_UABI_ENGINE_CLASS + 1]; + u64 class_last[I915_LAST_UABI_ENGINE_CLASS + 1]; + struct drm_i915_private *i915 = fpriv->dev_priv; + struct i915_drm_client *client = fpriv->client; + struct intel_engine_cs *engine; + bool over = usage > budget; + unsigned int i; + ktime_t unused; + int ret = 0; + u64 t; + + if (!supports_stats(i915)) + return -EINVAL; + + if (usage == 0 && budget == 0) + return 0; + +printk("i915_drm_cgroup_signal_budget client-id=%u over=%u (%llu/%llu) <%u>\n", + client->id, over, usage, budget, client->over_budget); + + if (over) { + client->over_budget++; + if (!client->over_budget) + client->over_budget = 2; + } else { + client->over_budget = 0; + memset(client->class_last, 0, sizeof(client->class_last)); + memset(client->throttle, 0, sizeof(client->throttle)); + return 0; + } + + memset(class_usage, 0, sizeof(class_usage)); + for_each_uabi_engine(engine, i915) + class_usage[engine->uabi_class] += + ktime_to_ns(intel_engine_get_busy_time(engine, &unused)); + + memcpy(class_last, client->class_last, sizeof(class_last)); + memcpy(client->class_last, class_usage, sizeof(class_last)); + + for (i = 0; i < ARRAY_SIZE(uabi_class_names); i++) + class_usage[i] -= class_last[i]; + + t = client->last; + client->last = ktime_get_raw_ns(); + t = client->last - t; + + if (client->over_budget == 1) + return 0; + + for (i = 0; i < ARRAY_SIZE(uabi_class_names); i++) { + u64 client_class_usage[I915_LAST_UABI_ENGINE_CLASS + 1]; + unsigned int capacity; + + if (!i915->engine_uabi_class_count[i]) + continue; + + t = DIV_ROUND_UP_ULL(t, 1000); + class_usage[i] = DIV_ROUND_CLOSEST_ULL(class_usage[i], 1000); + usage = DIV_ROUND_CLOSEST_ULL(class_usage[i] * 100ULL, + t * + i915->engine_uabi_class_count[i]); + if (usage <= 95) { + /* class not oversubsribed */ + if (client->throttle[i]) { + client->throttle[i] = false; +printk(" UN-throttling class%u (phys=%lld%%)\n", + i, usage); + } + continue; + } + + client_class_usage[i] = + get_class_active_ns(client, i, &capacity); + + if (client_class_usage[i] && !client->throttle[i]) { + ret |= 1; + client->throttle[i] = true; + /* + * QQQ maybe apply "strength" of throttling based on + * usage/budget? + */ +printk(" THROTTLING class%u (phys=%lld%% client=%lluus)\n", + i, usage, client_class_usage[i] / 1000); + } + } + + return ret; +} #endif #ifdef CONFIG_PROC_FS diff --git a/drivers/gpu/drm/i915/i915_drm_client.h b/drivers/gpu/drm/i915/i915_drm_client.h index 99b8ae01c183..b05afe01e68e 100644 --- a/drivers/gpu/drm/i915/i915_drm_client.h +++ b/drivers/gpu/drm/i915/i915_drm_client.h @@ -40,6 +40,13 @@ struct i915_drm_client { * @past_runtime: Accumulation of pphwsp runtimes from closed contexts. */ atomic64_t past_runtime[I915_LAST_UABI_ENGINE_CLASS + 1]; + +#ifdef CONFIG_CGROUP_DRM + bool throttle[I915_LAST_UABI_ENGINE_CLASS + 1]; + unsigned int over_budget; + u64 last; + u64 class_last[I915_LAST_UABI_ENGINE_CLASS + 1]; +#endif }; void i915_drm_clients_init(struct i915_drm_clients *clients, @@ -70,5 +77,7 @@ void i915_drm_clients_fini(struct i915_drm_clients *clients); unsigned int i915_drm_priority_levels(struct drm_file *file); u64 i915_drm_cgroup_get_active_time_us(struct drm_file *file); +int i915_drm_cgroup_signal_budget(struct drm_file *file, + u64 usage, u64 budget); #endif /* !__I915_DRM_CLIENT_H__ */