From patchwork Wed Nov 9 16:11:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 17652 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp424315wru; Wed, 9 Nov 2022 08:13:23 -0800 (PST) X-Google-Smtp-Source: AMsMyM5fvUxYW2owRbY5rezU0lE+a5NTYR766aUozlt7MUb9repycYQy6A0gVjsV/x7EkBof3GEu X-Received: by 2002:a50:fd81:0:b0:458:7489:34ea with SMTP id o1-20020a50fd81000000b00458748934eamr1215572edt.264.1668010403244; Wed, 09 Nov 2022 08:13:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668010403; cv=none; d=google.com; s=arc-20160816; b=lkLzezB6GeowDoVMYec7QaL/G3KL+NNplUkgVrBJ40ygIxMEaKaEzAFzH0LZEGyo4V nNQlwSoteQ8J7PmsT8qQAYD1R0jq9njgN9yDfo5XEBm6Ubvc8/97yG5o9n2iklij9+NG f5wYgwN5m746UQTKtl9WuPvUFz90mHij48iDQKejG1jIERlLUiHrgUrApOoFSwy1RwIx WifoGluaa7oa3jgREWZjLpAas9HRExAYsr8Yg9dvXIlhNF7vNoN39NctuuFPPRICsjLC qZ7KeksQ2BY+eArqH7LHhb57sISqNDDbst0F7yDjWs/xohA0ss3o2ZF6X391/qoJBg/D +ROQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=i+jUM9bFwZKtlbx25/UbI/tIcGDCzeaWR79IOZKA6tY=; b=J7dsP7KrAWN9yj6xs77Ef47rPU5Nyl0fTWyDjoys2hxiN8WrS/2exM3Xx+tlHhuxm9 xK1bFqRwbv3aoZVDBHClQnvPr8p5rm76up2ho4YSvCRdpt9GrdDg7SZNYkBtnosyD4ib Wbg5eDNcb6pUPrzpUSPRBd/eJjUmJkY6Xp09yLgT6kVIhoD38TD193Y6hOlLYbYxR6S2 sJ1lXrDlxRaA+l8p5y/zAcdp4uPxNZGAa8o7z+Nryp8ud3Jb8OEhL8LZD3ysd46f5Riu vrqGGfWYKAEeUG9/OGsTmjKn9AWZfSGfk+xHJ4cd/YDv1o4+/9WtXe6qvh/SLKkYebFe iptQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=GFPuXlHM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id sg33-20020a170907a42100b0078200f886bbsi5056734ejc.361.2022.11.09.08.12.57; Wed, 09 Nov 2022 08:13:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=GFPuXlHM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231494AbiKIQMF (ORCPT + 99 others); Wed, 9 Nov 2022 11:12:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58452 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230345AbiKIQMD (ORCPT ); Wed, 9 Nov 2022 11:12:03 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 87B6462CA; Wed, 9 Nov 2022 08:12:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668010322; x=1699546322; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=LhBrapaJ+vgi9kxPoWBSV/LQcGioxwEBpF/rBIIcwyQ=; b=GFPuXlHMibm5vpAndO/a82n3X3/y52qlHtAcSYd4+2LE2JLDTaIKdLjF BKpr3V9ASBRfBN2uo40aYYdeGx38lqI/DLO4rV0eUDoxGmJ4IzQZJObCJ sYxPK0yYe9M8Xz58Fs2bhcJa1pFbWe6F3WI6GexJhQAQB4DUByhFT4cgZ 1hN1rgX/CPVXutxFhZ55zCWhmUQxNCRW7uDWcj6mps707mbcDaDdswJW0 W/bAT5oyJqWYEFbR3nDJOZK2FkmZs0144oqLLmVPqTcnQLkXTR2BWJ9wu AagNfAEQ6p44fHUPuffXsgv17mtvCNkPUG8EcfIWCWPH1bl9nD51dyxSp Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="290735947" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="290735947" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:02 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="811684361" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="811684361" Received: from smurnane-mobl.ger.corp.intel.com (HELO localhost.localdomain) ([10.213.196.238]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:11:58 -0800 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 01/13] drm: Replace DRM_DEBUG with drm_dbg_core in file and ioctl handling Date: Wed, 9 Nov 2022 16:11:29 +0000 Message-Id: <20221109161141.2987173-2-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> References: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749035676642458294?= X-GMAIL-MSGID: =?utf-8?q?1749035676642458294?= From: Tvrtko Ursulin Replace the deprecated macro with the per-device one. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/drm_file.c | 21 +++++++++++---------- drivers/gpu/drm/drm_ioc32.c | 13 +++++++------ drivers/gpu/drm/drm_ioctl.c | 25 +++++++++++++------------ 3 files changed, 31 insertions(+), 28 deletions(-) diff --git a/drivers/gpu/drm/drm_file.c b/drivers/gpu/drm/drm_file.c index a8b4d918e9a3..ba5041137b29 100644 --- a/drivers/gpu/drm/drm_file.c +++ b/drivers/gpu/drm/drm_file.c @@ -245,10 +245,10 @@ void drm_file_free(struct drm_file *file) dev = file->minor->dev; - DRM_DEBUG("comm=\"%s\", pid=%d, dev=0x%lx, open_count=%d\n", - current->comm, task_pid_nr(current), - (long)old_encode_dev(file->minor->kdev->devt), - atomic_read(&dev->open_count)); + drm_dbg_core(dev, "comm=\"%s\", pid=%d, dev=0x%lx, open_count=%d\n", + current->comm, task_pid_nr(current), + (long)old_encode_dev(file->minor->kdev->devt), + atomic_read(&dev->open_count)); #ifdef CONFIG_DRM_LEGACY if (drm_core_check_feature(dev, DRIVER_LEGACY) && @@ -340,8 +340,8 @@ static int drm_open_helper(struct file *filp, struct drm_minor *minor) dev->switch_power_state != DRM_SWITCH_POWER_DYNAMIC_OFF) return -EINVAL; - DRM_DEBUG("comm=\"%s\", pid=%d, minor=%d\n", current->comm, - task_pid_nr(current), minor->index); + drm_dbg_core(dev, "comm=\"%s\", pid=%d, minor=%d\n", + current->comm, task_pid_nr(current), minor->index); priv = drm_file_alloc(minor); if (IS_ERR(priv)) @@ -450,11 +450,12 @@ EXPORT_SYMBOL(drm_open); void drm_lastclose(struct drm_device * dev) { - DRM_DEBUG("\n"); + drm_dbg_core(dev, "\n"); - if (dev->driver->lastclose) + if (dev->driver->lastclose) { dev->driver->lastclose(dev); - DRM_DEBUG("driver lastclose completed\n"); + drm_dbg_core(dev, "driver lastclose completed\n"); + } if (drm_core_check_feature(dev, DRIVER_LEGACY)) drm_legacy_dev_reinit(dev); @@ -485,7 +486,7 @@ int drm_release(struct inode *inode, struct file *filp) if (drm_dev_needs_global_mutex(dev)) mutex_lock(&drm_global_mutex); - DRM_DEBUG("open_count = %d\n", atomic_read(&dev->open_count)); + drm_dbg_core(dev, "open_count = %d\n", atomic_read(&dev->open_count)); drm_close_helper(filp); diff --git a/drivers/gpu/drm/drm_ioc32.c b/drivers/gpu/drm/drm_ioc32.c index 5d82891c3222..49a743f62b4a 100644 --- a/drivers/gpu/drm/drm_ioc32.c +++ b/drivers/gpu/drm/drm_ioc32.c @@ -972,6 +972,7 @@ long drm_compat_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) { unsigned int nr = DRM_IOCTL_NR(cmd); struct drm_file *file_priv = filp->private_data; + struct drm_device *dev = file_priv->minor->dev; drm_ioctl_compat_t *fn; int ret; @@ -986,14 +987,14 @@ long drm_compat_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) if (!fn) return drm_ioctl(filp, cmd, arg); - DRM_DEBUG("comm=\"%s\", pid=%d, dev=0x%lx, auth=%d, %s\n", - current->comm, task_pid_nr(current), - (long)old_encode_dev(file_priv->minor->kdev->devt), - file_priv->authenticated, - drm_compat_ioctls[nr].name); + drm_dbg_core(dev, "comm=\"%s\", pid=%d, dev=0x%lx, auth=%d, %s\n", + current->comm, task_pid_nr(current), + (long)old_encode_dev(file_priv->minor->kdev->devt), + file_priv->authenticated, + drm_compat_ioctls[nr].name); ret = (*fn)(filp, cmd, arg); if (ret) - DRM_DEBUG("ret = %d\n", ret); + drm_dbg_core(dev, "ret = %d\n", ret); return ret; } EXPORT_SYMBOL(drm_compat_ioctl); diff --git a/drivers/gpu/drm/drm_ioctl.c b/drivers/gpu/drm/drm_ioctl.c index ca2a6e6101dc..7c9d66ee917d 100644 --- a/drivers/gpu/drm/drm_ioctl.c +++ b/drivers/gpu/drm/drm_ioctl.c @@ -440,7 +440,7 @@ static int drm_setversion(struct drm_device *dev, void *data, struct drm_file *f int drm_noop(struct drm_device *dev, void *data, struct drm_file *file_priv) { - DRM_DEBUG("\n"); + drm_dbg_core(dev, "\n"); return 0; } EXPORT_SYMBOL(drm_noop); @@ -856,16 +856,16 @@ long drm_ioctl(struct file *filp, out_size = 0; ksize = max(max(in_size, out_size), drv_size); - DRM_DEBUG("comm=\"%s\" pid=%d, dev=0x%lx, auth=%d, %s\n", - current->comm, task_pid_nr(current), - (long)old_encode_dev(file_priv->minor->kdev->devt), - file_priv->authenticated, ioctl->name); + drm_dbg_core(dev, "comm=\"%s\" pid=%d, dev=0x%lx, auth=%d, %s\n", + current->comm, task_pid_nr(current), + (long)old_encode_dev(file_priv->minor->kdev->devt), + file_priv->authenticated, ioctl->name); /* Do not trust userspace, use our own definition */ func = ioctl->func; if (unlikely(!func)) { - DRM_DEBUG("no function\n"); + drm_dbg_core(dev, "no function\n"); retcode = -EINVAL; goto err_i1; } @@ -894,16 +894,17 @@ long drm_ioctl(struct file *filp, err_i1: if (!ioctl) - DRM_DEBUG("invalid ioctl: comm=\"%s\", pid=%d, dev=0x%lx, auth=%d, cmd=0x%02x, nr=0x%02x\n", - current->comm, task_pid_nr(current), - (long)old_encode_dev(file_priv->minor->kdev->devt), - file_priv->authenticated, cmd, nr); + drm_dbg_core(dev, + "invalid ioctl: comm=\"%s\", pid=%d, dev=0x%lx, auth=%d, cmd=0x%02x, nr=0x%02x\n", + current->comm, task_pid_nr(current), + (long)old_encode_dev(file_priv->minor->kdev->devt), + file_priv->authenticated, cmd, nr); if (kdata != stack_kdata) kfree(kdata); if (retcode) - DRM_DEBUG("comm=\"%s\", pid=%d, ret=%d\n", current->comm, - task_pid_nr(current), retcode); + drm_dbg_core(dev, "comm=\"%s\", pid=%d, ret=%d\n", + current->comm, task_pid_nr(current), retcode); return retcode; } EXPORT_SYMBOL(drm_ioctl); From patchwork Wed Nov 9 16:11:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 17653 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp424369wru; Wed, 9 Nov 2022 08:13:28 -0800 (PST) X-Google-Smtp-Source: AMsMyM5MwJDj+b3x3M3ZIJ0Ip61HlPPvFmTa3sD/ka9hS7zmlQAHL7/Qf/A/iUylubq8tfgvplFU X-Received: by 2002:a50:ec0a:0:b0:461:c0ca:a0bc with SMTP id g10-20020a50ec0a000000b00461c0caa0bcmr60824997edr.306.1668010408137; Wed, 09 Nov 2022 08:13:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668010408; cv=none; d=google.com; s=arc-20160816; b=HUCIg1IO4SirvlEGMKsDdg61grBWYguiiley7Nr6BGBLWq3NGmfUj1zaVs2Yx24bYr ZtU5WlFSMv0dPuFiO/7G8ldCRe2LCVwcjVc5EKS1WmHC0E4rPztVdcfbrjVmMkyWQ/pi cZ9ZyWBREJarEMpE1z1Ay+7UQlB9utg13CLIC2ENTg5jjF2IyYXwnCDkE+QKCSl5Ms9t md2GoVgCQR5GGjK+/aZ7pqgcAoeSZ7YIY0tAdU/ZpBATuHW9ep4TVHP4+Oh9YVpKhHF9 i2a0NPo6pKzaNaC23srohbG0WdEoDsctOuLnXEeB3aAc9CSkni8D4P8TtKq0srrSwF9C qyBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=qRk4p2HbEZZ1OcHa4nFMXHCL9FPdAHTsrM1YRCQsiCY=; b=AiplfXaKp/of2rFQ9diS/r+lZQdfqAuy2VvfxGy+ePLyHcIhkaiQAAn+jECOayDJpw eggP9eV0UoWb3JOj9cwGUgYUmNqr3FBpNImTccUsd4ZJCY2zsncEL5hhnezJITMY/nK0 7KDSNSKv6ZvEItdkGUTHYub4iqeiwZFQbrszz5M8QVCjIejc8nqt8bRDVlfG165NaNe+ Mv6DsZj8B3fSiZoYEVQLUdSd0OquOVhGdK1ri9ZPxaMECI34RR8AF4L0fZzBW6FSPMvA Qm1C3YCKMH99VNcUhV2midOyY7fQLWp01rWk76dMQe7tqB5r+/gQmP91yveAeXU/cvFU 8iGQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=J0P364hW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id qw13-20020a1709066a0d00b007ae75e99fffsi7943967ejc.487.2022.11.09.08.13.04; Wed, 09 Nov 2022 08:13:28 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=J0P364hW; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232054AbiKIQMV (ORCPT + 99 others); Wed, 9 Nov 2022 11:12:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58750 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231899AbiKIQMP (ORCPT ); Wed, 9 Nov 2022 11:12:15 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CFD8322B0C; Wed, 9 Nov 2022 08:12:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668010330; x=1699546330; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=65danGbq1zwJ9FB6InBTjiB49NsQltjw5lK60A9yBPk=; b=J0P364hWb86592KL1CS8EsMGK0mATqUa0Xg1FlNDgV9fbX1VhgB66VSE fnhK/6AWOg04EwCs5EtxDaghrImv7kfDOgTZQgdySJTdNGjuunbsrLqyv 36voFnS1RciyLsVcd20T4QgYnsmagpt+MSO/H864g92pgeOpsvzyljg8a f/wdeyeRxHtRWwxMoIywppisajGhlgSKOOd9BgsAUQio/THgQ0M81kvbW 6R1od1gXmr5+U/QvhuHB+DA2AVPj5sxsEr9AKg4hOk/0/hlVb/3mRor+l kmHLd8kSBQI4LMwfknGipNcCfabaiRogOx3jl/ev/MK6/2aR7H4hNdoUB A==; X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="290735959" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="290735959" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:09 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="811684393" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="811684393" Received: from smurnane-mobl.ger.corp.intel.com (HELO localhost.localdomain) ([10.213.196.238]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:02 -0800 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin , Zack Rusin , linux-graphics-maintainer@vmware.com, Alex Deucher Subject: [RFC 02/13] drm: Track clients by tgid and not tid Date: Wed, 9 Nov 2022 16:11:30 +0000 Message-Id: <20221109161141.2987173-3-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> References: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749035681731433726?= X-GMAIL-MSGID: =?utf-8?q?1749035681731433726?= From: Tvrtko Ursulin Thread group id (aka pid from userspace point of view) is a more interesting thing to show as an owner of a DRM fd, so track and show that instead of the thread id. In the next patch we will make the owner updated post file descriptor handover, which will also be tgid based to avoid ping-pong when multiple threads access the fd. Signed-off-by: Tvrtko Ursulin Cc: Zack Rusin Cc: linux-graphics-maintainer@vmware.com Cc: Alex Deucher Cc: "Christian König" --- drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 2 +- drivers/gpu/drm/drm_debugfs.c | 4 ++-- drivers/gpu/drm/drm_file.c | 2 +- drivers/gpu/drm/vmwgfx/vmwgfx_gem.c | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c index 8ef31d687ef3..4b940f8bd72b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c @@ -967,7 +967,7 @@ static int amdgpu_debugfs_gem_info_show(struct seq_file *m, void *unused) * Therefore, we need to protect this ->comm access using RCU. */ rcu_read_lock(); - task = pid_task(file->pid, PIDTYPE_PID); + task = pid_task(file->pid, PIDTYPE_TGID); seq_printf(m, "pid %8d command %s:\n", pid_nr(file->pid), task ? task->comm : ""); rcu_read_unlock(); diff --git a/drivers/gpu/drm/drm_debugfs.c b/drivers/gpu/drm/drm_debugfs.c index ee445f4605ba..42f657772025 100644 --- a/drivers/gpu/drm/drm_debugfs.c +++ b/drivers/gpu/drm/drm_debugfs.c @@ -80,7 +80,7 @@ static int drm_clients_info(struct seq_file *m, void *data) seq_printf(m, "%20s %5s %3s master a %5s %10s\n", "command", - "pid", + "tgid", "dev", "uid", "magic"); @@ -94,7 +94,7 @@ static int drm_clients_info(struct seq_file *m, void *data) bool is_current_master = drm_is_current_master(priv); rcu_read_lock(); /* locks pid_task()->comm */ - task = pid_task(priv->pid, PIDTYPE_PID); + task = pid_task(priv->pid, PIDTYPE_TGID); uid = task ? __task_cred(task)->euid : GLOBAL_ROOT_UID; seq_printf(m, "%20s %5d %3d %c %c %5d %10u\n", task ? task->comm : "", diff --git a/drivers/gpu/drm/drm_file.c b/drivers/gpu/drm/drm_file.c index ba5041137b29..5cde5014cea1 100644 --- a/drivers/gpu/drm/drm_file.c +++ b/drivers/gpu/drm/drm_file.c @@ -156,7 +156,7 @@ struct drm_file *drm_file_alloc(struct drm_minor *minor) if (!file) return ERR_PTR(-ENOMEM); - file->pid = get_pid(task_pid(current)); + file->pid = get_pid(task_tgid(current)); file->minor = minor; /* for compatibility root is always authenticated */ diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_gem.c b/drivers/gpu/drm/vmwgfx/vmwgfx_gem.c index ce609e7d758f..f2985337aa53 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_gem.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_gem.c @@ -260,7 +260,7 @@ static int vmw_debugfs_gem_info_show(struct seq_file *m, void *unused) * Therefore, we need to protect this ->comm access using RCU. */ rcu_read_lock(); - task = pid_task(file->pid, PIDTYPE_PID); + task = pid_task(file->pid, PIDTYPE_TGID); seq_printf(m, "pid %8d command %s:\n", pid_nr(file->pid), task ? task->comm : ""); rcu_read_unlock(); From patchwork Wed Nov 9 16:11:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 17664 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp437260wru; Wed, 9 Nov 2022 08:35:24 -0800 (PST) X-Google-Smtp-Source: AMsMyM6uLKh0b16NlLDgPQzy2gqvAQInNnHTwcuW8O0BDRK/WvXfaUppP9kG0PP3kxBX8NcI0K/w X-Received: by 2002:a17:906:7d8c:b0:7ae:159d:1146 with SMTP id v12-20020a1709067d8c00b007ae159d1146mr32840358ejo.528.1668011723969; Wed, 09 Nov 2022 08:35:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668011723; cv=none; d=google.com; s=arc-20160816; b=BZuFoDVbhZZ8eo9REAmVAJk7Ls89Rp3gMhv2l6nk5QaQ6zT4G/fgmiGW4Bxo2+GypB UfI6ht+PyEns7etZkaE9hTKqP/3eB9W4QG8t2zINZQTIJuaRIwqC9q30muqbmD6T50F6 QaixSFY6zVErGyj6zXXyZy+BQkeeZEXsTwtAYQj86JLctyLrXyHg69h8tgS7bhDMHhNM /UjmmRSuSMYgcpS5S/igfm5qPZnRoQblu6UtIRnJnG23cDI6UD59yqtcbPbcQrTYjR+V 6be0cL280p/okl3x3sLbgCEETTE6nuia4BDo3jwo5VPaZKBhpMTlgSGJVtNavff5hi2R aOGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=00Bgd2WL9bFw3nAUP7ePWGHB+4PRWXMQPyl9p64nluc=; b=Ap/sKZUaFjzgNFZjIeX6tBshOcsGx3EkNSEmpadRiLc3FYKLbNqth6IWLBI6J2mRYh 3ivK6dVkbzJMHHkiXSL4MSjz2G3o6uoDazd2f6Jon1JKwax8N+OqvSx7+CB1GgGh5yT7 WUdXeOThQfFgSOACsqwfOTLctPEf78f0NspSer5nS51P/4lN8Qfscjp3GSLrNLxBHGEo gtkt5Q+vuAz7pSrQKjL8aoPXjNpu+d0c92ObKQvQlvxFa7qWjnRFzUB+1rCpYz4+CNYX Dcu5RBT8LYu9Tef+WQdfH6FdogLu6wdLKXrDOtH0x/Q1wQLb0A57bYi9xULO5cx5m6LC /Oow== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=AF4toXuQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z18-20020a056402275200b00461fc07a835si18851838edd.18.2022.11.09.08.34.58; Wed, 09 Nov 2022 08:35:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=AF4toXuQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232238AbiKIQNj (ORCPT + 99 others); Wed, 9 Nov 2022 11:13:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59168 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232051AbiKIQMw (ORCPT ); Wed, 9 Nov 2022 11:12:52 -0500 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8443F22B35; Wed, 9 Nov 2022 08:12:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668010365; x=1699546365; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=408bXFKjUghNe6DBLZiEo8jtNRxH417j6r34+CGHn8g=; b=AF4toXuQM5GgjJUuIVp+4qpYRe5ydAmOM052cfofE9tIfisBEZ70Jzlw V4wc0Jo6BCrC+X2QLXAoJmAjZmRqeiGqIDdNhhTuG5nZsdpu5700ksDnd AjkY1c1BKL7CkIDbon7id58Bk5Vo7d+nnXIcpjskqrCUgBVchUUrkh3IB 9ipLZpVnT2gdt+09HjhYfFXx9Kag5z2rEQCKHDKiNnmokt9UfpPX+eP8F EIgrnmqr9xJ1A+QaYQOnQ51l6C1mWRNQratC9pPmO761qe8suacTL817o 6kQjBrx146R/bgCoM3Hpc8dQ/tqMqKNWTBwLBpgLtDAEKCF5kx+HWA8LI g==; X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="311015666" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="311015666" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:10 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="811684413" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="811684413" Received: from smurnane-mobl.ger.corp.intel.com (HELO localhost.localdomain) ([10.213.196.238]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:06 -0800 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 03/13] drm: Update file owner during use Date: Wed, 9 Nov 2022 16:11:31 +0000 Message-Id: <20221109161141.2987173-4-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> References: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-5.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749037061487193475?= X-GMAIL-MSGID: =?utf-8?q?1749037061487193475?= From: Tvrtko Ursulin With the typical model where the display server opends the file descriptor and then hands it over to the client we were showing stale data in debugfs. Fix it by updating the drm_file->pid on ioctl access from a different process. The field is also made RCU protected to allow for lockless readers. Update side is protected with dev->filelist_mutex. Signed-off-by: Tvrtko Ursulin Cc: "Christian König" --- drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 6 +++-- drivers/gpu/drm/drm_auth.c | 3 ++- drivers/gpu/drm/drm_debugfs.c | 10 +++++---- drivers/gpu/drm/drm_file.c | 30 +++++++++++++++++++++++++ drivers/gpu/drm/drm_ioctl.c | 3 +++ drivers/gpu/drm/nouveau/nouveau_drm.c | 5 ++++- drivers/gpu/drm/vmwgfx/vmwgfx_gem.c | 6 +++-- include/drm/drm_file.h | 13 +++++++++-- 8 files changed, 64 insertions(+), 12 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c index 4b940f8bd72b..d732ffb1c0d8 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c @@ -958,6 +958,7 @@ static int amdgpu_debugfs_gem_info_show(struct seq_file *m, void *unused) list_for_each_entry(file, &dev->filelist, lhead) { struct task_struct *task; struct drm_gem_object *gobj; + struct pid *pid; int id; /* @@ -967,8 +968,9 @@ static int amdgpu_debugfs_gem_info_show(struct seq_file *m, void *unused) * Therefore, we need to protect this ->comm access using RCU. */ rcu_read_lock(); - task = pid_task(file->pid, PIDTYPE_TGID); - seq_printf(m, "pid %8d command %s:\n", pid_nr(file->pid), + pid = rcu_dereference(file->pid); + task = pid_task(pid, PIDTYPE_TGID); + seq_printf(m, "pid %8d command %s:\n", pid_nr(pid), task ? task->comm : ""); rcu_read_unlock(); diff --git a/drivers/gpu/drm/drm_auth.c b/drivers/gpu/drm/drm_auth.c index cf92a9ae8034..2ed2585ded37 100644 --- a/drivers/gpu/drm/drm_auth.c +++ b/drivers/gpu/drm/drm_auth.c @@ -235,7 +235,8 @@ static int drm_new_set_master(struct drm_device *dev, struct drm_file *fpriv) static int drm_master_check_perm(struct drm_device *dev, struct drm_file *file_priv) { - if (file_priv->pid == task_pid(current) && file_priv->was_master) + if (file_priv->was_master && + rcu_access_pointer(file_priv->pid) == task_pid(current)) return 0; if (!capable(CAP_SYS_ADMIN)) diff --git a/drivers/gpu/drm/drm_debugfs.c b/drivers/gpu/drm/drm_debugfs.c index 42f657772025..cbcd79f01d50 100644 --- a/drivers/gpu/drm/drm_debugfs.c +++ b/drivers/gpu/drm/drm_debugfs.c @@ -90,15 +90,17 @@ static int drm_clients_info(struct seq_file *m, void *data) */ mutex_lock(&dev->filelist_mutex); list_for_each_entry_reverse(priv, &dev->filelist, lhead) { - struct task_struct *task; bool is_current_master = drm_is_current_master(priv); + struct task_struct *task; + struct pid *pid; - rcu_read_lock(); /* locks pid_task()->comm */ - task = pid_task(priv->pid, PIDTYPE_TGID); + rcu_read_lock(); /* Locks priv->pid and pid_task()->comm! */ + pid = rcu_dereference(priv->pid); + task = pid_task(pid, PIDTYPE_PID); uid = task ? __task_cred(task)->euid : GLOBAL_ROOT_UID; seq_printf(m, "%20s %5d %3d %c %c %5d %10u\n", task ? task->comm : "", - pid_vnr(priv->pid), + pid_vnr(pid), priv->minor->index, is_current_master ? 'y' : 'n', priv->authenticated ? 'y' : 'n', diff --git a/drivers/gpu/drm/drm_file.c b/drivers/gpu/drm/drm_file.c index 5cde5014cea1..4f5cff5c0bea 100644 --- a/drivers/gpu/drm/drm_file.c +++ b/drivers/gpu/drm/drm_file.c @@ -502,6 +502,36 @@ int drm_release(struct inode *inode, struct file *filp) } EXPORT_SYMBOL(drm_release); +void drm_file_update_pid(struct drm_file *filp) +{ + struct drm_device *dev; + struct pid *pid, *old; + + /* Master nodes are not expected to be passed between processes. */ + if (filp->was_master) + return; + + pid = task_tgid(current); + + /* + * Quick unlocked check since the model is a single handover followed by + * exclusive repeated use. + */ + if (pid == rcu_access_pointer(filp->pid)) + return; + + dev = filp->minor->dev; + mutex_lock(&dev->filelist_mutex); + old = rcu_replace_pointer(filp->pid, pid, 1); + mutex_unlock(&dev->filelist_mutex); + + if (pid != old) { + get_pid(pid); + synchronize_rcu(); + put_pid(old); + } +} + /** * drm_release_noglobal - release method for DRM file * @inode: device inode diff --git a/drivers/gpu/drm/drm_ioctl.c b/drivers/gpu/drm/drm_ioctl.c index 7c9d66ee917d..305b18d9d7b6 100644 --- a/drivers/gpu/drm/drm_ioctl.c +++ b/drivers/gpu/drm/drm_ioctl.c @@ -775,6 +775,9 @@ long drm_ioctl_kernel(struct file *file, drm_ioctl_t *func, void *kdata, struct drm_device *dev = file_priv->minor->dev; int retcode; + /* Update drm_file owner if fd was passed along. */ + drm_file_update_pid(file_priv); + if (drm_dev_is_unplugged(dev)) return -ENODEV; diff --git a/drivers/gpu/drm/nouveau/nouveau_drm.c b/drivers/gpu/drm/nouveau/nouveau_drm.c index fd99ec0f4257..17cd392acd69 100644 --- a/drivers/gpu/drm/nouveau/nouveau_drm.c +++ b/drivers/gpu/drm/nouveau/nouveau_drm.c @@ -1111,7 +1111,10 @@ nouveau_drm_open(struct drm_device *dev, struct drm_file *fpriv) } get_task_comm(tmpname, current); - snprintf(name, sizeof(name), "%s[%d]", tmpname, pid_nr(fpriv->pid)); + rcu_read_lock(); + snprintf(name, sizeof(name), "%s[%d]", + tmpname, pid_nr(rcu_dereference(fpriv->pid))); + rcu_read_unlock(); if (!(cli = kzalloc(sizeof(*cli), GFP_KERNEL))) { ret = -ENOMEM; diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_gem.c b/drivers/gpu/drm/vmwgfx/vmwgfx_gem.c index f2985337aa53..3853d9bb9ab8 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_gem.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_gem.c @@ -251,6 +251,7 @@ static int vmw_debugfs_gem_info_show(struct seq_file *m, void *unused) list_for_each_entry(file, &dev->filelist, lhead) { struct task_struct *task; struct drm_gem_object *gobj; + struct pid *pid; int id; /* @@ -260,8 +261,9 @@ static int vmw_debugfs_gem_info_show(struct seq_file *m, void *unused) * Therefore, we need to protect this ->comm access using RCU. */ rcu_read_lock(); - task = pid_task(file->pid, PIDTYPE_TGID); - seq_printf(m, "pid %8d command %s:\n", pid_nr(file->pid), + pid = rcu_dereference(file->pid); + task = pid_task(pid, PIDTYPE_TGID); + seq_printf(m, "pid %8d command %s:\n", pid_nr(pid), task ? task->comm : ""); rcu_read_unlock(); diff --git a/include/drm/drm_file.h b/include/drm/drm_file.h index d780fd151789..b8be69b551af 100644 --- a/include/drm/drm_file.h +++ b/include/drm/drm_file.h @@ -251,8 +251,15 @@ struct drm_file { /** @master_lookup_lock: Serializes @master. */ spinlock_t master_lookup_lock; - /** @pid: Process that opened this file. */ - struct pid *pid; + /** + * @pid: Process that is using this file. + * + * Must only be dereferenced under a rcu_read_lock or equivalent. + * + * Updates are guarded with dev->filelist_mutex and reference must be + * dropped after a RCU grace period to accommodate lockless readers. + */ + struct pid __rcu *pid; /** @magic: Authentication magic, see @authenticated. */ drm_magic_t magic; @@ -397,6 +404,8 @@ static inline bool drm_is_render_client(const struct drm_file *file_priv) return file_priv->minor->type == DRM_MINOR_RENDER; } +void drm_file_update_pid(struct drm_file *); + int drm_open(struct inode *inode, struct file *filp); ssize_t drm_read(struct file *filp, char __user *buffer, size_t count, loff_t *offset); From patchwork Wed Nov 9 16:11:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 17654 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp424415wru; Wed, 9 Nov 2022 08:13:32 -0800 (PST) X-Google-Smtp-Source: AMsMyM5hoUBQHGFn1JXGjc0m7vg3wvBNnapX+1avg/tRMiF6u66fQ6+VPy9ybu3yrfXRXEM7yOg2 X-Received: by 2002:a05:6402:27cf:b0:462:75b3:a277 with SMTP id c15-20020a05640227cf00b0046275b3a277mr63096375ede.175.1668010411924; Wed, 09 Nov 2022 08:13:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668010411; cv=none; d=google.com; s=arc-20160816; b=NV9zQCBg/qtjjIzEGxkdzdHkKObr3YL5kvEmn1MDK5NpuPVSy6vuf+LdRhUQYUZYXF HDW7AJ/7i3MgKtcZaLw9isy6GGgHplFDjAhx3JkmNspwBzWpbAE9L89CL/+qc8n1HAbD z0MHA7hGv4C84cTpaQHTbQr6RTsrQjDsceBRac4DlVe7/k3mwztppQdOW/bZaGGYjaRj Uxs2Fdo9o/oSc2n7UroD+TCsMtgCdtwwOvcDz+kGq0pJrRxszZfyFU7A5uH9A7mGIpOC nba+eeMy8OG8y7Reuh8ALmAwDtcW3/x912dQ/aHYH1pbWzSYfAM4NoJBIXmiIvakg8kq zcdA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=mJ/RjGQYjmHvoiSn91DMDt5Fn8MPQU1GeV6ZZ4CgEh8=; b=OkNcFf6/gd7j/McbtNqY2DTjoMxXhETPlF0HtiDGNrmuyQ0Wqs5zKVfziVbBBcc63q G/5Yrei8x7aSYB3X/X4r7HlFd0GxxQNeHqsp0zR+ZyfmPJ2bA1eaYSxiVEHXl5/BGG4X eX/fLXfCeXlTtuxY533I/M8pmEYouuuXWCPiAlrHGNyOVk9/zm8/3wxbUHcfYdf70jjW FjIP+xwIeQJlX9dg6NEDsKWHhlNPy4ORiPFzonyIhTsEiqTXi3OM4adOBJyDSSAJuv8C vnfjUSiT6jHxk3eXAdugD4hBtUn1nYtzZ2e5zWCRGzdW1MPw5N233JhYbiuhPvMzfyo8 ZwJg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=WTKMsaYJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id dm5-20020a170907948500b0073d6c0facdcsi17080251ejc.259.2022.11.09.08.13.07; Wed, 09 Nov 2022 08:13:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=WTKMsaYJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232087AbiKIQMZ (ORCPT + 99 others); Wed, 9 Nov 2022 11:12:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58734 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231964AbiKIQMS (ORCPT ); Wed, 9 Nov 2022 11:12:18 -0500 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C654A22BC4; Wed, 9 Nov 2022 08:12:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668010334; x=1699546334; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2BMu5cQFprtCxtPp+oUiMxCkYO0RYwczmemKP8KDzJQ=; b=WTKMsaYJ0KnqM1xg3jzS6hlD5dGy951FBSilNa2goOgtiB84Q9IOC1MC Wf7EgABXbjbtyJiY/zQkNM1q5wrCeNC8soTSC6/r1suN6dSX2z5HLpc68 PcHiohgCp62NSr39OFYqa3NaHsW7RP1NW79Nxay4Qk4RXKl9kj7mGOGH3 XRUrYKUj3WoW2cNKCEPVXwbdr9K5mWX7e8L18ghPEp1SblHeYCv9iomfr mB3cuidkTFymGArlO6bHqiGC6eFxx97rPnv7YmkKCv92gspXlTcOJKueF 4SglUyBw9DXi6WtnOG6qWSINDFvfkyPdF98ZWirFYjub06yUPdZepXoiU w==; X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="312181332" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="312181332" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:13 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="811684449" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="811684449" Received: from smurnane-mobl.ger.corp.intel.com (HELO localhost.localdomain) ([10.213.196.238]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:10 -0800 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 04/13] cgroup: Add the DRM cgroup controller Date: Wed, 9 Nov 2022 16:11:32 +0000 Message-Id: <20221109161141.2987173-5-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> References: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749035685763786113?= X-GMAIL-MSGID: =?utf-8?q?1749035685763786113?= From: Tvrtko Ursulin Skeleton controller without any functionality. Signed-off-by: Tvrtko Ursulin --- include/linux/cgroup_drm.h | 9 ++++++ include/linux/cgroup_subsys.h | 4 +++ init/Kconfig | 7 +++++ kernel/cgroup/Makefile | 1 + kernel/cgroup/drm.c | 54 +++++++++++++++++++++++++++++++++++ 5 files changed, 75 insertions(+) create mode 100644 include/linux/cgroup_drm.h create mode 100644 kernel/cgroup/drm.c diff --git a/include/linux/cgroup_drm.h b/include/linux/cgroup_drm.h new file mode 100644 index 000000000000..bf8abc6b8ebf --- /dev/null +++ b/include/linux/cgroup_drm.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright © 2022 Intel Corporation + */ + +#ifndef _CGROUP_DRM_H +#define _CGROUP_DRM_H + +#endif /* _CGROUP_DRM_H */ diff --git a/include/linux/cgroup_subsys.h b/include/linux/cgroup_subsys.h index 445235487230..49460494a010 100644 --- a/include/linux/cgroup_subsys.h +++ b/include/linux/cgroup_subsys.h @@ -65,6 +65,10 @@ SUBSYS(rdma) SUBSYS(misc) #endif +#if IS_ENABLED(CONFIG_CGROUP_DRM) +SUBSYS(drm) +#endif + /* * The following subsystems are not supported on the default hierarchy. */ diff --git a/init/Kconfig b/init/Kconfig index abf65098f1b6..70c08f340961 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1087,6 +1087,13 @@ config CGROUP_RDMA Attaching processes with active RDMA resources to the cgroup hierarchy is allowed even if can cross the hierarchy's limit. +config CGROUP_DRM + bool "DRM controller" + help + Provides the DRM subsystem controller. + + ... + config CGROUP_FREEZER bool "Freezer controller" help diff --git a/kernel/cgroup/Makefile b/kernel/cgroup/Makefile index 12f8457ad1f9..849bd2917477 100644 --- a/kernel/cgroup/Makefile +++ b/kernel/cgroup/Makefile @@ -6,4 +6,5 @@ obj-$(CONFIG_CGROUP_PIDS) += pids.o obj-$(CONFIG_CGROUP_RDMA) += rdma.o obj-$(CONFIG_CPUSETS) += cpuset.o obj-$(CONFIG_CGROUP_MISC) += misc.o +obj-$(CONFIG_CGROUP_DRM) += drm.o obj-$(CONFIG_CGROUP_DEBUG) += debug.o diff --git a/kernel/cgroup/drm.c b/kernel/cgroup/drm.c new file mode 100644 index 000000000000..b88c93661df3 --- /dev/null +++ b/kernel/cgroup/drm.c @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright © 2022 Intel Corporation + */ + +#include +#include +#include +#include + +struct drm_cgroup_state { + struct cgroup_subsys_state css; +}; + +static inline struct drm_cgroup_state * +css_to_drmcs(struct cgroup_subsys_state *css) +{ + return container_of(css, struct drm_cgroup_state, css); +} + +static void drmcs_free(struct cgroup_subsys_state *css) +{ + kfree(css_to_drmcs(css)); +} + +static struct drm_cgroup_state root_drmcs = { +}; + +static struct cgroup_subsys_state * +drmcs_alloc(struct cgroup_subsys_state *parent_css) +{ + struct drm_cgroup_state *drmcs; + + if (!parent_css) + return &root_drmcs.css; + + drmcs = kzalloc(sizeof(*drmcs), GFP_KERNEL); + if (!drmcs) + return ERR_PTR(-ENOMEM); + + return &drmcs->css; +} + +struct cftype files[] = { + { } /* Zero entry terminates. */ +}; + +struct cgroup_subsys drm_cgrp_subsys = { + .css_alloc = drmcs_alloc, + .css_free = drmcs_free, + .early_init = false, + .legacy_cftypes = files, + .dfl_cftypes = files, +}; From patchwork Wed Nov 9 16:11:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 17655 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp424558wru; Wed, 9 Nov 2022 08:13:47 -0800 (PST) X-Google-Smtp-Source: AA0mqf4nRPGKRTok4FP7SSHZzu/0YW6HhNBce1HerOwyyzU5wtM4rKCXHmEVqFjfNNbZqtH8ell9 X-Received: by 2002:aa7:d385:0:b0:461:8cd3:b38b with SMTP id x5-20020aa7d385000000b004618cd3b38bmr22081902edq.172.1668010427351; Wed, 09 Nov 2022 08:13:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668010427; cv=none; d=google.com; s=arc-20160816; b=A5vjEXwKppPPpjO1o74aBXWAGOSuHRTtO+fVFGxIyGd8zM2AekH/KtDsnJfVSFgNVA 0rZcm6yfrV7fOB2RJAoekA+HqH+/vYEYqlYlwVnsKrRbdoi35pmr/iCGjGMoD0EPRDKi SM2Wl94tDwiQTd5swAPF+NWmcIcRgnGc4h0wc0tKU0fE7gqvUY3bS7iJ1OEoWD0B1gVM MxmALtetn8pAuHeWQhgYfXN9MJWsSjLw5+oZxucCNYeGcN5vdnb9W37k54bHZirUogXM /6qSeMzcB4Tv3O+KBbt6PFLhcAA+79Nb0nHgeMrcewiqGjNVQB2KO89TDB4mL7V0X1hJ +NKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=z27N3gzt9zlvycQywnlAxpL4VRiozG1UiHdWCqkZ9eo=; b=zoDcN5pqAL4LetcVp1xy9Gw7InJgowVr/IeXIHDJlyREwOts+gu8PSIsRRXEtZEDO1 LewHGKTdY/iuDVhcA5hCdSkCaRo6jP2PfNlga+KOsyCN2QYkX6Ix/vrF9vbrnsxuzEBq ianpB++ipPviMbQjxlw/REEBZUxGHcIrz6pU5gQZ7Jrxj44gpnGmGvOV9BGmgLQEj1XX etUXhQBHCNp3+2FJyYM9CI50zfYoUTkMh7mygIe2EuELYjnDr2Yc4ull60TJM/22yKMg NDbsoC/hnh/deZbS2HJABVaXnOfSWJy2h+r66zzhK/iTcZAvQ87WsRBN52y1PwHcGren peAw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=CtuIL09I; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x13-20020a05640226cd00b00461727807a4si5244082edd.534.2022.11.09.08.13.22; Wed, 09 Nov 2022 08:13:47 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=CtuIL09I; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231599AbiKIQM3 (ORCPT + 99 others); Wed, 9 Nov 2022 11:12:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58818 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232051AbiKIQMT (ORCPT ); Wed, 9 Nov 2022 11:12:19 -0500 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 599D2220FA; Wed, 9 Nov 2022 08:12:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668010337; x=1699546337; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=DOi4RMupNEjpYyK4WovQ8kG/iXKjxHECtSroc0kXvAE=; b=CtuIL09I139syLEmVYuiqfQuO8R7ReTxij1KU7fqkZ2DseuIKLe+IXHU RH44Ltk2t0Y6ZGQS+cxmIXC6pn4DMkoK9QX5YVJlzCvDUs7F19ePIEJGF soGT3OyZ4zreF1ODbOEPJ63dsKtYMiwkk7/GCCHUtxkY9hf57JPDOx4K/ ifbiMGTNI9nOLbP6GfUEHDu87A76he+uL6ZVPEvMsvWX+a34l1wuULZuu X/HXlXZwr7E3PEg4hDw8IBVFarQvkIs8uwuKhoxrxe5RLwwOSzGOP4AZr rUZdc+6dNOsV1Lz9YBqYkhkUSMyqEN7DCCilJJgZa6MrKYByZ31c4x8w0 Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="312181345" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="312181345" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:17 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="811684480" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="811684480" Received: from smurnane-mobl.ger.corp.intel.com (HELO localhost.localdomain) ([10.213.196.238]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:13 -0800 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 05/13] drm/cgroup: Track clients per owning process Date: Wed, 9 Nov 2022 16:11:33 +0000 Message-Id: <20221109161141.2987173-6-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> References: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749035701546182520?= X-GMAIL-MSGID: =?utf-8?q?1749035701546182520?= From: Tvrtko Ursulin To enable propagation of settings from the cgroup drm controller to drm we need to start tracking which processes own which drm clients. Implement that by tracking the struct pid pointer of the owning process in a new XArray, pointing to a structure containing a list of associated struct drm_file pointers. Clients are added and removed under the filelist mutex and RCU list operations are used below it to allow for lockless lookup. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/Makefile | 1 + drivers/gpu/drm/drm_cgroup.c | 123 +++++++++++++++++++++++++++++++++++ drivers/gpu/drm/drm_file.c | 21 ++++-- include/drm/drm_clients.h | 44 +++++++++++++ include/drm/drm_file.h | 4 ++ 5 files changed, 189 insertions(+), 4 deletions(-) create mode 100644 drivers/gpu/drm/drm_cgroup.c create mode 100644 include/drm/drm_clients.h diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile index c44a54cadb61..4495dda2a720 100644 --- a/drivers/gpu/drm/Makefile +++ b/drivers/gpu/drm/Makefile @@ -59,6 +59,7 @@ drm-$(CONFIG_DRM_LEGACY) += \ drm_scatter.o \ drm_vm.o drm-$(CONFIG_DRM_LIB_RANDOM) += lib/drm_random.o +drm-$(CONFIG_CGROUP_DRM) += drm_cgroup.o drm-$(CONFIG_COMPAT) += drm_ioc32.o drm-$(CONFIG_DRM_PANEL) += drm_panel.o drm-$(CONFIG_OF) += drm_of.o diff --git a/drivers/gpu/drm/drm_cgroup.c b/drivers/gpu/drm/drm_cgroup.c new file mode 100644 index 000000000000..56aa8303974a --- /dev/null +++ b/drivers/gpu/drm/drm_cgroup.c @@ -0,0 +1,123 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright © 2022 Intel Corporation + */ + +#include +#include + +static DEFINE_XARRAY(drm_pid_clients); + +static void +__del_clients(struct drm_pid_clients *clients, + struct drm_file *file_priv, + unsigned long pid) +{ + list_del_rcu(&file_priv->clink); + if (atomic_dec_and_test(&clients->num)) { + xa_erase(&drm_pid_clients, pid); + kfree_rcu(clients, rcu); + } +} + +void drm_clients_close(struct drm_file *file_priv) +{ + struct drm_device *dev = file_priv->minor->dev; + struct drm_pid_clients *clients; + struct pid *pid; + + lockdep_assert_held(&dev->filelist_mutex); + + pid = rcu_access_pointer(file_priv->pid); + clients = xa_load(&drm_pid_clients, (unsigned long)pid); + if (WARN_ON_ONCE(!clients)) + return; + + __del_clients(clients, file_priv, (unsigned long)pid); +} + +static struct drm_pid_clients *__alloc_clients(void) +{ + struct drm_pid_clients *clients; + + clients = kmalloc(sizeof(*clients), GFP_KERNEL); + if (clients) { + atomic_set(&clients->num, 0); + INIT_LIST_HEAD(&clients->file_list); + init_rcu_head(&clients->rcu); + } + + return clients; +} + +int drm_clients_open(struct drm_file *file_priv) +{ + struct drm_device *dev = file_priv->minor->dev; + struct drm_pid_clients *clients; + bool new_client = false; + unsigned long pid; + + lockdep_assert_held(&dev->filelist_mutex); + + pid = (unsigned long)rcu_access_pointer(file_priv->pid); + clients = xa_load(&drm_pid_clients, pid); + if (!clients) { + clients = __alloc_clients(); + if (!clients) + return -ENOMEM; + new_client = true; + } + atomic_inc(&clients->num); + list_add_tail_rcu(&file_priv->clink, &clients->file_list); + if (new_client) { + void *xret; + + xret = xa_store(&drm_pid_clients, pid, clients, GFP_KERNEL); + if (xa_err(xret)) { + list_del_init(&file_priv->clink); + kfree(clients); + return PTR_ERR(clients); + } + } + + return 0; +} + +void +drm_clients_migrate(struct drm_file *file_priv, + unsigned long old, + unsigned long new) +{ + struct drm_device *dev = file_priv->minor->dev; + struct drm_pid_clients *existing_clients; + struct drm_pid_clients *clients; + + lockdep_assert_held(&dev->filelist_mutex); + + existing_clients = xa_load(&drm_pid_clients, new); + clients = xa_load(&drm_pid_clients, old); + + if (WARN_ON_ONCE(!clients)) + return; + else if (WARN_ON_ONCE(clients == existing_clients)) + return; + + __del_clients(clients, file_priv, old); + + if (!existing_clients) { + void *xret; + + clients = __alloc_clients(); + if (WARN_ON(!clients)) + return; + + xret = xa_store(&drm_pid_clients, new, clients, GFP_KERNEL); + if (WARN_ON(xa_err(xret))) + return; + } else { + clients = existing_clients; + } + + atomic_inc(&clients->num); + list_add_tail_rcu(&file_priv->clink, &clients->file_list); +} diff --git a/drivers/gpu/drm/drm_file.c b/drivers/gpu/drm/drm_file.c index 4f5cff5c0bea..16ca5b88f414 100644 --- a/drivers/gpu/drm/drm_file.c +++ b/drivers/gpu/drm/drm_file.c @@ -40,6 +40,7 @@ #include #include +#include #include #include #include @@ -298,6 +299,7 @@ static void drm_close_helper(struct file *filp) mutex_lock(&dev->filelist_mutex); list_del(&file_priv->lhead); + drm_clients_close(file_priv); mutex_unlock(&dev->filelist_mutex); drm_file_free(file_priv); @@ -349,10 +351,8 @@ static int drm_open_helper(struct file *filp, struct drm_minor *minor) if (drm_is_primary_client(priv)) { ret = drm_master_open(priv); - if (ret) { - drm_file_free(priv); - return ret; - } + if (ret) + goto err_free; } filp->private_data = priv; @@ -360,6 +360,9 @@ static int drm_open_helper(struct file *filp, struct drm_minor *minor) priv->filp = filp; mutex_lock(&dev->filelist_mutex); + ret = drm_clients_open(priv); + if (ret) + goto err_unlock; list_add(&priv->lhead, &dev->filelist); mutex_unlock(&dev->filelist_mutex); @@ -387,6 +390,13 @@ static int drm_open_helper(struct file *filp, struct drm_minor *minor) #endif return 0; + +err_unlock: + mutex_unlock(&dev->filelist_mutex); +err_free: + drm_file_free(priv); + + return ret; } /** @@ -523,6 +533,9 @@ void drm_file_update_pid(struct drm_file *filp) dev = filp->minor->dev; mutex_lock(&dev->filelist_mutex); old = rcu_replace_pointer(filp->pid, pid, 1); + if (pid != old) + drm_clients_migrate(filp, + (unsigned long)old, (unsigned long)pid); mutex_unlock(&dev->filelist_mutex); if (pid != old) { diff --git a/include/drm/drm_clients.h b/include/drm/drm_clients.h new file mode 100644 index 000000000000..fbb8cffdf7a9 --- /dev/null +++ b/include/drm/drm_clients.h @@ -0,0 +1,44 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright © 2022 Intel Corporation + */ + +#ifndef _DRM_CLIENTS_H_ +#define _DRM_CLIENTS_H_ + +#include + +struct drm_pid_clients { + atomic_t num; + struct list_head file_list; + struct rcu_head rcu; +}; + +#if IS_ENABLED(CONFIG_CGROUP_DRM) +void drm_clients_close(struct drm_file *file_priv); +int drm_clients_open(struct drm_file *file_priv); + +void +drm_clients_migrate(struct drm_file *file_priv, + unsigned long old, + unsigned long new); +#else +static inline void drm_clients_close(struct drm_file *file_priv) +{ +} + +static inline int drm_clients_open(struct drm_file *file_priv) +{ + return 0; +} + +static void +drm_clients_migrate(struct drm_file *file_priv, + unsigned long old, + unsigned long new) +{ + +} +#endif + +#endif diff --git a/include/drm/drm_file.h b/include/drm/drm_file.h index b8be69b551af..ad20aaad40f5 100644 --- a/include/drm/drm_file.h +++ b/include/drm/drm_file.h @@ -275,6 +275,10 @@ struct drm_file { /** @minor: &struct drm_minor for this file. */ struct drm_minor *minor; +#if IS_ENABLED(CONFIG_CGROUP_DRM) + struct list_head clink; +#endif + /** * @object_idr: * From patchwork Wed Nov 9 16:11:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 17656 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp424623wru; Wed, 9 Nov 2022 08:13:56 -0800 (PST) X-Google-Smtp-Source: AMsMyM7SbD1DKARhGqb9BgYVXTqPnxAHOKUs7DRIPhlHfHcpVkst15Xl/LV882uEK2//S5e8wVfW X-Received: by 2002:a05:6402:f2a:b0:461:eff7:bae8 with SMTP id i42-20020a0564020f2a00b00461eff7bae8mr60190564eda.322.1668010436308; Wed, 09 Nov 2022 08:13:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668010436; cv=none; d=google.com; s=arc-20160816; b=O6qWRWcfjWLi3ktnBbMALDH3xPbWcoDnIQpQGxA1M1DZoAo1EVipGeJQtYlwJ+J/BB k4oDLxcDHuO4wlxiGQnBa38/dSLU+Sui8AXeOXt/+hXfxH09GJTAhu5P7vOTaT9kdlqB R5Uwv2+LiOANpOupIKzjG8EWHGnXLcrk6yjLBtcrLsVwAisE097WOxmCcCfDpfSWT4xh AB2uoH7+1EjORLThdRSOTOZ/bLaGCwVG9+yAhIMMcAzHpuru4Ghh1M4l68UwblPhpAQl cmAS3OvyQFxksxNR/WquRti5x1YeqYkIWJNO+kW7DsYp/ywSBhgBKE8AguYGK1J+O7DD 13Zw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=snPMyv1TL2LRUAG4zp/I/NHREoeoxY5YOdxwQ0wGAOA=; b=WaA/tcKPz7BAVWn6PKmPU7Xk2+IFj/BXG3+Cq1ZaNWNJAowSboYUpAc6XkWzdkiUFl cDwQteFSaqn1M0wLLFHNYESEaI31cHYPq1+dtNQL41aU9tNHbzpQf4GeUK1YKldgKoLO zONQAvh//WDS/UZW2hRKgSIMq3NYbMrHKkTAzTwPGaC6BCfrAvZ2nhhQ/n5zJnaDqABq O1mH0EOrnTnkDqofhtkm99603L2b91ZhGP/g+87Baj/ly/PVIR4aQ9/srkB0AbxznLNP M5iFEvHwCQhWs9qSsa7YrWSAzX9UqaBdDl21Lz7rk7AqV2na978FF03UpEXim3LPZ7OK mZ9Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=EoAH1m8R; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b7-20020a056402350700b004580ae4abfcsi16638114edd.460.2022.11.09.08.13.27; Wed, 09 Nov 2022 08:13:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=EoAH1m8R; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232136AbiKIQMd (ORCPT + 99 others); Wed, 9 Nov 2022 11:12:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58756 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231920AbiKIQMV (ORCPT ); Wed, 9 Nov 2022 11:12:21 -0500 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BE3B310543; Wed, 9 Nov 2022 08:12:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668010340; x=1699546340; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KIk/9mO9BwW8K7kgNQr0OR9vwi+NAqyFfuSgODl/DSc=; b=EoAH1m8RfJaGmA0O8K/mo7xEbTHN0CSGPlcq1hfrg7+Obmkfd9J5/nrZ Tt0DVZ+GY0jSF8Jqqoz7I/9tmWAJKEPhg6/6xJDdS/MMd5wCuuUijsGtd NvMi7vtz8rj/Dcdbvm2I1vJqoynp4CGI7YlkhG54MAu2b5xlr+uhwHk+I bPAgC9b00k8RI7x9iwJbLaAMvYqyaoYs8Gej/k9KSNPPuU+9qYzyx/vgd RdV5bB1yascVM+G8hrsy9cAi9eFSTAhcLQkVLsMmF4F19RT6SlIZkJqq2 qCv4ECvS00e2mM9wq0wYg0BqueM7K5iAgo036E02m6ntYGEw1ciOLZAcp A==; X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="312181362" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="312181362" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:20 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="811684509" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="811684509" Received: from smurnane-mobl.ger.corp.intel.com (HELO localhost.localdomain) ([10.213.196.238]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:17 -0800 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 06/13] drm/cgroup: Allow safe external access to file_priv Date: Wed, 9 Nov 2022 16:11:34 +0000 Message-Id: <20221109161141.2987173-7-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> References: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749035711375596637?= X-GMAIL-MSGID: =?utf-8?q?1749035711375596637?= From: Tvrtko Ursulin Entry points from the cgroup subsystem into the drm cgroup controller will need to walk the file_priv structures associated with registered clients and since those are not RCU protected lets add a hack for now to make this safe. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/drm_cgroup.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/gpu/drm/drm_cgroup.c b/drivers/gpu/drm/drm_cgroup.c index 56aa8303974a..94e6f39b90c7 100644 --- a/drivers/gpu/drm/drm_cgroup.c +++ b/drivers/gpu/drm/drm_cgroup.c @@ -17,6 +17,13 @@ __del_clients(struct drm_pid_clients *clients, if (atomic_dec_and_test(&clients->num)) { xa_erase(&drm_pid_clients, pid); kfree_rcu(clients, rcu); + + /* + * FIXME: file_priv is not RCU protected so we add this hack + * to avoid any races with code which walks clients->file_list + * and accesses file_priv. + */ + synchronize_rcu(); } } From patchwork Wed Nov 9 16:11:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 17658 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp424719wru; Wed, 9 Nov 2022 08:14:07 -0800 (PST) X-Google-Smtp-Source: AMsMyM4DzWZuMWlC4RHAhpG2wWDJ3vea76F2QbbLr5TpMw060FBWi8aPeUFscOtq3RKLNq4+rANs X-Received: by 2002:a17:906:5242:b0:7ad:e161:b026 with SMTP id y2-20020a170906524200b007ade161b026mr1360319ejm.760.1668010447531; Wed, 09 Nov 2022 08:14:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668010447; cv=none; d=google.com; s=arc-20160816; b=yd3K6DiGhVfIWF3apzOR1bd8onOaoObMmvi2wk5iN22qAYvImSF43gJdqoFfS/DqBj 4lWjMmsWHueIzCX4393gTg6Nvyc6zkbHwTTcD3brasBB4jpPc7mj4N7gg6OGcBbJg1N4 y62syOYox8F+fNA6KOHOMq/Gg3eZ6HkNY2s41YWvhnDbQMXl+sRPrnP1o7VTxM/pxMqw bAdr06jPTaxdM9971KS7XezYDEX2ydlcfM+hdhtGt6CGt2IaURxUtcme8e0MMl6EgHW+ v98QZ+N64tNyXphTpW8J1k8Ryvi687u6SQL0FHYFcAxc5yqUWSnMwPSGrtK1ieLlWLxq G2ug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Li3I1jgDTWJVhE+FauG3+pXLM/qx4ADbsIPIr+LSDfU=; b=G3H+15v44ZXxVERXJnpM/3JvSmR/6RE5i5/LxNr9Pm7MCPjOM6Fud9ljCl76SQzL5e h7rJ2qRLFyTrpVeQB7XmNZakkPGln7t5FfIf5q/UwRJHo4vfgV6i/A11n0D82vVENy25 FnqzrGpHW5cpzufjd3zVgO85IZZMXEv3JZMg3DiMDwI5nDKW6hw4q4GGv3p+6zaDYWrX ql3EuhOHJkmj8W7nx40JXvRxcHCvLrUdhZaKGDqNbpdKf+3mpx24HnhHtdssMjQlqizt 3MRptfb/QZtDv2UFnJ2rPg6Qp4top1izz3D3YjKO5VfBmHjm3+ngzKr084x0+TeF2sVo jBUw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=KkXfG5UO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id c59-20020a509fc1000000b00462df2b0639si13607653edf.335.2022.11.09.08.13.39; Wed, 09 Nov 2022 08:14:07 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=KkXfG5UO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232160AbiKIQNB (ORCPT + 99 others); Wed, 9 Nov 2022 11:13:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58750 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232111AbiKIQMZ (ORCPT ); Wed, 9 Nov 2022 11:12:25 -0500 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4808210FE6; Wed, 9 Nov 2022 08:12:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668010344; x=1699546344; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+tkTnG1kbfQkuQWu7X2WwiAikHzraT6aROxd5PKF3OE=; b=KkXfG5UOC8FnwK3BUSvUAUaauAYhy9JDQd9nHEuvFTisnIwGpW5diy49 +s1YSZrd1qsSFXiTzjn3KbD1rEaLolOLaa8FBegD3HslpvvI9Sh6aY8d0 VOV9iW7CgBoJXRmJb6DcyOMRNWXrC0zPwm6XfCLbNyEg9yx5Qo8TDVdn/ 94WtnDDLyTvrEOwcUXrlfSBLbNY7/p8ArfPwQSM1OlmnrmEfolMovKDrJ kRhiAkpOHGzjteP/oieVOeHSRPqdefHjHSadXw/2sFjhgTx6VGlwwO33u BWWvJUwGcx3w5wiLKg16oW1QT3E5LEp4zkeBODYjAqn6K5J0KmZ+dXN+1 Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="312181384" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="312181384" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:24 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="811684529" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="811684529" Received: from smurnane-mobl.ger.corp.intel.com (HELO localhost.localdomain) ([10.213.196.238]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:20 -0800 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 07/13] drm/cgroup: Add ability to query drm cgroup GPU time Date: Wed, 9 Nov 2022 16:11:35 +0000 Message-Id: <20221109161141.2987173-8-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> References: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749035723249301460?= X-GMAIL-MSGID: =?utf-8?q?1749035723249301460?= From: Tvrtko Ursulin Add a driver callback and core helper which allow querying the time spent on GPUs for processes belonging to a group. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/drm_cgroup.c | 24 ++++++++++++++++++++++++ include/drm/drm_clients.h | 2 ++ include/drm/drm_drv.h | 28 ++++++++++++++++++++++++++++ 3 files changed, 54 insertions(+) diff --git a/drivers/gpu/drm/drm_cgroup.c b/drivers/gpu/drm/drm_cgroup.c index 94e6f39b90c7..06810b4c3ff1 100644 --- a/drivers/gpu/drm/drm_cgroup.c +++ b/drivers/gpu/drm/drm_cgroup.c @@ -128,3 +128,27 @@ drm_clients_migrate(struct drm_file *file_priv, atomic_inc(&clients->num); list_add_tail_rcu(&file_priv->clink, &clients->file_list); } + +u64 drm_pid_get_active_time_us(struct pid *pid) +{ + struct drm_pid_clients *clients; + u64 total = 0; + + rcu_read_lock(); + clients = xa_load(&drm_pid_clients, (unsigned long)pid); + if (clients) { + struct drm_file *fpriv; + + list_for_each_entry_rcu(fpriv, &clients->file_list, clink) { + const struct drm_cgroup_ops *cg_ops = + fpriv->minor->dev->driver->cg_ops; + + if (cg_ops && cg_ops->active_time_us) + total += cg_ops->active_time_us(fpriv); + } + } + rcu_read_unlock(); + + return total; +} +EXPORT_SYMBOL_GPL(drm_pid_get_active_time_us); diff --git a/include/drm/drm_clients.h b/include/drm/drm_clients.h index fbb8cffdf7a9..b9b8009c28a6 100644 --- a/include/drm/drm_clients.h +++ b/include/drm/drm_clients.h @@ -41,4 +41,6 @@ drm_clients_migrate(struct drm_file *file_priv, } #endif +u64 drm_pid_get_active_time_us(struct pid *pid); + #endif diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h index f6159acb8856..c09fe9bd517f 100644 --- a/include/drm/drm_drv.h +++ b/include/drm/drm_drv.h @@ -148,6 +148,24 @@ enum drm_driver_feature { DRIVER_KMS_LEGACY_CONTEXT = BIT(31), }; +/** + * struct drm_cgroup_ops + * + * This structure contains a number of callbacks that drivers can provide if + * they are able to support one or more of the functionalities implemented by + * the DRM cgroup controller. + */ +struct drm_cgroup_ops { + /** + * @active_time_us: + * + * Optional callback for reporting the GPU time consumed by this client. + * + * Used by the DRM core when queried by the DRM cgroup controller. + */ + u64 (*active_time_us) (struct drm_file *); +}; + /** * struct drm_driver - DRM driver structure * @@ -459,6 +477,16 @@ struct drm_driver { */ const struct file_operations *fops; +#ifdef CONFIG_CGROUP_DRM + /** + * @cg_ops: + * + * Optional pointer to driver callbacks facilitating integration with + * the DRM cgroup controller. + */ + const struct drm_cgroup_ops *cg_ops; +#endif + #ifdef CONFIG_DRM_LEGACY /* Everything below here is for legacy driver, never use! */ /* private: */ From patchwork Wed Nov 9 16:11:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 17657 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp424658wru; Wed, 9 Nov 2022 08:13:59 -0800 (PST) X-Google-Smtp-Source: AMsMyM7IWCxP2P9wRI6f8MwR/qqy6OrYP5RwtI1BP6SVvXPYP3xVpiqFUJMTTMr83gN3D6hy0Rok X-Received: by 2002:a17:907:aa1:b0:7ad:ca7e:a4d8 with SMTP id bz1-20020a1709070aa100b007adca7ea4d8mr50149877ejc.663.1668010439215; Wed, 09 Nov 2022 08:13:59 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668010439; cv=none; d=google.com; s=arc-20160816; b=SRwRD7lL2eN/Z4R88PVaTFZS9mANTd0LGAyd8OOq7CaFnJMFQYTWhJN+yTu3wiF9zE 8ZbSjXZvYBcr3Vr3EPTNmCOQiIK5IZLB57B2JK3fykgJAqKdoEcbzmFKumkpt/DZAnxQ zj/m632mk5qmB9a3AOKb0QqiLbVXmIbiWpBuMvp879QySeAyznVScofDu/tkabI/P484 nLQveiKfJl/FC8lifxjoEGK1rMC6NNRIavaTa+Djvstdfa4yl0XeJVlBLPCXl7u7+P4m XV7cOvm2PcLlBq3oKxwsz6sAQramYPjwEyXeNE3LoKtaAfNSc+LFlH3Z69RxB8eEp1ne X76Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=C7PiA0f0VaYmSgxSM9mZVSnviNVJlKm+TBJoFy1Yg4k=; b=J3/rICYLplTd5jOLk6dXZn4SiHWWSktAuysNKL3JkQ4yWfb2khDrj8GTG6DeAKXScl 82igIREeHBuLUu7px4rAj+i85IcUNEWk5sppoaDmZ0vjlnjeECn30AK6HNLDP/f2t4qm mn2y0f3fkDpVpHFIwHs7BlsRJrPGIenEXcwtuE2mm7VVcKp511MXk9olc475dR1chjEy fhyQxcbdXTI7mM6jHTd1alus3GFEInpZye9D4ZAe8BP6DZJTIjetmY3klrEqJ5edWlji 6+pIpNH1+UMbAaFWNaexhfOTIPYoekHW9j1W9c41CrQbr/nnsvpYSfiKVNiPkppsS/WP W4dA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YjQ+5XBr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e8-20020a056402148800b0045d088f4b79si14121269edv.258.2022.11.09.08.13.33; Wed, 09 Nov 2022 08:13:59 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YjQ+5XBr; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230014AbiKIQMy (ORCPT + 99 others); Wed, 9 Nov 2022 11:12:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58826 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232125AbiKIQM2 (ORCPT ); Wed, 9 Nov 2022 11:12:28 -0500 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C94C022B21; Wed, 9 Nov 2022 08:12:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668010347; x=1699546347; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Qq6Henm8EAobf6LwC/wUuKCvjc1IJ70N8x2PFzYMvRs=; b=YjQ+5XBr03tLQEV+adZgg5GKoWtQ9dsdG6v9oQZaokU8/FoXrWlP8GTM sP7R4QY9/Z6oK02iTeyboaw84UOJAnI9Pc1xr4oyDoriO+0j4R0DEq8U6 DyZ4wl27XRC3gxCxl/Tm7oN7blY8AW/gqaGGepy7lh7Bvtt7XuG0GcV7N MNfWiVOSgDT/ipusmPaUPYegKMOsNJ1InFoq0aI6x4MIaz7SIdeQkgOWD BpW8GGbS6tvQIJeOUwlxJKvhC6h3uHAKODTG72oRLWN9Bj0mjyWfN2pVF 8h+nohNk8m/T9WX/WPUK5+rBIvzwnoc02RQvQnBrU7mL37ErV6dKQLsSp A==; X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="312181402" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="312181402" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:27 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="811684540" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="811684540" Received: from smurnane-mobl.ger.corp.intel.com (HELO localhost.localdomain) ([10.213.196.238]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:24 -0800 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 08/13] drm/cgroup: Add over budget signalling callback Date: Wed, 9 Nov 2022 16:11:36 +0000 Message-Id: <20221109161141.2987173-9-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> References: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749035714657844010?= X-GMAIL-MSGID: =?utf-8?q?1749035714657844010?= From: Tvrtko Ursulin Add a new callback via which the drm cgroup controller is notifying the drm core that a certain process is above its allotted GPU time. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/drm_cgroup.c | 21 +++++++++++++++++++++ include/drm/drm_clients.h | 1 + include/drm/drm_drv.h | 8 ++++++++ 3 files changed, 30 insertions(+) diff --git a/drivers/gpu/drm/drm_cgroup.c b/drivers/gpu/drm/drm_cgroup.c index 06810b4c3ff1..e3854741c584 100644 --- a/drivers/gpu/drm/drm_cgroup.c +++ b/drivers/gpu/drm/drm_cgroup.c @@ -152,3 +152,24 @@ u64 drm_pid_get_active_time_us(struct pid *pid) return total; } EXPORT_SYMBOL_GPL(drm_pid_get_active_time_us); + +void drm_pid_signal_budget(struct pid *pid, u64 usage, u64 budget) +{ + struct drm_pid_clients *clients; + + rcu_read_lock(); + clients = xa_load(&drm_pid_clients, (unsigned long)pid); + if (clients) { + struct drm_file *fpriv; + + list_for_each_entry_rcu(fpriv, &clients->file_list, clink) { + const struct drm_cgroup_ops *cg_ops = + fpriv->minor->dev->driver->cg_ops; + + if (cg_ops && cg_ops->signal_budget) + cg_ops->signal_budget(fpriv, usage, budget); + } + } + rcu_read_unlock(); +} +EXPORT_SYMBOL_GPL(drm_pid_signal_budget); diff --git a/include/drm/drm_clients.h b/include/drm/drm_clients.h index b9b8009c28a6..356ee92792a6 100644 --- a/include/drm/drm_clients.h +++ b/include/drm/drm_clients.h @@ -42,5 +42,6 @@ drm_clients_migrate(struct drm_file *file_priv, #endif u64 drm_pid_get_active_time_us(struct pid *pid); +void drm_pid_signal_budget(struct pid *pid, u64 usage, u64 budget); #endif diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h index c09fe9bd517f..c30afe97f922 100644 --- a/include/drm/drm_drv.h +++ b/include/drm/drm_drv.h @@ -164,6 +164,14 @@ struct drm_cgroup_ops { * Used by the DRM core when queried by the DRM cgroup controller. */ u64 (*active_time_us) (struct drm_file *); + + /** + * @signal_budget: + * + * Optional callback used by the DRM core to forward over/under GPU time + * messages sent by the DRM cgroup controller. + */ + int (*signal_budget) (struct drm_file *, u64 used, u64 budget); }; /** From patchwork Wed Nov 9 16:11:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 17659 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp425094wru; Wed, 9 Nov 2022 08:14:49 -0800 (PST) X-Google-Smtp-Source: AMsMyM44cC1dggse4WgHrutf2EhKfSzFN+Y89gOBc7GHiuO1A7ssmAVGDOjf3+GgSMSJn/N6OO1Q X-Received: by 2002:a17:906:8a47:b0:7a3:86dd:d33b with SMTP id gx7-20020a1709068a4700b007a386ddd33bmr58589544ejc.67.1668010489250; Wed, 09 Nov 2022 08:14:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668010489; cv=none; d=google.com; s=arc-20160816; b=1C6AruD0Us1dLq+NMRjqxG/iosq+rXPYO9z5FEghkZywP8tOo++8RwUcaukbs96X1F KzjZgKWJ5K/m27OKHjtcjlQ/wrJ2vkVDZ7knRoNCxaiWGt5Z3e2K2qSfmxHdSa2I1pP+ VWUsBvcbZiRVtUMhxcd1x3D8A5J7DFY2UuPz4dJvbDiCx2L+rHksxl5ZlEUhUcx6LO6i T9lD6kcmB9nVXxQQwQ10nR+aK4uQGOltrvG50hUBJrJTUqn8zIjXQuyg9KaIj1LtpD8e gAwTHCikz4IKo20HAQ8RlGjpFA92Y3tEVE3grpJ0Cj0aabuwGO0XeXeOua8RcW5Du4pA nATg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=5BMmq1rhzP5+/n4dRcteCsuqqH1ge9r2cIrgxqEKClM=; b=x8OjhifiCn1Ike2kixHpjy5acddhBBqc51ZSN7KXc6IEogOdTDAq+O9wsxdA/R0q7h rXCy31hke2g2ltVGlbTCE66KnO63t9J4qohQH6Lwrg+F8gRGXGlckiLuS5qwJdK7arHV 1MUM+H6VQIkqoGp/QR8+/9+XVmt4T/2B+DDa7bkGvMn0pEDbw/BDQcjhyV5sjTO2WQLn Ogs/cTVH9n+AN4LRsc3tyQM9l+Gm4VMLvKrOcdbtUU3kgXFqr9QfuBxeQPo+zQkS5uT+ j4vJMHEskP8hEozGQT/1QNemaBstjazcEWydUw++pVCd7XnNoORLf1JVVbHEhTv3z0be R30g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ihKodDpY; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h7-20020a056402280700b00463ab1bf95bsi18898133ede.94.2022.11.09.08.14.23; Wed, 09 Nov 2022 08:14:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=ihKodDpY; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232167AbiKIQNG (ORCPT + 99 others); Wed, 9 Nov 2022 11:13:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232151AbiKIQMr (ORCPT ); Wed, 9 Nov 2022 11:12:47 -0500 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 12C6523E8F; Wed, 9 Nov 2022 08:12:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668010353; x=1699546353; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QhCzUoXOe6qmLG6Pf4Ep1QDjdcywTWaTd4/r+6EDODU=; b=ihKodDpY9JPEmtT4KZOPQXVqfc8u52eY4Ddj70IqK/T6QHn6TEQH+vK5 /BJ1yG/1ouq1dj+V+mzDs+Np7h6kywpP7yTXsmeUCFS+jQItglyyV293T /IV9Zq3no4+zxqkse3llm2cGQ9fldBKBpSHj2FWImpWTxtjAnZ+MWd6pg Unvdith2K+0NRBE/21atjDWiqegXSnaksyLFe4/UT3uoOZw0r93TIvlWi /ULr4CkqNku8VKA0qOQ8CPBXmLfYI1JfTbJmdGv7F0WF7f/k2S7bbo3B9 5gxLfVZK3s8GZhNQz2HCE2krKepwqjXqBTr8JLscdcXs/QaaDROTBhkrt w==; X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="312181433" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="312181433" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:32 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="811684566" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="811684566" Received: from smurnane-mobl.ger.corp.intel.com (HELO localhost.localdomain) ([10.213.196.238]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:27 -0800 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 09/13] drm/cgroup: Only track clients which are providing drm_cgroup_ops Date: Wed, 9 Nov 2022 16:11:37 +0000 Message-Id: <20221109161141.2987173-10-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> References: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749035766774257448?= X-GMAIL-MSGID: =?utf-8?q?1749035766774257448?= From: Tvrtko Ursulin To reduce the number of tracking going on, especially with drivers which will not support any sort of control from the drm cgroup controller side, lets express the funcionality as opt-in and use the presence of drm_cgroup_ops as activation criteria. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/drm_cgroup.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/drivers/gpu/drm/drm_cgroup.c b/drivers/gpu/drm/drm_cgroup.c index e3854741c584..d3050c744e3e 100644 --- a/drivers/gpu/drm/drm_cgroup.c +++ b/drivers/gpu/drm/drm_cgroup.c @@ -35,6 +35,9 @@ void drm_clients_close(struct drm_file *file_priv) lockdep_assert_held(&dev->filelist_mutex); + if (!dev->driver->cg_ops) + return; + pid = rcu_access_pointer(file_priv->pid); clients = xa_load(&drm_pid_clients, (unsigned long)pid); if (WARN_ON_ONCE(!clients)) @@ -66,6 +69,9 @@ int drm_clients_open(struct drm_file *file_priv) lockdep_assert_held(&dev->filelist_mutex); + if (!dev->driver->cg_ops) + return 0; + pid = (unsigned long)rcu_access_pointer(file_priv->pid); clients = xa_load(&drm_pid_clients, pid); if (!clients) { @@ -101,6 +107,9 @@ drm_clients_migrate(struct drm_file *file_priv, lockdep_assert_held(&dev->filelist_mutex); + if (!dev->driver->cg_ops) + return; + existing_clients = xa_load(&drm_pid_clients, new); clients = xa_load(&drm_pid_clients, old); From patchwork Wed Nov 9 16:11:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 17660 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp425308wru; Wed, 9 Nov 2022 08:15:13 -0800 (PST) X-Google-Smtp-Source: AMsMyM6fj/jZncQglD0eSGFVtegEquugusnTNu2SVwEqITVcoCl6TTa0U6E0imWkf04m8t0dL3Wl X-Received: by 2002:a17:902:d489:b0:187:2b1d:1101 with SMTP id c9-20020a170902d48900b001872b1d1101mr49950214plg.158.1668010513129; Wed, 09 Nov 2022 08:15:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668010513; cv=none; d=google.com; s=arc-20160816; b=o/lY7Sec1dUqR31FoSJrgCltAoIxbdDGeC6eeyCg5ISAKFl+d04eHZJxSHCCMYPp1z laS18gnef66GF5wP0MTp9i7e2G/kV2F3IQxxSELbbnMJosxzSApRQMrUp6W/KNypqEYu Q/pzRAxuQaCMiq7tuhwJHTeXWeM+LRxWrkVJWUg6linwr/nye21WmaFIaOrJad5kxKeV 2NQ8hQ0rc146H10ODXDbO03HxbjymD3cnEFqFHwjG4oAzz2ReHdJFsBsi1zJx3f/g7ED 81zga3PQctVNqtnA9SFgv7YNNNnAE8ge8vIGXOSSGgj1vl449V7lLfKWwfNzp32NvWuY YzSw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=YkKPPCbEBRE/kWEWtguueBKH+6EdPgrWKN1qODyjCmo=; b=aeoCxgRVLBw7vjMLItwI0LptKEY2imb+58+KngDeuB1OHclC66qG6QVR9HPcLZN/2r /r15JHUBuvKbZ0NnZh9gutGrwpzflfT/JmKJX7h3kTdxEEKQCV030Jw+Lfx+DDYGrT4k 0w1MXOzm6u4+FeoMR8WCyEeLjQDRlhav32g7mkJQ47zZcKl6SiLlv1E6EgakumBaoS60 lpHdp+UNbbNIHs8d1Z6FuvPvE5gCt0ne5BHC4kbyRmCmOcaoxfVerW+3rq8Hs76w24Cn 4YSPwsKSsk+yz7VK6kgakbZrHLV3fBB1NgkkSH3usB9vWsGziTGI70Hw0tkDRho2nN7B OFsQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Nur0QNLk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s17-20020a63d051000000b0046b15902616si17438878pgi.270.2022.11.09.08.14.57; Wed, 09 Nov 2022 08:15:13 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Nur0QNLk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231273AbiKIQNK (ORCPT + 99 others); Wed, 9 Nov 2022 11:13:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58872 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232157AbiKIQMt (ORCPT ); Wed, 9 Nov 2022 11:12:49 -0500 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CF3C823BFF; Wed, 9 Nov 2022 08:12:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668010354; x=1699546354; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xsH6SUJPl/aECTQLLwvLwzjjo/OdVfq3yYbEuLbpcjc=; b=Nur0QNLkADbBaix9OP9OgUxgd+zYPTvU/f9Kjk3FVRu7+1+udWPDCgFI PQ3D/h6jPpVCADRoq6w71mpWo49Re1qMzc4TU3k72KpNva1ga1YxtX6sy y49caY8V2ujcmuJv7ZV2RSp1om9UtNyAt7TK8dJSV1ZQ+0sDHrNKOBDeJ o+XC1j8bKRk/5lxvsRR5iJbqahEXncF4vAvQqwQAh9Z1vIMC26uKNlsXW l9ZVzp1B6tvqZTFbvwOD4nxbAO4vCBWJp4IIPoiXPL+VV1yN4I7tH8Vlw o0Xy2ONGkfz+e4A8VZfyM5NVJrr+3OgoCurHcgJ2ooUJXcxZipRNeaQin w==; X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="312181445" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="312181445" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:34 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="811684585" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="811684585" Received: from smurnane-mobl.ger.corp.intel.com (HELO localhost.localdomain) ([10.213.196.238]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:31 -0800 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 10/13] cgroup/drm: Client exit hook Date: Wed, 9 Nov 2022 16:11:38 +0000 Message-Id: <20221109161141.2987173-11-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> References: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749035792102569129?= X-GMAIL-MSGID: =?utf-8?q?1749035792102569129?= From: Tvrtko Ursulin We need the ability for DRM core to inform the cgroup controller when a client has closed a DRM file descriptor. This will allow us not needing to keep state relating to GPU time usage by tasks sets in the cgroup controller itself. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/drm_cgroup.c | 8 ++++++++ include/linux/cgroup_drm.h | 4 ++++ kernel/cgroup/drm.c | 13 +++++++++++++ 3 files changed, 25 insertions(+) diff --git a/drivers/gpu/drm/drm_cgroup.c b/drivers/gpu/drm/drm_cgroup.c index d3050c744e3e..31a1bf3c8a43 100644 --- a/drivers/gpu/drm/drm_cgroup.c +++ b/drivers/gpu/drm/drm_cgroup.c @@ -5,6 +5,7 @@ #include #include +#include static DEFINE_XARRAY(drm_pid_clients); @@ -31,6 +32,7 @@ void drm_clients_close(struct drm_file *file_priv) { struct drm_device *dev = file_priv->minor->dev; struct drm_pid_clients *clients; + struct task_struct *task; struct pid *pid; lockdep_assert_held(&dev->filelist_mutex); @@ -43,6 +45,12 @@ void drm_clients_close(struct drm_file *file_priv) if (WARN_ON_ONCE(!clients)) return; + task = get_pid_task(pid, PIDTYPE_PID); + if (task) { + drmcgroup_client_exited(task); + put_task_struct(task); + } + __del_clients(clients, file_priv, (unsigned long)pid); } diff --git a/include/linux/cgroup_drm.h b/include/linux/cgroup_drm.h index bf8abc6b8ebf..2f755b896136 100644 --- a/include/linux/cgroup_drm.h +++ b/include/linux/cgroup_drm.h @@ -6,4 +6,8 @@ #ifndef _CGROUP_DRM_H #define _CGROUP_DRM_H +struct task_struct; + +void drmcgroup_client_exited(struct task_struct *task); + #endif /* _CGROUP_DRM_H */ diff --git a/kernel/cgroup/drm.c b/kernel/cgroup/drm.c index b88c93661df3..e165d8f542cc 100644 --- a/kernel/cgroup/drm.c +++ b/kernel/cgroup/drm.c @@ -18,11 +18,24 @@ css_to_drmcs(struct cgroup_subsys_state *css) return container_of(css, struct drm_cgroup_state, css); } +static inline struct drm_cgroup_state *get_task_drmcs(struct task_struct *task) +{ + return css_to_drmcs(task_get_css(task, drm_cgrp_id)); +} + static void drmcs_free(struct cgroup_subsys_state *css) { kfree(css_to_drmcs(css)); } +void drmcgroup_client_exited(struct task_struct *task) +{ + struct drm_cgroup_state *drmcs = get_task_drmcs(task); + + css_put(&drmcs->css); +} +EXPORT_SYMBOL_GPL(drmcgroup_client_exited); + static struct drm_cgroup_state root_drmcs = { }; From patchwork Wed Nov 9 16:11:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 17661 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp425656wru; Wed, 9 Nov 2022 08:15:49 -0800 (PST) X-Google-Smtp-Source: AMsMyM5qvrwuXLKmDbOI88fIeze+vzCabBj1JPdoZyfSSu7Kukk5UCbNxL8qHR7mjUmdssKMQu0c X-Received: by 2002:a17:902:c950:b0:187:3da6:37ec with SMTP id i16-20020a170902c95000b001873da637ecmr42103276pla.139.1668010549203; Wed, 09 Nov 2022 08:15:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668010549; cv=none; d=google.com; s=arc-20160816; b=vCZ/bKHnbJuqVZlqIDYXN0W5NtRNiGyphS8FuNX/jqzyLtTa0BJn/EtaNLcLyNhw7w r1xS5e7KjKzhKbJOYKJQr3qjfv0VKqKJfbP2z5e7dNAQMLI05qAfrz9O6v9lcFDwAxbw VF4KyaVEdxsrPwZsIWRWVElt8e7UrGlDWOu9SKkLkVfCrgmX3S9FLevfCJVAFnYR7b+2 DlslZe4jJuJWKbfslYYg9zu1qB7sIg8AvTu5zUdpE1mb4lL5Gbg2uFHFdl2zuJDEQJmK CgReAKASJ7i7B4kReRMCX8bbPQlg9h//iafniE111/ikeYCiELQZq6cNDSP0oxVnYJ/+ A0Iw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=pUc5oSZgpXKwaipdFigkQt+h5HJPZ7MocpEEtdc84EM=; b=Z+j6okWqvKTiOBSBSu2SMeQRwoOgllQJcpd0PsxQiQ9S7RMpxBE4EERIU3eqYdzrN1 o1BDTshPVkkCOQVjmpriOLKPeNRPJcbAnshwziPs9HKeS2GlWumdrkb77eTN1osN+jk+ WOBhOZqHuuDm31zJCeq4k3sZS8Hau0cR6U49L5gJBuZKMZtXL0AXXCr3SuhIoj7Zalsd AhVIjDT4+S5ywC1thihcoqgjV8adm7VupMSNZZryp/nkM36YnHw4CEcNAaJWSOWJ2dsU Vx3LbviW0CvbGF7OhBln4RgN1Y3s4iep9S6o6DxC3esVEcrsc42WtxQeHdLyPeuXIV2T iR1Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Y1ZoNGzY; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q13-20020a056a00084d00b005637df79509si20429367pfk.197.2022.11.09.08.15.35; Wed, 09 Nov 2022 08:15:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Y1ZoNGzY; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232179AbiKIQNY (ORCPT + 99 others); Wed, 9 Nov 2022 11:13:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59602 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232181AbiKIQMt (ORCPT ); Wed, 9 Nov 2022 11:12:49 -0500 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7E5AA24BD5; Wed, 9 Nov 2022 08:12:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668010358; x=1699546358; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hC1EBIm+7jYNTOyjNm4NvKLmNZ3MeEatF+PTtOwzgNI=; b=Y1ZoNGzY6dEVo9pFOUYM+hPxsIXnCEqhhx+SjnchRRUyYKI10KBeY+x3 +NQdebRLBDRk5uAVVQuP+4inxI4nCtHDUmOua7RJOIjHpb/x3gf02LJvQ zQIviQf9Wpx78fYBQYS9J9himAblK84D/a7OYBuNgKIQjmYhMWNFun/zN npKTssi+oMYNQtVAQ0XUDqjv71mJgkTGpGWWhBZ1Q01M9UR5yqw0xc4Mu H85YR6nspv60lYCU/OEHU89q6K6vs1dDm6//ugmiuxdg6C+0Y7G1d/M9a jyaCTi8hlYAnhQGui3p16ro5hyzkEAuY7qMSWenCLY5lStLs4VBgwTqab g==; X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="312181468" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="312181468" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:38 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="811684593" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="811684593" Received: from smurnane-mobl.ger.corp.intel.com (HELO localhost.localdomain) ([10.213.196.238]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:34 -0800 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 11/13] cgroup/drm: Introduce weight based drm cgroup control Date: Wed, 9 Nov 2022 16:11:39 +0000 Message-Id: <20221109161141.2987173-12-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> References: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749035829569387758?= X-GMAIL-MSGID: =?utf-8?q?1749035829569387758?= From: Tvrtko Ursulin Similar to CPU scheduling, implement a concept of weight in the drm cgroup controller. Uses the same range and default as the CPU controller - CGROUP_WEIGHT_MIN, CGROUP_WEIGHT_DFL and CGROUP_WEIGHT_MAX. Later each cgroup is assigned a time budget proportionaly based on the relative weights of it's siblings. This time budget is in turn split by the group's children and so on. Children of the root cgroup will be exempt from split budgets and therefore compete for the GPU time independently and without weight based control. This will be used to implement a soft, or best effort signal from drm cgroup to drm core notifying about groups which are over their allotted budget. No guarantees that the limit can be enforced are provided or implied. Signed-off-by: Tvrtko Ursulin --- Documentation/admin-guide/cgroup-v2.rst | 37 ++ drivers/gpu/drm/Kconfig | 1 + init/Kconfig | 1 + kernel/cgroup/drm.c | 507 +++++++++++++++++++++++- 4 files changed, 545 insertions(+), 1 deletion(-) diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index dc254a3cb956..ab115fb4170e 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -2398,6 +2398,43 @@ HugeTLB Interface Files hugetlb pages of in this cgroup. Only active in use hugetlb pages are included. The per-node values are in bytes. +DRM +--- + +The DRM controller allows configuring scheduling soft limits. + +DRM scheduling soft limits +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Because of the heterogenous hardware and driver DRM capabilities, soft limits +are implemented as a loose co-operative (bi-directional) interface between the +controller and DRM core. + +The controller configures the GPU time allowed per group and periodically scans +the belonging tasks to detect the over budget condition, at which point it +invokes a callback notifying the DRM core of the condition. + +DRM core provides an API to query per process GPU utilization and 2nd API to +receive notification from the cgroup controller when the group enters or exits +the over budget condition. + +Individual DRM drivers which implement the interface are expected to act on this +in the best-effort manner only. There are no guarantees that the soft limits +will be respected. + +DRM scheduling soft limits interface files +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + drm.weight + Standard cgroup weight based control [1, 10000] used to configure the + relative distributing of GPU time between the sibling groups. + + drm.period_us (debugging aid during RFC only) + An integer representing the period with which the controller should look + at the GPU usage by the group and potentially send the over/under budget + signal. + Value of zero (defaul) disables the soft limit checking. + Misc ---- diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig index f30f99166531..e68a58237882 100644 --- a/drivers/gpu/drm/Kconfig +++ b/drivers/gpu/drm/Kconfig @@ -7,6 +7,7 @@ # menuconfig DRM tristate "Direct Rendering Manager (XFree86 4.1.0 and higher DRI support)" + default y if CGROUP_DRM=y depends on (AGP || AGP=n) && !EMULATED_CMPXCHG && HAS_DMA select DRM_NOMODESET select DRM_PANEL_ORIENTATION_QUIRKS diff --git a/init/Kconfig b/init/Kconfig index 70c08f340961..a39b005b9b8a 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1089,6 +1089,7 @@ config CGROUP_RDMA config CGROUP_DRM bool "DRM controller" + select DRM help Provides the DRM subsystem controller. diff --git a/kernel/cgroup/drm.c b/kernel/cgroup/drm.c index e165d8f542cc..c893c3738556 100644 --- a/kernel/cgroup/drm.c +++ b/kernel/cgroup/drm.c @@ -8,10 +8,38 @@ #include #include +#include + struct drm_cgroup_state { struct cgroup_subsys_state css; + + unsigned int weight; + unsigned int period_us; + + bool scanning_suspended; + unsigned int suspended_period_us; + + struct delayed_work scan_work; + + /* + * Below fields are owned and updated by the scan worker. Either the + * worker accesses them, or worker needs to be suspended and synced + * before they can be touched from the outside. + */ + ktime_t prev_timestamp; + + unsigned int sum_children_weights; + + u64 per_s_budget_ns; + u64 prev_active_us; + u64 active_us; + + bool over; + bool over_budget; }; +static DEFINE_MUTEX(drmcg_mutex); + static inline struct drm_cgroup_state * css_to_drmcs(struct cgroup_subsys_state *css) { @@ -23,20 +51,479 @@ static inline struct drm_cgroup_state *get_task_drmcs(struct task_struct *task) return css_to_drmcs(task_get_css(task, drm_cgrp_id)); } +static u64 drmcs_get_active_time_us(struct drm_cgroup_state *drmcs) +{ + struct cgroup *cgrp = drmcs->css.cgroup; + struct task_struct *task; + struct css_task_iter it; + u64 total = 0; + + css_task_iter_start(&cgrp->self, + CSS_TASK_ITER_PROCS | CSS_TASK_ITER_THREADED, + &it); + while ((task = css_task_iter_next(&it))) { + u64 time; + + /* Ignore kernel threads here. */ + if (task->flags & PF_KTHREAD) + continue; + + time = drm_pid_get_active_time_us(task_pid(task)); + total += time; + } + css_task_iter_end(&it); + + return total; +} + +static u64 +drmcs_read_weight(struct cgroup_subsys_state *css, struct cftype *cft) +{ + struct drm_cgroup_state *drmcs = css_to_drmcs(css); + + return drmcs->weight; +} + +static int +drmcs_write_weight(struct cgroup_subsys_state *css, struct cftype *cftype, + u64 weight) +{ + struct drm_cgroup_state *drmcs = css_to_drmcs(css); + int ret; + + if (weight < CGROUP_WEIGHT_MIN || weight > CGROUP_WEIGHT_MAX) + return -ERANGE; + + ret = mutex_lock_interruptible(&drmcg_mutex); + if (ret) + return ret; + drmcs->weight = weight; + mutex_unlock(&drmcg_mutex); + + return 0; +} + +static void +signal_drm_budget(struct drm_cgroup_state *drmcs, u64 usage, u64 budget) +{ + struct cgroup *cgrp = drmcs->css.cgroup; + struct task_struct *task; + struct css_task_iter it; + + css_task_iter_start(&cgrp->self, + CSS_TASK_ITER_PROCS | CSS_TASK_ITER_THREADED, + &it); + while ((task = css_task_iter_next(&it))) { + /* Ignore kernel threads here. */ + if (task->flags & PF_KTHREAD) + continue; + + drm_pid_signal_budget(task_pid(task), usage, budget); + } + css_task_iter_end(&it); +} + +static bool +__start_scanning(struct drm_cgroup_state *root, unsigned int period_us) +{ + struct cgroup_subsys_state *node; + bool ok = false; + + rcu_read_lock(); + + css_for_each_descendant_post(node, &root->css) { + struct drm_cgroup_state *drmcs = css_to_drmcs(node); + + if (!css_tryget_online(node)) + goto out; + + drmcs->active_us = 0; + drmcs->sum_children_weights = 0; + + if (node == &root->css) + drmcs->per_s_budget_ns = + DIV_ROUND_UP_ULL(NSEC_PER_SEC * period_us, + USEC_PER_SEC); + else + drmcs->per_s_budget_ns = 0; + + css_put(node); + } + + css_for_each_descendant_post(node, &root->css) { + struct drm_cgroup_state *drmcs = css_to_drmcs(node); + struct drm_cgroup_state *parent; + u64 active; + + if (!css_tryget_online(node)) + goto out; + if (!node->parent) { + css_put(node); + continue; + } + if (!css_tryget_online(node->parent)) { + css_put(node); + goto out; + } + parent = css_to_drmcs(node->parent); + + active = drmcs_get_active_time_us(drmcs); + if (active > drmcs->prev_active_us) + drmcs->active_us += active - drmcs->prev_active_us; + drmcs->prev_active_us = active; + + parent->active_us += drmcs->active_us; + parent->sum_children_weights += drmcs->weight; + + css_put(node); + css_put(&parent->css); + } + + ok = true; + +out: + rcu_read_unlock(); + + return ok; +} + +static void scan_worker(struct work_struct *work) +{ + struct drm_cgroup_state *root = + container_of(work, typeof(*root), scan_work.work); + struct cgroup_subsys_state *node; + unsigned int period_us; + ktime_t now; + + rcu_read_lock(); + + if (WARN_ON_ONCE(!css_tryget_online(&root->css))) { + rcu_read_unlock(); + return; + } + + now = ktime_get(); + period_us = ktime_to_us(ktime_sub(now, root->prev_timestamp)); + root->prev_timestamp = now; + + /* + * 1st pass - reset working values and update hierarchical weights and + * GPU utilisation. + */ + if (!__start_scanning(root, period_us)) + goto out_retry; /* + * Always come back later if scanner races with + * core cgroup management. (Repeated pattern.) + */ + + css_for_each_descendant_pre(node, &root->css) { + struct drm_cgroup_state *drmcs = css_to_drmcs(node); + struct cgroup_subsys_state *css; + unsigned int over_weights = 0; + u64 unused_ns = 0; + + if (!css_tryget_online(node)) + goto out_retry; + + /* + * 2nd pass - calculate initial budgets, mark over budget + * siblings and add up unused budget for the group. + */ + css_for_each_child(css, &drmcs->css) { + struct drm_cgroup_state *sibling = css_to_drmcs(css); + + if (!css_tryget_online(css)) { + css_put(node); + goto out_retry; + } + + sibling->per_s_budget_ns = + DIV_ROUND_UP_ULL(drmcs->per_s_budget_ns * + sibling->weight, + drmcs->sum_children_weights); + + sibling->over = sibling->active_us * NSEC_PER_USEC > + sibling->per_s_budget_ns; + if (sibling->over) + over_weights += sibling->weight; + else + unused_ns += sibling->per_s_budget_ns - + sibling->active_us; + + css_put(css); + } + + /* + * 3rd pass - spread unused budget according to relative weights + * of over budget siblings. + */ + css_for_each_child(css, &drmcs->css) { + struct drm_cgroup_state *sibling = css_to_drmcs(css); + + if (!css_tryget_online(css)) { + css_put(node); + goto out_retry; + } + + if (sibling->over) { + u64 budget_ns = + DIV_ROUND_UP_ULL(unused_ns * + sibling->weight, + over_weights); + sibling->per_s_budget_ns += budget_ns; + sibling->over = + sibling->active_us * NSEC_PER_USEC > + sibling->per_s_budget_ns; + } + + css_put(css); + } + + css_put(node); + } + + /* + * 4th pass - send out over/under budget notifications. + */ + css_for_each_descendant_post(node, &root->css) { + struct drm_cgroup_state *drmcs = css_to_drmcs(node); + + if (!css_tryget_online(node)) + goto out_retry; + + if (drmcs->over || drmcs->over_budget) + signal_drm_budget(drmcs, + drmcs->active_us * NSEC_PER_USEC, + drmcs->per_s_budget_ns); + drmcs->over_budget = drmcs->over; + + css_put(node); + } + +out_retry: + rcu_read_unlock(); + + period_us = READ_ONCE(root->period_us); + if (period_us) + schedule_delayed_work(&root->scan_work, + usecs_to_jiffies(period_us)); + + css_put(&root->css); +} + +static void start_scanning(struct drm_cgroup_state *drmcs, u64 period_us) +{ + drmcs->period_us = (unsigned int)period_us; + WARN_ON_ONCE(!__start_scanning(drmcs, 0)); + drmcs->prev_timestamp = ktime_get(); + mod_delayed_work(system_wq, &drmcs->scan_work, + usecs_to_jiffies(period_us)); +} + +static void stop_scanning(struct drm_cgroup_state *drmcs) +{ + drmcs->period_us = 0; + cancel_delayed_work_sync(&drmcs->scan_work); + if (drmcs->over_budget) { + /* + * Signal under budget when scanning goes off so drivers + * correctly update their state. + */ + signal_drm_budget(drmcs, 0, drmcs->per_s_budget_ns); + drmcs->over_budget = false; + } +} + +static struct drm_cgroup_state *drmcs_scanner(struct drm_cgroup_state *drmcs) +{ + while (drmcs->css.cgroup->level > 1) + drmcs = css_to_drmcs(drmcs->css.parent); + + return drmcs; +} + +static void start_suspend_scanning(struct drm_cgroup_state *drmcs) +{ + drmcs = drmcs_scanner(drmcs); + + if (drmcs->scanning_suspended) + return; + + drmcs->scanning_suspended = true; + drmcs->suspended_period_us = drmcs->period_us; + drmcs->period_us = 0; +} + +static void finish_suspend_scanning(struct drm_cgroup_state *drmcs) +{ + drmcs = drmcs_scanner(drmcs); + + if (drmcs->suspended_period_us) + cancel_delayed_work_sync(&drmcs->scan_work); +} + +static void resume_scanning(struct drm_cgroup_state *drmcs) +{ + drmcs = drmcs_scanner(drmcs); + + if (!drmcs->scanning_suspended) + return; + + drmcs->scanning_suspended = false; + if (drmcs->suspended_period_us) { + start_scanning(drmcs, drmcs->suspended_period_us); + drmcs->suspended_period_us = 0; + } +} + static void drmcs_free(struct cgroup_subsys_state *css) { - kfree(css_to_drmcs(css)); + struct drm_cgroup_state *drmcs = css_to_drmcs(css); + + stop_scanning(drmcs); + + kfree(drmcs); +} + +static int drmcs_can_attach(struct cgroup_taskset *tset) +{ + struct cgroup_subsys_state *new_css; + struct task_struct *task; + int ret; + + /* + * As processes are getting moved between groups we need to ensure + * both that the old group does not see a sudden downward jump in the + * GPU utilisation, and that the new group does not see a sudden jump + * up with all the GPU time clients belonging to the migrated process + * have accumulated. + * + * To achieve that we suspend the scanner until the migration is + * completed where the resume at the end ensures both groups start + * observing GPU utilisation from a reset state. + */ + + ret = mutex_lock_interruptible(&drmcg_mutex); + if (ret) + return ret; + + cgroup_taskset_for_each(task, new_css, tset) { + start_suspend_scanning(css_to_drmcs(task_css(task, + drm_cgrp_id))); + start_suspend_scanning(css_to_drmcs(new_css)); + } + + mutex_unlock(&drmcg_mutex); + + cgroup_taskset_for_each(task, new_css, tset) { + finish_suspend_scanning(css_to_drmcs(task_css(task, + drm_cgrp_id))); + finish_suspend_scanning(css_to_drmcs(new_css)); + } + + return 0; +} + +static void tset_resume_scanning(struct cgroup_taskset *tset) +{ + struct cgroup_subsys_state *new_css; + struct task_struct *task; + + mutex_lock(&drmcg_mutex); + cgroup_taskset_for_each(task, new_css, tset) { + resume_scanning(css_to_drmcs(task_css(task, drm_cgrp_id))); + resume_scanning(css_to_drmcs(new_css)); + } + mutex_unlock(&drmcg_mutex); +} + +static void drmcs_attach(struct cgroup_taskset *tset) +{ + tset_resume_scanning(tset); +} + +static void drmcs_cancel_attach(struct cgroup_taskset *tset) +{ + tset_resume_scanning(tset); +} + +static u64 +drmcs_read_period_us(struct cgroup_subsys_state *css, struct cftype *cft) +{ + struct drm_cgroup_state *drmcs = css_to_drmcs(css); + + return drmcs->period_us; +} + +static int +drmcs_write_period_us(struct cgroup_subsys_state *css, struct cftype *cftype, + u64 period_us) +{ + struct drm_cgroup_state *drmcs = css_to_drmcs(css); + int ret; + + if (WARN_ON_ONCE(!css->parent)) + return -EINVAL; + if (css->cgroup->level != 1) + return -EINVAL; + if ((period_us && period_us < 500000) || period_us > USEC_PER_SEC * 60) + return -EINVAL; + + ret = mutex_lock_interruptible(&drmcg_mutex); + if (ret) + return ret; + + if (!drmcs->scanning_suspended) { + if (period_us) + start_scanning(drmcs, period_us); + else + stop_scanning(drmcs); + } else { + /* + * If scanning is temporarily suspended just update the period + * which will apply once resumed, or simply skip resuming in + * case of disabling. + */ + drmcs->suspended_period_us = period_us; + if (!period_us) + drmcs->scanning_suspended = false; + } + + mutex_unlock(&drmcg_mutex); + + return 0; } void drmcgroup_client_exited(struct task_struct *task) { struct drm_cgroup_state *drmcs = get_task_drmcs(task); + /* + * Since we are not tracking accumulated GPU time for each cgroup, + * avoid jumps in group observed GPU usage by re-setting the scanner + * at a point when GPU usage can suddenly jump down. + * + * Downside is clients can influence the effectiveness of the over- + * budget scanning by continuously closing DRM file descriptors but for + * now we do not worry about it. + */ + + mutex_lock(&drmcg_mutex); + start_suspend_scanning(drmcs); + mutex_unlock(&drmcg_mutex); + + finish_suspend_scanning(drmcs); + + mutex_lock(&drmcg_mutex); + resume_scanning(drmcs); + mutex_unlock(&drmcg_mutex); + css_put(&drmcs->css); } EXPORT_SYMBOL_GPL(drmcgroup_client_exited); static struct drm_cgroup_state root_drmcs = { + .weight = CGROUP_WEIGHT_DFL, }; static struct cgroup_subsys_state * @@ -51,16 +538,34 @@ drmcs_alloc(struct cgroup_subsys_state *parent_css) if (!drmcs) return ERR_PTR(-ENOMEM); + drmcs->weight = CGROUP_WEIGHT_DFL; + INIT_DELAYED_WORK(&drmcs->scan_work, scan_worker); + return &drmcs->css; } struct cftype files[] = { + { + .name = "weight", + .flags = CFTYPE_NOT_ON_ROOT, + .read_u64 = drmcs_read_weight, + .write_u64 = drmcs_write_weight, + }, + { + .name = "period_us", + .flags = CFTYPE_NOT_ON_ROOT, + .read_u64 = drmcs_read_period_us, + .write_u64 = drmcs_write_period_us, + }, { } /* Zero entry terminates. */ }; struct cgroup_subsys drm_cgrp_subsys = { .css_alloc = drmcs_alloc, .css_free = drmcs_free, + .can_attach = drmcs_can_attach, + .attach = drmcs_attach, + .cancel_attach = drmcs_cancel_attach, .early_init = false, .legacy_cftypes = files, .dfl_cftypes = files, From patchwork Wed Nov 9 16:11:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 17665 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp438954wru; Wed, 9 Nov 2022 08:38:47 -0800 (PST) X-Google-Smtp-Source: AA0mqf5N+Gm3Jy3z3zJuDwY+rYEQMJl+7uUDdO6wy9tS8BWnens3mfrdogHI0vS0YqP4eT3bi0nf X-Received: by 2002:a17:907:6087:b0:7ad:e132:bb86 with SMTP id ht7-20020a170907608700b007ade132bb86mr4346767ejc.408.1668011926782; Wed, 09 Nov 2022 08:38:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668011926; cv=none; d=google.com; s=arc-20160816; b=eqkdY4WWVESp+LtIn6/gWsBCExCVgd/5IA1uIeDZ011nQnLfKBwq4FH24VjUxwvSEF IYmZ13/OlR+xgNHjVUjrci/7x4XziRw6yRmuoexylf69qVG1zNdC6yYj1CZHDzWsZvwU /rw2QBvkRKIAsjbuD7HA3ITX5fJT+iKdNxoQAsGejX/ySwUs690mk8US1gP8RuboZxqR ldostqXkOYQXD69m3sL3riFej4S1bxe7IP2hZd+9qPIEkYBgavMVReoIskYoxC3TbE60 ZZDdWw6loG7s39KVCBxC20YIRiWyuiSCF4YlHDwYuWXQqdrinMdZmlmBYUTOe5ADkwAa xtSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=mBbkZxiSWYsSj7xBwfHgUxxHik/rUkCzttJ2tKz9c6c=; b=ca95n5aUu30icnVDH9PTqcRzxc4/bGfLk47NMJ7R3GZKs3Tq4gNWrpz59iOJKjtTJg INQGQS8v4sLcaBuDI+N3bYKRPc8IP3o9ThKG6HkGStWBivrqUCZ08OFsz934YRFNq3Uq 7DXc+8p6s6DIw1dRHzYglAg395WenUFBZgEyzZUyGr87vSOaUVBa5vq+oNgg2qEpwqUb MhT2k3WdfeDFqMJ4B+hg7w6yfATqaL+7rBhGKUsPxNajW3bzH9yDk59WLslPEkhuiAAV CI9vRCwxBqmTcVgvrCZ1JHQ1ttmICIBktIJXqKyQaP7wMachdXTYSG8lxzN4cXs0Ncgh GwGw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="Y9/O9zHE"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id hs36-20020a1709073ea400b0078d9c0e8971si16724807ejc.752.2022.11.09.08.38.22; Wed, 09 Nov 2022 08:38:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="Y9/O9zHE"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232129AbiKIQNa (ORCPT + 99 others); Wed, 9 Nov 2022 11:13:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59130 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231707AbiKIQMw (ORCPT ); Wed, 9 Nov 2022 11:12:52 -0500 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 32489252A4; Wed, 9 Nov 2022 08:12:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668010363; x=1699546363; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=a7D9TOFF4QZXnog+7lQ4amXbM/ZFMku48rQ12ekKnqE=; b=Y9/O9zHEv0YDnZiVgHF+fitrsC+2nBK4bcbXZLcNlMRatha3B9RR4JhX mjz3H3c87B4ybfRbKLEo57QwKeL0ORdK0bRiNbj3rRt0gZnKwEHdcxoRG bpHJEtCTu7u+nbhweFAL0vkmZpb9FDx7ESfcSMSsduoDebwE6sDAevGrK 25k4btrvz6q/MGeXZpOizCB8dJTn8uE7BejzRL8g4Fmk9ZA4S/j1i9yWl 2p8/P6YM9ruI0BAGGAISs+7imyMlRLkUOOZKeZ5Flf30jdWkzbOIF7MbX rJK8iRqdpRzUWM0o+SqoBAC2PBVujxY2hIRKKtHXTh/+6PBX261klMBOg Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="312181477" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="312181477" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:43 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="811684604" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="811684604" Received: from smurnane-mobl.ger.corp.intel.com (HELO localhost.localdomain) ([10.213.196.238]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:38 -0800 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 12/13] drm/i915: Wire up with drm controller GPU time query Date: Wed, 9 Nov 2022 16:11:40 +0000 Message-Id: <20221109161141.2987173-13-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> References: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749037274062287735?= X-GMAIL-MSGID: =?utf-8?q?1749037274062287735?= From: Tvrtko Ursulin Implement the drm_cgroup_ops->active_time_us callback. Signed-off-by: Tvrtko Ursulin --- drivers/gpu/drm/i915/i915_driver.c | 10 ++++ drivers/gpu/drm/i915/i915_drm_client.c | 76 ++++++++++++++++++++++---- drivers/gpu/drm/i915/i915_drm_client.h | 2 + 3 files changed, 78 insertions(+), 10 deletions(-) diff --git a/drivers/gpu/drm/i915/i915_driver.c b/drivers/gpu/drm/i915/i915_driver.c index c3d43f9b1e45..96a7da5258c4 100644 --- a/drivers/gpu/drm/i915/i915_driver.c +++ b/drivers/gpu/drm/i915/i915_driver.c @@ -1894,6 +1894,12 @@ static const struct drm_ioctl_desc i915_ioctls[] = { DRM_IOCTL_DEF_DRV(I915_GEM_VM_DESTROY, i915_gem_vm_destroy_ioctl, DRM_RENDER_ALLOW), }; +#ifdef CONFIG_CGROUP_DRM +static const struct drm_cgroup_ops i915_drm_cgroup_ops = { + .active_time_us = i915_drm_cgroup_get_active_time_us, +}; +#endif + /* * Interface history: * @@ -1922,6 +1928,10 @@ static const struct drm_driver i915_drm_driver = { .lastclose = i915_driver_lastclose, .postclose = i915_driver_postclose, +#ifdef CONFIG_CGROUP_DRM + .cg_ops = &i915_drm_cgroup_ops, +#endif + .prime_handle_to_fd = drm_gem_prime_handle_to_fd, .prime_fd_to_handle = drm_gem_prime_fd_to_handle, .gem_prime_import = i915_gem_prime_import, diff --git a/drivers/gpu/drm/i915/i915_drm_client.c b/drivers/gpu/drm/i915/i915_drm_client.c index b09d1d386574..c9754cb0277f 100644 --- a/drivers/gpu/drm/i915/i915_drm_client.c +++ b/drivers/gpu/drm/i915/i915_drm_client.c @@ -75,7 +75,7 @@ void i915_drm_clients_fini(struct i915_drm_clients *clients) xa_destroy(&clients->xarray); } -#ifdef CONFIG_PROC_FS +#if defined(CONFIG_PROC_FS) || defined(CONFIG_CGROUP_DRM) static const char * const uabi_class_names[] = { [I915_ENGINE_CLASS_RENDER] = "render", [I915_ENGINE_CLASS_COPY] = "copy", @@ -100,22 +100,78 @@ static u64 busy_add(struct i915_gem_context *ctx, unsigned int class) return total; } -static void -show_client_class(struct seq_file *m, - struct i915_drm_client *client, - unsigned int class) +static u64 get_class_active_ns(struct i915_drm_client *client, + unsigned int class, + unsigned int *capacity) { - const struct list_head *list = &client->ctx_list; - u64 total = atomic64_read(&client->past_runtime[class]); - const unsigned int capacity = - client->clients->i915->engine_uabi_class_count[class]; struct i915_gem_context *ctx; + u64 total; + + *capacity = + client->clients->i915->engine_uabi_class_count[class]; + if (!*capacity) + return 0; + + total = atomic64_read(&client->past_runtime[class]); rcu_read_lock(); - list_for_each_entry_rcu(ctx, list, client_link) + list_for_each_entry_rcu(ctx, &client->ctx_list, client_link) total += busy_add(ctx, class); rcu_read_unlock(); + return total; +} +#endif + +#ifdef CONFIG_CGROUP_DRM +static bool supports_stats(struct drm_i915_private *i915) +{ + if (GRAPHICS_VER(i915) < 8) + return false; + + /* temporary... */ + if (intel_uc_uses_guc_submission(&to_gt(i915)->uc)) + return false; + + return true; +} + +u64 i915_drm_cgroup_get_active_time_us(struct drm_file *file) +{ + struct drm_i915_file_private *fpriv = file->driver_priv; + struct i915_drm_client *client = fpriv->client; + unsigned int i; + u64 busy = 0; + + if (!supports_stats(client->clients->i915)) + return 0; + + for (i = 0; i < ARRAY_SIZE(uabi_class_names); i++) { + unsigned int capacity; + u64 b; + + b = get_class_active_ns(client, i, &capacity); + if (capacity) { + b = DIV_ROUND_UP_ULL(b, capacity * 1000); + busy += b; + } + } + + return busy; +} +#endif + +#ifdef CONFIG_PROC_FS +static void +show_client_class(struct seq_file *m, + struct i915_drm_client *client, + unsigned int class) +{ + unsigned int capacity; + u64 total; + + total = get_class_active_ns(client, class, &capacity); + if (capacity) seq_printf(m, "drm-engine-%s:\t%llu ns\n", uabi_class_names[class], total); diff --git a/drivers/gpu/drm/i915/i915_drm_client.h b/drivers/gpu/drm/i915/i915_drm_client.h index 69496af996d9..c8439eaa89be 100644 --- a/drivers/gpu/drm/i915/i915_drm_client.h +++ b/drivers/gpu/drm/i915/i915_drm_client.h @@ -65,4 +65,6 @@ void i915_drm_client_fdinfo(struct seq_file *m, struct file *f); void i915_drm_clients_fini(struct i915_drm_clients *clients); +u64 i915_drm_cgroup_get_active_time_us(struct drm_file *file); + #endif /* !__I915_DRM_CLIENT_H__ */ From patchwork Wed Nov 9 16:11:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tvrtko Ursulin X-Patchwork-Id: 17663 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp432222wru; Wed, 9 Nov 2022 08:27:44 -0800 (PST) X-Google-Smtp-Source: AMsMyM5HhABzwljihX1MpkqnEa5NHOQmedTKnsvkCXnlacV8Ec1VClTipmmbkYAhZLKG64iMTz55 X-Received: by 2002:a17:902:ba8d:b0:187:11c7:e878 with SMTP id k13-20020a170902ba8d00b0018711c7e878mr55905806pls.67.1668011264084; Wed, 09 Nov 2022 08:27:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668011264; cv=none; d=google.com; s=arc-20160816; b=smtyV6viscYEs8c35nNXW+aaF/tzl3PqouKq3sexQZALq9kwxwkoDcOgScvgsw/yl1 j2cmxfXexz4Z1oKYze20knAaCED/5+/LY8c11CVpF5DpGQMHAsACyBujm++swXN4pwOD P0ro8SB0A3sk+FWafeKBK8keUu00kqpgSzbLiPb5On4DrjGmQGpHBa2S/H7E8GW5YuIR cc/aFYhtSXkrajNuUK38DSw7Q5CU4sLMVNnNMG6MATFf1BJghLnTueF+r14n5wywuBED LXwp/2Lm0Kv0ZAE2yAoTQLK+/9oab0k3GSkhjxsqmTm+tnbh0AZvNx4YGLEEtsn5E+I2 f4QA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=M0HKEWlCp+86nck9EJ+TgZ1HyoQhV5CqXkdne1ULo0Q=; b=gqjNje+gaUoRnO3w8HjTAth1blDA0CJ+1wxV5bYnx3LohfJ9YGSPegzvW5DA7CpEjw lZyKBFkFImq4uhMcUJAZAw2pfxiTNnTDdHA+7AV2jP4JX3EjMg24Qqja4kCAizchxb4T nNkXJc7VRl/Ci+8HHyH6mkZ+r8E/QPf9ZVL84hROCMrYsqAuZYGVuEFOLMZW5eudyPAb L4eWA2LFwQgGCyw140ebkC5LEG++gCOVhZWVLyb9CaDWaEZP/sPXeZTzC6IHyTifLHnF Swo78g1xQpAWR0hl4+xYu9UzqaO9Fd/9UBfnwBGYSNAEQFDKJr28JD2zoMPl5NeBSQwS PWLA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=natJ81G2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id lp15-20020a17090b4a8f00b0021414ac5caasi2084593pjb.124.2022.11.09.08.27.30; Wed, 09 Nov 2022 08:27:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=natJ81G2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232228AbiKIQNf (ORCPT + 99 others); Wed, 9 Nov 2022 11:13:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59166 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231899AbiKIQMw (ORCPT ); Wed, 9 Nov 2022 11:12:52 -0500 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BB2AF25C5D; Wed, 9 Nov 2022 08:12:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668010365; x=1699546365; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=x6XVT4VrLhMh4IJBu+3B412tbpXC3M4uZJfhwf8sKQM=; b=natJ81G2NaVCBnu9icWN+86xLgvJFJDDMfk1uB/ck7eZaTex0aUl2TZN 2FjlkKdmvRcrV6PU6Mp7R7xS/Mpzt2LkidI6HuqQZLZCIk3ooKM/nPkiu S6JBUoMWAWN5NbGRDWUpAAdw+5xwZLwbhZK1NvL0QCDVxhF0RM2mDiQIb TUetC6TGjLHg2aDJLql0YVP4CpWsrulvFh52fE8pXGZ65s3YFdYcvRcRy ktFrFwmW+isM9EtrFAtws66M5oQgiLFzdlqiVg7KjLvOJlLSYOXAnyes2 LZtCmQZPQPRStnVod08Chn80uuAN4IQiGMV/3uXK9fYtdrluASpk2AkS7 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="312181498" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="312181498" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:45 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="811684610" X-IronPort-AV: E=Sophos;i="5.96,151,1665471600"; d="scan'208";a="811684610" Received: from smurnane-mobl.ger.corp.intel.com (HELO localhost.localdomain) ([10.213.196.238]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Nov 2022 08:12:41 -0800 From: Tvrtko Ursulin To: Intel-gfx@lists.freedesktop.org Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo , Johannes Weiner , Zefan Li , Dave Airlie , Daniel Vetter , Rob Clark , =?utf-8?q?St=C3=A9phane_Marchesin?= , "T . J . Mercier" , Kenny.Ho@amd.com, =?utf-8?q?Chris?= =?utf-8?q?tian_K=C3=B6nig?= , Brian Welty , Tvrtko Ursulin Subject: [RFC 13/13] drm/i915: Implement cgroup controller over budget throttling Date: Wed, 9 Nov 2022 16:11:41 +0000 Message-Id: <20221109161141.2987173-14-tvrtko.ursulin@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> References: <20221109161141.2987173-1-tvrtko.ursulin@linux.intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HK_RANDOM_ENVFROM,HK_RANDOM_FROM, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE, SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749036579466100383?= X-GMAIL-MSGID: =?utf-8?q?1749036579466100383?= From: Tvrtko Ursulin When notified by the drm core we are over our allotted time budget, i915 instance will check if any of the GPU engines it is reponsible for is fully saturated. If it is, and the client in question is using that engine, it will throttle it. For now throttling is done simplistically by lowering the scheduling priority while client is throttled. Signed-off-by: Tvrtko Ursulin --- .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 37 +++++++- drivers/gpu/drm/i915/i915_driver.c | 1 + drivers/gpu/drm/i915/i915_drm_client.c | 93 +++++++++++++++++++ drivers/gpu/drm/i915/i915_drm_client.h | 11 +++ 4 files changed, 141 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c index 1160723c9d2d..280ed90d5001 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c @@ -6,6 +6,7 @@ #include #include +#include #include #include @@ -3015,6 +3016,40 @@ static void retire_requests(struct intel_timeline *tl, struct i915_request *end) break; } +#ifdef CONFIG_CGROUP_DRM +static unsigned int +__get_class(struct drm_i915_file_private *fpriv, const struct i915_request *rq) +{ + unsigned int class; + + class = rq->context->engine->uabi_class; + + if (WARN_ON_ONCE(class >= ARRAY_SIZE(fpriv->client->throttle))) + class = 0; + + return class; +} + +static void copy_priority(struct i915_sched_attr *attr, + const struct i915_execbuffer *eb, + const struct i915_request *rq) +{ + struct drm_i915_file_private *file_priv = eb->file->driver_priv; + + *attr = eb->gem_context->sched; + if (file_priv->client->throttle[__get_class(file_priv, rq)]) + attr->priority -= + 1 + prandom_u32_max(-I915_CONTEXT_MIN_USER_PRIORITY / 2); +} +#else +static void copy_priority(struct i915_sched_attr *attr, + const struct i915_execbuffer *eb, + const struct i915_request *rq) +{ + *attr = eb->gem_context->sched; +} +#endif + static int eb_request_add(struct i915_execbuffer *eb, struct i915_request *rq, int err, bool last_parallel) { @@ -3031,7 +3066,7 @@ static int eb_request_add(struct i915_execbuffer *eb, struct i915_request *rq, /* Check that the context wasn't destroyed before submission */ if (likely(!intel_context_is_closed(eb->context))) { - attr = eb->gem_context->sched; + copy_priority(&attr, eb, rq); } else { /* Serialise with context_close via the add_to_timeline */ i915_request_set_error_once(rq, -ENOENT); diff --git a/drivers/gpu/drm/i915/i915_driver.c b/drivers/gpu/drm/i915/i915_driver.c index 96a7da5258c4..d6c0501af3f4 100644 --- a/drivers/gpu/drm/i915/i915_driver.c +++ b/drivers/gpu/drm/i915/i915_driver.c @@ -1897,6 +1897,7 @@ static const struct drm_ioctl_desc i915_ioctls[] = { #ifdef CONFIG_CGROUP_DRM static const struct drm_cgroup_ops i915_drm_cgroup_ops = { .active_time_us = i915_drm_cgroup_get_active_time_us, + .signal_budget = i915_drm_cgroup_signal_budget, }; #endif diff --git a/drivers/gpu/drm/i915/i915_drm_client.c b/drivers/gpu/drm/i915/i915_drm_client.c index c9754cb0277f..81144c8e8d05 100644 --- a/drivers/gpu/drm/i915/i915_drm_client.c +++ b/drivers/gpu/drm/i915/i915_drm_client.c @@ -4,6 +4,7 @@ */ #include +#include #include #include @@ -159,6 +160,98 @@ u64 i915_drm_cgroup_get_active_time_us(struct drm_file *file) return busy; } + +int i915_drm_cgroup_signal_budget(struct drm_file *file, u64 usage, u64 budget) +{ + struct drm_i915_file_private *fpriv = file->driver_priv; + u64 class_usage[I915_LAST_UABI_ENGINE_CLASS + 1]; + u64 class_last[I915_LAST_UABI_ENGINE_CLASS + 1]; + struct drm_i915_private *i915 = fpriv->dev_priv; + struct i915_drm_client *client = fpriv->client; + struct intel_engine_cs *engine; + bool over = usage > budget; + unsigned int i; + ktime_t unused; + int ret = 0; + u64 t; + + if (!supports_stats(i915)) + return -EINVAL; + + if (usage == 0 && budget == 0) + return 0; + +printk("i915_drm_cgroup_signal_budget client-id=%u over=%u (%llu/%llu) <%u>\n", + client->id, over, usage, budget, client->over_budget); + + if (over) { + client->over_budget++; + if (!client->over_budget) + client->over_budget = 2; + } else { + client->over_budget = 0; + memset(client->class_last, 0, sizeof(client->class_last)); + memset(client->throttle, 0, sizeof(client->throttle)); + return 0; + } + + memset(class_usage, 0, sizeof(class_usage)); + for_each_uabi_engine(engine, i915) + class_usage[engine->uabi_class] += + ktime_to_ns(intel_engine_get_busy_time(engine, &unused)); + + memcpy(class_last, client->class_last, sizeof(class_last)); + memcpy(client->class_last, class_usage, sizeof(class_last)); + + for (i = 0; i < ARRAY_SIZE(uabi_class_names); i++) + class_usage[i] -= class_last[i]; + + t = client->last; + client->last = ktime_get_raw_ns(); + t = client->last - t; + + if (client->over_budget == 1) + return 0; + + for (i = 0; i < ARRAY_SIZE(uabi_class_names); i++) { + u64 client_class_usage[I915_LAST_UABI_ENGINE_CLASS + 1]; + unsigned int capacity; + + if (!i915->engine_uabi_class_count[i]) + continue; + + t = DIV_ROUND_UP_ULL(t, 1000); + class_usage[i] = DIV_ROUND_CLOSEST_ULL(class_usage[i], 1000); + usage = DIV_ROUND_CLOSEST_ULL(class_usage[i] * 100ULL, + t * + i915->engine_uabi_class_count[i]); + if (usage <= 95) { + /* class not oversubsribed */ + if (client->throttle[i]) { + client->throttle[i] = false; +printk(" UN-throttling class%u (phys=%lld%%)\n", + i, usage); + } + continue; + } + + client_class_usage[i] = + get_class_active_ns(client, i, &capacity); + + if (client_class_usage[i] && !client->throttle[i]) { + ret |= 1; + client->throttle[i] = true; + /* + * QQQ maybe apply "strength" of throttling based on + * usage/budget? + */ +printk(" THROTTLING class%u (phys=%lld%% client=%lluus)\n", + i, usage, client_class_usage[i] / 1000); + } + } + + return ret; +} #endif #ifdef CONFIG_PROC_FS diff --git a/drivers/gpu/drm/i915/i915_drm_client.h b/drivers/gpu/drm/i915/i915_drm_client.h index c8439eaa89be..1331cab7ad5e 100644 --- a/drivers/gpu/drm/i915/i915_drm_client.h +++ b/drivers/gpu/drm/i915/i915_drm_client.h @@ -15,6 +15,8 @@ #define I915_LAST_UABI_ENGINE_CLASS I915_ENGINE_CLASS_COMPUTE +struct drm_file; + struct drm_i915_private; struct i915_drm_clients { @@ -38,6 +40,13 @@ struct i915_drm_client { * @past_runtime: Accumulation of pphwsp runtimes from closed contexts. */ atomic64_t past_runtime[I915_LAST_UABI_ENGINE_CLASS + 1]; + +#ifdef CONFIG_CGROUP_DRM + bool throttle[I915_LAST_UABI_ENGINE_CLASS + 1]; + unsigned int over_budget; + u64 last; + u64 class_last[I915_LAST_UABI_ENGINE_CLASS + 1]; +#endif }; void i915_drm_clients_init(struct i915_drm_clients *clients, @@ -66,5 +75,7 @@ void i915_drm_client_fdinfo(struct seq_file *m, struct file *f); void i915_drm_clients_fini(struct i915_drm_clients *clients); u64 i915_drm_cgroup_get_active_time_us(struct drm_file *file); +int i915_drm_cgroup_signal_budget(struct drm_file *file, + u64 usage, u64 budget); #endif /* !__I915_DRM_CLIENT_H__ */