From patchwork Thu Feb 29 02:14:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 208164 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2097:b0:108:e6aa:91d0 with SMTP id gs23csp129857dyb; Wed, 28 Feb 2024 18:15:36 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXB+JW/UKVMWvA2+M1+vLP+udR3lj6op12TpW87FZq19vZPN/O0M2xxWT/SxLNAQIvP0S95ynkxnL1LpEa1DGjqsdJWZw== X-Google-Smtp-Source: AGHT+IEBK6e+xK+mARce+vu4Jn17RWAjzjeLw+OQ056+l71dibGE/4MVdrPAHr65M23qdOUKsBKw X-Received: by 2002:a17:903:2810:b0:1dc:c161:bce6 with SMTP id kp16-20020a170903281000b001dcc161bce6mr666357plb.15.1709172936122; Wed, 28 Feb 2024 18:15:36 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1709172936; cv=pass; d=google.com; s=arc-20160816; b=PGhM1WYCkXo7/w2/Pb8hO+5Qr8cvp+3KDkm70nx6bCSPHIO+ExeXY0P6ix2MbiRlS9 a3GV/0SGG1AbxXS/jEMTFgYZIvAYFmR8hZkpKMo10f+i6RoZ1MqkYfqH96Nl+AhTELGf 2z1A/wjHeEaQ/afDfneNZsI7jHVMg5rpkciW4wIs69aemhQH2dBoSWmDAzSLxOjkyJG1 mvSgTcOLFYytWk0f+tygbIad641H+MPZBLKET8dmCnxSiJ9WV8Au71uTuxVAJGolMmxP 9tJtjPlzSqCfOYPPPI5Mz4m5hRCBq7aIQAgkk+r00cumOQBDYzNJq65dl0gFw8RCxkk9 BU8A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=Iw2m/lgdJeZzD7aZQt9cEYEaJ4jQS/QQ20bKstBiolo=; fh=Ieh26qMz7ii+wAfD3rSAC/AGrlk2wkd+lASkLp3ZwYY=; b=YEkAq2ZbEdEVFRw8hPDWp9afoXHQPlQpOydmSVqSTVk1u0ZgVB+r66+jbhn9peBRuQ 05cd3XcyyYCCFpKtDUaM+zb5scGjHhEJJgjZl+0UwrAATthG0k8EGqZ4P49we1N7dulM UWbKggXcMHavvfhLRt9GPV4HC5Yel3/jx3y9XfUvBftUegTSPBOjbE3mg+37MwT3HJvF M9F5+y8Oko9qIqi8W9dntUWpz4q1DsdGGtMWA9sHQwcdOyLKHHEL9+/3vIHfx+VuWWth s+UYMspdnntwaP7FxT5BzU/lK4y9XcaWmTFT7DHiI8xWf2mDGGsGmjI2QbLtqKpZITyz r/PA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=KvHs9CMY; arc=pass (i=1 spf=pass spfdomain=redhat.com dkim=pass dkdomain=redhat.com dmarc=pass fromdomain=redhat.com); spf=pass (google.com: domain of linux-kernel+bounces-86036-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-86036-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id f11-20020a170902e98b00b001da2a2c90e4si291600plb.438.2024.02.28.18.15.36 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Feb 2024 18:15:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-86036-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=KvHs9CMY; arc=pass (i=1 spf=pass spfdomain=redhat.com dkim=pass dkdomain=redhat.com dmarc=pass fromdomain=redhat.com); spf=pass (google.com: domain of linux-kernel+bounces-86036-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-86036-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id E2AA1288AD2 for ; Thu, 29 Feb 2024 02:15:35 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9522D381C4; Thu, 29 Feb 2024 02:15:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="KvHs9CMY" Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C485364C7 for ; Thu, 29 Feb 2024 02:14:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709172900; cv=none; b=ogjpbug/0P1belvjaZgjRnYQOtevlkat6W4+g/v+6UmwXZUOLwWLS5Y0J0+7FKJo/iVw4vvZIzHvBNmAkD1jtdAHx+evLee1sQy5w3vU9XQ6T1l8pn8OKHTmLyiUHY46CgIDPJ358gdQAtokJ9uZHiQxR2MsS7AtjGuMuwbkLtw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709172900; c=relaxed/simple; bh=ZJ/fljn/Qv75UAjs1VLdo2uoQDDup88nyCXUQF/cxT8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=djT7hwh57Ug+GVINRKyN5GWl6I5Lc/NaBRdV6rdt15R20x6/6gdyu6K/DICxC6EUvq3Kaxl7IYF25LyVNtfqV32MxdIWp8khbQg7PvS/KTv2hrQNTNe0ZELxPZqeBv/l610iKD2VjQCv03WRto8SCUaUmJOqLXumdghJqA7xV8o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=KvHs9CMY; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1709172897; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Iw2m/lgdJeZzD7aZQt9cEYEaJ4jQS/QQ20bKstBiolo=; b=KvHs9CMYJG2m8hT1txY1x2P6wpsJHXRIUmmoZLk0MwIspg/Vb2699V//7551ZLKXOWy42S Y7rIodH56djWFLi3nyFvYZ/Zp9sjMxBQ9aJ2V08kJR4K/fYppZRiVe/zIC9vHHYveeYBv+ TWnGEyr3C/e5PFrweKeac3UTKK0r46Y= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-616-SgMkZ3S1OoO-MwiCMniSfA-1; Wed, 28 Feb 2024 21:14:53 -0500 X-MC-Unique: SgMkZ3S1OoO-MwiCMniSfA-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4AB33863012; Thu, 29 Feb 2024 02:14:52 +0000 (UTC) Received: from llong.com (unknown [10.22.9.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id BDDBC492BE8; Thu, 29 Feb 2024 02:14:50 +0000 (UTC) From: Waiman Long To: Zefan Li , Tejun Heo , Johannes Weiner , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Daniel Bristot de Oliveira , Valentin Schneider , Frederic Weisbecker Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Cestmir Kalina , Costa Shulyupin , Waiman Long Subject: [PATCH 1/2] sched/isolation: Exclude dynamically isolated CPUs from housekeeping masks Date: Wed, 28 Feb 2024 21:14:13 -0500 Message-Id: <20240229021414.508972-2-longman@redhat.com> In-Reply-To: <20240229021414.508972-1-longman@redhat.com> References: <20240229021414.508972-1-longman@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.10 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1792197720306932364 X-GMAIL-MSGID: 1792197720306932364 The housekeeping CPU masks, set up by the "isolcpus" and "nohz_full" boot command line options, are used at boot time to exclude selected CPUs from running some kernel background processes to minimize disturbance to latency sensitive userspace applications. Some of housekeeping CPU masks are also checked at run time to avoid using those isolated CPUs. The cpuset subsystem is now able to dynamically create a set of isolated CPUs to be used in isolated cpuset partitions. The long term goal is to make the degree of isolation as close as possible to what can be done statically using those boot command line options. This patch is a step in that direction by making the housekeeping CPU mask APIs exclude the dynamically isolated CPUs when they are called at run time. The housekeeping CPU masks will fall back to the bootup default when all the dynamically isolated CPUs are released. A new housekeeping_exlude_isolcpus() function is added which is to be called by the cpuset subsystem to provide a list of isolated CPUs to be excluded. Signed-off-by: Waiman Long --- include/linux/sched/isolation.h | 8 +++ kernel/sched/isolation.c | 101 +++++++++++++++++++++++++++++++- 2 files changed, 108 insertions(+), 1 deletion(-) diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h index 2b461129d1fa..d64fa4e60138 100644 --- a/include/linux/sched/isolation.h +++ b/include/linux/sched/isolation.h @@ -27,6 +27,8 @@ extern bool housekeeping_enabled(enum hk_type type); extern void housekeeping_affine(struct task_struct *t, enum hk_type type); extern bool housekeeping_test_cpu(int cpu, enum hk_type type); extern void __init housekeeping_init(void); +extern int housekeeping_exlude_isolcpus(const struct cpumask *isolcpus, + unsigned long flags); #else @@ -54,6 +56,12 @@ static inline bool housekeeping_test_cpu(int cpu, enum hk_type type) } static inline void housekeeping_init(void) { } + +static inline int housekeeping_exlude_isolcpus(struct cpumask *isolcpus, + unsigned long flags) +{ + return -EOPNOTSUPP; +} #endif /* CONFIG_CPU_ISOLATION */ static inline bool housekeeping_cpu(int cpu, enum hk_type type) diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c index 373d42c707bc..f47ff04ac77c 100644 --- a/kernel/sched/isolation.c +++ b/kernel/sched/isolation.c @@ -28,7 +28,8 @@ struct housekeeping { unsigned long flags; }; -static struct housekeeping housekeeping; +static struct housekeeping housekeeping __read_mostly; +static struct housekeeping housekeeping_boot __read_mostly; bool housekeeping_enabled(enum hk_type type) { @@ -239,3 +240,101 @@ static int __init housekeeping_isolcpus_setup(char *str) return housekeeping_setup(str, flags); } __setup("isolcpus=", housekeeping_isolcpus_setup); + +/* + * Save init housekeeping masks to housekeeping_boot + */ +static int housekeeping_copy2_boot(void) +{ + enum hk_type type; + + housekeeping_boot.flags = housekeeping.flags; + for_each_set_bit(type, &housekeeping.flags, HK_TYPE_MAX) { + if (!alloc_cpumask_var(&housekeeping_boot.cpumasks[type], + GFP_KERNEL)) + return -ENOMEM; + cpumask_copy(housekeeping_boot.cpumasks[type], + housekeeping.cpumasks[type]); + } + return 0; +} + +/* + * Exclude the given dynamically isolated CPUs from the housekeeping CPUs + * External synchronization is required to make sure that concurrent call to + * this function will not happen. + * + * Return: 0 if successful, an error code if not + */ +int housekeeping_exlude_isolcpus(const struct cpumask *isolcpus, unsigned long flags) +{ + static unsigned long alloc_flags; + static cpumask_var_t tmp_mask; + static bool excluded; /* @true if some CPUs have been excluded */ + static bool inited; /* @true if called before */ + + bool mask_empty = !isolcpus || cpumask_empty(isolcpus); + enum hk_type type; + + lockdep_assert_cpus_held(); + + if (mask_empty && (!inited || !excluded)) + return 0; + + if (unlikely(!inited)) { + if (!alloc_cpumask_var(&tmp_mask, GFP_KERNEL)) + return -ENOMEM; + if (housekeeping.flags) { + int err = housekeeping_copy2_boot(); + + if (err) + return err; + } + alloc_flags = housekeeping.flags; + inited = true; + } + + if (mask_empty) { + excluded = false; + + /* + * Reset housekeeping to bootup default + */ + for_each_set_bit(type, &housekeeping_boot.flags, HK_TYPE_MAX) + cpumask_copy(housekeeping.cpumasks[type], + housekeeping_boot.cpumasks[type]); + + WRITE_ONCE(housekeeping.flags, housekeeping_boot.flags); + if (!housekeeping_boot.flags && + static_branch_unlikely(&housekeeping_overridden)) + static_key_disable_cpuslocked(&housekeeping_overridden.key); + return 0; + } + + /* + * Setting up the new housekeeping cpumasks + */ + for_each_set_bit(type, &flags, HK_TYPE_MAX) { + const struct cpumask *src_mask; + + if (!(BIT(type) & alloc_flags)) { + if (!alloc_cpumask_var(&housekeeping.cpumasks[type], GFP_KERNEL)) + return -ENOMEM; + alloc_flags |= BIT(type); + } + src_mask = (BIT(type) & housekeeping_boot.flags) + ? housekeeping_boot.cpumasks[type] : cpu_possible_mask; + /* + * Make sure there is at least one online housekeeping CPU + */ + cpumask_andnot(tmp_mask, src_mask, isolcpus); + if (!cpumask_intersects(tmp_mask, cpu_online_mask)) + return -EINVAL; /* Invalid isolated CPUs */ + cpumask_copy(housekeeping.cpumasks[type], tmp_mask); + } + WRITE_ONCE(housekeeping.flags, housekeeping_boot.flags | flags); + excluded = true; + if (!static_branch_unlikely(&housekeeping_overridden)) + static_key_enable_cpuslocked(&housekeeping_overridden.key); + return 0; +} From patchwork Thu Feb 29 02:14:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 208165 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2097:b0:108:e6aa:91d0 with SMTP id gs23csp129882dyb; Wed, 28 Feb 2024 18:15:41 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWqgGe/oS82k8pPhrd3yUR0Jv+GTrXTdM5aopXeDNGIX0KdxrtNTPDRkOGxwEaplD9tak/TtUE5NpOlZLC9EPhArrfu1w== X-Google-Smtp-Source: AGHT+IETD6Ae/+kUpULvpi3/xUfA1+PFishXeatOyMfwE/oVa26vJ5KqRtbwZvWLSgEg7jri5t4W X-Received: by 2002:a17:90a:b889:b0:29b:299:6f8b with SMTP id o9-20020a17090ab88900b0029b02996f8bmr994767pjr.27.1709172941687; Wed, 28 Feb 2024 18:15:41 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1709172941; cv=pass; d=google.com; s=arc-20160816; b=Ud+SXBOuX+SkHB+TvLP8Py5P5bIvq7oCbdAaXJ2p10duhTV2GAx44wOmR+e2M0j4EF 04l37COfHknlFB73Zt7Logkk/GT7SuzRDlX6c82adNDkyvCthzOAZaMA0Fyt65JzTjyX AsjRTxC9UzbY99iAd2Fz2o0zkjWkxnw6YxVuLypd432b93/KsvmWZh4qUHVFHS11/B8P yATHKf7u8Pdq4HuZ46qo+CZfVBrAAjPcEVvX/zmxVcijk6HBLbNR1HfXsqOKXcBcdpgO tKKiXpOXy6A4Kddo67iI/bZGEdYlThtUYYIZ5ujty9BvjrgsR2OmNfJVl6taR9ASev8w pR7A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=/XKfDwEYq77yh2Tx7xxyUZeYeDXKbwbjelgL1h1OpUk=; fh=Ieh26qMz7ii+wAfD3rSAC/AGrlk2wkd+lASkLp3ZwYY=; b=I0TQ9I+Ig8fgNZ12rW5ULmQYSFTQ0r9bfELZCyOG6CtLa2Khgsvn4BxGuFOtl/VtNT 3jWvgKW2rS9/sa+ghXAE/yyWCVTRgf38afhnnNjTlAy1cIqXv4M9Om0Uw2NguvvbofOi JLkNIcQ9Bt9XwTf005YY9uQDtBIopW+0MRV/EtrfPkOfByL7RtH0iWeslmPpi8wcDCwR 6weKf4862wjdLstoKtcDnUAUc58hlvxCqyRVONMRoCbjE/kzHjkhRrpQ4GU1w1jGGOya MdSqgy2zCURskK3t+EWK8/gP+VXurcTSths5hNx0xevXE3KZMSklX/Kc7oYxxQujB4iV 1G6A==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=iCMWisb7; arc=pass (i=1 spf=pass spfdomain=redhat.com dkim=pass dkdomain=redhat.com dmarc=pass fromdomain=redhat.com); spf=pass (google.com: domain of linux-kernel+bounces-86037-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-86037-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id e63-20020a17090a6fc500b00296416a8f10si2490781pjk.112.2024.02.28.18.15.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Feb 2024 18:15:41 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-86037-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=iCMWisb7; arc=pass (i=1 spf=pass spfdomain=redhat.com dkim=pass dkdomain=redhat.com dmarc=pass fromdomain=redhat.com); spf=pass (google.com: domain of linux-kernel+bounces-86037-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-86037-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 78B792889B4 for ; Thu, 29 Feb 2024 02:15:41 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6AD4B38384; Thu, 29 Feb 2024 02:15:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="iCMWisb7" Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CDCB536AFF for ; Thu, 29 Feb 2024 02:14:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709172901; cv=none; b=cNVUIHJZ1OWavto36r5/pODRS7+3aHSTrlztUbI6TmJlQ3PtfkRWvBT8y+RLDvEpKN2AvprT1iMVrZOBPbdhy6qRrIo2ku46cNuMK7tUPmIo9jMAmBvfsX52cpWS/KZ1z12j+yXICEhBX4MFZh8CXqanR42QcYiGVY5nCSx5z1w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709172901; c=relaxed/simple; bh=sakExV8Zhp5Sh/q1+gBQuscarUS25o1Em4K3tkAC4hY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=uiwa4v5PoZ9gg6b9+1iKDu1urNRutxQTCMbPYZVAMZ20aDsOr6Eo6042FW3XD1thQjPpFN2gZAZiZCdBJ0lsxi3W2VfI4Z7/LW/E2+S7qNvtoMPnr5cVQIs2FRpAZWlXNlwwaYUl5r9Xj81Ubgf81vhkYz0erT5PUWrKJzuV7GM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=iCMWisb7; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1709172898; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/XKfDwEYq77yh2Tx7xxyUZeYeDXKbwbjelgL1h1OpUk=; b=iCMWisb7YdMJFW1meLGylIuuQDACZaR+KNXB233LUevdoF3ej/6KaWAOUgjH3sp0vQB/p3 HA1rZC6UWl1bhN30G9ozPbw+t1jqGvaDO4y6B5TEZVSEacz9BV1sh7nJqMYqfqzjW1dIPw 7QQ2wmrvW0vfbuUbIjsYouK2ce2bFN4= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-658-hYHPAaMkPg2jcoflFBPneA-1; Wed, 28 Feb 2024 21:14:54 -0500 X-MC-Unique: hYHPAaMkPg2jcoflFBPneA-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E47AA85A588; Thu, 29 Feb 2024 02:14:53 +0000 (UTC) Received: from llong.com (unknown [10.22.9.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5AE30492BFA; Thu, 29 Feb 2024 02:14:52 +0000 (UTC) From: Waiman Long To: Zefan Li , Tejun Heo , Johannes Weiner , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Daniel Bristot de Oliveira , Valentin Schneider , Frederic Weisbecker Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Cestmir Kalina , Costa Shulyupin , Waiman Long Subject: [PATCH 2/2] cgroup/cpuset: Exclude isolated CPUs from housekeeping CPU masks Date: Wed, 28 Feb 2024 21:14:14 -0500 Message-Id: <20240229021414.508972-3-longman@redhat.com> In-Reply-To: <20240229021414.508972-1-longman@redhat.com> References: <20240229021414.508972-1-longman@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.10 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1792197726645116429 X-GMAIL-MSGID: 1792197726645116429 Call the newly introduced housekeeping_exlude_isolcpus() function to exclude isolated CPUs from the selected housekeeping CPU masks. This is in addition to the exclusion of isolated CPUs from the workqueue unbound CPU mask. Right now only HK_TYPE_TIMER and HK_TYPE_RCU CPU masks are updated, but more may be added in the future when appropriate. Signed-off-by: Waiman Long --- kernel/cgroup/cpuset.c | 30 +++++++++++++++++++++++------- 1 file changed, 23 insertions(+), 7 deletions(-) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index ba36c073304a..d2541af71c22 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -214,6 +214,11 @@ static cpumask_var_t isolated_cpus; /* List of remote partition root children */ static struct list_head remote_children; +/* + * The set of housekeeping flags to be updated for CPU isolation + */ +#define HOUSEKEEPING_FLAGS (BIT(HK_TYPE_TIMER) | BIT(HK_TYPE_RCU)) + /* * Partition root states: * @@ -1505,7 +1510,15 @@ static bool partition_xcpus_del(int old_prs, struct cpuset *parent, return isolcpus_updated; } -static void update_unbound_workqueue_cpumask(bool isolcpus_updated) +/** + * update_isolation_cpumasks - Update external isolation CPU masks + * @isolcpus_updated - @true if isolation CPU masks update needed + * + * The following external CPU masks will be updated if necessary: + * - workqueue unbound cpumask + * - housekeeping cpumasks + */ +static void update_isolation_cpumasks(bool isolcpus_updated) { int ret; @@ -1515,7 +1528,10 @@ static void update_unbound_workqueue_cpumask(bool isolcpus_updated) return; ret = workqueue_unbound_exclude_cpumask(isolated_cpus); - WARN_ON_ONCE(ret < 0); + if (WARN_ON_ONCE(ret < 0)) + return; + ret = housekeeping_exlude_isolcpus(isolated_cpus, HOUSEKEEPING_FLAGS); + WARN_ON_ONCE((ret < 0) && (ret != -EOPNOTSUPP)); } /** @@ -1609,7 +1625,7 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs, parent->child_ecpus_count--; } spin_unlock_irq(&callback_lock); - update_unbound_workqueue_cpumask(isolcpus_updated); + update_isolation_cpumasks(isolcpus_updated); /* * Proprogate changes in top_cpuset's effective_cpus down the hierarchy. @@ -1645,7 +1661,7 @@ static void remote_partition_disable(struct cpuset *cs, struct tmpmasks *tmp) cs->prs_err = PERR_INVCPUS; reset_partition_data(cs); spin_unlock_irq(&callback_lock); - update_unbound_workqueue_cpumask(isolcpus_updated); + update_isolation_cpumasks(isolcpus_updated); /* * Proprogate changes in top_cpuset's effective_cpus down the hierarchy. @@ -1697,7 +1713,7 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *newmask, if (deleting) isolcpus_updated += partition_xcpus_del(prs, NULL, tmp->delmask); spin_unlock_irq(&callback_lock); - update_unbound_workqueue_cpumask(isolcpus_updated); + update_isolation_cpumasks(isolcpus_updated); /* * Proprogate changes in top_cpuset's effective_cpus down the hierarchy. @@ -2067,7 +2083,7 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd, WARN_ON_ONCE(parent->nr_subparts < 0); } spin_unlock_irq(&callback_lock); - update_unbound_workqueue_cpumask(isolcpus_updated); + update_isolation_cpumasks(isolcpus_updated); if ((old_prs != new_prs) && (cmd == partcmd_update)) update_partition_exclusive(cs, new_prs); @@ -3131,7 +3147,7 @@ static int update_prstate(struct cpuset *cs, int new_prs) else if (new_xcpus_state) partition_xcpus_newstate(old_prs, new_prs, cs->effective_xcpus); spin_unlock_irq(&callback_lock); - update_unbound_workqueue_cpumask(new_xcpus_state); + update_isolation_cpumasks(new_xcpus_state); /* Force update if switching back to member */ update_cpumasks_hier(cs, &tmpmask, !new_prs ? HIER_CHECKALL : 0);