From patchwork Sat Nov 12 22:19:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 19267 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp1454642wru; Sat, 12 Nov 2022 14:48:19 -0800 (PST) X-Google-Smtp-Source: AA0mqf5+eIQD+w3ZnqerBDm1NJVz+2YgsyB9f//HusDXsWnO3MpU/aPsa01xEdog0luI+EV+MW6f X-Received: by 2002:a63:c001:0:b0:46f:e657:7d25 with SMTP id h1-20020a63c001000000b0046fe6577d25mr6505424pgg.347.1668293298823; Sat, 12 Nov 2022 14:48:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668293298; cv=none; d=google.com; s=arc-20160816; b=A1AEF8O0t+owahezfC3o/HU6VRY/EfQrwwZm3TMwoVxVchbgAEov43ecL1a5Q3KKZ9 dJGwmODSrOUpPoYbGBhiZh4zjVw7qZ3S72XbFenbc8h4D+uDbyDy+rC1Ng7NDW1InG84 Khhwvi6k4gu1inZfhEjnP3oZvr48h+t9PahFV/yf8EO90E516GSbRISPvfHnkiogYcWP OMC4JJVltRPqHIKf3yL9YCdManT/juRqWTFAJbb28Xj3NyuwGXucGGDotL3ll7LdEvQK cDfjff+/QxubApIaxpHG5xZMgGF7DC5nbjoXSyMH84StB3swCTrX7qdgpB+aKP8+S+0w roLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=BEeh1aXYPMoNUSRUbgR3jz4w6CXvrkqhiV75ZDaEEV4=; b=TDIZnaNDc79EXuQNGyrrmWjbsp/bBtDGklnZV/Ek+a77Yd6IaSkgrbO2Mc87SCmv9I 5C3XWK9kVG06xnb9rapWV+Dtzufn3dCbK3Yiz6FVZ1b/VspKBqA9duXj9g5dUNUxrg0t wK9EaMkleRQw2Us0VFPGXt1+oQOKF4EqKUFJFN3o8WOBN7euVnDya8cSN3f37NbrP5Cv bU/OZMGe6iOTAWcXCKeyADBXYjxLgVgdqxEtmA9EpwY5cQgrVgcPLJQsRZkNNJIeAkT4 vmhjycVWcweLgSBTHE4z/156kCmWuW+yNMsVH0EyurVqnbojGKvr0H/rS5pLi9zRYRvG kn+w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=bG1Dr5JX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id oc4-20020a17090b1c0400b00213cb58e47bsi3882544pjb.8.2022.11.12.14.48.04; Sat, 12 Nov 2022 14:48:18 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=bG1Dr5JX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235139AbiKLWWG (ORCPT + 99 others); Sat, 12 Nov 2022 17:22:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40326 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234941AbiKLWWE (ORCPT ); Sat, 12 Nov 2022 17:22:04 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C00D3FCC3 for ; Sat, 12 Nov 2022 14:20:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1668291615; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BEeh1aXYPMoNUSRUbgR3jz4w6CXvrkqhiV75ZDaEEV4=; b=bG1Dr5JX9LKI7mF1SvOUrNIelMNkpMPjku/e17sIRR5PTtpdUNZ32VZ0yD2Xb6l3+0xJcQ TFxuNCpGe3eR5GBN4QTEKzGZs2gPBZ0Zxn4f1gJm7WN0B16xK6aT9xBw8j25BT9kLIenqu TDGwFh2EZB5oTOROw1su28dpTNTgx3M= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-654-8zaKV3QnNFCJ7_2jNM1Ukw-1; Sat, 12 Nov 2022 17:20:09 -0500 X-MC-Unique: 8zaKV3QnNFCJ7_2jNM1Ukw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F25DE3806703; Sat, 12 Nov 2022 22:20:08 +0000 (UTC) Received: from llong.com (unknown [10.22.8.110]) by smtp.corp.redhat.com (Postfix) with ESMTP id A31B8C15BB2; Sat, 12 Nov 2022 22:20:08 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Sebastian Andrzej Siewior , Waiman Long Subject: [PATCH 1/2] cgroup/cpuset: Skip spread flags update on v2 Date: Sat, 12 Nov 2022 17:19:38 -0500 Message-Id: <20221112221939.1272764-2-longman@redhat.com> In-Reply-To: <20221112221939.1272764-1-longman@redhat.com> References: <20221112221939.1272764-1-longman@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749332314205235929?= X-GMAIL-MSGID: =?utf-8?q?1749332314205235929?= Cpuset v2 has no spread flags to set. So we can skip spread flags update if cpuset v2 is being used. Also change the name to cpuset_update_task_spread_flags() to indicate that there are multiple spread flags. Signed-off-by: Waiman Long --- kernel/cgroup/cpuset.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index b474289c15b8..2525905cdf48 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -550,11 +550,15 @@ static void guarantee_online_mems(struct cpuset *cs, nodemask_t *pmask) /* * update task's spread flag if cpuset's page/slab spread flag is set * - * Call with callback_lock or cpuset_rwsem held. + * Call with callback_lock or cpuset_rwsem held. The check can be skipped + * if on default hierarchy. */ -static void cpuset_update_task_spread_flag(struct cpuset *cs, +static void cpuset_update_task_spread_flags(struct cpuset *cs, struct task_struct *tsk) { + if (cgroup_subsys_on_dfl(cpuset_cgrp_subsys)) + return; + if (is_spread_page(cs)) task_set_spread_page(tsk); else @@ -2153,7 +2157,7 @@ static void update_tasks_flags(struct cpuset *cs) css_task_iter_start(&cs->css, 0, &it); while ((task = css_task_iter_next(&it))) - cpuset_update_task_spread_flag(cs, task); + cpuset_update_task_spread_flags(cs, task); css_task_iter_end(&it); } @@ -2530,7 +2534,7 @@ static void cpuset_attach(struct cgroup_taskset *tset) WARN_ON_ONCE(set_cpus_allowed_ptr(task, cpus_attach)); cpuset_change_task_nodemask(task, &cpuset_attach_nodemask_to); - cpuset_update_task_spread_flag(cs, task); + cpuset_update_task_spread_flags(cs, task); } /* From patchwork Sat Nov 12 22:19:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 19266 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp1452155wru; Sat, 12 Nov 2022 14:38:03 -0800 (PST) X-Google-Smtp-Source: AA0mqf7NRE/Huy6SiXbMRS2gWY8z5tDpf8GGCxeL96Xk1p+eoRfEiIRvrHMTA971EAzAFdW75qfg X-Received: by 2002:a17:906:d1d0:b0:7ad:88f8:6a53 with SMTP id bs16-20020a170906d1d000b007ad88f86a53mr6374991ejb.61.1668292682877; Sat, 12 Nov 2022 14:38:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668292682; cv=none; d=google.com; s=arc-20160816; b=t4UKF8ot6Y5llrfXaJ6sKAPJlo9l7MuS+s921ZIwEJqFpW/WenRZ/l7yIcYWDQQOWz J0a+Gi68XItEc4FbyEnI+FcNNS/Tw/wvQNYrZiSQz6gG7UOSzyb5zM0m8a/vGsLMkOtJ 3U2MsmLRfknElvhzSPaSWWxr4NR6w4FWXTtxMW0bt08NKNdGq9cYCmsDHnTOTOYZiTwu Lol7TTiSY/Mxdk+eqP+vxc9hePS+Te7jMhJt0Rke4nhDDwvKA69zXupskoCSnWXi7ZCC AbFhlzimGZuozk5SjRUp1j1RosGBfiaii3cY0Yv9bOQBG8PvBcE/0193ILZkb/BIQmL0 C1gQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=oU8Uioe5AqDTtnEdw4klUnNv+nnvIMtyMCYaMWGnjIg=; b=ICeN37E9Pu32Ihq3t3l+vXkNuC1YA4UoqJ6fRtghL/+0qWSmRc5pHjTnhSsg1rprCT wViJesMbySjZgpZkufQN6dmsLoDsvaqi2Y3gOXHLsXuAYjT6nRavMkntoHWIyoO9Q0v/ L6ixQP5WNiwoo7+IlHL2FMte+wwxwiPJJSleh9rpjtKfHEz5z+mYL+nSvYtEDgWngRaq +JESDXq05KG5XKB/2ab/fxeuxsueu/qb9VpEBXaK+VObaXXZzWf2pSSlq0awvSHv0qJP YGStXyrHZjAKbj0qyO0H0yc6DqC0uIipHo1LLERMDH9gUWnRwXyAGsPwDId3zN5v+Hk5 TAJQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=T9kB3I1A; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n4-20020aa7c444000000b004618343b140si5385115edr.199.2022.11.12.14.37.37; Sat, 12 Nov 2022 14:38:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=T9kB3I1A; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234996AbiKLWVI (ORCPT + 99 others); Sat, 12 Nov 2022 17:21:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40292 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234695AbiKLWVF (ORCPT ); Sat, 12 Nov 2022 17:21:05 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D4118F5BA for ; Sat, 12 Nov 2022 14:20:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1668291613; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oU8Uioe5AqDTtnEdw4klUnNv+nnvIMtyMCYaMWGnjIg=; b=T9kB3I1Ac4NN1SjCe7OCN46rxLAug84APZXi3tDF92Ju50ILuZyDrz2pusRVBlPQKLAHE9 1ev/QTb0DAQe1CiH267IOp730cZl+Ala9rYE+RwSRESz3QzkJqJgPIhLL+P8UUG0yUb5Vm 0HIa3n8TnOaAiK204nm4Zny/b2FRMQk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-646-LKRu5Ry_N2a_dV9ShLO1uQ-1; Sat, 12 Nov 2022 17:20:09 -0500 X-MC-Unique: LKRu5Ry_N2a_dV9ShLO1uQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5D31C8027EA; Sat, 12 Nov 2022 22:20:09 +0000 (UTC) Received: from llong.com (unknown [10.22.8.110]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0D1ACC15BA4; Sat, 12 Nov 2022 22:20:09 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Sebastian Andrzej Siewior , Waiman Long Subject: [PATCH 2/2] cgroup/cpuset: Optimize cpuset_attach() on v2 Date: Sat, 12 Nov 2022 17:19:39 -0500 Message-Id: <20221112221939.1272764-3-longman@redhat.com> In-Reply-To: <20221112221939.1272764-1-longman@redhat.com> References: <20221112221939.1272764-1-longman@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749331668354178867?= X-GMAIL-MSGID: =?utf-8?q?1749331668354178867?= It was found that with the default hierarchy, enabling cpuset in the child cgroups can trigger a cpuset_attach() call in each of the child cgroups that have tasks with no change in effective cpus and mems. If there are many processes in those child cgroups, it will burn quite a lot of cpu cycles iterating all the tasks without doing useful work. Optimizing this case by comparing between the old and new cpusets and skip useless update if there is no change in effective cpus and mems. Also mems_allowed are less likely to be changed than cpus_allowed. So skip changing mm if there is no change in effective_mems and CS_MEMORY_MIGRATE is not set. By inserting some instrumentation code and running a simple command in a container 200 times in a cgroup v2 system, it was found that all the cpuset_attach() calls are skipped (401 times in total) as there was no change in effective cpus and mems. Signed-off-by: Waiman Long --- kernel/cgroup/cpuset.c | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 2525905cdf48..b8361f55ef36 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -2513,12 +2513,28 @@ static void cpuset_attach(struct cgroup_taskset *tset) struct cgroup_subsys_state *css; struct cpuset *cs; struct cpuset *oldcs = cpuset_attach_old_cs; + bool cpus_updated, mems_updated; cgroup_taskset_first(tset, &css); cs = css_cs(css); lockdep_assert_cpus_held(); /* see cgroup_attach_lock() */ percpu_down_write(&cpuset_rwsem); + cpus_updated = !cpumask_equal(cs->effective_cpus, + oldcs->effective_cpus); + mems_updated = !nodes_equal(cs->effective_mems, oldcs->effective_mems); + + /* + * In the default hierarchy, enabling cpuset in the child cgroups + * will trigger a number of cpuset_attach() calls with no change + * in effective cpus and mems. In that case, we can optimize out + * by skipping the task iteration and update. + */ + if (cgroup_subsys_on_dfl(cpuset_cgrp_subsys) && + !cpus_updated && !mems_updated) { + cpuset_attach_nodemask_to = cs->effective_mems; + goto out; + } guarantee_online_mems(cs, &cpuset_attach_nodemask_to); @@ -2539,9 +2555,14 @@ static void cpuset_attach(struct cgroup_taskset *tset) /* * Change mm for all threadgroup leaders. This is expensive and may - * sleep and should be moved outside migration path proper. + * sleep and should be moved outside migration path proper. Skip it + * if there is no change in effective_mems and CS_MEMORY_MIGRATE is + * not set. */ cpuset_attach_nodemask_to = cs->effective_mems; + if (!is_memory_migrate(cs) && !mems_updated) + goto out; + cgroup_taskset_for_each_leader(leader, css, tset) { struct mm_struct *mm = get_task_mm(leader); @@ -2564,6 +2585,7 @@ static void cpuset_attach(struct cgroup_taskset *tset) } } +out: cs->old_mems_allowed = cpuset_attach_nodemask_to; cs->attach_in_progress--;