From patchwork Mon Nov 27 23:45:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 170503 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce62:0:b0:403:3b70:6f57 with SMTP id o2csp3550743vqx; Mon, 27 Nov 2023 15:46:19 -0800 (PST) X-Google-Smtp-Source: AGHT+IHAsaqtIkfqxJ76Lg61H8+BKg3dcFDbjRyF/Az2KA8+DMa/I92MbMHEabd8J/CfebKe8u4v X-Received: by 2002:a05:6a20:1444:b0:187:23f4:d061 with SMTP id a4-20020a056a20144400b0018723f4d061mr11196947pzi.37.1701128779258; Mon, 27 Nov 2023 15:46:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701128779; cv=none; d=google.com; s=arc-20160816; b=eNnaGCCFtBvaR6ueacnytc9RogpJMOhtoO+9Tcz5wUdYUil6KvEy6cbKc2hHPimAV1 Aowgf1pt8ORQZZoCbOBnL91gfakY/qHVJ2zL70tv93KDmAe5ZqgbRwhQvrN/0HDjAEAL Kpcjihcg0pZFabralX7s9zNjQ9QxEroh16OOYcMBikoiZVgoxpqcZ/09VfWrKDzEuq8e nqB3gVlF11PdtojXPbaRKovRkO1YCj3zAPMQ1KChjn74PYXlJ9423MjFjm1kIWiH40so /GpSTHO+x+k4wkGWDDHOSixKRXTfKoQoIHW7u0VeX80HQ81aws6iWc6m/lizDFsuzGZG iNkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=jvxvLBlp9fV8n96zZT2Ez0ns0pnZxld9OCN8MtepcpQ=; fh=5ynFD2G6LA0fdRuAOSZxbBxHoIIG4U3xrwbnkupZs28=; b=ilaA1WtA+uJIWpqVWMW9F+Zg2qJcW/SQoYN9RlLQW+BEdT42JJYlrEaYculbrUEClO 9id5dSZv52GWzgrHWA9d2A4GI2LR/mzm9znZtMigTeS+rXUOwZP3vMJ0gffVhlTm5LYe sf+K7FCwSX5bF2GKcBWvoBB65ZEEwj3KYUOTvRRkILaVmtbRaXEViRJrLxLpuFoQgxdg Ihj9YBHNkU7W6tjR1NN9Le0ikkpTjaH4FABS1ZkpyAXEtNc6SZWHtgp4i37iF016qexH 1T16alr88YJdXlCFQnqvDkRZbWM6+kCk+HjKSixSfZZGeQtq/U46vbUxaVY0qzf5mYIx CJrA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=NdYYdNoU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from morse.vger.email (morse.vger.email. [23.128.96.31]) by mx.google.com with ESMTPS id o37-20020a634e65000000b005be029a66d1si10649383pgl.806.2023.11.27.15.46.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Nov 2023 15:46:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=NdYYdNoU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id 53E27811216E; Mon, 27 Nov 2023 15:46:13 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233721AbjK0Xp7 (ORCPT + 99 others); Mon, 27 Nov 2023 18:45:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35516 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231459AbjK0Xp5 (ORCPT ); Mon, 27 Nov 2023 18:45:57 -0500 Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6986B1B6; Mon, 27 Nov 2023 15:46:03 -0800 (PST) Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1cfc3f50504so15148395ad.3; Mon, 27 Nov 2023 15:46:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701128763; x=1701733563; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jvxvLBlp9fV8n96zZT2Ez0ns0pnZxld9OCN8MtepcpQ=; b=NdYYdNoUqcFgDD1XreOzzSy4sZVc2MqDQsK2+y1Du34+I07vGndVbfuopiP5OWsML0 kg/1gNPwVZSJAcoC5PzG+BB7PU6VZM0e4s49kXhm4n+gedLSYw8d28SetjzO5ZU2SAmA +PMkZZkGR5TAj55g/u1iW5TCSHweZni2b1sCdGXTzHyvcMsIElFVtoZcD52xWmhkDxrp CEY+IgonLsE8Rr+LLyyBsSBAX8shgKgx1YYTmgt6ctvpSzbVT8WF1f46SnV58kQTxOTh X/ynxOsz++uGYkLZK+dxRgykOSoezIR6ITlDi1ggX20TuX4konCN/1vrA7jMrYvWkyHg dVYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701128763; x=1701733563; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jvxvLBlp9fV8n96zZT2Ez0ns0pnZxld9OCN8MtepcpQ=; b=Y/WYSlOtLl6gXj/u2XV4Xfvi3wKr0Dw5wVnCMnWzSX6iihwxFsBIAlPEKWwq/QsLTU 2pDmZqwKjfaTJXLAsequLgeOKiWAed8C6AQP5jiG9Dm2SpXyvSC1bQVQukNUUWp+sXrH RBu97KJHT/l0suoW8FKDJo3GCVJ8myu6qDavor/gaUyU3CND18Fc5QzpWuVOicB7SVYg JZoKskLz0gt8Y+8cKXC5A5uHunQYahxHSDZyT1kYKHPCoIqpvYNGzmG4WEntJMvganZc xoUsVFNYHM90i80WmBJvHs7SNH+d4ovlPfQaP9PWoOJOvRgY5OMTxt7Ozlpz8wb2oI2h EfTg== X-Gm-Message-State: AOJu0Yx9Qjku4wd9BnZ8Vyv1XB8Ab6/+ZgGjpXbw1LPzofGiUOsDS17p 9lE5FQNYNGFcP4KhCPDxsX4= X-Received: by 2002:a17:902:b28c:b0:1cc:3fc9:1802 with SMTP id u12-20020a170902b28c00b001cc3fc91802mr11577380plr.61.1701128762852; Mon, 27 Nov 2023 15:46:02 -0800 (PST) Received: from localhost (fwdproxy-prn-011.fbsv.net. [2a03:2880:ff:b::face:b00c]) by smtp.gmail.com with ESMTPSA id u13-20020a170902e80d00b001cfb14a09a4sm5507314plg.126.2023.11.27.15.46.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Nov 2023 15:46:02 -0800 (PST) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, cerasuolodomenico@gmail.com, yosryahmed@google.com, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, chrisl@kernel.org, linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, shuah@kernel.org Subject: [PATCH v7 2/6] memcontrol: add a new function to traverse online-only memcg hierarchy Date: Mon, 27 Nov 2023 15:45:56 -0800 Message-Id: <20231127234600.2971029-3-nphamcs@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231127234600.2971029-1-nphamcs@gmail.com> References: <20231127234600.2971029-1-nphamcs@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Mon, 27 Nov 2023 15:46:13 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783762810378239595 X-GMAIL-MSGID: 1783762810378239595 The new zswap writeback scheme requires an online-only memcg hierarchy traversal. Add this functionality via the new mem_cgroup_iter_online() function - the old mem_cgroup_iter() is a special case of this new function. Suggested-by: Andrew Morton Signed-off-by: Nhat Pham Acked-by: Johannes Weiner --- include/linux/memcontrol.h | 13 +++++++++++++ mm/memcontrol.c | 29 +++++++++++++++++++++++++---- 2 files changed, 38 insertions(+), 4 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 7bdcf3020d7a..5bfe41840e80 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -833,6 +833,10 @@ static inline void mem_cgroup_put(struct mem_cgroup *memcg) struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *, struct mem_cgroup *, struct mem_cgroup_reclaim_cookie *); +struct mem_cgroup *mem_cgroup_iter_online(struct mem_cgroup *root, + struct mem_cgroup *prev, + struct mem_cgroup_reclaim_cookie *reclaim, + bool online); void mem_cgroup_iter_break(struct mem_cgroup *, struct mem_cgroup *); void mem_cgroup_scan_tasks(struct mem_cgroup *memcg, int (*)(struct task_struct *, void *), void *arg); @@ -1386,6 +1390,15 @@ mem_cgroup_iter(struct mem_cgroup *root, return NULL; } +static inline struct mem_cgroup * +mem_cgroup_iter_online(struct mem_cgroup *root, + struct mem_cgroup *prev, + struct mem_cgroup_reclaim_cookie *reclaim, + bool online) +{ + return NULL; +} + static inline void mem_cgroup_iter_break(struct mem_cgroup *root, struct mem_cgroup *prev) { diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 470821d1ba1a..b1fb96233fa1 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1111,10 +1111,11 @@ struct mem_cgroup *get_mem_cgroup_from_current(void) } /** - * mem_cgroup_iter - iterate over memory cgroup hierarchy + * mem_cgroup_iter_online - iterate over memory cgroup hierarchy * @root: hierarchy root * @prev: previously returned memcg, NULL on first invocation * @reclaim: cookie for shared reclaim walks, NULL for full walks + * @online: whether to skip offline memcgs * * Returns references to children of the hierarchy below @root, or * @root itself, or %NULL after a full round-trip. @@ -1123,13 +1124,16 @@ struct mem_cgroup *get_mem_cgroup_from_current(void) * invocations for reference counting, or use mem_cgroup_iter_break() * to cancel a hierarchy walk before the round-trip is complete. * + * Caller can skip offline memcgs by passing true for @online. + * * Reclaimers can specify a node in @reclaim to divide up the memcgs * in the hierarchy among all concurrent reclaimers operating on the * same node. */ -struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, +struct mem_cgroup *mem_cgroup_iter_online(struct mem_cgroup *root, struct mem_cgroup *prev, - struct mem_cgroup_reclaim_cookie *reclaim) + struct mem_cgroup_reclaim_cookie *reclaim, + bool online) { struct mem_cgroup_reclaim_iter *iter; struct cgroup_subsys_state *css = NULL; @@ -1199,7 +1203,8 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, * is provided by the caller, so we know it's alive * and kicking, and don't take an extra reference. */ - if (css == &root->css || css_tryget(css)) { + if (css == &root->css || (!online && css_tryget(css)) || + css_tryget_online(css)) { memcg = mem_cgroup_from_css(css); break; } @@ -1228,6 +1233,22 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, return memcg; } +/** + * mem_cgroup_iter - iterate over memory cgroup hierarchy + * @root: hierarchy root + * @prev: previously returned memcg, NULL on first invocation + * @reclaim: cookie for shared reclaim walks, NULL for full walks + * + * Perform an iteration on the memory cgroup hierarchy without skipping + * offline memcgs. + */ +struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, + struct mem_cgroup *prev, + struct mem_cgroup_reclaim_cookie *reclaim) +{ + return mem_cgroup_iter_online(root, prev, reclaim, false); +} + /** * mem_cgroup_iter_break - abort a hierarchy walk prematurely * @root: hierarchy root