From patchwork Mon Nov 27 19:36:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 170373 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce62:0:b0:403:3b70:6f57 with SMTP id o2csp3407325vqx; Mon, 27 Nov 2023 11:37:25 -0800 (PST) X-Google-Smtp-Source: AGHT+IGTvXpqXOe+mFOFtI/g0bsdP8QlQRG+4ZTp3UH5dUwOvldGomZ2oK2+sidH6G/no9J5zGkt X-Received: by 2002:a05:6a21:3116:b0:18b:8158:86ea with SMTP id yz22-20020a056a21311600b0018b815886eamr17858054pzb.10.1701113845193; Mon, 27 Nov 2023 11:37:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701113845; cv=none; d=google.com; s=arc-20160816; b=MpR5Mw2n87yrsKaXyJRnV4gZ7W+Sx/roa8TYA8R7Nk0eagFC1ETBvBz1/U2sWkIvy7 zG5DxJ+KerUMaxwwyiNP2RH0GIaF3uBDlbiWE3J35tDeUeV0lMk1t6zpjs3yMcdb6ZIK 8Tey8gMl328h5AMeLkBagbIzfcnLdwgX1nz4se1lCtErlpjL3ASNRrWJZb2X3VNXKqz5 1noP37uDRPLksWBf0tmAk0XD7PNe7VclEgPMQUOFdfX2SSZNX5MVnwmCjG13h4m/+3n3 wLxsa6mLHGRNwssWA+6LzOebYTrhvmSa67OX1NR0S4nrimIJC+6f8g307SlZ2W0ADuuB G78w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Yk3Ra7LxPSTcjlQasqmaE6aQE/pvNfDE/tLSZ7yZw8U=; fh=5ynFD2G6LA0fdRuAOSZxbBxHoIIG4U3xrwbnkupZs28=; b=ND2N8svZkMhhHexYe+zwITMjBkQ1frVIJdoep0dUwr9MplCzKQ27kgx+Q/JmpluY2E ZY9N/O+74kVn56X3W8haeydfdirQfO+uG9ZdV+MAmM7LEySP95ynRsBtuuvblxfCtigt w8H2Cp4H54gN8cFJsDk/K6mDrUOBmZNvmKtimT8p6gdko+vsQejn4aOt4+MvUd07zSdZ P2kqx9DQJwe385vz0AvmmMmu44jy3quQ3YbAVh8wE06GjM1CFzq7fhjJWwU52uhevvQc IF6k5lpuo1nnBTeUSEekoGT8hOR7enBFldWRJS50rKycYXF6QUb6EdAMzbRt72brOtR+ n/Og== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=LCKQBSnj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id bc3-20020a170902930300b001cfc4194cb7si3852274plb.634.2023.11.27.11.37.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Nov 2023 11:37:25 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=LCKQBSnj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 37439806CF76; Mon, 27 Nov 2023 11:37:24 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232922AbjK0ThH (ORCPT + 99 others); Mon, 27 Nov 2023 14:37:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50428 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232852AbjK0ThA (ORCPT ); Mon, 27 Nov 2023 14:37:00 -0500 Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E63B710CB; Mon, 27 Nov 2023 11:37:06 -0800 (PST) Received: by mail-pj1-x102a.google.com with SMTP id 98e67ed59e1d1-2859551886cso2466944a91.2; Mon, 27 Nov 2023 11:37:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701113826; x=1701718626; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Yk3Ra7LxPSTcjlQasqmaE6aQE/pvNfDE/tLSZ7yZw8U=; b=LCKQBSnjW4kVZH+ox4SLDtgl76dXgqYdAsE0wo3eAtKfvZnwUsAZygTSJevp6exVgV IC4qIjlhn3L6y809RYqttPJCMsGvP3LbZkynMyKcH2Vnpz+nijqSWU2ZVjKQmCaFySFR zYFuT2AkPqJC0qhpiVJy0q/lvLPxVSRXGpHdzFqmP/7xbwHAjt3YA5nrJA5CY5acT4w6 ASceD2DT6AaArhHzt/tLzStasehBWXi19ZgbGshv5YiW6ZO8cmUyC3uODc29/8blYkH/ +4A4fdlJv+A4Nyztzf7DfZDRZaKbEzY/8L3GgsdLWRQHiQWjEkdDcgLn6EObuifJHw3/ bWyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701113826; x=1701718626; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Yk3Ra7LxPSTcjlQasqmaE6aQE/pvNfDE/tLSZ7yZw8U=; b=M7MSEaK+4OYtyhwJr9jOf9ZNJedXXsuTCnVqbpWtBOiBGd0FXiYYqVYXaNXHbKbllh famKvmA4cq2Qpr9CH3emAMBDIV5RAMSft2Yc1qYOzpQ18Z1gZwi3oyd+M7JwG4bhoLge Y83SOuduhnvTPG8coRsI4OijwfseWlBFhw+A1ZkT2IR9ANPujM/1kUtD9GHMlRplQgIf GfwGqFl33TYXHZouUmaTqXxEsA0E2QJ+yxiATFkGPS0L+3lrX8h9y0/Vd6hmRLwwT+Ld /wmWsKkJH8C4+iGWLNv4T6Ey09kdelU0r3EO3xomY0VSxwRRkUzeMRX44RluNHiDG4GN c3KQ== X-Gm-Message-State: AOJu0YxtCGaLx9M1N2Z0+BQ8sPX0TL94WcI3nN+AmmN85e5B7fWmC1R/ PGjEX47QWoTM/rYv2XPc7Hw= X-Received: by 2002:a17:90b:3b92:b0:285:a72f:966f with SMTP id pc18-20020a17090b3b9200b00285a72f966fmr8454941pjb.46.1701113826185; Mon, 27 Nov 2023 11:37:06 -0800 (PST) Received: from localhost (fwdproxy-prn-026.fbsv.net. [2a03:2880:ff:1a::face:b00c]) by smtp.gmail.com with ESMTPSA id a21-20020a17090ad81500b002851e283c21sm7822831pjv.12.2023.11.27.11.37.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Nov 2023 11:37:05 -0800 (PST) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, cerasuolodomenico@gmail.com, yosryahmed@google.com, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, chrisl@kernel.org, linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, shuah@kernel.org Subject: [PATCH v6 2/6] memcontrol: allows mem_cgroup_iter() to check for onlineness Date: Mon, 27 Nov 2023 11:36:59 -0800 Message-Id: <20231127193703.1980089-3-nphamcs@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231127193703.1980089-1-nphamcs@gmail.com> References: <20231127193703.1980089-1-nphamcs@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Mon, 27 Nov 2023 11:37:24 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783747151331478794 X-GMAIL-MSGID: 1783747151331478794 The new zswap writeback scheme requires an online-only memcg hierarchy traversal. Add a new parameter to mem_cgroup_iter() to check for onlineness before returning. Signed-off-by: Nhat Pham --- include/linux/memcontrol.h | 4 ++-- mm/memcontrol.c | 17 ++++++++++------- mm/shrinker.c | 4 ++-- mm/vmscan.c | 26 +++++++++++++------------- 4 files changed, 27 insertions(+), 24 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 7bdcf3020d7a..86adce081a08 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -832,7 +832,7 @@ static inline void mem_cgroup_put(struct mem_cgroup *memcg) struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *, struct mem_cgroup *, - struct mem_cgroup_reclaim_cookie *); + struct mem_cgroup_reclaim_cookie *, bool online); void mem_cgroup_iter_break(struct mem_cgroup *, struct mem_cgroup *); void mem_cgroup_scan_tasks(struct mem_cgroup *memcg, int (*)(struct task_struct *, void *), void *arg); @@ -1381,7 +1381,7 @@ static inline struct lruvec *folio_lruvec_lock_irqsave(struct folio *folio, static inline struct mem_cgroup * mem_cgroup_iter(struct mem_cgroup *root, struct mem_cgroup *prev, - struct mem_cgroup_reclaim_cookie *reclaim) + struct mem_cgroup_reclaim_cookie *reclaim, bool online) { return NULL; } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 564aa8f25b71..a1f051adaa15 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -221,14 +221,14 @@ enum res_type { * be used for reference counting. */ #define for_each_mem_cgroup_tree(iter, root) \ - for (iter = mem_cgroup_iter(root, NULL, NULL); \ + for (iter = mem_cgroup_iter(root, NULL, NULL, false); \ iter != NULL; \ - iter = mem_cgroup_iter(root, iter, NULL)) + iter = mem_cgroup_iter(root, iter, NULL, false)) #define for_each_mem_cgroup(iter) \ - for (iter = mem_cgroup_iter(NULL, NULL, NULL); \ + for (iter = mem_cgroup_iter(NULL, NULL, NULL, false); \ iter != NULL; \ - iter = mem_cgroup_iter(NULL, iter, NULL)) + iter = mem_cgroup_iter(NULL, iter, NULL, false)) static inline bool task_is_dying(void) { @@ -1115,6 +1115,7 @@ struct mem_cgroup *get_mem_cgroup_from_current(void) * @root: hierarchy root * @prev: previously returned memcg, NULL on first invocation * @reclaim: cookie for shared reclaim walks, NULL for full walks + * @online: skip offline memcgs * * Returns references to children of the hierarchy below @root, or * @root itself, or %NULL after a full round-trip. @@ -1129,7 +1130,8 @@ struct mem_cgroup *get_mem_cgroup_from_current(void) */ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, struct mem_cgroup *prev, - struct mem_cgroup_reclaim_cookie *reclaim) + struct mem_cgroup_reclaim_cookie *reclaim, + bool online) { struct mem_cgroup_reclaim_iter *iter; struct cgroup_subsys_state *css = NULL; @@ -1199,7 +1201,8 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, * is provided by the caller, so we know it's alive * and kicking, and don't take an extra reference. */ - if (css == &root->css || css_tryget(css)) { + if (css == &root->css || (!online && css_tryget(css)) || + css_tryget_online(css)) { memcg = mem_cgroup_from_css(css); break; } @@ -1812,7 +1815,7 @@ static int mem_cgroup_soft_reclaim(struct mem_cgroup *root_memcg, excess = soft_limit_excess(root_memcg); while (1) { - victim = mem_cgroup_iter(root_memcg, victim, &reclaim); + victim = mem_cgroup_iter(root_memcg, victim, &reclaim, false); if (!victim) { loop++; if (loop >= 2) { diff --git a/mm/shrinker.c b/mm/shrinker.c index dd91eab43ed3..54f5d3aa4f27 100644 --- a/mm/shrinker.c +++ b/mm/shrinker.c @@ -160,7 +160,7 @@ static int expand_shrinker_info(int new_id) new_size = shrinker_unit_size(new_nr_max); old_size = shrinker_unit_size(shrinker_nr_max); - memcg = mem_cgroup_iter(NULL, NULL, NULL); + memcg = mem_cgroup_iter(NULL, NULL, NULL, false); do { ret = expand_one_shrinker_info(memcg, new_size, old_size, new_nr_max); @@ -168,7 +168,7 @@ static int expand_shrinker_info(int new_id) mem_cgroup_iter_break(NULL, memcg); goto out; } - } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL)) != NULL); + } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL, false)) != NULL); out: if (!ret) shrinker_nr_max = new_nr_max; diff --git a/mm/vmscan.c b/mm/vmscan.c index d8c3338fee0f..9a65ee3a1bb7 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -397,10 +397,10 @@ static unsigned long drop_slab_node(int nid) unsigned long freed = 0; struct mem_cgroup *memcg = NULL; - memcg = mem_cgroup_iter(NULL, NULL, NULL); + memcg = mem_cgroup_iter(NULL, NULL, NULL, false); do { freed += shrink_slab(GFP_KERNEL, nid, memcg, 0); - } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL)) != NULL); + } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL, false)) != NULL); return freed; } @@ -3935,7 +3935,7 @@ static void lru_gen_age_node(struct pglist_data *pgdat, struct scan_control *sc) if (!min_ttl || sc->order || sc->priority == DEF_PRIORITY) return; - memcg = mem_cgroup_iter(NULL, NULL, NULL); + memcg = mem_cgroup_iter(NULL, NULL, NULL, false); do { struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat); @@ -3945,7 +3945,7 @@ static void lru_gen_age_node(struct pglist_data *pgdat, struct scan_control *sc) } cond_resched(); - } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL))); + } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL, false))); /* * The main goal is to OOM kill if every generation from all memcgs is @@ -5037,7 +5037,7 @@ static void lru_gen_change_state(bool enabled) else static_branch_disable_cpuslocked(&lru_gen_caps[LRU_GEN_CORE]); - memcg = mem_cgroup_iter(NULL, NULL, NULL); + memcg = mem_cgroup_iter(NULL, NULL, NULL, false); do { int nid; @@ -5061,7 +5061,7 @@ static void lru_gen_change_state(bool enabled) } cond_resched(); - } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL))); + } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL, false))); unlock: mutex_unlock(&state_mutex); put_online_mems(); @@ -5164,7 +5164,7 @@ static void *lru_gen_seq_start(struct seq_file *m, loff_t *pos) if (!m->private) return ERR_PTR(-ENOMEM); - memcg = mem_cgroup_iter(NULL, NULL, NULL); + memcg = mem_cgroup_iter(NULL, NULL, NULL, false); do { int nid; @@ -5172,7 +5172,7 @@ static void *lru_gen_seq_start(struct seq_file *m, loff_t *pos) if (!nr_to_skip--) return get_lruvec(memcg, nid); } - } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL))); + } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL, false))); return NULL; } @@ -5195,7 +5195,7 @@ static void *lru_gen_seq_next(struct seq_file *m, void *v, loff_t *pos) nid = next_memory_node(nid); if (nid == MAX_NUMNODES) { - memcg = mem_cgroup_iter(NULL, memcg, NULL); + memcg = mem_cgroup_iter(NULL, memcg, NULL, false); if (!memcg) return NULL; @@ -5798,7 +5798,7 @@ static void shrink_node_memcgs(pg_data_t *pgdat, struct scan_control *sc) struct mem_cgroup *target_memcg = sc->target_mem_cgroup; struct mem_cgroup *memcg; - memcg = mem_cgroup_iter(target_memcg, NULL, NULL); + memcg = mem_cgroup_iter(target_memcg, NULL, NULL, false); do { struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat); unsigned long reclaimed; @@ -5855,7 +5855,7 @@ static void shrink_node_memcgs(pg_data_t *pgdat, struct scan_control *sc) sc->nr_scanned - scanned, sc->nr_reclaimed - reclaimed); - } while ((memcg = mem_cgroup_iter(target_memcg, memcg, NULL))); + } while ((memcg = mem_cgroup_iter(target_memcg, memcg, NULL, false))); } static void shrink_node(pg_data_t *pgdat, struct scan_control *sc) @@ -6522,12 +6522,12 @@ static void kswapd_age_node(struct pglist_data *pgdat, struct scan_control *sc) if (!inactive_is_low(lruvec, LRU_INACTIVE_ANON)) return; - memcg = mem_cgroup_iter(NULL, NULL, NULL); + memcg = mem_cgroup_iter(NULL, NULL, NULL, false); do { lruvec = mem_cgroup_lruvec(memcg, pgdat); shrink_active_list(SWAP_CLUSTER_MAX, lruvec, sc, LRU_ACTIVE_ANON); - memcg = mem_cgroup_iter(NULL, memcg, NULL); + memcg = mem_cgroup_iter(NULL, memcg, NULL, false); } while (memcg); }