From patchwork Thu Oct 19 22:53:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roman Gushchin X-Patchwork-Id: 155732 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:2010:b0:403:3b70:6f57 with SMTP id fe16csp698442vqb; Thu, 19 Oct 2023 15:54:40 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHvTtoP2rZ4upPOIZIcOdCLY/eFs+IgdsLK4zM9ZqdPa94HRdVR96SXSAu3+0qeBI21qFcl X-Received: by 2002:a05:6870:b254:b0:1e9:b79a:c6c6 with SMTP id b20-20020a056870b25400b001e9b79ac6c6mr338117oam.7.1697756079798; Thu, 19 Oct 2023 15:54:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1697756079; cv=none; d=google.com; s=arc-20160816; b=hfE+Cx79srVC6ynVwWpZ323EiK9T4TLMHNTxjGcffsFhVUN3nFz4FD6zO4X9+z337x rE2MgYwzT5hcFxpXvWsFVXKKPChj5t9nc+WtjiqzqGvGI2EH8hZuUtYseQv6M+enkJtp B8KooMHEftIf0SDm85hpGoSQ/jvHyEu+x3wqyDR6EU/FVk2lrI7QVrzqRE4h3hvGZ1op mAsBlIFv/dFUmwwHAETGGbxu4+K5YLjw8vzsqkgDewLzDF1GJjBLsmNnFWw3E+KEMJjf YOSzSFAejADAF0Op53O7eUMgjg8gqgKvMrxilhI07rlvrLWkR1yshHEdyeC1JmHRTkcx izIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=TjRbekCofEjC4WvfGU9ZZivZap7GfKCofqK2R8XbFtA=; fh=0sD7u+6ZeVLabXSJ0/ta2yf8mUfUXWwh1sJBhHvFyso=; b=TDlpuIBHpX7m+9qmASrZi9E4HyVF2olODkShtFCS1v3b8fN1dLfwIYx5cS9lx741FH 0WczrZudfygM7QEjtgxmB9BpQl3uIOLreGoz2FdzZH8/LEJLhPgb0UGSXtNaLls08pqv rSlXZdJofrsmRUgbzwZJAHd+wqJ1T2Ageqi9VZBkY4MOpn31qiLPhYcdd/ikuPdGkqVK 2TWoYPswp10PocJcoNbBjHju0vE5hEzJc4lJl3B7CNNP3cUnqt44bXLjUR//VBGzrCgf Qh51qWBts3/7Jf7HVIyHrYxd90rY8bz/YWLR7VIJSFM14Z/UZLNuqzI+zgeG2GDH6f/l 6Jjw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=CSSCOtFv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id t10-20020a63b70a000000b005898b54186esi509900pgf.96.2023.10.19.15.54.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Oct 2023 15:54:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=CSSCOtFv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 7ADA58280D5B; Thu, 19 Oct 2023 15:54:38 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346663AbjJSWyb (ORCPT + 26 others); Thu, 19 Oct 2023 18:54:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58886 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346716AbjJSWy0 (ORCPT ); Thu, 19 Oct 2023 18:54:26 -0400 Received: from out-201.mta0.migadu.com (out-201.mta0.migadu.com [91.218.175.201]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C9DED130 for ; Thu, 19 Oct 2023 15:54:20 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1697756059; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TjRbekCofEjC4WvfGU9ZZivZap7GfKCofqK2R8XbFtA=; b=CSSCOtFvi7Ic1PfkshMhl604bLC9wtck8ncGfvc+CVnVI76s2gWESUbI5kqm+wkB4NEvbZ f6YjpCeVY9hDbH+wgY8JXIJC3BqE3wtExzGADF2pr5NRWaoryj7dR2Vq4KjLwkGq+VQNHQ phUAsoG9TN53NmVeOn9ebjkFEG27Nx4= From: Roman Gushchin To: Andrew Morton Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Dennis Zhou , David Rientjes , Vlastimil Babka , Naresh Kamboju , Roman Gushchin Subject: [PATCH v5 3/6] mm: kmem: make memcg keep a reference to the original objcg Date: Thu, 19 Oct 2023 15:53:43 -0700 Message-ID: <20231019225346.1822282-4-roman.gushchin@linux.dev> In-Reply-To: <20231019225346.1822282-1-roman.gushchin@linux.dev> References: <20231019225346.1822282-1-roman.gushchin@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Thu, 19 Oct 2023 15:54:38 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1780226279086195893 X-GMAIL-MSGID: 1780226279086195893 Keep a reference to the original objcg object for the entire life of a memcg structure. This allows to simplify the synchronization on the kernel memory allocation paths: pinning a (live) memcg will also pin the corresponding objcg. The memory overhead of this change is minimal because object cgroups usually outlive their corresponding memory cgroups even without this change, so it's only an additional pointer per memcg. Signed-off-by: Roman Gushchin (Cruise) Tested-by: Naresh Kamboju Acked-by: Shakeel Butt Reviewed-by: Vlastimil Babka --- include/linux/memcontrol.h | 8 +++++++- mm/memcontrol.c | 5 +++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index ab94ad4597d0..277690af383d 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -299,7 +299,13 @@ struct mem_cgroup { #ifdef CONFIG_MEMCG_KMEM int kmemcg_id; - struct obj_cgroup __rcu *objcg; + /* + * memcg->objcg is wiped out as a part of the objcg repaprenting + * process. memcg->orig_objcg preserves a pointer (and a reference) + * to the original objcg until the end of live of memcg. + */ + struct obj_cgroup __rcu *objcg; + struct obj_cgroup *orig_objcg; /* list of inherited objcgs, protected by objcg_lock */ struct list_head objcg_list; #endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 4c4b1f85f939..d964b91f00c8 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3817,6 +3817,8 @@ static int memcg_online_kmem(struct mem_cgroup *memcg) objcg->memcg = memcg; rcu_assign_pointer(memcg->objcg, objcg); + obj_cgroup_get(objcg); + memcg->orig_objcg = objcg; static_branch_enable(&memcg_kmem_online_key); @@ -5311,6 +5313,9 @@ static void __mem_cgroup_free(struct mem_cgroup *memcg) { int node; + if (memcg->orig_objcg) + obj_cgroup_put(memcg->orig_objcg); + for_each_node(node) free_mem_cgroup_per_node_info(memcg, node); kfree(memcg->vmstats);