From patchwork Fri Oct 6 18:46:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 149381 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a888:0:b0:403:3b70:6f57 with SMTP id x8csp528661vqo; Fri, 6 Oct 2023 11:47:39 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEFt64Zr/ITxSFlAUkjFOAEVAdJZh/FuyzYXeCMoGzd7dTBAN9ILQceZLcopF8e2GqAgaXU X-Received: by 2002:a17:90a:cc01:b0:279:47d:318a with SMTP id b1-20020a17090acc0100b00279047d318amr8485907pju.29.1696618059132; Fri, 06 Oct 2023 11:47:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696618059; cv=none; d=google.com; s=arc-20160816; b=Mg1386K2nMPCfR/podZiaSMvwIK1aNZ7+hxN31wjY+sQCdRDTSD2F4ZB96Ngdfsp/E n3VF4YggbWxWRRodh3ve3vBF6BBYSk1HUbpV81HDoD6wk3g+bZPWkZZAz+bCNJYkjPeM 1z+BzepAgfCh3LSaM2amYKC3mBnZs4BhsedDM9dyozDHy8kEEcbLNEW47Xcst4W+I7cB xhplwncGaXJv4eyMl8fabpssY25/Rumjm2s4yz7BmX+SLSlSj9a3vCbmBQHX9Wizg/eW vdOcEdWedgn3V1ye0PPQf2ui6mzRuAfCpIsDpkHCoMUPMeJp1al0jk169HWu3k4jBAnd HIOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=IQ3fDKoP0HxAKgFx5HjhrMjKnZYWAS/nwLzGqRn4tZU=; fh=iuOdG2UsaRKUpHOR3B0vXOw1fow3AOmqKEKZ7KuRmaE=; b=zLZaNS/QKXViyyNxtZ25HiXtfzXfMdz63gmwgy+UVTkVee7EyOWx4tFOo02AFs4usb PSX11uFTl15B2kvchSJp/AiaqVsaingwBskufw8Hd3ZgFgpUNx1Pdc0ry0lGg8Y3n88n WdH0uC0Dkqg3dguFCZ5Y+LPYHscmEots3j/7L05Ot+miXSGk45C8leWSod12xnuWBiIs AflAjFqrCsFAXlUGR+jiE1YTWP1OK8QjbOO7+G1jUK/rfDU+4RBDJf8QnCO9PX/VvIx1 rwBebvT2VPnmxaFnc9T1UQuN+IvX08kzy1r+f1bTfQHQUZuEsbQ4Cto8KycUxRtk3SZ7 Torg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=neycuRVK; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id y16-20020a17090264d000b001c755810f80si4126327pli.28.2023.10.06.11.47.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Oct 2023 11:47:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=neycuRVK; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 9FE9580E7123; Fri, 6 Oct 2023 11:47:00 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233254AbjJFSqf (ORCPT + 18 others); Fri, 6 Oct 2023 14:46:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50740 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233232AbjJFSqe (ORCPT ); Fri, 6 Oct 2023 14:46:34 -0400 Received: from mail-pl1-x630.google.com (mail-pl1-x630.google.com [IPv6:2607:f8b0:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 57AB8CF; Fri, 6 Oct 2023 11:46:32 -0700 (PDT) Received: by mail-pl1-x630.google.com with SMTP id d9443c01a7336-1c877f27e8fso19769075ad.1; Fri, 06 Oct 2023 11:46:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696617992; x=1697222792; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IQ3fDKoP0HxAKgFx5HjhrMjKnZYWAS/nwLzGqRn4tZU=; b=neycuRVKvb8N1AEPcwUgrEdxwVeqzRrMSJJ7QeWYChnBlSU09nA7R6vcFcc6kJcpL6 FsFLkoJmhwpkiR1EpQQMhUXqY4YATGPRaNxAbY3Bh1EyeU50kUZT8Eu74a2rbok89ICh iZAFJO3KIFJwOuMtNVbO19H7CDVuauxLps9z8iiHIGNBaIH7D7ju/FZK6MgKYmpOOgkD gygXjN7InzXmS+diHyrGdvY8cuGLeObzqkOyaEY64PMXC0lHiuI6DkLVFJBKLjCc+1o9 mAaj5Vo9urjt8yie7nG1fOt9NZNcyZUSubOb8NBxZk3fSip/YAJkAnLdvTTiuvhmEV5L 90vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696617992; x=1697222792; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IQ3fDKoP0HxAKgFx5HjhrMjKnZYWAS/nwLzGqRn4tZU=; b=ADQBCnIzOVlD3LdB0SyYXwNXoKwIoXva5gQ7TWQp7YRPNRcV0ed15O/uIZKgFKIo5N vVV1agHdq7PfNCJFkVVmXn8Az2KmLWUFxr3WSOZUrUbPjcuSMBM3EKzZH0hxuSDhuYZS 5AK7U+ffnv1VQJ0Yw+NKMPoiR2kUCU1B7yLH4orqVL0vaYM1iD6zAOZ4tiXAYuGNEc2U z/gMQbtGZ3RplX3xnQWoN+EX1udywXuiRYn0ryeRtaX7qTkLt6vc4WkBJwHqCZ/yGe/U MHNZfJZp4JwNS8d9HlhH1Tk6FVF3MSdneRNS2iwc/7ed8E4Wyt9OQmM8CK9kOpYjAFrf jyyQ== X-Gm-Message-State: AOJu0Yyqk2tyGlIOpTdajicZvs1dJoGBNJjZaCOmNUiFjB+wqJgYjOWf gFg7k16n7R//QX+SdCfl8YA= X-Received: by 2002:a17:90a:b891:b0:273:441a:dae6 with SMTP id o17-20020a17090ab89100b00273441adae6mr8808492pjr.19.1696617991566; Fri, 06 Oct 2023 11:46:31 -0700 (PDT) Received: from localhost (fwdproxy-prn-000.fbsv.net. [2a03:2880:ff::face:b00c]) by smtp.gmail.com with ESMTPSA id s20-20020a170902989400b001c5b8087fe5sm4250869plp.94.2023.10.06.11.46.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Oct 2023 11:46:31 -0700 (PDT) From: Nhat Pham To: akpm@linux-foundation.org Cc: riel@surriel.com, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, tj@kernel.org, lizefan.x@bytedance.com, shuah@kernel.org, mike.kravetz@oracle.com, yosryahmed@google.com, fvdl@google.com, linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org Subject: [PATCH v4 1/4] memcontrol: add helpers for hugetlb memcg accounting Date: Fri, 6 Oct 2023 11:46:26 -0700 Message-Id: <20231006184629.155543-2-nphamcs@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231006184629.155543-1-nphamcs@gmail.com> References: <20231006184629.155543-1-nphamcs@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Fri, 06 Oct 2023 11:47:00 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779032977825522042 X-GMAIL-MSGID: 1779032977825522042 This patch exposes charge committing and cancelling as parts of the memory controller interface. These functionalities are useful when the try_charge() and commit_charge() stages have to be separated by other actions in between (which can fail). One such example is the new hugetlb accounting behavior in the following patch. The patch also adds a helper function to obtain a reference to the current task's memcg. Acked-by: Michal Hocko Acked-by: Johannes Weiner Signed-off-by: Nhat Pham --- include/linux/memcontrol.h | 21 ++++++++++++++ mm/memcontrol.c | 59 ++++++++++++++++++++++++++++++-------- 2 files changed, 68 insertions(+), 12 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index e0cfab58ab71..42bf7e9b1a2f 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -653,6 +653,8 @@ static inline bool mem_cgroup_below_min(struct mem_cgroup *target, page_counter_read(&memcg->memory); } +void mem_cgroup_commit_charge(struct folio *folio, struct mem_cgroup *memcg); + int __mem_cgroup_charge(struct folio *folio, struct mm_struct *mm, gfp_t gfp); /** @@ -704,6 +706,8 @@ static inline void mem_cgroup_uncharge_list(struct list_head *page_list) __mem_cgroup_uncharge_list(page_list); } +void mem_cgroup_cancel_charge(struct mem_cgroup *memcg, unsigned int nr_pages); + void mem_cgroup_migrate(struct folio *old, struct folio *new); /** @@ -760,6 +764,8 @@ struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p); struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm); +struct mem_cgroup *get_mem_cgroup_from_current(void); + struct lruvec *folio_lruvec_lock(struct folio *folio); struct lruvec *folio_lruvec_lock_irq(struct folio *folio); struct lruvec *folio_lruvec_lock_irqsave(struct folio *folio, @@ -1245,6 +1251,11 @@ static inline bool mem_cgroup_below_min(struct mem_cgroup *target, return false; } +static inline void mem_cgroup_commit_charge(struct folio *folio, + struct mem_cgroup *memcg) +{ +} + static inline int mem_cgroup_charge(struct folio *folio, struct mm_struct *mm, gfp_t gfp) { @@ -1269,6 +1280,11 @@ static inline void mem_cgroup_uncharge_list(struct list_head *page_list) { } +static inline void mem_cgroup_cancel_charge(struct mem_cgroup *memcg, + unsigned int nr_pages) +{ +} + static inline void mem_cgroup_migrate(struct folio *old, struct folio *new) { } @@ -1306,6 +1322,11 @@ static inline struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm) return NULL; } +static inline struct mem_cgroup *get_mem_cgroup_from_current(void) +{ + return NULL; +} + static inline struct mem_cgroup *mem_cgroup_from_css(struct cgroup_subsys_state *css) { diff --git a/mm/memcontrol.c b/mm/memcontrol.c index d1a322a75172..0219befeae38 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1086,6 +1086,27 @@ struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm) } EXPORT_SYMBOL(get_mem_cgroup_from_mm); +/** + * get_mem_cgroup_from_current - Obtain a reference on current task's memcg. + */ +struct mem_cgroup *get_mem_cgroup_from_current(void) +{ + struct mem_cgroup *memcg; + + if (mem_cgroup_disabled()) + return NULL; + +again: + rcu_read_lock(); + memcg = mem_cgroup_from_task(current); + if (!css_tryget(&memcg->css)) { + rcu_read_unlock(); + goto again; + } + rcu_read_unlock(); + return memcg; +} + static __always_inline bool memcg_kmem_bypass(void) { /* Allow remote memcg charging from any context. */ @@ -2873,7 +2894,12 @@ static inline int try_charge(struct mem_cgroup *memcg, gfp_t gfp_mask, return try_charge_memcg(memcg, gfp_mask, nr_pages); } -static inline void cancel_charge(struct mem_cgroup *memcg, unsigned int nr_pages) +/** + * mem_cgroup_cancel_charge() - cancel an uncommitted try_charge() call. + * @memcg: memcg previously charged. + * @nr_pages: number of pages previously charged. + */ +void mem_cgroup_cancel_charge(struct mem_cgroup *memcg, unsigned int nr_pages) { if (mem_cgroup_is_root(memcg)) return; @@ -2898,6 +2924,22 @@ static void commit_charge(struct folio *folio, struct mem_cgroup *memcg) folio->memcg_data = (unsigned long)memcg; } +/** + * mem_cgroup_commit_charge - commit a previously successful try_charge(). + * @folio: folio to commit the charge to. + * @memcg: memcg previously charged. + */ +void mem_cgroup_commit_charge(struct folio *folio, struct mem_cgroup *memcg) +{ + css_get(&memcg->css); + commit_charge(folio, memcg); + + local_irq_disable(); + mem_cgroup_charge_statistics(memcg, folio_nr_pages(folio)); + memcg_check_events(memcg, folio_nid(folio)); + local_irq_enable(); +} + #ifdef CONFIG_MEMCG_KMEM /* * The allocated objcg pointers array is not accounted directly. @@ -6105,7 +6147,7 @@ static void __mem_cgroup_clear_mc(void) /* we must uncharge all the leftover precharges from mc.to */ if (mc.precharge) { - cancel_charge(mc.to, mc.precharge); + mem_cgroup_cancel_charge(mc.to, mc.precharge); mc.precharge = 0; } /* @@ -6113,7 +6155,7 @@ static void __mem_cgroup_clear_mc(void) * we must uncharge here. */ if (mc.moved_charge) { - cancel_charge(mc.from, mc.moved_charge); + mem_cgroup_cancel_charge(mc.from, mc.moved_charge); mc.moved_charge = 0; } /* we must fixup refcnts and charges */ @@ -7020,20 +7062,13 @@ void mem_cgroup_calculate_protection(struct mem_cgroup *root, static int charge_memcg(struct folio *folio, struct mem_cgroup *memcg, gfp_t gfp) { - long nr_pages = folio_nr_pages(folio); int ret; - ret = try_charge(memcg, gfp, nr_pages); + ret = try_charge(memcg, gfp, folio_nr_pages(folio)); if (ret) goto out; - css_get(&memcg->css); - commit_charge(folio, memcg); - - local_irq_disable(); - mem_cgroup_charge_statistics(memcg, nr_pages); - memcg_check_events(memcg, folio_nid(folio)); - local_irq_enable(); + mem_cgroup_commit_charge(folio, memcg); out: return ret; } From patchwork Fri Oct 6 18:46:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 149379 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a888:0:b0:403:3b70:6f57 with SMTP id x8csp528453vqo; Fri, 6 Oct 2023 11:47:11 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEZIqCcOTOshUE2obe3NpgXWVF5hPhd+xKj1/Kp+Fwpg1PHFrsdJA6oPPmay3goue+cqtkQ X-Received: by 2002:a05:6a00:2e23:b0:68a:59c6:c0a6 with SMTP id fc35-20020a056a002e2300b0068a59c6c0a6mr11389332pfb.24.1696618030785; Fri, 06 Oct 2023 11:47:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696618030; cv=none; d=google.com; s=arc-20160816; b=OyudjQVy3l256oI0TgXS8kt+SWskf13zBPK9wdo0y+wjuZGo9482CoXrsWFOltwj8A WbcT9TpSerb84vXz/QvIwQ+Imaf7U+OMyutMdrLK49fTBrlbIJsCtBDK3kypCMb8SlWv xkxTL/88CC4ezzMR02h0xrGH85oDypcHp8sJsJb+qhppawyRfXLfjDHdZDQ9fYJgU6R4 hBLMwuE7iS7flEpZuxEVTa/3nxuHL3IW+QMCcHCWScZ4Qb0lMIljCeHiluRK+4ymTk5s kNwYp2Mwg/1e6EsEU8Hm9CCCZBX2dNqgyADWPos+gffHWe3opFfbD7oHi7JRFAtsUbO4 95dA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=6PLbRh1GdQSuzgMGxcjYajdp+r42f9X7B8/Cs8IIw1E=; fh=iuOdG2UsaRKUpHOR3B0vXOw1fow3AOmqKEKZ7KuRmaE=; b=MDFsfcYbGbKqZZvDZ0weYT3ZL2hi6ZeB1GY9Uu9prrGbjCWrOunhmeBxmo4/aN7tIB paFdnOLEsFMeeWisDlMQXccjiriiL8INkeFT6uHtXnl0jsl2S1f8EeudL4TpTT5Egl3v QlEYQFRIJUJHwQpYXNtOJDkiN90d8XsV/npK6/lCZ8t1+YOPMdEHQ4rq6tx6LxwqxBK1 VKOR/YhnDKTMuL4kE4RLovg7C+whpR0GncKbOgMU7MCeIQNyuGwCYteUEbsNNCwV3jE0 mYDBVYacGo4409O+WQ99o19PNz4H+Jw3Tet8tUnD6MD8QXc7aA+H/U9a8p0Gf+cACYSh 8aSQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b="FA8/2iUm"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id p12-20020a056a000b4c00b0069022a03d42si2122362pfo.283.2023.10.06.11.47.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Oct 2023 11:47:10 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b="FA8/2iUm"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 0EC0A8184521; Fri, 6 Oct 2023 11:47:07 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233304AbjJFSqp (ORCPT + 18 others); Fri, 6 Oct 2023 14:46:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50776 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233130AbjJFSqf (ORCPT ); Fri, 6 Oct 2023 14:46:35 -0400 Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6FA86C2; Fri, 6 Oct 2023 11:46:34 -0700 (PDT) Received: by mail-pg1-x52c.google.com with SMTP id 41be03b00d2f7-53fbf2c42bfso1889872a12.3; Fri, 06 Oct 2023 11:46:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696617994; x=1697222794; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6PLbRh1GdQSuzgMGxcjYajdp+r42f9X7B8/Cs8IIw1E=; b=FA8/2iUmvjVAYzzY1Nv6MpC7piEEiiiYT7jeohXVNotEEmCRsRfYlmfVJk1EF5ROOc E07EBGZLk1gX73IBv7/ndrcAssQcTi/+kA58T0Ny92pjt5YczpofF7kibwHqDjiquFof 0yKDm+5n0WhsqhgwZb68G3UXqWXF/DstyKR00F5wa9K86bzo6+7vbFqyuDsqLsRp/B0H EvZ2GGbECSoiEfq4//vVANcc8dZ6SN836eirWahfyu4b49nM/1/XyasXARnabwtzbYFw ZqAtpuUDSJT2PLMuH5hyU3krqv3ZudCUnvFWDhzyXhWxN6PSC1lED6khBVdOmeVdR52w QbIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696617994; x=1697222794; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6PLbRh1GdQSuzgMGxcjYajdp+r42f9X7B8/Cs8IIw1E=; b=jgajFpSzm51uxIi+uiG9BNhDwMUZ7RP3dwMQK0/z72tvEdVDKU4QLTVISCZDFjjkOh MTfMwvEOlP4PoiXSqaAGkwrCGqOM9LHR3ozPu5s32cVve2bY0Xe+ff8UKy8W0oKEaP2z qszLBOXrDwAXU9zb3q6uH7XwfypSyzCpX+AAcY0gJWpcFilgp+/Rg8uGB45dRvNn7GX+ 0eawB4fdHg+j9XfY162mvsdaCp2AefES4WcB/hIcecfBx9Eg6MSVZO5WJaR2fZs8HNKF NagWWPwYqW3mmaPJn818LPAEE7Y8xYG0Szy85JY+epwpZwgEHPkPYxgQ+lEYUN4EXkQk sPZQ== X-Gm-Message-State: AOJu0Yy7Y7IHzQ+cVMyFPrF7QRW+j93N+lzA6NwNStxwZdrirdVPQ2yj wvs7dur+vvmqDA2VenKx+Jw= X-Received: by 2002:a05:6a21:798a:b0:16c:bd7e:d524 with SMTP id bh10-20020a056a21798a00b0016cbd7ed524mr566168pzc.57.1696617993719; Fri, 06 Oct 2023 11:46:33 -0700 (PDT) Received: from localhost (fwdproxy-prn-015.fbsv.net. [2a03:2880:ff:f::face:b00c]) by smtp.gmail.com with ESMTPSA id g22-20020aa78756000000b00682669dc19bsm1800647pfo.201.2023.10.06.11.46.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Oct 2023 11:46:32 -0700 (PDT) From: Nhat Pham To: akpm@linux-foundation.org Cc: riel@surriel.com, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, tj@kernel.org, lizefan.x@bytedance.com, shuah@kernel.org, mike.kravetz@oracle.com, yosryahmed@google.com, fvdl@google.com, linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org Subject: [PATCH v4 2/4] memcontrol: only transfer the memcg data for migration Date: Fri, 6 Oct 2023 11:46:27 -0700 Message-Id: <20231006184629.155543-3-nphamcs@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231006184629.155543-1-nphamcs@gmail.com> References: <20231006184629.155543-1-nphamcs@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=3.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_SBL_CSS, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Fri, 06 Oct 2023 11:47:07 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779032948088449716 X-GMAIL-MSGID: 1779032948088449716 For most migration use cases, only transfer the memcg data from the old folio to the new folio, and clear the old folio's memcg data. No charging and uncharging will be done. This shaves off some work on the migration path, and avoids the temporary double charging of a folio during its migration. The only exception is replace_page_cache_folio(), which will use the old mem_cgroup_migrate() (now renamed to mem_cgroup_replace_folio). In that context, the isolation of the old page isn't quite as thorough as with migration, so we cannot use our new implementation directly. This patch is the result of the following discussion on the new hugetlb memcg accounting behavior: https://lore.kernel.org/lkml/20231003171329.GB314430@monkey/ Suggested-by: Johannes Weiner Signed-off-by: Nhat Pham Acked-by: Johannes Weiner --- include/linux/memcontrol.h | 7 +++++++ mm/filemap.c | 2 +- mm/memcontrol.c | 40 +++++++++++++++++++++++++++++++++++--- 3 files changed, 45 insertions(+), 4 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 42bf7e9b1a2f..5daf14da3759 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -708,6 +708,8 @@ static inline void mem_cgroup_uncharge_list(struct list_head *page_list) void mem_cgroup_cancel_charge(struct mem_cgroup *memcg, unsigned int nr_pages); +void mem_cgroup_replace_folio(struct folio *old, struct folio *new); + void mem_cgroup_migrate(struct folio *old, struct folio *new); /** @@ -1285,6 +1287,11 @@ static inline void mem_cgroup_cancel_charge(struct mem_cgroup *memcg, { } +static inline void mem_cgroup_replace_folio(struct folio *old, + struct folio *new) +{ +} + static inline void mem_cgroup_migrate(struct folio *old, struct folio *new) { } diff --git a/mm/filemap.c b/mm/filemap.c index 9481ffaf24e6..673745219c82 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -819,7 +819,7 @@ void replace_page_cache_folio(struct folio *old, struct folio *new) new->mapping = mapping; new->index = offset; - mem_cgroup_migrate(old, new); + mem_cgroup_replace_folio(old, new); xas_lock_irq(&xas); xas_store(&xas, new); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 0219befeae38..b9c479d768e2 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -7281,16 +7281,17 @@ void __mem_cgroup_uncharge_list(struct list_head *page_list) } /** - * mem_cgroup_migrate - Charge a folio's replacement. + * mem_cgroup_replace_folio - Charge a folio's replacement. * @old: Currently circulating folio. * @new: Replacement folio. * * Charge @new as a replacement folio for @old. @old will - * be uncharged upon free. + * be uncharged upon free. This is only used by the page cache + * (in replace_page_cache_folio()). * * Both folios must be locked, @new->mapping must be set up. */ -void mem_cgroup_migrate(struct folio *old, struct folio *new) +void mem_cgroup_replace_folio(struct folio *old, struct folio *new) { struct mem_cgroup *memcg; long nr_pages = folio_nr_pages(new); @@ -7329,6 +7330,39 @@ void mem_cgroup_migrate(struct folio *old, struct folio *new) local_irq_restore(flags); } +/** + * mem_cgroup_migrate - Transfer the memcg data from the old to the new folio. + * @old: Currently circulating folio. + * @new: Replacement folio. + * + * Transfer the memcg data from the old folio to the new folio for migration. + * The old folio's data info will be cleared. Note that the memory counters + * will remain unchanged throughout the process. + * + * Both folios must be locked, @new->mapping must be set up. + */ +void mem_cgroup_migrate(struct folio *old, struct folio *new) +{ + struct mem_cgroup *memcg; + + VM_BUG_ON_FOLIO(!folio_test_locked(old), old); + VM_BUG_ON_FOLIO(!folio_test_locked(new), new); + VM_BUG_ON_FOLIO(folio_test_anon(old) != folio_test_anon(new), new); + VM_BUG_ON_FOLIO(folio_nr_pages(old) != folio_nr_pages(new), new); + + if (mem_cgroup_disabled()) + return; + + memcg = folio_memcg(old); + VM_WARN_ON_ONCE_FOLIO(!memcg, old); + if (!memcg) + return; + + /* Transfer the charge and the css ref */ + commit_charge(new, memcg); + old->memcg_data = 0; +} + DEFINE_STATIC_KEY_FALSE(memcg_sockets_enabled_key); EXPORT_SYMBOL(memcg_sockets_enabled_key); From patchwork Fri Oct 6 18:46:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 149380 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a888:0:b0:403:3b70:6f57 with SMTP id x8csp528594vqo; Fri, 6 Oct 2023 11:47:30 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFCS/Ap4RVgVlyS+zowDBHTXv3QGc0P2SlBVBmm5rmfHYgC+3KW5cdLGyUzH1T2yeqyNJFq X-Received: by 2002:a9d:6e8a:0:b0:6bb:1071:ea72 with SMTP id a10-20020a9d6e8a000000b006bb1071ea72mr8632809otr.36.1696618050038; Fri, 06 Oct 2023 11:47:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696618050; cv=none; d=google.com; s=arc-20160816; b=R3+MmShpH8vw42dG3Qk8ACSKJE8jKjRxRqLC7qKIJxGkvF6i01jM+tINdHOshddzVL 0IBUpcX2U0rdp4srzvBH2ogDTXWWHWuWomu71BOdxBTsvMZa2M3YbYYtLYmPh6PJFfBu eJTjyL+v8cPD/UPDIXL3F3T7B76dcNaBp8Mc0HCfCaCjt3bM49gXphzFUbj31YRe3f3z ZGvJYYpd3wxgcINrFYibC9L+5fOv85dLp5TNdREFtJTmIbm66HJHS6af4hxMHGRybUuw ePYYplkIHlGO1HaN90S75U9LGaqAD4xkCHbLIteEs/6Z3lnO3GKfr4hmqAQUldc7WSiA XU9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=vY+Soyh4ezPzEsiTwAjxm3PLfRjJVUJ94CEs1TUSu9g=; fh=iuOdG2UsaRKUpHOR3B0vXOw1fow3AOmqKEKZ7KuRmaE=; b=Rityl5ozrgxbRzO18z1KDpUpTt85M27y15IaMpyi/KX5Iw4yMVCsjIwY63nFv3mLDG c55+b3qoXgD/oFBLZTNp7fy5MQ+wNnAL4xT6DEJkZK4tjN2BRIZfKwbsB1uIKMXhTv+8 5XoYOZa4sfKB121SJtTL7pidHenzkecBVZtQL5I4V2n66E1NHUbiZ1CnpKz8Nnl88m/c PcBX19pMZTdWTkcb0amwEKr/nLhEjCJ0C/7xbr1e62eO9Xoe57VE0YPesBGDgVesNldQ aVLJW18s2nN4g01XxHkKsjqgRMEHc06lW7hwdj8gTKWZJXGSngtQuECuL4MfRzg9LAi3 m2qA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=J4fXz1Xk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id e2-20020a637442000000b00578a7f5a0b2si4146306pgn.403.2023.10.06.11.47.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Oct 2023 11:47:29 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=J4fXz1Xk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 51CB0818453D; Fri, 6 Oct 2023 11:47:27 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233308AbjJFSqv (ORCPT + 18 others); Fri, 6 Oct 2023 14:46:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52672 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233269AbjJFSqn (ORCPT ); Fri, 6 Oct 2023 14:46:43 -0400 Received: from mail-oi1-x22a.google.com (mail-oi1-x22a.google.com [IPv6:2607:f8b0:4864:20::22a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C5B85D6; Fri, 6 Oct 2023 11:46:36 -0700 (PDT) Received: by mail-oi1-x22a.google.com with SMTP id 5614622812f47-3ae5ee80c0dso1535581b6e.3; Fri, 06 Oct 2023 11:46:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696617996; x=1697222796; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vY+Soyh4ezPzEsiTwAjxm3PLfRjJVUJ94CEs1TUSu9g=; b=J4fXz1Xk5EqxkJOGtF3SFwrKH5hRiQasPMJP75AvlgwJxXuNhr3Y/is41FytLfI/xT RVUqLYfLOW9MOKhosnnjAPgxyntvxNTcci9sNyAGTOuv4yPyXhslt5eSTm4+qiTHJLRH eG+LMzfx1iSY6uZUmuFKlF0hWPQrTK+dpBRFduvxoz55HCqO44btJas9yoJEGXaC4P4M qFzHdfwV+SxvCSkfaLWZhSrResgIgQhDddcBScuX1U4lVLIsLfxrG3YkmgP2c9JWlMFB IyfjuvMV8TtsNZXrgPjd0y/29d79OfIRRnvnXG46o/EU450yCIW1bs1qI/p+WBB3L2qQ YVvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696617996; x=1697222796; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vY+Soyh4ezPzEsiTwAjxm3PLfRjJVUJ94CEs1TUSu9g=; b=BB9O1P9OFoXsOAPeAvFgC+1CK7QftiqAuzG6hxuQZv5PXFApprQ4a0FcJPkSPocPQ9 qyt1Y+3oYWDK2eTpvCvEDVuVT4z1RQgQf2oGNfZ7ZEhcz5IvrtAXflWyDslDIp4/YPK3 WKDQXwAuNPHjSw16dfbmmo4s6ltE9dgtPJMwwfYI+AgWfibwis6IQ8/53mWS5AzHC2u/ tZrjgn2Q7qECYtznxVLZnrCSg45vt5/9RbNJibVUVcS1uFFN12F3DK3sHTnXsz69Zr46 hqgHyLJy5AiM/wXg5/o7bNuAh1Ybq9QDgboPrqnK3pVqwlk9nFLAJsToybvoydOzjvLP b4Ew== X-Gm-Message-State: AOJu0YyS4lpthtTdxz/nAVUqpNEdQUlUEoIwyQbdE7hf+10Dbymanxu6 +bm6zly2L+8xo9QQzFIBSxo= X-Received: by 2002:a54:4812:0:b0:3ad:c497:1336 with SMTP id j18-20020a544812000000b003adc4971336mr9048765oij.16.1696617995913; Fri, 06 Oct 2023 11:46:35 -0700 (PDT) Received: from localhost (fwdproxy-prn-013.fbsv.net. [2a03:2880:ff:d::face:b00c]) by smtp.gmail.com with ESMTPSA id a24-20020a637058000000b00581048ffc13sm3722085pgn.81.2023.10.06.11.46.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Oct 2023 11:46:35 -0700 (PDT) From: Nhat Pham To: akpm@linux-foundation.org Cc: riel@surriel.com, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, tj@kernel.org, lizefan.x@bytedance.com, shuah@kernel.org, mike.kravetz@oracle.com, yosryahmed@google.com, fvdl@google.com, linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org Subject: [PATCH v4 4/4] selftests: add a selftest to verify hugetlb usage in memcg Date: Fri, 6 Oct 2023 11:46:29 -0700 Message-Id: <20231006184629.155543-5-nphamcs@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231006184629.155543-1-nphamcs@gmail.com> References: <20231006184629.155543-1-nphamcs@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=3.0 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_SBL_CSS, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Fri, 06 Oct 2023 11:47:27 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779032968101290026 X-GMAIL-MSGID: 1779032968101290026 This patch add a new kselftest to demonstrate and verify the new hugetlb memcg accounting behavior. Signed-off-by: Nhat Pham --- MAINTAINERS | 2 + tools/testing/selftests/cgroup/.gitignore | 1 + tools/testing/selftests/cgroup/Makefile | 2 + .../selftests/cgroup/test_hugetlb_memcg.c | 234 ++++++++++++++++++ 4 files changed, 239 insertions(+) create mode 100644 tools/testing/selftests/cgroup/test_hugetlb_memcg.c diff --git a/MAINTAINERS b/MAINTAINERS index bf0f54c24f81..ce9f40bcc2ba 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -5269,6 +5269,7 @@ S: Maintained F: mm/memcontrol.c F: mm/swap_cgroup.c F: tools/testing/selftests/cgroup/memcg_protection.m +F: tools/testing/selftests/cgroup/test_hugetlb_memcg.c F: tools/testing/selftests/cgroup/test_kmem.c F: tools/testing/selftests/cgroup/test_memcontrol.c @@ -9652,6 +9653,7 @@ F: include/linux/hugetlb.h F: mm/hugetlb.c F: mm/hugetlb_vmemmap.c F: mm/hugetlb_vmemmap.h +F: tools/testing/selftests/cgroup/test_hugetlb_memcg.c HVA ST MEDIA DRIVER M: Jean-Christophe Trotin diff --git a/tools/testing/selftests/cgroup/.gitignore b/tools/testing/selftests/cgroup/.gitignore index af8c3f30b9c1..2732e0b29271 100644 --- a/tools/testing/selftests/cgroup/.gitignore +++ b/tools/testing/selftests/cgroup/.gitignore @@ -7,4 +7,5 @@ test_kill test_cpu test_cpuset test_zswap +test_hugetlb_memcg wait_inotify diff --git a/tools/testing/selftests/cgroup/Makefile b/tools/testing/selftests/cgroup/Makefile index c27f05f6ce9b..00b441928909 100644 --- a/tools/testing/selftests/cgroup/Makefile +++ b/tools/testing/selftests/cgroup/Makefile @@ -14,6 +14,7 @@ TEST_GEN_PROGS += test_kill TEST_GEN_PROGS += test_cpu TEST_GEN_PROGS += test_cpuset TEST_GEN_PROGS += test_zswap +TEST_GEN_PROGS += test_hugetlb_memcg LOCAL_HDRS += $(selfdir)/clone3/clone3_selftests.h $(selfdir)/pidfd/pidfd.h @@ -27,3 +28,4 @@ $(OUTPUT)/test_kill: cgroup_util.c $(OUTPUT)/test_cpu: cgroup_util.c $(OUTPUT)/test_cpuset: cgroup_util.c $(OUTPUT)/test_zswap: cgroup_util.c +$(OUTPUT)/test_hugetlb_memcg: cgroup_util.c diff --git a/tools/testing/selftests/cgroup/test_hugetlb_memcg.c b/tools/testing/selftests/cgroup/test_hugetlb_memcg.c new file mode 100644 index 000000000000..f0fefeb4cc24 --- /dev/null +++ b/tools/testing/selftests/cgroup/test_hugetlb_memcg.c @@ -0,0 +1,234 @@ +// SPDX-License-Identifier: GPL-2.0 +#define _GNU_SOURCE + +#include +#include +#include +#include +#include +#include +#include "../kselftest.h" +#include "cgroup_util.h" + +#define ADDR ((void *)(0x0UL)) +#define FLAGS (MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB) +/* mapping 8 MBs == 4 hugepages */ +#define LENGTH (8UL*1024*1024) +#define PROTECTION (PROT_READ | PROT_WRITE) + +/* borrowed from mm/hmm-tests.c */ +static long get_hugepage_size(void) +{ + int fd; + char buf[2048]; + int len; + char *p, *q, *path = "/proc/meminfo", *tag = "Hugepagesize:"; + long val; + + fd = open(path, O_RDONLY); + if (fd < 0) { + /* Error opening the file */ + return -1; + } + + len = read(fd, buf, sizeof(buf)); + close(fd); + if (len < 0) { + /* Error in reading the file */ + return -1; + } + if (len == sizeof(buf)) { + /* Error file is too large */ + return -1; + } + buf[len] = '\0'; + + /* Search for a tag if provided */ + if (tag) { + p = strstr(buf, tag); + if (!p) + return -1; /* looks like the line we want isn't there */ + p += strlen(tag); + } else + p = buf; + + val = strtol(p, &q, 0); + if (*q != ' ') { + /* Error parsing the file */ + return -1; + } + + return val; +} + +static int set_file(const char *path, long value) +{ + FILE *file; + int ret; + + file = fopen(path, "w"); + if (!file) + return -1; + ret = fprintf(file, "%ld\n", value); + fclose(file); + return ret; +} + +static int set_nr_hugepages(long value) +{ + return set_file("/proc/sys/vm/nr_hugepages", value); +} + +static unsigned int check_first(char *addr) +{ + return *(unsigned int *)addr; +} + +static void write_data(char *addr) +{ + unsigned long i; + + for (i = 0; i < LENGTH; i++) + *(addr + i) = (char)i; +} + +static int hugetlb_test_program(const char *cgroup, void *arg) +{ + char *test_group = (char *)arg; + void *addr; + long old_current, expected_current, current; + int ret = EXIT_FAILURE; + + old_current = cg_read_long(test_group, "memory.current"); + set_nr_hugepages(20); + current = cg_read_long(test_group, "memory.current"); + if (current - old_current >= MB(2)) { + ksft_print_msg( + "setting nr_hugepages should not increase hugepage usage.\n"); + ksft_print_msg("before: %ld, after: %ld\n", old_current, current); + return EXIT_FAILURE; + } + + addr = mmap(ADDR, LENGTH, PROTECTION, FLAGS, 0, 0); + if (addr == MAP_FAILED) { + ksft_print_msg("fail to mmap.\n"); + return EXIT_FAILURE; + } + current = cg_read_long(test_group, "memory.current"); + if (current - old_current >= MB(2)) { + ksft_print_msg("mmap should not increase hugepage usage.\n"); + ksft_print_msg("before: %ld, after: %ld\n", old_current, current); + goto out_failed_munmap; + } + old_current = current; + + /* read the first page */ + check_first(addr); + expected_current = old_current + MB(2); + current = cg_read_long(test_group, "memory.current"); + if (!values_close(expected_current, current, 5)) { + ksft_print_msg("memory usage should increase by around 2MB.\n"); + ksft_print_msg( + "expected memory: %ld, actual memory: %ld\n", + expected_current, current); + goto out_failed_munmap; + } + + /* write to the whole range */ + write_data(addr); + current = cg_read_long(test_group, "memory.current"); + expected_current = old_current + MB(8); + if (!values_close(expected_current, current, 5)) { + ksft_print_msg("memory usage should increase by around 8MB.\n"); + ksft_print_msg( + "expected memory: %ld, actual memory: %ld\n", + expected_current, current); + goto out_failed_munmap; + } + + /* unmap the whole range */ + munmap(addr, LENGTH); + current = cg_read_long(test_group, "memory.current"); + expected_current = old_current; + if (!values_close(expected_current, current, 5)) { + ksft_print_msg("memory usage should go back down.\n"); + ksft_print_msg( + "expected memory: %ld, actual memory: %ld\n", + expected_current, current); + return ret; + } + + ret = EXIT_SUCCESS; + return ret; + +out_failed_munmap: + munmap(addr, LENGTH); + return ret; +} + +static int test_hugetlb_memcg(char *root) +{ + int ret = KSFT_FAIL; + char *test_group; + + test_group = cg_name(root, "hugetlb_memcg_test"); + if (!test_group || cg_create(test_group)) { + ksft_print_msg("fail to create cgroup.\n"); + goto out; + } + + if (cg_write(test_group, "memory.max", "100M")) { + ksft_print_msg("fail to set cgroup memory limit.\n"); + goto out; + } + + /* disable swap */ + if (cg_write(test_group, "memory.swap.max", "0")) { + ksft_print_msg("fail to disable swap.\n"); + goto out; + } + + if (!cg_run(test_group, hugetlb_test_program, (void *)test_group)) + ret = KSFT_PASS; +out: + cg_destroy(test_group); + free(test_group); + return ret; +} + +int main(int argc, char **argv) +{ + char root[PATH_MAX]; + int ret = EXIT_SUCCESS, has_memory_hugetlb_acc; + + has_memory_hugetlb_acc = proc_mount_contains("memory_hugetlb_accounting"); + if (has_memory_hugetlb_acc < 0) + ksft_exit_skip("Failed to query cgroup mount option\n"); + else if (!has_memory_hugetlb_acc) + ksft_exit_skip("memory hugetlb accounting is disabled\n"); + + /* Unit is kB! */ + if (get_hugepage_size() != 2048) { + ksft_print_msg("test_hugetlb_memcg requires 2MB hugepages\n"); + ksft_test_result_skip("test_hugetlb_memcg\n"); + return ret; + } + + if (cg_find_unified_root(root, sizeof(root))) + ksft_exit_skip("cgroup v2 isn't mounted\n"); + + switch (test_hugetlb_memcg(root)) { + case KSFT_PASS: + ksft_test_result_pass("test_hugetlb_memcg\n"); + break; + case KSFT_SKIP: + ksft_test_result_skip("test_hugetlb_memcg\n"); + break; + default: + ret = EXIT_FAILURE; + ksft_test_result_fail("test_hugetlb_memcg\n"); + break; + } + + return ret; +}