From patchwork Fri Aug 4 07:57:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhongkun He X-Patchwork-Id: 131012 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:44a:b0:3f2:4152:657d with SMTP id ez10csp115297vqb; Fri, 4 Aug 2023 01:39:34 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHcNW92T2V3DeKK2ParQKS0KilLGtt9STdb5XZVGtjpb1BSYYO8igne2bjqStdGIRp8G3Tr X-Received: by 2002:a17:906:32d9:b0:982:c69c:8c30 with SMTP id k25-20020a17090632d900b00982c69c8c30mr992614ejk.55.1691138373771; Fri, 04 Aug 2023 01:39:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691138373; cv=none; d=google.com; s=arc-20160816; b=V6389liv59l0rO6WZzVo5TufC2iskoJ9x37dynmwdR8LG5wryBO1Bv0CkMf8Tlb7BD 8tslgtfr64iWVKb255oF36+/pHu/bTSMa8p45KBcbq7/oXLUaYEeq+ytQYn69n1C1r9o HsC9IXGRzC/MzOmWbDLkK18WLX+zEiNczbT1KuGNHtVcXC65ELAVj/X4mPo/tYreMvfr J/rCJLn2/kKh46K8SJZn/CGtabOM5idzDgpsRfcXfMcXzt8nbDZ2NzZEfrO5BhQhmdw0 wlwU+iO13z3uj1oXNOVI5Cq1NAqeHjKaNpOmpjwyIepLFTMDwpaRTEvUO9rTRAxtF4kb tuqQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=Z802CX/jhRxr4i7C0Kq8FNtLQD0asQherflJFZ/9BtY=; fh=512KhH5uQWMkzdsfYlnl6sN/+Kp7rI1F537sZxMX9nk=; b=sOfhDPmJFfxw1W3kXap54ueqGjT4HckKFPpRvj2rb72L4lBsApj1oQp9P0mFI6dGFK AhB+1bo5gR4FBrEzSuxlnURjNPUlFQAYckGZ4YuOD/TX7UBzb4rj+4kPSPrI59MpHsyC imNVtY8d8SCjAXfIkrXrsBbhJIIx12BM1aAy/ugv3OPN+A5gYdOZlfDq8bvr0bmOH29d m2xsd38ZB517ZmjhFZaO6oFoNk8L3qHzi0sv6Vr4A1+A9mLHsYxFijNzVxuDF+jt2nxt bxdh99KoW09dQtGWo46cRv6bZgYr6XPT/puyi4kiFVsTOrmuam02IvAmY8HQmBmNnr0G GL+g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=lCi7sEZa; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h23-20020a1709060f5700b0099bcf69f506si1292278ejj.202.2023.08.04.01.39.10; Fri, 04 Aug 2023 01:39:33 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=lCi7sEZa; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234259AbjHDH6H (ORCPT + 99 others); Fri, 4 Aug 2023 03:58:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46450 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232676AbjHDH6B (ORCPT ); Fri, 4 Aug 2023 03:58:01 -0400 Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9C4BD1B2 for ; Fri, 4 Aug 2023 00:58:00 -0700 (PDT) Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1bb9e6c2a90so15535895ad.1 for ; Fri, 04 Aug 2023 00:58:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1691135880; x=1691740680; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=Z802CX/jhRxr4i7C0Kq8FNtLQD0asQherflJFZ/9BtY=; b=lCi7sEZaWt6+fEBpNPAu+/u1pxqMPkL9EUjiIKr2BYY5B8ttIBXJF78aA9AS+vBMc5 8lXOg0V+i6Y9jrpLdS8sdTZiMEYD8JAiQJSfcj7I5ZNPtaqyTvUqcGX74AVy/bIGXlOn lM7guK0QY9FGsSsGw8Bb8xjpjqnRgdTQ/knqfcu6iFRVEarlV0dzqNJYtu8Fd7i1X7+r vaMRa8oJkxBmT1iMVJb864MVy0fQdGblOvBIw4ONNZcg8rV61BlsUZNI8HzwDbs1CczS ph5ck4WD5ulsLnWc8AavPNJvvlfMgiR8KNh8WKuLBWB2hPs9lGHg4MI5qozYgCAVQ8J4 osdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691135880; x=1691740680; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Z802CX/jhRxr4i7C0Kq8FNtLQD0asQherflJFZ/9BtY=; b=QderayL4/IAbYGhnIOhCMxQ21AIxIbTl87IwVP0eDm5aKmZxcKNySTwo+S0wLojo8q sv46QxgIbdaCkW4rwb1iRe9hZ/KARgVNZzU2IBFkRZnskVMD4ACEMMN8dKQhcczfSgWk LaECigVeqXjnkFvlYBcW2QR0/Vs3KnvhGNDc+QdagnLQB6tGvevFolfw3fDxx3fg0Zsc Fn5yKpAqPh6fSw2Qwr2cLTuGh9980HvDaNQFRZrFzeocIYsZIC7Mrh07g3ssrlN8cx5l a9MmwLk+I/q+YSynRm+7Y3+H7qyMF22LDuUoQy6OArvTLGbFL+hi18LvILqb2f5jU2uV 8MBg== X-Gm-Message-State: AOJu0YyoyfpRy3TtNGccNcSHXIEDd8E+dNmGCZFs6t1pc71o0gJ/Wpxt ePLlo0Yiqk1zDWL0nUkDmahRDQ== X-Received: by 2002:a17:902:f7c1:b0:1bc:41e4:af57 with SMTP id h1-20020a170902f7c100b001bc41e4af57mr1056464plw.48.1691135880054; Fri, 04 Aug 2023 00:58:00 -0700 (PDT) Received: from Tower.bytedance.net ([203.208.167.146]) by smtp.gmail.com with ESMTPSA id q17-20020a170902dad100b001bb9aadfb04sm1093113plx.220.2023.08.04.00.57.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Aug 2023 00:57:59 -0700 (PDT) From: Zhongkun He To: minchan@kernel.org, senozhatsky@chromium.org, mhocko@suse.com Cc: david@redhat.com, yosryahmed@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Zhongkun He Subject: [RFC PATCH RESEND v2 2/2] zram: charge the compressed RAM to the page's memcgroup Date: Fri, 4 Aug 2023 15:57:51 +0800 Message-Id: <20230804075751.208045-1-hezhongkun.hzk@bytedance.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773287111300502685 X-GMAIL-MSGID: 1773287111300502685 The compressed RAM is currently charged to kernel, not to any memory cgroup. This patch can charge the pages regardless of direct or indirect zram usage. Direct zram usage by process within a cgroup will fail to charge if there is no memory. Indirect zram usage by process within a cgroup via swap in PF_MEMALLOC context, wqwill charge successfully. This allows some limit overrun, but not enough to matter in practice.Charge compressed page once, mean a page will be freed.the size of compressed page is less than or equal to the page to be freed. The numbers of excess depend on the compression ratio only. The maximum amount will not exceed 400KB, and will be smaller than the hard limit finally, So not an unbounded way. Signed-off-by: Zhongkun He --- drivers/block/zram/zram_drv.c | 45 +++++++++++++++++++++++++++++++++++ drivers/block/zram/zram_drv.h | 1 + 2 files changed, 46 insertions(+) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 5676e6dd5b16..3aa352940b9b 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -33,6 +33,7 @@ #include #include #include +#include #include "zram_drv.h" @@ -135,6 +136,18 @@ static void zram_set_obj_size(struct zram *zram, zram->table[index].flags = (flags << ZRAM_FLAG_SHIFT) | size; } +static inline void zram_set_obj_cgroup(struct zram *zram, u32 index, + struct obj_cgroup *objcg) +{ + zram->table[index].objcg = objcg; +} + +static inline struct obj_cgroup *zram_get_obj_cgroup(struct zram *zram, + u32 index) +{ + return zram->table[index].objcg; +} + static inline bool zram_allocated(struct zram *zram, u32 index) { return zram_get_obj_size(zram, index) || @@ -1256,6 +1269,7 @@ static bool zram_meta_alloc(struct zram *zram, u64 disksize) static void zram_free_page(struct zram *zram, size_t index) { unsigned long handle; + struct obj_cgroup *objcg; #ifdef CONFIG_ZRAM_MEMORY_TRACKING zram->table[index].ac_time = 0; @@ -1289,6 +1303,13 @@ static void zram_free_page(struct zram *zram, size_t index) goto out; } + objcg = zram_get_obj_cgroup(zram, index); + if (objcg) { + obj_cgroup_uncharge_zram(objcg, zram_get_obj_size(zram, index)); + obj_cgroup_put(objcg); + zram_set_obj_cgroup(zram, index, NULL); + } + handle = zram_get_handle(zram, index); if (!handle) return; @@ -1419,6 +1440,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) struct zcomp_strm *zstrm; unsigned long element = 0; enum zram_pageflags flags = 0; + struct obj_cgroup *objcg; mem = kmap_atomic(page); if (page_same_filled(mem, &element)) { @@ -1494,6 +1516,14 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return -ENOMEM; } + objcg = get_obj_cgroup_from_page(page); + if (objcg && obj_cgroup_charge_zram(objcg, GFP_KERNEL, comp_len)) { + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zs_free(zram->mem_pool, handle); + obj_cgroup_put(objcg); + return -ENOMEM; + } + dst = zs_map_object(zram->mem_pool, handle, ZS_MM_WO); src = zstrm->buffer; @@ -1526,6 +1556,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) } else { zram_set_handle(zram, index, handle); zram_set_obj_size(zram, index, comp_len); + zram_set_obj_cgroup(zram, index, objcg); } zram_slot_unlock(zram, index); @@ -1575,6 +1606,7 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, u32 threshold, u32 prio, u32 prio_max) { struct zcomp_strm *zstrm = NULL; + struct obj_cgroup *objcg; unsigned long handle_old; unsigned long handle_new; unsigned int comp_len_old; @@ -1669,6 +1701,16 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, if (threshold && comp_len_new >= threshold) return 0; + objcg = zram_get_obj_cgroup(zram, index); + if (objcg) { + obj_cgroup_get(objcg); + if (obj_cgroup_charge_zram(objcg, GFP_KERNEL, comp_len_new)) { + zcomp_stream_put(zram->comps[prio]); + obj_cgroup_put(objcg); + return -ENOMEM; + } + } + /* * No direct reclaim (slow path) for handle allocation and no * re-compression attempt (unlike in zram_write_bvec()) since @@ -1683,6 +1725,8 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, __GFP_MOVABLE); if (IS_ERR_VALUE(handle_new)) { zcomp_stream_put(zram->comps[prio]); + obj_cgroup_uncharge_zram(objcg, comp_len_new); + obj_cgroup_put(objcg); return PTR_ERR((void *)handle_new); } @@ -1696,6 +1740,7 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, zram_set_handle(zram, index, handle_new); zram_set_obj_size(zram, index, comp_len_new); zram_set_priority(zram, index, prio); + zram_set_obj_cgroup(zram, index, objcg); atomic64_add(comp_len_new, &zram->stats.compr_data_size); atomic64_inc(&zram->stats.pages_stored); diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index ca7a15bd4845..959d721d5474 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -72,6 +72,7 @@ struct zram_table_entry { #ifdef CONFIG_ZRAM_MEMORY_TRACKING ktime_t ac_time; #endif + struct obj_cgroup *objcg; }; struct zram_stats {