From patchwork Mon Jul 24 06:22:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhongkun He X-Patchwork-Id: 124634 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9010:0:b0:3e4:2afc:c1 with SMTP id l16csp1607607vqg; Sun, 23 Jul 2023 23:34:56 -0700 (PDT) X-Google-Smtp-Source: APBJJlHZG/OlFIiCA2AoqHvPiOX/j3R/7CcH6WSCPjLhpih8TIQUZHQDyoDhcZNOXXzWMEmEpmfq X-Received: by 2002:aa7:ce1a:0:b0:522:30cc:a1ec with SMTP id d26-20020aa7ce1a000000b0052230cca1ecmr1002878edv.0.1690180496044; Sun, 23 Jul 2023 23:34:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690180496; cv=none; d=google.com; s=arc-20160816; b=H7DxOTqc4b3m1PXsO5TNwofcYEdl5BsyV7na2CceMsKgv34ZRyfx9XT44nGlsvTJoC qn7LMPdTJSbVftwEfHaViXs0YFddlWUdbO+qMkvh44uYYUFDaTjQFinZ6mw90LFoWyux wjPuqI994giephBsN7D3R+qEMvOy1HLSZ6ctHgM0iqokEGOphYXjVREKvzIyA/lwlBGI sbXOB6xPVCi672NiqnqP9r1nuJbY5iy4jWH2x2eaWFltGiLKpREdqdyyCBA9vbE6/1hC 8N7Af9d3yAHONgPW4HXldFd9YdjzuYe2snt0l7sEXer8iMSvqajKMIuZN3jU/fn+leyG 0ouw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=zMxhq1yiZbeuYK1cuOPzx7JANSxn1juq0pUBihEL+Wk=; fh=512KhH5uQWMkzdsfYlnl6sN/+Kp7rI1F537sZxMX9nk=; b=fUePxZXfQY+kCdaAWSHVAfRdLcI05xuGvlW9sR4vCSL87S3n1RkKRzcXmL6knVR4NT uJ9ZhynRXIqiYjHlo6Gk3iU3A2GjdI+I9CRXMh16rqz7+u9N/A4fNW7Kylg5NUEqJ3lV Hn/MBPtsGbCinTmGbzayHKA4Pkt36yHeJCS49WEPaWkPXqwFxHjRTWr2G5WaKYZnQt5v ycz4DdEsc9lLWiDLrAENAr4QI30vPkBkXKHvpeyHIMDnQGULMTCi6JaxS1iHhjVKMKDZ dc/ycBMk33/N6KcqRYWxllVj2r0xsYovnHKuiyE1VERrSEE4NdMuuex4E7JOmtIPYItf QnKQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=KFYjcOVL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w18-20020aa7d292000000b0051bf69845desi5991507edq.64.2023.07.23.23.34.31; Sun, 23 Jul 2023 23:34:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=KFYjcOVL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229705AbjGXGWg (ORCPT + 99 others); Mon, 24 Jul 2023 02:22:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54108 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230095AbjGXGWc (ORCPT ); Mon, 24 Jul 2023 02:22:32 -0400 Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 15EA1B9 for ; Sun, 23 Jul 2023 23:22:11 -0700 (PDT) Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-1b8ad356f03so19026855ad.1 for ; Sun, 23 Jul 2023 23:22:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1690179730; x=1690784530; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=zMxhq1yiZbeuYK1cuOPzx7JANSxn1juq0pUBihEL+Wk=; b=KFYjcOVLyGA1d4rfaoLPdLB/e6IjDH5icirSxJhNketSHU37mMtGFferynYbZDdVDr SBHHg1RIVmDI1fNLSXGw64oy5igBfOWadAkh1lGrBGeXMBa1ulxQGdtmi8/J7UhGyGZ5 PEqbCcd8mvm1vPkw4Rr/lF18ynZhCmEe2Nseic1Cu+mBDAkH1xnTwwZWgykAo+8kBVSo K+5y+jfTjdVUDLBZ1x9Lr90D1oTQILAGJl5dG+myCccy2uIl1c+6SRN9JGvOolhslNK4 nVA38dojo5rsWjEmeGe9RFjEZoFNh+SK6kqQBoGX1ixHQeZl5xBToafr9ey+tHo56CfZ eOtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690179730; x=1690784530; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=zMxhq1yiZbeuYK1cuOPzx7JANSxn1juq0pUBihEL+Wk=; b=j5SqbRjlAxMeiCzQ+Y32dumE5KXWgXn2q/+HnNjhs82K7Ca+H/PUb/lNhsLqlRyMB/ /AFNIPNjwyC0BcGAxb3v6mMQcqQIfeyBaTsupOWo+ZNkBPtHH9RuXj5fmrJLEgXKXmF3 d5ccTf0ZmhQ6q/XwLPym9JaJFApbkmvSQ4cVUAdyTzzKPWH5spU/3gcq2gfjrO2rBqVF 2WIeMym4CYutJJrdQ1v8moJggkDHWPiAOl4Tq/Pw74Bpz2K5YvGgUCNEqBrI+qZ0vGQt Aku4SQO7ko/NQRIm01kKIaz9+YUAXKNX/S9IQExEYf7LOzX+MevFvHkkng8tYq5nCCUb L5BQ== X-Gm-Message-State: ABy/qLYYWMaoEdRKUcoXXpDJZ/kYBrGHZQz0dS6BX6/QD7rNV+GwvslK XLc+oIs+JsKvBcLXlAZBQ6VIBQ== X-Received: by 2002:a17:902:e995:b0:1b6:ab53:c7a5 with SMTP id f21-20020a170902e99500b001b6ab53c7a5mr6059293plb.46.1690179730618; Sun, 23 Jul 2023 23:22:10 -0700 (PDT) Received: from Tower.bytedance.net ([203.208.167.146]) by smtp.gmail.com with ESMTPSA id l14-20020a170902f68e00b001b7e63cfa19sm7983504plg.234.2023.07.23.23.22.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Jul 2023 23:22:09 -0700 (PDT) From: Zhongkun He To: minchan@kernel.org, senozhatsky@chromium.org, mhocko@suse.com Cc: david@redhat.com, yosryahmed@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Zhongkun He Subject: [RFC PATCH v2 1/2] memcg: Add support for zram object charge Date: Mon, 24 Jul 2023 14:22:02 +0800 Message-Id: <20230724062202.2244188-1-hezhongkun.hzk@bytedance.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772282704149517425 X-GMAIL-MSGID: 1772282704149517425 The compressed RAM(ZRAM) is currently charged to kernel, not to any memory cgroup, which can escape their cgroup memory containment if the memory of a task is limited by memcgroup. it will swap out the memory to zram swap device when the memory is insufficient. In that case, the memory hard limit will be invalidated. So, it should makes sense to charge the compressed RAM to the page's memory cgroup. As we know, zram can be used in two ways, direct and indirect, this patchset can charge memory in both cases. Direct zram usage by process within a cgroup will fail to charge if there is no memory. Indirect zram usage by process within a cgroup via swap in PF_MEMALLOC context, will charge successfully. Signed-off-by: Zhongkun He --- include/linux/memcontrol.h | 12 ++++++++++++ mm/memcontrol.c | 24 ++++++++++++++++++++++++ 2 files changed, 36 insertions(+) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 5818af8eca5a..24bac877bc83 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1819,6 +1819,9 @@ static inline void count_objcg_event(struct obj_cgroup *objcg, rcu_read_unlock(); } +int obj_cgroup_charge_zram(struct obj_cgroup *objcg, gfp_t gfp, + size_t size); +void obj_cgroup_uncharge_zram(struct obj_cgroup *objcg, size_t size); #else static inline bool mem_cgroup_kmem_disabled(void) { @@ -1880,6 +1883,15 @@ static inline void count_objcg_event(struct obj_cgroup *objcg, { } +int obj_cgroup_charge_zram(struct obj_cgroup *objcg, gfp_t gfp, + size_t size) +{ + return 0; +} + +void obj_cgroup_uncharge_zram(struct obj_cgroup *objcg, size_t size) +{ +} #endif /* CONFIG_MEMCG_KMEM */ #if defined(CONFIG_MEMCG_KMEM) && defined(CONFIG_ZSWAP) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index e8ca4bdcb03c..118544acf895 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3059,6 +3059,7 @@ struct obj_cgroup *get_obj_cgroup_from_page(struct page *page) } return objcg; } +EXPORT_SYMBOL(get_obj_cgroup_from_page); static void memcg_account_kmem(struct mem_cgroup *memcg, int nr_pages) { @@ -3409,6 +3410,29 @@ void obj_cgroup_uncharge(struct obj_cgroup *objcg, size_t size) refill_obj_stock(objcg, size, true); } +int obj_cgroup_charge_zram(struct obj_cgroup *objcg, gfp_t gfp, + size_t size) +{ + if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) + return 0; + + /* + * Indirect zram usage in PF_MEMALLOC, charging must succeed. + * Direct zram usage, charging may failed. + */ + return obj_cgroup_charge(objcg, gfp, size); +} +EXPORT_SYMBOL(obj_cgroup_charge_zram); + +void obj_cgroup_uncharge_zram(struct obj_cgroup *objcg, size_t size) +{ + if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) + return; + + obj_cgroup_uncharge(objcg, size); +} +EXPORT_SYMBOL(obj_cgroup_uncharge_zram); + #endif /* CONFIG_MEMCG_KMEM */ /* From patchwork Mon Jul 24 06:22:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhongkun He X-Patchwork-Id: 124635 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9010:0:b0:3e4:2afc:c1 with SMTP id l16csp1607768vqg; Sun, 23 Jul 2023 23:35:22 -0700 (PDT) X-Google-Smtp-Source: APBJJlHjKJlHgC3v9DWAUSckVef6pW95Q2Ct+nfVUkNwQQRwkqLMG5ooqNIPgvtvkoLYA8+KGLmk X-Received: by 2002:a17:907:2cf6:b0:993:fba5:cdf1 with SMTP id hz22-20020a1709072cf600b00993fba5cdf1mr9273380ejc.22.1690180522750; Sun, 23 Jul 2023 23:35:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690180522; cv=none; d=google.com; s=arc-20160816; b=XjwZoChEuqnf4fW4UHlfbqh0qn310ijG6kpAHOs5nC9/HKbPAV4HRLEtYexfUQFAuJ DFywN+wipC8FXaCYMp0JGKlUpggQsywDWYhTpYgDy11ziSDPfM9pwgqxdsltPegv6YDd UBwhxNvtQ7VvLcoxNb7FOV/OvRO3ZFwYf5Pehxq1vzdTRxcY2X3/mgCuLWHoD4if8l0Q uoc30g9hWatXypQs1S4AkGMqFi/X8JYYyjKO2vCG3ZytV4LIoWoN4YNHvUjevQmfQF6N nAVM36E1Gfmrr4Gw4zqqWYnAP2fjXEgXypTE8A7o/XP00u/XA9ZcMNMJOB15feDhkAlh SyVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=vQ0+FHFKuAIc1z56nGRfE+KN8lTLE329HHeSqqWKPM0=; fh=512KhH5uQWMkzdsfYlnl6sN/+Kp7rI1F537sZxMX9nk=; b=xj0knzDJEPxOg5yDrTSNAy5UcymkycswOOAD2vE9YxrnELPrHv+8WkkktMxv3GirJJ 5IXsMro+av1pzjqYaVQyLZJ+rdU/rSQ8JRz2aMWCkyW3Rpega2xuaOJJb69bGHg4M+5w dEQj3HsigfGlQdPw9XR69BSiRErjSc2g36j1HOGAij6ATwmmu5sixTIpnNpPO3pD5HdB uvA4eWxrIVjSEBKGavE5FDFJgNZKsbP9QmL7RFtY8KmoUXjPkonT+O+gClq/tZO8Gk0i sUkdPEbJj7qvJCTuJCxaTikGLSd0qs1mUvY8vUIw+f2CoKurmAc0Jazxd830joT1A33h YkMw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b="g8XFmi/X"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id qo13-20020a170907212d00b00992be132d85si5715939ejb.853.2023.07.23.23.34.57; Sun, 23 Jul 2023 23:35:22 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b="g8XFmi/X"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230001AbjGXGWy (ORCPT + 99 others); Mon, 24 Jul 2023 02:22:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54218 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229504AbjGXGWx (ORCPT ); Mon, 24 Jul 2023 02:22:53 -0400 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ED644E55 for ; Sun, 23 Jul 2023 23:22:27 -0700 (PDT) Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-1bba2318546so5261195ad.1 for ; Sun, 23 Jul 2023 23:22:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1690179747; x=1690784547; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=vQ0+FHFKuAIc1z56nGRfE+KN8lTLE329HHeSqqWKPM0=; b=g8XFmi/Xu7Q/IPaOlyMdCeZDLC2qDIUrES87VDacv5L4RIvIl2r0gW/KhtEcTINWF6 85uL6rrA8LQjGriLE4rjl7XVCGU8czhLCTXmr4/dBUFZmwaukf40MyoppTsVSXGPnMED yuLs+xCiBxSTLrm8Wp+vlyQRp022Dqdc5p/LMHi5gv8ZOTY7L8QxjhIb1B56LA/FH6ml b4vSr/xbX35wV5C08h/v8anQ2+KiEZecKkvJfOPAvGWtQd/3dA75o4GonVIh8w3yJFiC QQonyK5SW6Ifk7GmdSnw1GrPioh0+m5Mf///5dFHYd8C1LlCS2KyorB9zGkd8VXm/aPc Ge3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690179747; x=1690784547; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=vQ0+FHFKuAIc1z56nGRfE+KN8lTLE329HHeSqqWKPM0=; b=CtGtSRIqJRppVGxrnDCLHt5q/1S0xQBS6MAnissE6AEeWH/lDtJ4KWz2zi3rMnw8tK Z7M5FQxFo9uAfgX8GBhTQtMVxoJReh9CCMucF1LoQ9OTgMTHHPQuMQx3JmVFGJZBH5jT 00k9E1Q0RWuA7SAJTSfpF76B0Zab/on4dPgXF+9aVIU4PicrzK37ts23KHcJ649MQr+j 1vZxinva0RGw5SHNC1SVjkNGuF5usM7aVXDLXSJ9JImwZK+bPO+HhLhu/Rxps5533deS avZktoQKRDMadP4Fh4CuKgSSfgUzcoUjMtXjFr3iAtnILJ7NV7pJzWp3viZsTUtud8AZ Qk9A== X-Gm-Message-State: ABy/qLapGRgluAWOsy5D5JfYZWnfXnJsU3OSqjBbAPO1O+7HLPDJnxRd aN1da9Kln+5zd9noTy7NChq2JQ== X-Received: by 2002:a17:902:d2cd:b0:1b8:a389:43ef with SMTP id n13-20020a170902d2cd00b001b8a38943efmr12250391plc.24.1690179747454; Sun, 23 Jul 2023 23:22:27 -0700 (PDT) Received: from Tower.bytedance.net ([203.208.167.146]) by smtp.gmail.com with ESMTPSA id ix20-20020a170902f81400b001b80760fd04sm1443236plb.112.2023.07.23.23.22.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Jul 2023 23:22:26 -0700 (PDT) From: Zhongkun He To: minchan@kernel.org, senozhatsky@chromium.org, mhocko@suse.com Cc: david@redhat.com, yosryahmed@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Zhongkun He Subject: [RFC PATCH v2 2/2] zram: charge the compressed RAM to the page's memcgroup Date: Mon, 24 Jul 2023 14:22:19 +0800 Message-Id: <20230724062219.2244240-1-hezhongkun.hzk@bytedance.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772282731722354423 X-GMAIL-MSGID: 1772282731722354423 The compressed RAM is currently charged to kernel, not to any memory cgroup. This patch can charge the pages regardless of direct or indirect zram usage. Direct zram usage by process within a cgroup will fail to charge if there is no memory. Indirect zram usage by process within a cgroup via swap in PF_MEMALLOC context, wqwill charge successfully. This allows some limit overrun, but not enough to matter in practice.Charge compressed page once, mean a page will be freed.the size of compressed page is less than or equal to the page to be freed. The numbers of excess depend on the compression ratio only. The maximum amount will not exceed 400KB, and will be smaller than the hard limit finally, So not an unbounded way. Signed-off-by: Zhongkun He --- drivers/block/zram/zram_drv.c | 46 +++++++++++++++++++++++++++++++++++ drivers/block/zram/zram_drv.h | 1 + 2 files changed, 47 insertions(+) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 5676e6dd5b16..1e685a430c95 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -33,6 +33,7 @@ #include #include #include +#include #include "zram_drv.h" @@ -135,6 +136,18 @@ static void zram_set_obj_size(struct zram *zram, zram->table[index].flags = (flags << ZRAM_FLAG_SHIFT) | size; } +static inline void zram_set_obj_cgroup(struct zram *zram, u32 index, + struct obj_cgroup *objcg) +{ + zram->table[index].objcg = objcg; +} + +static inline struct obj_cgroup *zram_get_obj_cgroup(struct zram *zram, + u32 index) +{ + return zram->table[index].objcg; +} + static inline bool zram_allocated(struct zram *zram, u32 index) { return zram_get_obj_size(zram, index) || @@ -1256,6 +1269,7 @@ static bool zram_meta_alloc(struct zram *zram, u64 disksize) static void zram_free_page(struct zram *zram, size_t index) { unsigned long handle; + struct obj_cgroup *objcg; #ifdef CONFIG_ZRAM_MEMORY_TRACKING zram->table[index].ac_time = 0; @@ -1289,6 +1303,13 @@ static void zram_free_page(struct zram *zram, size_t index) goto out; } + objcg = zram_get_obj_cgroup(zram, index); + if (objcg) { + obj_cgroup_uncharge_zram(objcg, zram_get_obj_size(zram, index)); + obj_cgroup_put(objcg); + zram_set_obj_cgroup(zram, index, NULL); + } + handle = zram_get_handle(zram, index); if (!handle) return; @@ -1419,6 +1440,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) struct zcomp_strm *zstrm; unsigned long element = 0; enum zram_pageflags flags = 0; + struct obj_cgroup *objcg; mem = kmap_atomic(page); if (page_same_filled(mem, &element)) { @@ -1494,6 +1516,14 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return -ENOMEM; } + objcg = get_obj_cgroup_from_page(page); + if (objcg && obj_cgroup_charge_zram(objcg, comp_len)) { + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zs_free(zram->mem_pool, handle); + obj_cgroup_put(objcg); + return -ENOMEM; + } + dst = zs_map_object(zram->mem_pool, handle, ZS_MM_WO); src = zstrm->buffer; @@ -1526,6 +1556,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) } else { zram_set_handle(zram, index, handle); zram_set_obj_size(zram, index, comp_len); + zram_set_obj_cgroup(zram, index, objcg); } zram_slot_unlock(zram, index); @@ -1575,6 +1606,7 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, u32 threshold, u32 prio, u32 prio_max) { struct zcomp_strm *zstrm = NULL; + struct obj_cgroup *objcg; unsigned long handle_old; unsigned long handle_new; unsigned int comp_len_old; @@ -1669,6 +1701,17 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, if (threshold && comp_len_new >= threshold) return 0; + objcg = zram_get_obj_cgroup(zram, index); + if (objcg) { + obj_cgroup_get(objcg); + if (obj_cgroup_charge_zram(objcg, GFP_KERNEL, comp_len_new)) { + zcomp_stream_put(zram->comps[prio]); + obj_cgroup_put(objcg); + return -ENOMEM; + } + } + /* * No direct reclaim (slow path) for handle allocation and no * re-compression attempt (unlike in zram_write_bvec()) since @@ -1683,6 +1726,8 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, __GFP_MOVABLE); if (IS_ERR_VALUE(handle_new)) { zcomp_stream_put(zram->comps[prio]); + obj_cgroup_uncharge_zram(objcg, comp_len_new); + obj_cgroup_put(objcg); return PTR_ERR((void *)handle_new); } @@ -1696,6 +1741,7 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, zram_set_handle(zram, index, handle_new); zram_set_obj_size(zram, index, comp_len_new); zram_set_priority(zram, index, prio); + zram_set_obj_cgroup(zram, index, objcg); atomic64_add(comp_len_new, &zram->stats.compr_data_size); atomic64_inc(&zram->stats.pages_stored); diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index ca7a15bd4845..959d721d5474 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -72,6 +72,7 @@ struct zram_table_entry { #ifdef CONFIG_ZRAM_MEMORY_TRACKING ktime_t ac_time; #endif + struct obj_cgroup *objcg; }; struct zram_stats {