From patchwork Mon Oct 24 16:12:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 9908 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp578947wru; Mon, 24 Oct 2022 10:53:47 -0700 (PDT) X-Google-Smtp-Source: AMsMyM48z0GqgZQ9xsUhGjYdex2qj6buqoaOwBsYfL5pjSEWw4e5oaVrseazs59K3hWL70yowjgI X-Received: by 2002:a17:907:168e:b0:7a1:6786:f16 with SMTP id hc14-20020a170907168e00b007a167860f16mr10937764ejc.590.1666634026973; Mon, 24 Oct 2022 10:53:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666634026; cv=none; d=google.com; s=arc-20160816; b=mvj1OF2Z8gMve6If1lMCzOjmU+5kpYiF0FyMRIODOH7UaDW7kIdVbSRM0r46P0Jglt yZss2mWauxuY+x1nDtHEhPt21DPOM2XP1o79XM6aT/jLe7ANf7G+H4NqbysvUkw4IBjE E578lN80B6e/4X1nH7kK24i0k858DQk4CM5b7bkbF7wYqpCeJktwzf9aeqLpydzl8Y6C 1nVWdHmuD4N5sRBtDb+V1gBj0yYeL8PtKD0oMxKAuycQb+ecfTWQkRStJ1u8zfAxTpN8 5a6wouS21rPyinbaN/DuiOcSS1yuAmiJHyqotDjb8d93QjWJANWHvN2lxHHIXq5UDtR6 NAdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=TpgHV4GWqLolLG30Yz1Ob4AwCWf0sAfvb4vxV3+C6wA=; b=v0XGw9J6QxBXkaKEIjomHcGNeI/h4kXnlJqjf/jpccMEwaQcLr8yMhrh3IziQJk3D0 U+h3E0c5POjMK/5Rwe8axxEWDVsdTYKCvL1tZIEe4A7yfKg8M64c7+ZLCdAha1KFDEs9 8CYAAVS2nZ9trZAAVg40jkYISA5RaHY3B4VJZHl5AhjXJVrRa5ib/KgMScDgvNKXgasd GE+un4ojRDeBo7RbVuLxeJXlUkeJxC6E/dVw5DM5RBxt2LBY9jtdvJrcD0asmfCqYe2G LckRNo8vPXUyRAqpzDdLqw3z4XPUt/JfmO18ugbywMl4iMoUw+WYg9GWgQ8P/aKHEjxP vJxw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=HHq1aywf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id qx9-20020a170906fcc900b0078d473448afsi296647ejb.233.2022.10.24.10.53.22; Mon, 24 Oct 2022 10:53:46 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=HHq1aywf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233571AbiJXRks (ORCPT + 99 others); Mon, 24 Oct 2022 13:40:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59576 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232688AbiJXRjg (ORCPT ); Mon, 24 Oct 2022 13:39:36 -0400 Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 21BDAB7F74 for ; Mon, 24 Oct 2022 09:15:22 -0700 (PDT) Received: by mail-pj1-f53.google.com with SMTP id u8-20020a17090a5e4800b002106dcdd4a0so13426250pji.1 for ; Mon, 24 Oct 2022 09:15:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TpgHV4GWqLolLG30Yz1Ob4AwCWf0sAfvb4vxV3+C6wA=; b=HHq1aywfjQlvNx1trvPuFYc7YhDbHmizdWfEjkHxsuiMMNa4XT8SX+3kiRaPvXKYJ9 Gcfika5ITCGEo/p9ddI26Gw7tJLsbxBqBua4mzeF21fBQ1CiHJEV3/D9jVwMk8O5DMA6 sjwB+tvPAv3DAaPWnB7kMjc7bM/IAQfFEYRaM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TpgHV4GWqLolLG30Yz1Ob4AwCWf0sAfvb4vxV3+C6wA=; b=nEuB4jpZIkJNzumF7M8JJOv9WF3hp9EFncX2ZGaPYazJG1NUmzyJKpskJITVJ/3u/k 1GyAzEUKkEtX4amGVKFuG2UN4J6uT3bMPYNai5AYcy50KQNNl9snlcnowIvCWwIgQg7f Z156Se+X0mG7YvQSQhjix6Q/Smq/xMYgjEMAHpgu512n9qyB3Oc6lRFlAlUUqeVfiUOx PXJl5ymazTwxqwIZfF4bKKnxuS306ZB9aeo7HrsI5uoX5ZV1ZIT0NPM5ghmahXaU/+1y q/72mwigZSFkEeMLe+claSk9DT8Ahw0my0byupfvqaVOdUI6/ZNH+YXrS4cyHc0dZw75 Uj8Q== X-Gm-Message-State: ACrzQf1o+0Vb3bk1n/eRtuXnM8T+668UkWJF9bfSdlDwzZyzgVpQCnSe itT1KoCHM61AJGr47JKjBCpOS9hU3kI5+g== X-Received: by 2002:a17:902:c40f:b0:186:b221:f378 with SMTP id k15-20020a170902c40f00b00186b221f378mr3371050plk.69.1666627948153; Mon, 24 Oct 2022 09:12:28 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:5f9c:c5bc:902f:3da4]) by smtp.gmail.com with ESMTPSA id u70-20020a627949000000b0056b8726d2d3sm5162pfc.157.2022.10.24.09.12.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Oct 2022 09:12:27 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCH 1/6] zsmalloc: turn zspage order into runtime variable Date: Tue, 25 Oct 2022 01:12:08 +0900 Message-Id: <20221024161213.3221725-2-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.0.135.g90850a2211-goog In-Reply-To: <20221024161213.3221725-1-senozhatsky@chromium.org> References: <20221024161213.3221725-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747592441226258298?= X-GMAIL-MSGID: =?utf-8?q?1747592441226258298?= zsmalloc has 255 size classes. Size classes contain a number of zspages, which store objects of the same size. zspage can consist of up to four physical pages. The exact (most optimal) zspage size is calculated for each size class during zsmalloc pool creation. As a reasonable optimization, zsmalloc merges size classes that have similar characteristics: number of pages per zspage and number of objects zspage can store. For example, let's look at the following size classes: class size almost_full almost_empty obj_allocated obj_used pages_used pages_per_zspage freeable .. 94 1536 0 0 0 0 0 3 0 100 1632 0 0 0 0 0 2 0 .. Size classes #95-99 are merged with size class #100. That is, each time we store an object of size, say, 1568 bytes instead of using class #96 we end up storing it in size class #100. Class #100 is for objects of 1632 bytes in size, hence every 1568 bytes object wastes 1632-1568 bytes. Class #100 zspages consist of 2 physical pages and can hold 5 objects. When we need to store, say, 13 objects of size 1568 we end up allocating three zspages; in other words, 6 physical pages. However, if we'll look closer at size class #96 (which should hold objects of size 1568 bytes) and trace get_pages_per_zspage(): pages per zspage wasted bytes used% 1 960 76 2 352 95 3 1312 89 4 704 95 5 96 99 We'd notice that the most optimal zspage configuration for this class is when it consists of 5 physical pages, but currently we never let zspages to consists of more than 4 pages. A 5 page class #96 configuration would store 13 objects of size 1568 in a single zspage, allocating 5 physical pages, as opposed to 6 physical pages that class #100 will allocate. A higher order zspage for class #96 also changes its key characteristics: pages per-zspage and objects per-zspage. As a result classes #96 and #100 are not merged anymore, which gives us more compact zsmalloc. Let's take a closer look at the bottom of /sys/kernel/debug/zsmalloc/zram0/classes: class size almost_full almost_empty obj_allocated obj_used pages_used pages_per_zspage freeable ... 202 3264 0 0 0 0 0 4 0 254 4096 0 0 0 0 0 1 0 ... For exactly same reason - maximum 4 pages per zspage - the last non-huge size class is #202, which stores objects of size 3264 bytes. Any object larger than 3264 bytes, hence, is considered to be huge and lands in size class #254, which uses a whole physical page to store every object. To put it slightly differently - objects in huge classes don't share physical pages. 3264 bytes is too low of a watermark and we have too many huge classes: classes from #203 to #254. Similarly to class size #96 above, higher order zspages change key characteristics for some of those huge size classes and thus those classes become normal classes, where stored objects share physical pages. We move huge class watermark with higher order zspages. For order 3, huge class watermark becomes 3632 bytes: class size almost_full almost_empty obj_allocated obj_used pages_used pages_per_zspage freeable ... 202 3264 0 0 0 0 0 4 0 211 3408 0 0 0 0 0 5 0 217 3504 0 0 0 0 0 6 0 222 3584 0 0 0 0 0 7 0 225 3632 0 0 0 0 0 8 0 254 4096 0 0 0 0 0 1 0 ... For order 4, huge class watermark becomes 3840 bytes: class size almost_full almost_empty obj_allocated obj_used pages_used pages_per_zspage freeable ... 202 3264 0 0 0 0 0 4 0 206 3328 0 0 0 0 0 13 0 207 3344 0 0 0 0 0 9 0 208 3360 0 0 0 0 0 14 0 211 3408 0 0 0 0 0 5 0 212 3424 0 0 0 0 0 16 0 214 3456 0 0 0 0 0 11 0 217 3504 0 0 0 0 0 6 0 219 3536 0 0 0 0 0 13 0 222 3584 0 0 0 0 0 7 0 223 3600 0 0 0 0 0 15 0 225 3632 0 0 0 0 0 8 0 228 3680 0 0 0 0 0 9 0 230 3712 0 0 0 0 0 10 0 232 3744 0 0 0 0 0 11 0 234 3776 0 0 0 0 0 12 0 235 3792 0 0 0 0 0 13 0 236 3808 0 0 0 0 0 14 0 238 3840 0 0 0 0 0 15 0 254 4096 0 0 0 0 0 1 0 ... TESTS ===== 1) ChromeOS memory pressure test ----------------------------------------------------------------------------- Our standard memory pressure test, that is designed with the reproducibility in mind. zram is configured as a swap device, lzo-rle compression algorithm. We captured /sys/block/zram0/mm_stat after every test and rebooted device. Columns per (Documentation/admin-guide/blockdev/zram.rst) orig_data_size mem_used_total mem_used_max pages_compacted compr_data_size mem_limit same_pages huge_pages ORDER 2 (BASE) 10353639424 2981711944 3166896128 0 3543158784 579494 825135 123707 10168573952 2932288347 3106541568 0 3499085824 565187 853137 126153 9950461952 2815911234 3035693056 0 3441090560 586696 748054 122103 9892335616 2779566152 2943459328 0 3514736640 591541 650696 119621 9993949184 2814279212 3021357056 0 3336421376 582488 711744 121273 9953226752 2856382009 3025649664 0 3512893440 564559 787861 123034 9838448640 2785481728 2997575680 0 3367219200 573282 777099 122739 ORDER 3 9509138432 2706941227 2823393280 0 3389587456 535856 1011472 90223 10105245696 2882368370 3013095424 0 3296165888 563896 1059033 94808 9531236352 2666125512 2867650560 0 3396173824 567117 1126396 88807 9561812992 2714536764 2956652544 0 3310505984 548223 827322 90992 9807470592 2790315707 2908053504 0 3378315264 563670 1020933 93725 10178371584 2948838782 3071209472 0 3329548288 548533 954546 90730 9925165056 2849839413 2958274560 0 3336978432 551464 1058302 89381 ORDER 4 9444515840 2613362645 2668232704 0 3396759552 573735 1162207 83475 10129108992 2925888488 3038351360 0 3499597824 555634 1231542 84525 9876594688 2786692282 2897006592 0 3469463552 584835 1290535 84133 10012909568 2649711847 2801512448 0 3171323904 675405 750728 80424 10120966144 2866742402 2978639872 0 3257815040 587435 1093981 83587 9578790912 2671245225 2802270208 0 3376353280 545548 1047930 80895 10108588032 2888433523 2983960576 0 3316641792 571445 1290640 81402 First, we establish that order 3 and 4 don't cause any statistically significant change in `orig_data_size` (number of bytes we store during the test), in other words larger zspages don't cause regressions. T-test for order 3: x order-2-stored + order-3-stored +-----------------------------------------------------------------------------+ |+ + + + x x + x x + x+ x| | |________________________AM__|_________M_____A____|__________| | +-----------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 7 9.8384486e+09 1.0353639e+10 9.9532268e+09 1.0021519e+10 1.7916718e+08 + 7 9.5091384e+09 1.0178372e+10 9.8074706e+09 9.8026344e+09 2.7856206e+08 No difference proven at 95.0% confidence T-test for order 4: x order-2-stored + order-4-stored +-----------------------------------------------------------------------------+ | + | |+ + x +x xx x + ++ x x| | |__________________|____A____M____M____________|_| | +-----------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 7 9.8384486e+09 1.0353639e+10 9.9532268e+09 1.0021519e+10 1.7916718e+08 + 7 9.4445158e+09 1.0129109e+10 1.001291e+10 9.8959249e+09 2.7947784e+08 No difference proven at 95.0% confidence Next we establish that there is a statistically significant improvement in `mem_used_total` metrics. T-test for order 3: x order-2-usedmem + order-3-usedmem +-----------------------------------------------------------------------------+ |+ + + x ++ x + xx x + x x| | |_________________A__M__|____________|__A________________| | +-----------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 7 2.9434593e+09 3.1668961e+09 3.0256497e+09 3.0424532e+09 73235062 + 7 2.8233933e+09 3.0712095e+09 2.9566525e+09 2.9426185e+09 84630851 Difference at 95.0% confidence -9.98347e+07 +/- 9.21744e+07 -3.28139% +/- 3.02961% (Student's t, pooled s = 7.91383e+07) T-test for order 4: x order-2-usedmem + order-4-usedmem +-----------------------------------------------------------------------------+ | + x | |+ + + x ++ x x * x x| | |__________________A__M__________|_____|_M__A__________| | +-----------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 7 2.9434593e+09 3.1668961e+09 3.0256497e+09 3.0424532e+09 73235062 + 7 2.6682327e+09 3.0383514e+09 2.8970066e+09 2.8814248e+09 1.3098053e+08 Difference at 95.0% confidence -1.61028e+08 +/- 1.23591e+08 -5.29272% +/- 4.0622% (Student's t, pooled s = 1.06111e+08) Order 3 zspages also show statistically significant improvement in `mem_used_max` metrics. T-test for order 3: x order-2-maxmem + order-3-maxmem +-----------------------------------------------------------------------------+ |+ + + x+ x + + + x x x x| | |________M__A_________|_|_____________________A___________M____________| | +-----------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 7 3.3364214e+09 3.5431588e+09 3.4990858e+09 3.4592294e+09 80073158 + 7 3.2961659e+09 3.3961738e+09 3.3369784e+09 3.3481822e+09 39840377 Difference at 95.0% confidence -1.11047e+08 +/- 7.36589e+07 -3.21017% +/- 2.12934% (Student's t, pooled s = 6.32415e+07) Order 4 zspages, on the other hand, do not show any statistically significant improvement in `mem_used_max` metrics. T-test for order 4: x order-2-maxmem + order-4-maxmem +-----------------------------------------------------------------------------+ |+ + + x x + + x + * x x| | |_______________________A___M________________A_|_____M_______| | +-----------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 7 3.3364214e+09 3.5431588e+09 3.4990858e+09 3.4592294e+09 80073158 + 7 3.1713239e+09 3.4995978e+09 3.3763533e+09 3.3554221e+09 1.1609062e+08 No difference proven at 95.0% confidence Overall, with sufficient level of confidence order 3 zspages appear to be beneficial for these particular use-case and data patterns. Rather expectedly we also observed lower numbers of huge-pages when zsmalloc is configured with order 3 and order 4 zspages, for the reason already explained. 2) Synthetic test ----------------------------------------------------------------------------- Test untars linux-6.0.tar.xz and compiles the kernel. zram is configured as a block device with ext4 file system, lzo-rle compression algorithm. We captured /sys/block/zram0/mm_stat after every test and rebooted VM. orig_data_size mem_used_total mem_used_max pages_compacted compr_data_size mem_limit same_pages huge_pages ORDER 2 (BASE) 1691807744 628091753 655187968 0 655187968 59 0 34042 34043 1691803648 628089105 655159296 0 655159296 60 0 34043 34043 1691795456 628087429 655151104 0 655151104 59 0 34046 34046 1691799552 628093723 655216640 0 655216640 60 0 34044 34044 ORDER 3 1691787264 627781464 641740800 0 641740800 59 0 33591 33591 1691795456 627794239 641789952 0 641789952 59 0 33591 33591 1691811840 627788466 641691648 0 641691648 60 0 33591 33591 1691791360 627790682 641781760 0 641781760 59 0 33591 33591 ORDER 4 1691807744 627729506 639627264 0 639627264 59 0 33432 33432 1691820032 627731485 639606784 0 639606784 59 0 33432 33432 1691799552 627725753 639623168 0 639623168 59 0 33432 33433 1691820032 627734080 639746048 0 639746048 61 0 33432 33432 Order 3 and order 4 show statistically significant improvement in `mem_used_total` metrics. T-test for order 3: x order-2-usedmem-comp + order-3-usedmem-comp +-----------------------------------------------------------------------------+ |++ x| |++ x| |AM A| +-----------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 4 6.551511e+08 6.5521664e+08 6.5518797e+08 6.5517875e+08 29795.878 + 4 6.4169165e+08 6.4178995e+08 6.4178176e+08 6.4175104e+08 45056 Difference at 95.0% confidence -1.34277e+07 +/- 66089.8 -2.04947% +/- 0.0100873% (Student's t, pooled s = 38195.8) T-test for order 4: x order-2-usedmem-comp + order-4-usedmem-comp +-----------------------------------------------------------------------------+ |+ x| |+ x| |++ x| |A| A| +-----------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 4 6.551511e+08 6.5521664e+08 6.5518797e+08 6.5517875e+08 29795.878 + 4 6.3960678e+08 6.3974605e+08 6.3962726e+08 6.3965082e+08 64101.637 Difference at 95.0% confidence -1.55279e+07 +/- 86486.9 -2.37003% +/- 0.0132005% (Student's t, pooled s = 49984.1) Order 3 and order 4 show statistically significant improvement in `mem_used_max` metrics. T-test for order 3: x order-2-maxmem-comp + order-3-maxmem-comp +-----------------------------------------------------------------------------+ |++ x| |++ x| |AM A| +-----------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 4 6.551511e+08 6.5521664e+08 6.5518797e+08 6.5517875e+08 29795.878 + 4 6.4169165e+08 6.4178995e+08 6.4178176e+08 6.4175104e+08 45056 Difference at 95.0% confidence -1.34277e+07 +/- 66089.8 -2.04947% +/- 0.0100873% (Student's t, pooled s = 38195.8) T-test for order 4: x order-2-maxmem-comp + order-4-maxmem-comp +-----------------------------------------------------------------------------+ |+ x| |+ x| |++ x| |A| A| +-----------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 4 6.551511e+08 6.5521664e+08 6.5518797e+08 6.5517875e+08 29795.878 + 4 6.3960678e+08 6.3974605e+08 6.3962726e+08 6.3965082e+08 64101.637 Difference at 95.0% confidence -1.55279e+07 +/- 86486.9 -2.37003% +/- 0.0132005% (Student's t, pooled s = 49984.1) This test tends to benefit more from order 4 zspages, due to test's data patterns. Data patterns that generate a considerable number of badly compressible objects benefit from higher `huge_class_size` watermark, which is achieved with order 4 zspages. Signed-off-by: Sergey Senozhatsky --- include/linux/zsmalloc.h | 13 ++++++++ mm/zsmalloc.c | 72 +++++++++++++++++++++++----------------- 2 files changed, 55 insertions(+), 30 deletions(-) diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h index 2a430e713ce5..2110b140e0fa 100644 --- a/include/linux/zsmalloc.h +++ b/include/linux/zsmalloc.h @@ -33,6 +33,19 @@ enum zs_mapmode { */ }; +#define ZS_PAGE_ORDER_2 2 +#define ZS_PAGE_ORDER_3 3 +#define ZS_PAGE_ORDER_4 4 + +/* + * A single 'zspage' is composed of up to 2^N discontiguous 0-order (single) + * pages. ZS_MAX_PAGE_ORDER defines upper limit on N, ZS_MIN_PAGE_ORDER + * defines lower limit on N. ZS_DEFAULT_PAGE_ORDER is recommended value. + */ +#define ZS_MIN_PAGE_ORDER ZS_PAGE_ORDER_2 +#define ZS_MAX_PAGE_ORDER ZS_PAGE_ORDER_4 +#define ZS_DEFAULT_PAGE_ORDER ZS_PAGE_ORDER_2 + struct zs_pool_stats { /* How many pages were migrated (freed) */ atomic_long_t pages_compacted; diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 6645506b0b14..6ffa32b8b6c8 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -74,12 +74,7 @@ */ #define ZS_ALIGN 8 -/* - * A single 'zspage' is composed of up to 2^N discontiguous 0-order (single) - * pages. ZS_MAX_ZSPAGE_ORDER defines upper limit on N. - */ -#define ZS_MAX_ZSPAGE_ORDER 2 -#define ZS_MAX_PAGES_PER_ZSPAGE (_AC(1, UL) << ZS_MAX_ZSPAGE_ORDER) +#define ZS_MAX_PAGES_PER_ZSPAGE (_AC(1, UL) << ZS_MAX_PAGE_ORDER) #define ZS_HANDLE_SIZE (sizeof(unsigned long)) @@ -124,10 +119,8 @@ #define ISOLATED_BITS 3 #define MAGIC_VAL_BITS 8 -#define MAX(a, b) ((a) >= (b) ? (a) : (b)) -/* ZS_MIN_ALLOC_SIZE must be multiple of ZS_ALIGN */ -#define ZS_MIN_ALLOC_SIZE \ - MAX(32, (ZS_MAX_PAGES_PER_ZSPAGE << PAGE_SHIFT >> OBJ_INDEX_BITS)) +#define ZS_MIN_ALLOC_SIZE 32U + /* each chunk includes extra space to keep handle */ #define ZS_MAX_ALLOC_SIZE PAGE_SIZE @@ -141,12 +134,10 @@ * determined). NOTE: all those class sizes must be set as multiple of * ZS_ALIGN to make sure link_free itself never has to span 2 pages. * - * ZS_MIN_ALLOC_SIZE and ZS_SIZE_CLASS_DELTA must be multiple of ZS_ALIGN - * (reason above) + * pool->min_alloc_size (ZS_MIN_ALLOC_SIZE) and ZS_SIZE_CLASS_DELTA must + * be multiple of ZS_ALIGN (reason above) */ #define ZS_SIZE_CLASS_DELTA (PAGE_SIZE >> CLASS_BITS) -#define ZS_SIZE_CLASSES (DIV_ROUND_UP(ZS_MAX_ALLOC_SIZE - ZS_MIN_ALLOC_SIZE, \ - ZS_SIZE_CLASS_DELTA) + 1) enum fullness_group { ZS_EMPTY, @@ -230,12 +221,16 @@ struct link_free { struct zs_pool { const char *name; - struct size_class *size_class[ZS_SIZE_CLASSES]; + struct size_class **size_class; struct kmem_cache *handle_cachep; struct kmem_cache *zspage_cachep; atomic_long_t pages_allocated; + u32 num_size_classes; + u32 min_alloc_size; + u32 max_pages_per_zspage; + struct zs_pool_stats stats; /* Compact classes */ @@ -523,15 +518,15 @@ static void set_zspage_mapping(struct zspage *zspage, * classes depending on its size. This function returns index of the * size class which has chunk size big enough to hold the given size. */ -static int get_size_class_index(int size) +static int get_size_class_index(struct zs_pool *pool, int size) { int idx = 0; - if (likely(size > ZS_MIN_ALLOC_SIZE)) - idx = DIV_ROUND_UP(size - ZS_MIN_ALLOC_SIZE, + if (likely(size > pool->min_alloc_size)) + idx = DIV_ROUND_UP(size - pool->min_alloc_size, ZS_SIZE_CLASS_DELTA); - return min_t(int, ZS_SIZE_CLASSES - 1, idx); + return min_t(int, pool->num_size_classes - 1, idx); } /* type can be of enum type class_stat_type or fullness_group */ @@ -591,7 +586,7 @@ static int zs_stats_size_show(struct seq_file *s, void *v) "obj_allocated", "obj_used", "pages_used", "pages_per_zspage", "freeable", "objs_per_zspage"); - for (i = 0; i < ZS_SIZE_CLASSES; i++) { + for (i = 0; i < pool->num_size_classes; i++) { class = pool->size_class[i]; if (class->index != i) @@ -777,13 +772,13 @@ static enum fullness_group fix_fullness_group(struct size_class *class, * link together 3 PAGE_SIZE sized pages to form a zspage * since then we can perfectly fit in 8 such objects. */ -static int get_pages_per_zspage(int class_size) +static int get_pages_per_zspage(struct zs_pool *pool, int class_size) { int i, max_usedpc = 0; /* zspage order which gives maximum used size per KB */ int max_usedpc_order = 1; - for (i = 1; i <= ZS_MAX_PAGES_PER_ZSPAGE; i++) { + for (i = 1; i <= pool->max_pages_per_zspage; i++) { int zspage_size; int waste, usedpc; @@ -1410,7 +1405,7 @@ unsigned long zs_malloc(struct zs_pool *pool, size_t size, gfp_t gfp) /* extra space in chunk to keep the handle */ size += ZS_HANDLE_SIZE; - class = pool->size_class[get_size_class_index(size)]; + class = pool->size_class[get_size_class_index(pool, size)]; /* class->lock effectively protects the zpage migration */ spin_lock(&class->lock); @@ -1959,7 +1954,7 @@ static void async_free_zspage(struct work_struct *work) struct zs_pool *pool = container_of(work, struct zs_pool, free_work); - for (i = 0; i < ZS_SIZE_CLASSES; i++) { + for (i = 0; i < pool->num_size_classes; i++) { class = pool->size_class[i]; if (class->index != i) continue; @@ -2108,7 +2103,7 @@ unsigned long zs_compact(struct zs_pool *pool) struct size_class *class; unsigned long pages_freed = 0; - for (i = ZS_SIZE_CLASSES - 1; i >= 0; i--) { + for (i = pool->num_size_classes - 1; i >= 0; i--) { class = pool->size_class[i]; if (class->index != i) continue; @@ -2152,7 +2147,7 @@ static unsigned long zs_shrinker_count(struct shrinker *shrinker, struct zs_pool *pool = container_of(shrinker, struct zs_pool, shrinker); - for (i = ZS_SIZE_CLASSES - 1; i >= 0; i--) { + for (i = pool->num_size_classes - 1; i >= 0; i--) { class = pool->size_class[i]; if (class->index != i) continue; @@ -2199,6 +2194,22 @@ struct zs_pool *zs_create_pool(const char *name) if (!pool) return NULL; + pool->max_pages_per_zspage = 1U << ZS_MIN_PAGE_ORDER; + /* min_alloc_size must be multiple of ZS_ALIGN */ + pool->min_alloc_size = (pool->max_pages_per_zspage << PAGE_SHIFT) >> + OBJ_INDEX_BITS; + pool->min_alloc_size = max(pool->min_alloc_size, ZS_MIN_ALLOC_SIZE); + + pool->num_size_classes = + DIV_ROUND_UP(ZS_MAX_ALLOC_SIZE - pool->min_alloc_size, + ZS_SIZE_CLASS_DELTA) + 1; + + pool->size_class = kmalloc_array(pool->num_size_classes, + sizeof(struct size_class *), + GFP_KERNEL | __GFP_ZERO); + if (!pool->size_class) + goto err; + init_deferred_free(pool); rwlock_init(&pool->migrate_lock); @@ -2213,17 +2224,17 @@ struct zs_pool *zs_create_pool(const char *name) * Iterate reversely, because, size of size_class that we want to use * for merging should be larger or equal to current size. */ - for (i = ZS_SIZE_CLASSES - 1; i >= 0; i--) { + for (i = pool->num_size_classes - 1; i >= 0; i--) { int size; int pages_per_zspage; int objs_per_zspage; struct size_class *class; int fullness = 0; - size = ZS_MIN_ALLOC_SIZE + i * ZS_SIZE_CLASS_DELTA; + size = pool->min_alloc_size + i * ZS_SIZE_CLASS_DELTA; if (size > ZS_MAX_ALLOC_SIZE) size = ZS_MAX_ALLOC_SIZE; - pages_per_zspage = get_pages_per_zspage(size); + pages_per_zspage = get_pages_per_zspage(pool, size); objs_per_zspage = pages_per_zspage * PAGE_SIZE / size; /* @@ -2307,7 +2318,7 @@ void zs_destroy_pool(struct zs_pool *pool) zs_flush_migration(pool); zs_pool_stat_destroy(pool); - for (i = 0; i < ZS_SIZE_CLASSES; i++) { + for (i = 0; i < pool->num_size_classes; i++) { int fg; struct size_class *class = pool->size_class[i]; @@ -2327,6 +2338,7 @@ void zs_destroy_pool(struct zs_pool *pool) } destroy_cache(pool); + kfree(pool->size_class); kfree(pool->name); kfree(pool); } From patchwork Mon Oct 24 16:12:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 9905 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp575060wru; Mon, 24 Oct 2022 10:42:03 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5OALXOaDNSNwMCrhRcYfGQULqisA9ouuV73aeC0SyAWHvPx9Q/aWo3mfMOjfUNU67PRm/E X-Received: by 2002:a05:6a00:2409:b0:54e:a3ad:d32d with SMTP id z9-20020a056a00240900b0054ea3add32dmr34466311pfh.70.1666633312156; Mon, 24 Oct 2022 10:41:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666633312; cv=none; d=google.com; s=arc-20160816; b=q6Bd6ti6yvQ+XEyTiJouhOhnRmqUKiYcqHrWMeP/0iV+OVbGdaAiRePuwHOv6I3Rj6 d20zgyk7nwLbVbaVS3d6x5WeuDfxzB0uTeIbboUHigvgiWZEBqCUBwEK3NZUePDReQjf GCwXQ1DEVg/bSlAe5nzKc0vKn5ZdfEvima8WMVfgWOjxPN04f/S1OsGp63tMt4ObaIjD D11yN2saWLP0zThBpLW7SSPJfNV5eft/l752fu42agPJZXnZksTAY8DjyzNRu1zuvcMU DfZSZd6Rnh9PG3HaTHq0s+61QZL2fv3oov2/j2FNVJt83cgbGweQinDF+ea2WQxIpdny Xqdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=f2Veyg4Nf5OS8EqVCK8SMJvhKgZdYRnABHlUXCuatrY=; b=0oimwZ7+5tOQz9JWQKg5TgQpyoq2B/sRz66maIZKIpPlgajjbTFVAqJYXPZBOASLF3 VvDilrV5Kqee92ImPZBpsCRp6w2EvMYgdXsgh4TZN3kp+jhyirD01SsM7zQTsrvbj/1V fCWKOjOst2kvm4/dTN+MueWS0RdkWiLz5tFo0ACae+RKDnB+DzZL5O3Ua5jpd6kmDlLu +tu4vWCu0q6HQasq4UW5u47yyydG4QCr+VAaWy4sUYQSayvP0NJzQxOmkcf59zgbMhmZ 2BHx1I0gk/XEBaG4UpRcZsoWLU7pZnhkl0c0+Unyj/fT1psQhyWuP8VXU81Q9wyIjq5W 4cyQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=HofNrlVO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id r18-20020a170903411200b0017f75654a51si105657pld.245.2022.10.24.10.41.37; Mon, 24 Oct 2022 10:41:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=HofNrlVO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233877AbiJXRjx (ORCPT + 99 others); Mon, 24 Oct 2022 13:39:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57090 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232364AbiJXRiw (ORCPT ); Mon, 24 Oct 2022 13:38:52 -0400 Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 32267F55 for ; Mon, 24 Oct 2022 09:14:33 -0700 (PDT) Received: by mail-pg1-x52d.google.com with SMTP id h2so2937228pgp.4 for ; Mon, 24 Oct 2022 09:14:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=f2Veyg4Nf5OS8EqVCK8SMJvhKgZdYRnABHlUXCuatrY=; b=HofNrlVOtqi8ZwPhhW9uLCwc+6uzwE1KBoMF3VcMdZDnR7LFHq8ikzjf7Hv4G3zQLU rp/BxL8DdIY/NMbC/qDx8YBTVkcfegpnrh/dhIKWoX2D7HzG0gKjqqYmxflQmBzFy/jx E0/pLtZAe47U0RHpL3vFn+ytPI+h7Q5K7/eyQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=f2Veyg4Nf5OS8EqVCK8SMJvhKgZdYRnABHlUXCuatrY=; b=W59lsSlWhP2XnkIqIszkaPke2+66Q1CSg+j6PcFNgvwqfngC2bVCHLnMGbC1d8tpC4 RJgaIJ5B+IrbMDu0dSUvpfR7xWy+65dinIhX+7a78zwp4niSeS55uERFjH1YXRJ9fXml Wdipa1F1UldgUeajmdP1SbPz7Rbbuo11MqGhf3plOFx2rFzWsaS/p7vkiJG/DOxu7w9V FSUlHDTy+HvVB+msscdHMMTURtE2/bXo5n64TuG1iMirQNJQmg1rlQEur8HJyooMPwM9 oMjiHdOIKCcalLpmjOZmb6sMy1kn8Ouzd9tdAUIfLhTt/G38Yy1nsOVPUgSE+VdQ3pZN +18Q== X-Gm-Message-State: ACrzQf2rsdqUoem1T91rRI5/RiEkOOtV/4kmcHfk6BDDrY2ZlvEt4bOt nkyAUZkQw3FFOcu8dGFETpTqph0CMwid+Q== X-Received: by 2002:a63:6b49:0:b0:46a:fcba:308f with SMTP id g70-20020a636b49000000b0046afcba308fmr29088458pgc.8.1666627950628; Mon, 24 Oct 2022 09:12:30 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:5f9c:c5bc:902f:3da4]) by smtp.gmail.com with ESMTPSA id u70-20020a627949000000b0056b8726d2d3sm5162pfc.157.2022.10.24.09.12.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Oct 2022 09:12:30 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCH 2/6] zsmalloc/zram: pass zspage order to zs_create_pool() Date: Tue, 25 Oct 2022 01:12:09 +0900 Message-Id: <20221024161213.3221725-3-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.0.135.g90850a2211-goog In-Reply-To: <20221024161213.3221725-1-senozhatsky@chromium.org> References: <20221024161213.3221725-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747591691576433966?= X-GMAIL-MSGID: =?utf-8?q?1747591691576433966?= Allow zsmalloc pool owner to specify max zspage (during pool creation), so that different pools can have different characteristics. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 3 ++- include/linux/zsmalloc.h | 2 +- mm/zsmalloc.c | 11 ++++++++--- 3 files changed, 11 insertions(+), 5 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 364323713393..e3ef542f9618 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1253,7 +1253,8 @@ static bool zram_meta_alloc(struct zram *zram, u64 disksize) if (!zram->table) return false; - zram->mem_pool = zs_create_pool(zram->disk->disk_name); + zram->mem_pool = zs_create_pool(zram->disk->disk_name, + ZS_DEFAULT_PAGE_ORDER); if (!zram->mem_pool) { vfree(zram->table); return false; diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h index 2110b140e0fa..4a92c5e186ad 100644 --- a/include/linux/zsmalloc.h +++ b/include/linux/zsmalloc.h @@ -53,7 +53,7 @@ struct zs_pool_stats { struct zs_pool; -struct zs_pool *zs_create_pool(const char *name); +struct zs_pool *zs_create_pool(const char *name, u32 zspage_order); void zs_destroy_pool(struct zs_pool *pool); unsigned long zs_malloc(struct zs_pool *pool, size_t size, gfp_t flags); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 6ffa32b8b6c8..fa55e0c66f8d 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -369,7 +369,7 @@ static void *zs_zpool_create(const char *name, gfp_t gfp, * different contexts and its caller must provide a valid * gfp mask. */ - return zs_create_pool(name); + return zs_create_pool(name, ZS_DEFAULT_PAGE_ORDER); } static void zs_zpool_destroy(void *pool) @@ -2177,6 +2177,7 @@ static int zs_register_shrinker(struct zs_pool *pool) /** * zs_create_pool - Creates an allocation pool to work from. * @name: pool name to be created + * @zspage_order: maximum order of zspage * * This function must be called before anything when using * the zsmalloc allocator. @@ -2184,17 +2185,21 @@ static int zs_register_shrinker(struct zs_pool *pool) * On success, a pointer to the newly created pool is returned, * otherwise NULL. */ -struct zs_pool *zs_create_pool(const char *name) +struct zs_pool *zs_create_pool(const char *name, u32 zspage_order) { int i; struct zs_pool *pool; struct size_class *prev_class = NULL; + if (WARN_ON(zspage_order < ZS_MIN_PAGE_ORDER || + zspage_order > ZS_MAX_PAGE_ORDER)) + return NULL; + pool = kzalloc(sizeof(*pool), GFP_KERNEL); if (!pool) return NULL; - pool->max_pages_per_zspage = 1U << ZS_MIN_PAGE_ORDER; + pool->max_pages_per_zspage = 1U << zspage_order; /* min_alloc_size must be multiple of ZS_ALIGN */ pool->min_alloc_size = (pool->max_pages_per_zspage << PAGE_SHIFT) >> OBJ_INDEX_BITS; From patchwork Mon Oct 24 16:12:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 9911 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp581868wru; Mon, 24 Oct 2022 11:01:37 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7IWSZEwtHdFOO+rVdFzoI7dD1OA+G6Yu+MaZn/jLhMY72FCxJ27QOkLaCvEQFBA97jkMSJ X-Received: by 2002:a63:86c1:0:b0:46f:1554:1f31 with SMTP id x184-20020a6386c1000000b0046f15541f31mr4191664pgd.244.1666634497010; Mon, 24 Oct 2022 11:01:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666634497; cv=none; d=google.com; s=arc-20160816; b=sN8jo+ZBPjC26zAvQ4n07TXpbv36VXmXX7o9WB1i8otI1ByX9VnCdEsAe9iiRCzbP1 ZMBNiI2pMdlUB5eSlvu5X+wt0+KbDcT3Vn8PnHJtc+FCGyXiNTe13MtS3iINRPx3Dseq l2rc8q++3PRIAen5LbvgTG1e5fmhg+12ahFmRuviA8MkbVM1lczPOAkzMpPT1kckbksF 16KxfT752zjTSN1NYmkVM/wC4iCFj3Dd9B5vnLPRtPsKzgXjHzHyAlHjuKc6QKrswL5r AQCaKd8oWIDs8eqsnEG/vemrBi1vRYz8MCtDUaH5+xHwjT9HzY0fgV8/UD4g3dmjGAkN A1XQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=w9VjWrNf/YPNyhVI/eZNl14XBpdcpcyeDNi5KSdPpF8=; b=itis9SN7oQBV7gzp5rSds2ismREfFOWUpsDL0qrsa95KjzC5sQZx7SHOHiMTXBvWJf dDYrQoRaYrcoMB4pHFMiUUx60rR3xxox50LCblGM/Z/Y6UfVe2fkVUw17vCSe5PNIDs/ r/mL2n6KCDnlJa9BZll6ezekWuEsNRPx7gctbtYws81DN655ey01sb9VcYDrmG9j1OQI xgkfJuzG0560djcjJxf2BvuuX1WO2j09FW6lbzQCFjaintkrqRgJ6eLin4rCKHjHTFpJ vz/NaY0M1zQfp45/AcMSkTCoJFPAMJS010LL6QHjHw9kTHWExVYBZWLA7dNOdt1cG8p7 eb8w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=DFrjiwdN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i12-20020aa78d8c000000b00536bbfa4994si247884pfr.345.2022.10.24.11.01.22; Mon, 24 Oct 2022 11:01:36 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=DFrjiwdN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233491AbiJXRjp (ORCPT + 99 others); Mon, 24 Oct 2022 13:39:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44556 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234764AbiJXRjP (ORCPT ); Mon, 24 Oct 2022 13:39:15 -0400 Received: from mail-pg1-x534.google.com (mail-pg1-x534.google.com [IPv6:2607:f8b0:4864:20::534]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 244EA48C9B for ; Mon, 24 Oct 2022 09:14:54 -0700 (PDT) Received: by mail-pg1-x534.google.com with SMTP id b5so9051237pgb.6 for ; Mon, 24 Oct 2022 09:14:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=w9VjWrNf/YPNyhVI/eZNl14XBpdcpcyeDNi5KSdPpF8=; b=DFrjiwdNQ/wdO9nKOl8hS0v7J4HbfNZAVkr+2ftKxzzo+O9vReSREpw1NYFHbezRKn eBEwBH2umkuDsdKyeiDvfWjEGRH5JhiZsjtymd15Sgevf3KIL3zwgLkmB/+KZ0YUfKiA Ek3WOVt4CrLCmoyFBzoDGXdqxrFsXBd1kkZJE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=w9VjWrNf/YPNyhVI/eZNl14XBpdcpcyeDNi5KSdPpF8=; b=pNaYHuoO/kCS2jEbKGE4oRz5vvhRV0hs2vAErxe5wZKorvRUzmWuS2h0sqldN85Lft 7P24Wnx/w7PAwGdg8QUReIpyOC68wXQc+SYhYRVSuHukqCAJB8m8qsq09Dk6hfGYcSqw wx7qNHyNikVUuD6AQMvV5hgIyBKOFuVC6Gk5CPy3gdLiVtGyU1ITwdi2O7X/jdn1O3sD 4SPhFdrTqpuQqewidq+BOfX3cUnhIjgFVZg1hM+sbtIngKU6aRXsMUTjndfy3MWks0+m oF7b6xsobWEkoQTfjJE+igayx0dA2iCPLbD3BqFcRHLDPdutmEFNOLsnGRTFBgKcygwr wDHA== X-Gm-Message-State: ACrzQf1ZdrOIWlAKxYlXwFk3tKdIzU5FUHrZpbRPjJM7cqIuwKySUJUt 78taaAq+jfBLqD9tFGRmFuBk0jvzRnB7BA== X-Received: by 2002:a63:914b:0:b0:46e:dbd5:ae15 with SMTP id l72-20020a63914b000000b0046edbd5ae15mr11048144pge.94.1666627952967; Mon, 24 Oct 2022 09:12:32 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:5f9c:c5bc:902f:3da4]) by smtp.gmail.com with ESMTPSA id u70-20020a627949000000b0056b8726d2d3sm5162pfc.157.2022.10.24.09.12.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Oct 2022 09:12:32 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCH 3/6] zram: add pool_page_order device attribute Date: Tue, 25 Oct 2022 01:12:10 +0900 Message-Id: <20221024161213.3221725-4-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.0.135.g90850a2211-goog In-Reply-To: <20221024161213.3221725-1-senozhatsky@chromium.org> References: <20221024161213.3221725-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747592934068396489?= X-GMAIL-MSGID: =?utf-8?q?1747592934068396489?= Add a new sysfs knob that allows user-space to set zsmalloc page order value on per-device basis. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 43 ++++++++++++++++++++++++++++++++++- drivers/block/zram/zram_drv.h | 2 ++ 2 files changed, 44 insertions(+), 1 deletion(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index e3ef542f9618..517dae4ff21c 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1186,6 +1186,44 @@ static ssize_t mm_stat_show(struct device *dev, return ret; } +static ssize_t pool_page_order_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + u32 val; + struct zram *zram = dev_to_zram(dev); + + down_read(&zram->init_lock); + val = zram->pool_page_order; + up_read(&zram->init_lock); + + return scnprintf(buf, PAGE_SIZE, "%d\n", val); +} + +static ssize_t pool_page_order_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct zram *zram = dev_to_zram(dev); + u32 val; + + if (kstrtou32(buf, 10, &val)) + return -EINVAL; + + if (val < ZS_MIN_PAGE_ORDER || val > ZS_MAX_PAGE_ORDER) + return -EINVAL; + + down_read(&zram->init_lock); + if (init_done(zram)) { + up_read(&zram->init_lock); + return -EINVAL; + } + + zram->pool_page_order = val; + up_read(&zram->init_lock); + + return len; +} + #ifdef CONFIG_ZRAM_WRITEBACK #define FOUR_K(x) ((x) * (1 << (PAGE_SHIFT - 12))) static ssize_t bd_stat_show(struct device *dev, @@ -1254,7 +1292,7 @@ static bool zram_meta_alloc(struct zram *zram, u64 disksize) return false; zram->mem_pool = zs_create_pool(zram->disk->disk_name, - ZS_DEFAULT_PAGE_ORDER); + zram->pool_page_order); if (!zram->mem_pool) { vfree(zram->table); return false; @@ -2176,6 +2214,7 @@ static DEVICE_ATTR_RW(writeback_limit_enable); static DEVICE_ATTR_RW(recomp_algorithm); static DEVICE_ATTR_WO(recompress); #endif +static DEVICE_ATTR_RW(pool_page_order); static struct attribute *zram_disk_attrs[] = { &dev_attr_disksize.attr, @@ -2203,6 +2242,7 @@ static struct attribute *zram_disk_attrs[] = { &dev_attr_recomp_algorithm.attr, &dev_attr_recompress.attr, #endif + &dev_attr_pool_page_order.attr, NULL, }; @@ -2240,6 +2280,7 @@ static int zram_add(void) goto out_free_idr; } + zram->pool_page_order = ZS_DEFAULT_PAGE_ORDER; zram->disk->major = zram_major; zram->disk->first_minor = device_id; zram->disk->minors = 1; diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index 09b9ceb5dfa3..076d5b17a954 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -120,6 +120,8 @@ struct zram { */ u64 disksize; /* bytes */ const char *comp_algs[ZRAM_MAX_ZCOMPS]; + + u32 pool_page_order; /* * zram is claimed so open request will be failed */ From patchwork Mon Oct 24 16:12:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 10124 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp633552wru; Mon, 24 Oct 2022 13:14:41 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7k6Q4iAeXEk+LqF0oXlJka6KWYwwnlmCXTX8jIEl+LQO40M8umjC8KdEHeXTb5uuX2JMPE X-Received: by 2002:a17:902:f789:b0:17f:8cb6:7da3 with SMTP id q9-20020a170902f78900b0017f8cb67da3mr34740988pln.167.1666642481365; Mon, 24 Oct 2022 13:14:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666642481; cv=none; d=google.com; s=arc-20160816; b=PxhEjWyCUQ/R54NBwEDK7ItPW6KaRn6Wvt+6bx4U3iTILWr2/Jg9RCAA9ZAtsSDdnn TU/agZrl/8GjYie01iLQWDFPl9gRbCFi3gpza2uDPA4fjsVFFNaVsObZ5sVBsj+zMt9e b/wInpG1tcG0DvzbmQFxLtLOYdBjisULuFsjXaM9EfTs/ElShbh4v4psNNPn7vptJMl2 fVhBPjItPhUx39omubJZkU3WiP+B5Hw+O2m+59nHiHLPJwGUMR1X0U0UwwIsQQYA5RIA jDn+S1iARvPP2xSkX3bayprb2WQtilhc7h8QjReUbpaOiq5QuR6Dd/7Tyn4JejM/SU1V +sEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=0+L/wgZJq25QWZOfBJI7Y8G4kNdlVFzsE8gcKEq0Mfs=; b=zDn3aei6YE8pvpNZ34rnMutk5GYoClvoYSHwoOJugHx1M20xv5gOws+5XJ3WaamJde aVuwWKZ7HfKCRI0uIbkzNig6nUpOyH5F4wFMiD/1W8al45b8wnmuoS/AGjOuqB7421VP 5nvfxYoKhoTzAae+PzyIp33CsE/sO/aPlzj2g8Od3uPZBw7do/QjhdtqaQOvrAzhzCNt 9yBokYgrFUA4kBxT44o+dVuVIml224Rjj8IbukLmpSHlEg/oav1LQSWKE+o7PTHVpY+K ea6kmjPU2ibwTjgvMSDPfObw+liVf3fAAzbq3CzXdrB2xbLL+bAGn4XHfRZcYQPfIxXJ /J6Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=ZT+bcVMB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u4-20020a170902e80400b00178431e09bdsi541943plg.230.2022.10.24.13.14.28; Mon, 24 Oct 2022 13:14:41 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=ZT+bcVMB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229817AbiJXUEg (ORCPT + 99 others); Mon, 24 Oct 2022 16:04:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43754 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233964AbiJXUD5 (ORCPT ); Mon, 24 Oct 2022 16:03:57 -0400 Received: from mail-oi1-x22f.google.com (mail-oi1-x22f.google.com [IPv6:2607:f8b0:4864:20::22f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7FC4938A0E for ; Mon, 24 Oct 2022 11:24:55 -0700 (PDT) Received: by mail-oi1-x22f.google.com with SMTP id o64so11672035oib.12 for ; Mon, 24 Oct 2022 11:24:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0+L/wgZJq25QWZOfBJI7Y8G4kNdlVFzsE8gcKEq0Mfs=; b=ZT+bcVMBNk7fpdzG51qvU+nw1MQlFcyYzv6wxeZnY1j+dre6v7vQ6lHY5Ifk344dlP gdAvAMVQmF+7ROzaIojNXfakKOHyzc9qvCV2j2xBa1B8l4HMqv5zJmx3MP2QUkXEn+gr ee7UlBpMa++JOciZw2MCsyIYwsJxjL7k2zx7k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0+L/wgZJq25QWZOfBJI7Y8G4kNdlVFzsE8gcKEq0Mfs=; b=Jjlz1RE4I/lTm/HGBYemcmdE4O+zt7pDtKwvXYQvBlr4Sg9VBcmAqBZAQge3ldkKXb 9Jng1Ud4jAwDWQUHO6E16yuxACE5CtgnxHCfnBP7mvAxloE98D1leK+q9SLK35UBLnEX HqSrhFw1TMxO7xyIYEi+GTntOLkAcx2oQFAshMq2WeV8mFD5b9xTvRsj2EcL3KCTnxcT iDcBytMJ+ApIsua5mDhA24SMmAh3T+zJz83wq3f8GRKnw6HVx51nrpLdQ3iq8ZmKe4wi 9lZQMmNU2BK1FdGgYu4W60R2VsUSHxBBLz+EUQTNz2GcM49kyQRaDYHeZX0lOeV6DMlM YW6A== X-Gm-Message-State: ACrzQf1AbAii47LLWGLotKlOmaZMptnvOfHNvu6qfjoHTS0CeELlzdQU olSvfzFSMNgzI2hmX98+pu1d7WYcl3PTUg== X-Received: by 2002:a17:90b:2317:b0:213:26a3:246f with SMTP id mt23-20020a17090b231700b0021326a3246fmr3082381pjb.148.1666627955871; Mon, 24 Oct 2022 09:12:35 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:5f9c:c5bc:902f:3da4]) by smtp.gmail.com with ESMTPSA id u70-20020a627949000000b0056b8726d2d3sm5162pfc.157.2022.10.24.09.12.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Oct 2022 09:12:35 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCH 4/6] Documentation: document zram pool_page_order attribute Date: Tue, 25 Oct 2022 01:12:11 +0900 Message-Id: <20221024161213.3221725-5-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.0.135.g90850a2211-goog In-Reply-To: <20221024161213.3221725-1-senozhatsky@chromium.org> References: <20221024161213.3221725-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747601306605545715?= X-GMAIL-MSGID: =?utf-8?q?1747601306605545715?= Provide a simple documentation for zram pool_page_order device attribute. Signed-off-by: Sergey Senozhatsky --- Documentation/admin-guide/blockdev/zram.rst | 31 ++++++++++++++++----- 1 file changed, 24 insertions(+), 7 deletions(-) diff --git a/Documentation/admin-guide/blockdev/zram.rst b/Documentation/admin-guide/blockdev/zram.rst index 010fb05a5999..cd12a5982ae0 100644 --- a/Documentation/admin-guide/blockdev/zram.rst +++ b/Documentation/admin-guide/blockdev/zram.rst @@ -112,7 +112,24 @@ to list all of them using, for instance, /proc/crypto or any other method. This, however, has an advantage of permitting the usage of custom crypto compression modules (implementing S/W or H/W compression). -4) Set Disksize +4) Set maximum pool page order +============================== + +zsmalloc pages can consist of up to 2^N physical pages. The exact size +is calculated per each zsmalloc size class during zsmalloc pool creation. +ZRAM provides pool_page_order device attribute to see or change N. + +Examples:: + + #show current maximum zsmalloc page order + cat /sys/block/zramX/pool_page_order + 2 + + #set maximum zsmalloc page order + echo 3 > /sys/block/zramX/pool_page_order + + +5) Set Disksize =============== Set disk size by writing the value to sysfs node 'disksize'. @@ -132,7 +149,7 @@ There is little point creating a zram of greater than twice the size of memory since we expect a 2:1 compression ratio. Note that zram uses about 0.1% of the size of the disk when not in use so a huge zram is wasteful. -5) Set memory limit: Optional +6) Set memory limit: Optional ============================= Set memory limit by writing the value to sysfs node 'mem_limit'. @@ -151,7 +168,7 @@ Examples:: # To disable memory limit echo 0 > /sys/block/zram0/mem_limit -6) Activate +7) Activate =========== :: @@ -162,7 +179,7 @@ Examples:: mkfs.ext4 /dev/zram1 mount /dev/zram1 /tmp -7) Add/remove zram devices +8) Add/remove zram devices ========================== zram provides a control interface, which enables dynamic (on-demand) device @@ -182,7 +199,7 @@ execute:: echo X > /sys/class/zram-control/hot_remove -8) Stats +9) Stats ======== Per-device statistics are exported as various nodes under /sys/block/zram/ @@ -283,7 +300,7 @@ a single line of text and contains the following stats separated by whitespace: Unit: 4K bytes ============== ============================================================= -9) Deactivate +10) Deactivate ============= :: @@ -291,7 +308,7 @@ a single line of text and contains the following stats separated by whitespace: swapoff /dev/zram0 umount /dev/zram1 -10) Reset +11) Reset ========= Write any positive value to 'reset' sysfs node:: From patchwork Mon Oct 24 16:12:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 9906 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp578301wru; Mon, 24 Oct 2022 10:51:38 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7JDGuoxN1cp3As04Npni8xTYiI6sQ40p8n67otx7ScH583TId6fyU39glfU5QdV8oZfBXW X-Received: by 2002:a17:902:d70e:b0:178:2d9d:ba7b with SMTP id w14-20020a170902d70e00b001782d9dba7bmr34926401ply.90.1666633898665; Mon, 24 Oct 2022 10:51:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666633898; cv=none; d=google.com; s=arc-20160816; b=B3rK9VRqlzO8Gxx6vO/viS6ZKa9D22PWJ1reHndmE4Aog3QVPhU8XA95RCRjq+rDEk uOviAYTDbTqWIe3rI21+T84wlmsNr205hboyGpRZoYCtOF/cv3vMWQNuogRNWtzdHVdt wwYma7rRl9/hGAcKwvN/Uh7SzkjNklsiUyiLffU7emREWQLDKqgnKE1r2Kehq0u/i5Bi 1QnY1dooxgTBR1bCTydljlGd5b5h9J0iIQGKFKwdlLMkw356ALpjJP4ZOJyqKZ2MgRXs yKlHNaXKvGmQkHp0wug0qBULbRH4MWq0lD7HwNMLkaM6gXJHiEhcJwbD22voblzg/3Bf NAsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=NPUA49DxSROeuVXUf5+5dYRiCRYwBmJzY7QehiBfLmI=; b=i9MNCRcbzgqWF8E7uTOIlL54fzMVXTG/3dS+rprGK+oDdXid7Sv0l6ZScI9qnFrB2N i/0Jwt0ejRROMdssJmQyfr5RhNoKJuDDzEWnu8aISdmNGVl+dmjxq8xGKOic0xkeBIol Yq1VCkONb9dOQsrZ+6MkVQtKsM/jR1NXeksGMR/PVn4cnUxiDqVpMOhIAayqzafO2j7H mHKhMSfk9pas8zbndtne0ouxEK2BUv1WS3cFWtB5I6EtM9DEgwhzNxHEd4K2SUNavlcg ir3hpLUSr+dD9F5onHHRjIzHEDmKltCLqjkMJGK9Ufjrh6IIkOPn13qC+UDR/1i7CdyI FFew== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b="EV/G2vlr"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t14-20020a635f0e000000b004344b581925si47391pgb.879.2022.10.24.10.51.24; Mon, 24 Oct 2022 10:51:38 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b="EV/G2vlr"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231290AbiJXRsq (ORCPT + 99 others); Mon, 24 Oct 2022 13:48:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232560AbiJXRs2 (ORCPT ); Mon, 24 Oct 2022 13:48:28 -0400 Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 22F8FBEAFF for ; Mon, 24 Oct 2022 09:25:10 -0700 (PDT) Received: by mail-pg1-f172.google.com with SMTP id 78so9047785pgb.13 for ; Mon, 24 Oct 2022 09:25:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NPUA49DxSROeuVXUf5+5dYRiCRYwBmJzY7QehiBfLmI=; b=EV/G2vlr9ho5Ulkc8sroUJ+aLbRJoO3MY36vm4QL6xhuYyNjpL7Ar87vrlrAZ5Svm8 rbhIxyNU/WCYSM4dKZqJeI1a0HC2RhQzSx8q4R8RoowxZFSpPbFNOdbB5yGxYkkko3BM jE8lRkc7rPtLERNeVljXsu9Q7FuZGB/Z+DT7o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NPUA49DxSROeuVXUf5+5dYRiCRYwBmJzY7QehiBfLmI=; b=iGx6WnWYTmJljeqBa7FtZhIdSKl3+4KYJwgxEo9LBJBaAvtwSKpktkK6d67XRIXOOG XkxFJEsbbZOG/Nu2CQgHwauoCrKvNfPDWi3L9M4/Cf08VjKQHSHwFgvolY+sJ14QxmqS vmSmSc4kQLCnj3hlCgHKuy4N2fBoSO14ZuiVS25VbNuhjavVt9OSMIUgOBXpz64fGRkv KRmjTeD7nKr8An560JQWmDTjwpc7fqq5N/6ufm/PYsQ4ZICNW6dr/fDJJf1VPu/NaJul mH78Qan2enI/diQlJ77pqIP1PHFatjherp2BnL7oQF70lZooLFbjxRVOBLCPcEU+06xU 65iA== X-Gm-Message-State: ACrzQf1VQwf5+mgqxcpB/El5NJdlIyWl+qGk6IYOyFmZqDLOhsXMt4TC tW/PI7EwjTuZCvUXDp8c90vP+K5VbnuqJg== X-Received: by 2002:a63:7304:0:b0:46a:faa5:700f with SMTP id o4-20020a637304000000b0046afaa5700fmr29794719pgc.253.1666627958807; Mon, 24 Oct 2022 09:12:38 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:5f9c:c5bc:902f:3da4]) by smtp.gmail.com with ESMTPSA id u70-20020a627949000000b0056b8726d2d3sm5162pfc.157.2022.10.24.09.12.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Oct 2022 09:12:38 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCH 5/6] zsmalloc: break out of loop when found perfect zspage order Date: Tue, 25 Oct 2022 01:12:12 +0900 Message-Id: <20221024161213.3221725-6-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.0.135.g90850a2211-goog In-Reply-To: <20221024161213.3221725-1-senozhatsky@chromium.org> References: <20221024161213.3221725-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747592306844496659?= X-GMAIL-MSGID: =?utf-8?q?1747592306844496659?= If we found zspage configuration that gives us perfect 100% used percentage (zero wasted space) then there is no point it trying any other configuration Signed-off-by: Sergey Senozhatsky --- mm/zsmalloc.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index fa55e0c66f8d..40a09b1f63b5 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -790,6 +790,9 @@ static int get_pages_per_zspage(struct zs_pool *pool, int class_size) max_usedpc = usedpc; max_usedpc_order = i; } + + if (usedpc == 100) + break; } return max_usedpc_order; From patchwork Mon Oct 24 16:12:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 9903 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp574206wru; Mon, 24 Oct 2022 10:40:06 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7Kd+JEBY+4bfsDt2K/lrKx3OZdP/QWha6Q7SVv7XccVlEkBZASoeO8WLAQV8vOy+wr2dxb X-Received: by 2002:a17:902:f789:b0:17f:8cb6:7da3 with SMTP id q9-20020a170902f78900b0017f8cb67da3mr34181452pln.167.1666633206670; Mon, 24 Oct 2022 10:40:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666633206; cv=none; d=google.com; s=arc-20160816; b=MaQzO06GJAHftqptb5bJ7rlcFemVSgwcylJwhINLeZ5AMAsLa8+WdWIDFEGMzpQVj7 OqBoXRUnEbMREuu5QVkXZ+0drCp+lu34VRGytX9C60t5cxI3Uya3wyLklyLtniMq+ypt yQjev66Dqf7u5rwtY11IsJwwABfVX6RzHwcHHIgeO4VpRLe3yYN8G8NOjsyp+4pyk7v7 UwiW4ylqkaRM64yjW48OIJ+sQBGcnsMLqZFmYtDMh2Ovey51M5zwpf3K2hpFFShhri0c /5yj0tmfYejIdje9VJL+1sZEl20S4HyoU/Kq0rovZBkk2hC1qPSQZv34bWDd6SXgKomM yKFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=6K/W/tGQbEGTcjFYFF7obhZCetWTg/joFwdMrcbbg88=; b=qG5uUTtChB3YEWx7Hj7rlPNUQERX7TmJlr12iSspPciK2FKdvDQ9oJflmC9m1jZGO+ pI3Z5he2+5gE9zI7Wk1puayubEK9um2sUubb1T6KxXvvPr/qsyR0kVJ81bPOemDx0beo KJwIBZ9vZ5i/z8mCrITvpFJay/3TLwdCbBdS2qKGM3JUagpw0xZGcX9dY6HcqhLihz1A EcEdotZDjZtnyfTyVla1ukfSxXVRAS5ZT5sj/rEZfi2czeUMKdeSjNzOrHunt+1c2VJf NerIV6GDe7FmGdwWAkR/mQ0supYXSdfATPwHXiciA161gGR7fNKTAKWHj/MGlnXXn2Yi a4Yg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b="h/ytl5e9"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id nu15-20020a17090b1b0f00b001f303395c8esi11092222pjb.141.2022.10.24.10.39.51; Mon, 24 Oct 2022 10:40:06 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b="h/ytl5e9"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231405AbiJXRje (ORCPT + 99 others); Mon, 24 Oct 2022 13:39:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55494 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233852AbiJXRi5 (ORCPT ); Mon, 24 Oct 2022 13:38:57 -0400 Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com [IPv6:2607:f8b0:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9C8893341F for ; Mon, 24 Oct 2022 09:14:44 -0700 (PDT) Received: by mail-pg1-x529.google.com with SMTP id q1so9036358pgl.11 for ; Mon, 24 Oct 2022 09:14:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6K/W/tGQbEGTcjFYFF7obhZCetWTg/joFwdMrcbbg88=; b=h/ytl5e9/k/342yB1H8BeaL02HxFJn4GIfjrkkGet6ZLG4cPmCYS/DpaC6IXsHL+CF Dqi4snGmU88lbGI0yHxHTbbvOMuiht6icq3YEZzVWdCQhtsW7U7x/xCJt2ThlpE5kCpa Ai42HwHKaNshye+8lyPpeRjxySViQ+q8azKSc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6K/W/tGQbEGTcjFYFF7obhZCetWTg/joFwdMrcbbg88=; b=Il9EaMoTwJSi+spYhN9sFPMGvPSEGGuAp/EmmwuCUAQPdcMO0qB8CCPQ/+AHEtKCFb M1DykAqNn+K3iICO/xbRqnLxu1ohN6im2Ftu3QGbA5x0GifybuKGGB/xuagFj0RdcSFg FnAzulVHwzi/49H9mPDM/dd3c8u5Rv09fPD/x/4Zk5Z3Jz0nVTGPE+n1JB4lAfZMmVHF rw9yI9m5BmVLfw2Kf4LRkk+qfaDFRm9VGfhkSBuzAVR+FCrG9AlXF6G877RP42CwS/a7 rzlQEMwWzSYYiK+BQG7p4RpRUSUJZtqt1e4zD4eqNHY3U+A1EMR/nS3wX5tFoPnl+o1b FB6g== X-Gm-Message-State: ACrzQf1PieDyh7a0bF+vn3cE24qzOaXmeVkT51KOWEuyT9d3D8tDrs49 rkkcOn4DjtnFy9uGsc2cLvrmwqNJ3BGxHQ== X-Received: by 2002:a05:6a00:1799:b0:56c:1495:70bc with SMTP id s25-20020a056a00179900b0056c149570bcmr119687pfg.12.1666627961483; Mon, 24 Oct 2022 09:12:41 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:5f9c:c5bc:902f:3da4]) by smtp.gmail.com with ESMTPSA id u70-20020a627949000000b0056b8726d2d3sm5162pfc.157.2022.10.24.09.12.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Oct 2022 09:12:40 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCH 6/6] zsmalloc: make sure we select best zspage size Date: Tue, 25 Oct 2022 01:12:13 +0900 Message-Id: <20221024161213.3221725-7-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.0.135.g90850a2211-goog In-Reply-To: <20221024161213.3221725-1-senozhatsky@chromium.org> References: <20221024161213.3221725-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747591581023727055?= X-GMAIL-MSGID: =?utf-8?q?1747591581023727055?= We currently decide the best zspage size by looking at used percentage value. This is not exactly enough as zspage usage percentage calculation is not accurate. For example, let's look at size class 208 pages per zspage wasted bytes used% 1 144 96 2 80 99 3 16 99 4 160 99 We will select 2 page per zspage configuration, as it is the first one to reach 99%. However, 3 pages per zspage wastes less memory. Hence we need to also consider wasted space metrics when device zspage size. Additionally, rename max_usedpc_order because it does not hold zspage order, it holds maximum pages per-zspage value. Signed-off-by: Sergey Senozhatsky --- mm/zsmalloc.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 40a09b1f63b5..5de56f4cd16a 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -775,8 +775,9 @@ static enum fullness_group fix_fullness_group(struct size_class *class, static int get_pages_per_zspage(struct zs_pool *pool, int class_size) { int i, max_usedpc = 0; - /* zspage order which gives maximum used size per KB */ - int max_usedpc_order = 1; + /* zspage size which gives maximum used size per KB */ + int pages_per_zspage = 1; + int min_waste = INT_MAX; for (i = 1; i <= pool->max_pages_per_zspage; i++) { int zspage_size; @@ -788,14 +789,19 @@ static int get_pages_per_zspage(struct zs_pool *pool, int class_size) if (usedpc > max_usedpc) { max_usedpc = usedpc; - max_usedpc_order = i; + pages_per_zspage = i; } if (usedpc == 100) break; + + if (waste < min_waste) { + min_waste = waste; + pages_per_zspage = i; + } } - return max_usedpc_order; + return pages_per_zspage; } static struct zspage *get_zspage(struct page *page)