From patchwork Mon Oct 31 05:41:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13111 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp2127658wru; Sun, 30 Oct 2022 22:41:50 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5VDZd6RzkoJ5p4Zh2fkWrnac5lPGdt7xtVvUWvtsKZ0ncjfp5cIqivuUV4FR6GCuwSQe5G X-Received: by 2002:a17:90b:4d91:b0:213:f1b:dab5 with SMTP id oj17-20020a17090b4d9100b002130f1bdab5mr12835868pjb.95.1667194909880; Sun, 30 Oct 2022 22:41:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667194909; cv=none; d=google.com; s=arc-20160816; b=C4rHF6WTwNzNnrqAkN9fwuSrQNV0lrJWQrAUNbqKn4qE04RbYvA/phENkZ7YXu5A1c 1ztCQzH7nKJ36QLXoZLUIKx/4zcfoUxsVK+XqVvqXGw4rPG4xh2SiMonWJfwQmIynEm7 rqlshaRznmPDTz1LrZyokvQx+IPR+NkGz9UVp61ofXZOt/71p6WBFf7ScdYvvToDSD/L bUqBe0+EQ0pcsYrw0YWRSz/063+B2vZqLuq5aMQ9vrEr/A3BsfR90UAOOje3Mugfqw45 KJ0d+3XtoDO//IJ6XbmCmLKB4+9M2j63o5p/fgvYEjbdMhENVyQc3Dk/+46l1BAfO/wY oeXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=9v+j7YIhXu2E5hbe6+G2o2cWhQoPLQTweQOnVfkUv2E=; b=0Gdagc0UVj0Sw0ZJqqlkRciVq2gM99Q321fluk0QX2yGFWJmDof+/V7o/TGZy0J7z8 LnVka126z8txZFnlNx+IETPmHYubXHPEhRY6wTDQy/IjjYYIUGpvu/lAmUJ5zEoVyIcJ nWmfmFGCOUnYWn97x+a0kfNiIrPB1OYBPgMjVKF4DueqD704j+WHv/Z3XozwQFVsBLdN SiRRcdS/pJ4G4oDuPnY/FRNOVw9VDZbyd7zn+kLT6VG6k1KMxPe/6pnSrTvvchyBXyed fjHhe29fPvvXQL3emy7kewa/xBHVOtS5MiQ617VKugy2eAoakWnFUnUl9J8c39dsmYOD anTQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=HHA2qvPE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q15-20020a170902dacf00b001825b1375ebsi8712275plx.544.2022.10.30.22.41.36; Sun, 30 Oct 2022 22:41:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=HHA2qvPE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229761AbiJaFlX (ORCPT + 99 others); Mon, 31 Oct 2022 01:41:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53284 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229779AbiJaFlV (ORCPT ); Mon, 31 Oct 2022 01:41:21 -0400 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3255FAE6F for ; Sun, 30 Oct 2022 22:41:20 -0700 (PDT) Received: by mail-pf1-x429.google.com with SMTP id k15so1727404pfg.2 for ; Sun, 30 Oct 2022 22:41:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9v+j7YIhXu2E5hbe6+G2o2cWhQoPLQTweQOnVfkUv2E=; b=HHA2qvPEXFBXVL0y35D/gKbkMA3J8JE7u+qse7jccRNYRVFeZl1ZI8qGbNMOVHXSHf v40uZihi/8P1v4QSHUvJCAjLlzOeQ9qeN0ngXzT5Xo41WcZ2vMNTKZiUV4bOmY+SSXQ3 Byk0Lm156jYYKOSKcsI020PV+WW+rdYOEU3jQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9v+j7YIhXu2E5hbe6+G2o2cWhQoPLQTweQOnVfkUv2E=; b=lyO1tFrDitDsQhK3O5IxaDSTg8jmc3SdqxlA6t9tly3QA0nFrxAfB8ZkSLYcs0MIAZ fyYIYhvnFU0MQmzYOi6BNAPfRyOg+pu7Ka4qroyeFE+mQOTmhVidtc2zRBJFzxi1CK9g 0vuVwGtP7QySbgaKy2b+AtKrbAK6Hev5j5jAlFqmgS89A0GvRiB5szxculJyN2O9OGyy qG+bXhPIRlbBOjp56jL9rG2ubbQJjAXUNRtfp3fPZGcHndWbk8Mk10LfME1c/l9IvfJe jvM5ROE7TezL2f2ijfXWMd1hmDpmGEffFrjO3yHrj89hGOcOFThRCXGGYQhFIRPYFubU /eew== X-Gm-Message-State: ACrzQf0gmoA5RDUqJoz74THhK7jjVGYa619sDD1zYXfk2pGr6Filidv+ Jw9uAmPEqGw2zkkKD7Y7XDyLaQ== X-Received: by 2002:a62:e40e:0:b0:56b:add7:fc22 with SMTP id r14-20020a62e40e000000b0056badd7fc22mr12547815pfh.63.1667194879706; Sun, 30 Oct 2022 22:41:19 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:7616:afe0:ba6c:96f4]) by smtp.gmail.com with ESMTPSA id w12-20020aa79a0c000000b0056befcd7958sm3573308pfj.84.2022.10.30.22.41.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 22:41:19 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexey Romanov , Sergey Senozhatsky Subject: [PATCHv4 1/9] zram: add size class equals check into recompression Date: Mon, 31 Oct 2022 14:41:00 +0900 Message-Id: <20221031054108.541190-2-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.1.273.g43a17bfeac-goog In-Reply-To: <20221031054108.541190-1-senozhatsky@chromium.org> References: <20221031054108.541190-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748180570053138881?= X-GMAIL-MSGID: =?utf-8?q?1748180570053138881?= From: Alexey Romanov It makes no sense for us to recompress the object if it will be in the same size class. We anyway don't get any memory gain. But, at the same time, we get a CPU time overhead when inserting this object into zspage and decompressing it afterwards. Signed-off-by: Alexey Romanov Reviewed-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 5 +++++ include/linux/zsmalloc.h | 2 ++ mm/zsmalloc.c | 21 +++++++++++++++++++++ 3 files changed, 28 insertions(+) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 364323713393..fd31beb6491a 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1632,6 +1632,8 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, unsigned long handle_next; unsigned int comp_len_next; unsigned int comp_len_prev; + unsigned int class_index_prev; + unsigned int class_index_next; struct zcomp_strm *zstrm; void *src, *dst; int ret; @@ -1656,6 +1658,8 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, ret = zcomp_compress(zstrm, src, &comp_len_next); kunmap_atomic(src); + class_index_prev = zs_lookup_class_index(zram->mem_pool, comp_len_prev); + class_index_next = zs_lookup_class_index(zram->mem_pool, comp_len_next); /* * Either a compression error or we failed to compressed the object * in a way that will save us memory. Mark the object so that we @@ -1663,6 +1667,7 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, */ if (comp_len_next >= huge_class_size || comp_len_next >= comp_len_prev || + class_index_next >= class_index_prev || ret) { zram_set_flag(zram, index, ZRAM_RECOMP_SKIP); zram_clear_flag(zram, index, ZRAM_IDLE); diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h index 2a430e713ce5..a48cd0ffe57d 100644 --- a/include/linux/zsmalloc.h +++ b/include/linux/zsmalloc.h @@ -55,5 +55,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle); unsigned long zs_get_total_pages(struct zs_pool *pool); unsigned long zs_compact(struct zs_pool *pool); +unsigned int zs_lookup_class_index(struct zs_pool *pool, unsigned int size); + void zs_pool_stats(struct zs_pool *pool, struct zs_pool_stats *stats); #endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index d03941cace2c..065744b7e9d8 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1205,6 +1205,27 @@ static bool zspage_full(struct size_class *class, struct zspage *zspage) return get_zspage_inuse(zspage) == class->objs_per_zspage; } +/** + * zs_lookup_class_index() - Returns index of the zsmalloc &size_class + * that hold objects of the provided size. + * @pool: zsmalloc pool to use + * @size: object size + * + * Context: Any context. + * + * Return: the index of the zsmalloc &size_class that hold objects of the + * provided size. + */ +unsigned int zs_lookup_class_index(struct zs_pool *pool, unsigned int size) +{ + struct size_class *class; + + class = pool->size_class[get_size_class_index(size)]; + + return class->index; +} +EXPORT_SYMBOL_GPL(zs_lookup_class_index); + unsigned long zs_get_total_pages(struct zs_pool *pool) { return atomic_long_read(&pool->pages_allocated); From patchwork Mon Oct 31 05:41:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13112 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp2127702wru; Sun, 30 Oct 2022 22:41:57 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4AExxfh+hsxiWu6M74rSGJ4lyrnm9KtM86CnmwkyUKUO7QzCakHdWhGRbhICgllMmQfoPQ X-Received: by 2002:a17:902:e5c5:b0:186:5f09:8468 with SMTP id u5-20020a170902e5c500b001865f098468mr12907573plf.122.1667194917295; Sun, 30 Oct 2022 22:41:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667194917; cv=none; d=google.com; s=arc-20160816; b=LANcdidkwYB834Sgqvo0rCg/bp+VlDN6z2KnwWqqVKv9PrL0QeKL0ErjBxTZ5f1jBH 2VOWKQ3UyRfyRZoVomt5tz3pmY6XMl4NjOqbwMtD4mYFpw4LjG/JQyboMeMcsyKDgwis vsvaS5e3Vq2InbAt40UYhFK6ylqrO/0XdVyReQjcvxAHIe9Q+xwVzwTBwtcQB0QHjOQC uDBlGrz8BidJ6D4oiLpyEP2Vmex2yqyAALtmjsvVMR7NxzjH+gI3AUKPn1M+h1F9k73a CFBmTzMXekfaEe1EKgqUb3/x/lf6uV6jrR8cU9N4u+yV5FjsGGOrFZKmlus8S+OBYY42 XZOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=3t+2U4oymRETQLfWsFIy7oUT4aEsjQJ2d8KM7HrwZlo=; b=ulMA7JrcheGzMiHdhXTTuPuQ97COh7H0KUZBZvXHmbIyWmBBdns+0rKWOHkh+oqkn8 Jxs9DULE6Sj9Pr9yNJXLp61re9aqj/BUyznQme2sM5j1NtE/cAr3H5Wg0jcjsoNEKOFN m/mQj31cGPij2SQBPgZrUB/dtglw8y8qjldkrBcLhiR8qqcNGpXENWLj9aImUKF5cx3G zejnT6/jIPX6BbC6l8Yb2/X7FDTHzJ3kXgakLG5nhcvnhtxrps3huzCcaw6QOvSl+Dpo v+fi1ii5YLjpBd5W2wldYMR8HntYTlzMiQyJ/6hAtPnPydQTzqpIRrtkxKcVLyIfVaRj 2/mg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=L4rUPJZ9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y2-20020a655a02000000b0045f7fc4b812si7811426pgs.295.2022.10.30.22.41.44; Sun, 30 Oct 2022 22:41:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=L4rUPJZ9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229826AbiJaFlb (ORCPT + 99 others); Mon, 31 Oct 2022 01:41:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53338 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229781AbiJaFlZ (ORCPT ); Mon, 31 Oct 2022 01:41:25 -0400 Received: from mail-pl1-x62c.google.com (mail-pl1-x62c.google.com [IPv6:2607:f8b0:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 513D7AE6F for ; Sun, 30 Oct 2022 22:41:23 -0700 (PDT) Received: by mail-pl1-x62c.google.com with SMTP id 4so9868998pli.0 for ; Sun, 30 Oct 2022 22:41:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3t+2U4oymRETQLfWsFIy7oUT4aEsjQJ2d8KM7HrwZlo=; b=L4rUPJZ98rg6T8HBwQPxMu2mPUCPtOdrHP+01h2AXUlEaRUUhYB5W0GAoS7rCYONw+ 7qmUHXM1PB2C7I9Yos4xfxi8x1s4Lpg8mLIMlKNG7rhp0jG7Dh5OpCVFIJsYtWKRl+SR hpCoEUa7d3ZWV6gAVAG6n6fNwcZV5fL9QU2PA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3t+2U4oymRETQLfWsFIy7oUT4aEsjQJ2d8KM7HrwZlo=; b=jXlZmhYu+ltyX3qnREtV7G/1V1RkIDvzDmBMKPd1D9S+AJtKl0tFvQjQmSHMHZFoQN 59HVk8WDcCYdmBrEVltWpKu2aIF80gasGOCw9CS896tTs4ZQu/+rTSP8pZQ+70d/vT+i tQzkx80BT9BOd/wwogvyJqiw9xLSn8uac8sgz/cJ6teQh5KffdqKcfk4M3tqB3s4KIGj +5W+tZGck2v5uHX/aFAY3vrkLKu29RQhiJR5qoVcydn5e79GfBlVyHddIKPwzykoCuRA +WkQGS3O8CUsz8m+jTL/9UWKBCMGYwscZR+bRLUdpekfVNYEevG8TNatUj8/A3Te8beM PDRw== X-Gm-Message-State: ACrzQf0gIIsYOGK0d5AFT/kzyTDgFxIvK++b+0DM+pAQGr8BqSYegTxC NocLY9rvJ17vybqHizwGaIlp+fd5xuLIOg== X-Received: by 2002:a17:90b:1642:b0:213:f368:648b with SMTP id il2-20020a17090b164200b00213f368648bmr1057816pjb.173.1667194882600; Sun, 30 Oct 2022 22:41:22 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:7616:afe0:ba6c:96f4]) by smtp.gmail.com with ESMTPSA id w12-20020aa79a0c000000b0056befcd7958sm3573308pfj.84.2022.10.30.22.41.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 22:41:22 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv4 2/9] zsmalloc: turn zspage order into runtime variable Date: Mon, 31 Oct 2022 14:41:01 +0900 Message-Id: <20221031054108.541190-3-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.1.273.g43a17bfeac-goog In-Reply-To: <20221031054108.541190-1-senozhatsky@chromium.org> References: <20221031054108.541190-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748180577274124901?= X-GMAIL-MSGID: =?utf-8?q?1748180577274124901?= zsmalloc has 255 size classes. Size classes contain a number of zspages, which store objects of the same size. zspage can consist of up to four physical pages. The exact (most optimal) zspage size is calculated for each size class during zsmalloc pool creation. As a reasonable optimization, zsmalloc merges size classes that have similar characteristics: number of pages per zspage and number of objects zspage can store. For example, let's look at the following size classes: class size almost_full almost_empty obj_allocated obj_used pages_used pages_per_zspage freeable .. 94 1536 0 0 0 0 0 3 0 100 1632 0 0 0 0 0 2 0 .. Size classes #95-99 are merged with size class #100. That is, each time we store an object of size, say, 1568 bytes instead of using class #96 we end up storing it in size class #100. Class #100 is for objects of 1632 bytes in size, hence every 1568 bytes object wastes 1632-1568 bytes. Class #100 zspages consist of 2 physical pages and can hold 5 objects. When we need to store, say, 13 objects of size 1568 we end up allocating three zspages; in other words, 6 physical pages. However, if we'll look closer at size class #96 (which should hold objects of size 1568 bytes) and trace get_pages_per_zspage(): pages per zspage wasted bytes used% 1 960 76 2 352 95 3 1312 89 4 704 95 5 96 99 We'd notice that the most optimal zspage configuration for this class is when it consists of 5 physical pages, but currently we never let zspages to consists of more than 4 pages. A 5 page class #96 configuration would store 13 objects of size 1568 in a single zspage, allocating 5 physical pages, as opposed to 6 physical pages that class #100 will allocate. A higher order zspage for class #96 also changes its key characteristics: pages per-zspage and objects per-zspage. As a result classes #96 and #100 are not merged anymore, which gives us more compact zsmalloc. Of course the described effect does not apply only to size classes #96 and We still merge classes, but less often so. In other words classes are grouped in a more compact way, which decreases memory wastage: zspage order # unique size classes 2 69 3 123 4 191 Let's take a closer look at the bottom of /sys/kernel/debug/zsmalloc/zram0/classes: class size almost_full almost_empty obj_allocated obj_used pages_used pages_per_zspage freeable ... 202 3264 0 0 0 0 0 4 0 254 4096 0 0 0 0 0 1 0 ... For exactly same reason - maximum 4 pages per zspage - the last non-huge size class is #202, which stores objects of size 3264 bytes. Any object larger than 3264 bytes, hence, is considered to be huge and lands in size class #254, which uses a whole physical page to store every object. To put it slightly differently - objects in huge classes don't share physical pages. 3264 bytes is too low of a watermark and we have too many huge classes: classes from #203 to #254. Similarly to class size #96 above, higher order zspages change key characteristics for some of those huge size classes and thus those classes become normal classes, where stored objects share physical pages. Hence yet another consequence of higher order zspages: we move the huge size class watermark with higher order zspages, have less huge classes and store large objects in a more compact way. For order 3, huge class watermark becomes 3632 bytes: class size almost_full almost_empty obj_allocated obj_used pages_used pages_per_zspage freeable ... 202 3264 0 0 0 0 0 4 0 211 3408 0 0 0 0 0 5 0 217 3504 0 0 0 0 0 6 0 222 3584 0 0 0 0 0 7 0 225 3632 0 0 0 0 0 8 0 254 4096 0 0 0 0 0 1 0 ... For order 4, huge class watermark becomes 3840 bytes: class size almost_full almost_empty obj_allocated obj_used pages_used pages_per_zspage freeable ... 202 3264 0 0 0 0 0 4 0 206 3328 0 0 0 0 0 13 0 207 3344 0 0 0 0 0 9 0 208 3360 0 0 0 0 0 14 0 211 3408 0 0 0 0 0 5 0 212 3424 0 0 0 0 0 16 0 214 3456 0 0 0 0 0 11 0 217 3504 0 0 0 0 0 6 0 219 3536 0 0 0 0 0 13 0 222 3584 0 0 0 0 0 7 0 223 3600 0 0 0 0 0 15 0 225 3632 0 0 0 0 0 8 0 228 3680 0 0 0 0 0 9 0 230 3712 0 0 0 0 0 10 0 232 3744 0 0 0 0 0 11 0 234 3776 0 0 0 0 0 12 0 235 3792 0 0 0 0 0 13 0 236 3808 0 0 0 0 0 14 0 238 3840 0 0 0 0 0 15 0 254 4096 0 0 0 0 0 1 0 ... TESTS ===== Test untars linux-6.0.tar.xz and compiles the kernel. zram is configured as a block device with ext4 file system, lzo-rle compression algorithm. We captured /sys/block/zram0/mm_stat after every test and rebooted the VM. orig_data_size mem_used_total mem_used_max pages_compacted compr_data_size mem_limit same_pages huge_pages ORDER 2 (BASE) zspage 1691791360 628086729 655171584 0 655171584 60 0 34043 1691787264 628089196 655175680 0 655175680 60 0 34046 1691803648 628098840 655187968 0 655187968 59 0 34047 1691795456 628091503 655183872 0 655183872 60 0 34044 1691799552 628086877 655183872 0 655183872 60 0 34047 ORDER 3 zspage 1691803648 627792993 641794048 0 641794048 60 0 33591 1691787264 627779342 641708032 0 641708032 59 0 33591 1691811840 627786616 641769472 0 641769472 60 0 33591 1691803648 627794468 641818624 0 641818624 59 0 33592 1691783168 627780882 641794048 0 641794048 61 0 33591 ORDER 4 zspage 1691803648 627726635 639655936 0 639655936 60 0 33435 1691811840 627733348 639643648 0 639643648 61 0 33434 1691795456 627726290 639614976 0 639614976 60 0 33435 1691803648 627730458 639688704 0 639688704 60 0 33434 1691811840 627727771 639688704 0 639688704 60 0 33434 Order 3 and order 4 show statistically significant improvement in `mem_used_max` metrics. T-test for order 3: x order-2-maxmem + order-3-maxmem N Min Max Median Avg Stddev x 5 6.5517158e+08 6.5518797e+08 6.5518387e+08 6.551806e+08 6730.4157 + 5 6.4170803e+08 6.4181862e+08 6.4179405e+08 6.4177684e+08 42210.666 Difference at 95.0% confidence -1.34038e+07 +/- 44080.7 -2.04581% +/- 0.00672802% (Student's t, pooled s = 30224.5) T-test for order 4: x order-2-maxmem + order-4-maxmem N Min Max Median Avg Stddev x 5 6.5517158e+08 6.5518797e+08 6.5518387e+08 6.551806e+08 6730.4157 + 5 6.3961498e+08 6.396887e+08 6.3965594e+08 6.3965839e+08 31408.602 Difference at 95.0% confidence -1.55222e+07 +/- 33126.2 -2.36915% +/- 0.00505604% (Student's t, pooled s = 22713.4) This test tends to benefit more from order 4 zspages, due to test's data patterns. zsmalloc object distribution analysis ============================================================================= Order 2 (4 pages per zspage) tends to put many objects in size class 2048, which is merged with size classes #112-#125: class size almost_full almost_empty obj_allocated obj_used pages_used pages_per_zspage freeable ... 71 1168 0 0 6146 6146 1756 2 0 74 1216 0 1 4560 4552 1368 3 0 76 1248 0 1 2938 2934 904 4 0 83 1360 0 0 10971 10971 3657 1 0 91 1488 0 0 16126 16126 5864 4 0 94 1536 0 1 5912 5908 2217 3 0 100 1632 0 0 11990 11990 4796 2 0 107 1744 0 1 15771 15768 6759 3 0 111 1808 0 1 10386 10380 4616 4 0 126 2048 0 0 45444 45444 22722 1 0 144 2336 0 0 47446 47446 27112 4 0 151 2448 1 0 10760 10759 6456 3 0 168 2720 0 0 10173 10173 6782 2 0 190 3072 0 1 1700 1697 1275 3 0 202 3264 0 1 290 286 232 4 0 254 4096 0 0 34051 34051 34051 1 0 Order 3 (8 pages per zspage) changed pool characteristics and unmerged some of the size classes, which resulted in less objects being put into size class 2048, because there are lower size classes are now available for more compact object storage: class size almost_full almost_empty obj_allocated obj_used pages_used pages_per_zspage freeable ... 71 1168 0 1 2996 2994 856 2 0 72 1184 0 1 1632 1609 476 7 0 73 1200 1 0 1445 1442 425 5 0 74 1216 0 0 1510 1510 453 3 0 75 1232 0 1 1495 1479 455 7 0 76 1248 0 1 1456 1451 448 4 0 78 1280 0 1 3040 3033 950 5 0 79 1296 0 1 1584 1571 504 7 0 83 1360 0 0 6375 6375 2125 1 0 84 1376 0 1 1817 1796 632 8 0 87 1424 0 1 6020 6006 2107 7 0 88 1440 0 1 2108 2101 744 6 0 89 1456 0 1 2072 2064 740 5 0 91 1488 0 1 4169 4159 1516 4 0 92 1504 0 1 2014 2007 742 7 0 94 1536 0 1 3904 3900 1464 3 0 95 1552 0 1 1890 1873 720 8 0 96 1568 0 1 1963 1958 755 5 0 97 1584 0 1 1980 1974 770 7 0 100 1632 0 1 6190 6187 2476 2 0 103 1680 0 0 6477 6477 2667 7 0 104 1696 0 1 2256 2253 940 5 0 105 1712 0 1 2356 2340 992 8 0 107 1744 1 0 4697 4696 2013 3 0 110 1792 0 1 7744 7734 3388 7 0 111 1808 0 1 2655 2649 1180 4 0 114 1856 0 1 8371 8365 3805 5 0 116 1888 1 0 5863 5862 2706 6 0 117 1904 0 1 2955 2942 1379 7 0 118 1920 0 1 3009 2997 1416 8 0 126 2048 0 0 25276 25276 12638 1 0 128 2080 0 1 6060 6052 3232 8 0 129 2096 1 0 3081 3080 1659 7 0 134 2176 0 1 14835 14830 7912 8 0 135 2192 0 1 2769 2758 1491 7 0 137 2224 0 1 5082 5077 2772 6 0 140 2272 0 1 7236 7232 4020 5 0 144 2336 0 1 8428 8423 4816 4 0 147 2384 0 1 5316 5313 3101 7 0 151 2448 0 1 5445 5443 3267 3 0 155 2512 0 0 4121 4121 2536 8 0 158 2560 0 1 2208 2205 1380 5 0 160 2592 0 0 1133 1133 721 7 0 168 2720 0 0 2712 2712 1808 2 0 177 2864 1 0 1100 1098 770 7 0 180 2912 0 1 189 183 135 5 0 184 2976 0 1 176 166 128 8 0 190 3072 0 0 252 252 189 3 0 197 3184 0 1 198 192 154 7 0 202 3264 0 1 100 96 80 4 0 211 3408 0 1 210 208 175 5 0 217 3504 0 1 98 94 84 6 0 222 3584 0 0 104 104 91 7 0 225 3632 0 1 54 50 48 8 0 254 4096 0 0 33591 33591 33591 1 0 Note, the huge size watermark is above 3632 and there are a number of new normal classes available that previously were merged with the huge class. For instance, size class #211 holds 210 objects of size 3408 and uses 175 physical pages, while previously for those objects we would have used 210 physical pages. Signed-off-by: Sergey Senozhatsky --- include/linux/zsmalloc.h | 12 +++++++ mm/zsmalloc.c | 73 +++++++++++++++++++++++----------------- 2 files changed, 54 insertions(+), 31 deletions(-) diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h index a48cd0ffe57d..6cd1d95b928a 100644 --- a/include/linux/zsmalloc.h +++ b/include/linux/zsmalloc.h @@ -33,6 +33,18 @@ enum zs_mapmode { */ }; +#define ZS_PAGE_ORDER_2 2 +#define ZS_PAGE_ORDER_4 4 + +/* + * A single 'zspage' is composed of up to 2^N discontiguous 0-order (single) + * pages. ZS_MAX_PAGE_ORDER defines upper limit on N, ZS_MIN_PAGE_ORDER + * defines lower limit on N. ZS_DEFAULT_PAGE_ORDER is recommended value. + */ +#define ZS_MIN_PAGE_ORDER ZS_PAGE_ORDER_2 +#define ZS_MAX_PAGE_ORDER ZS_PAGE_ORDER_4 +#define ZS_DEFAULT_PAGE_ORDER ZS_PAGE_ORDER_2 + struct zs_pool_stats { /* How many pages were migrated (freed) */ atomic_long_t pages_compacted; diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 065744b7e9d8..a9773566f85b 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -74,12 +74,7 @@ */ #define ZS_ALIGN 8 -/* - * A single 'zspage' is composed of up to 2^N discontiguous 0-order (single) - * pages. ZS_MAX_ZSPAGE_ORDER defines upper limit on N. - */ -#define ZS_MAX_ZSPAGE_ORDER 2 -#define ZS_MAX_PAGES_PER_ZSPAGE (_AC(1, UL) << ZS_MAX_ZSPAGE_ORDER) +#define ZS_MAX_PAGES_PER_ZSPAGE (_AC(1, UL) << ZS_MAX_PAGE_ORDER) #define ZS_HANDLE_SIZE (sizeof(unsigned long)) @@ -124,10 +119,8 @@ #define ISOLATED_BITS 3 #define MAGIC_VAL_BITS 8 -#define MAX(a, b) ((a) >= (b) ? (a) : (b)) -/* ZS_MIN_ALLOC_SIZE must be multiple of ZS_ALIGN */ -#define ZS_MIN_ALLOC_SIZE \ - MAX(32, (ZS_MAX_PAGES_PER_ZSPAGE << PAGE_SHIFT >> OBJ_INDEX_BITS)) +#define ZS_MIN_ALLOC_SIZE 32U + /* each chunk includes extra space to keep handle */ #define ZS_MAX_ALLOC_SIZE PAGE_SIZE @@ -141,12 +134,10 @@ * determined). NOTE: all those class sizes must be set as multiple of * ZS_ALIGN to make sure link_free itself never has to span 2 pages. * - * ZS_MIN_ALLOC_SIZE and ZS_SIZE_CLASS_DELTA must be multiple of ZS_ALIGN - * (reason above) + * pool->min_alloc_size (ZS_MIN_ALLOC_SIZE) and ZS_SIZE_CLASS_DELTA must + * be multiple of ZS_ALIGN (reason above) */ #define ZS_SIZE_CLASS_DELTA (PAGE_SIZE >> CLASS_BITS) -#define ZS_SIZE_CLASSES (DIV_ROUND_UP(ZS_MAX_ALLOC_SIZE - ZS_MIN_ALLOC_SIZE, \ - ZS_SIZE_CLASS_DELTA) + 1) enum fullness_group { ZS_EMPTY, @@ -230,12 +221,15 @@ struct link_free { struct zs_pool { const char *name; - struct size_class *size_class[ZS_SIZE_CLASSES]; + struct size_class **size_class; struct kmem_cache *handle_cachep; struct kmem_cache *zspage_cachep; atomic_long_t pages_allocated; + u32 num_size_classes; + u32 min_alloc_size; + struct zs_pool_stats stats; /* Compact classes */ @@ -523,15 +517,15 @@ static void set_zspage_mapping(struct zspage *zspage, * classes depending on its size. This function returns index of the * size class which has chunk size big enough to hold the given size. */ -static int get_size_class_index(int size) +static int get_size_class_index(struct zs_pool *pool, int size) { int idx = 0; - if (likely(size > ZS_MIN_ALLOC_SIZE)) - idx = DIV_ROUND_UP(size - ZS_MIN_ALLOC_SIZE, + if (likely(size > pool->min_alloc_size)) + idx = DIV_ROUND_UP(size - pool->min_alloc_size, ZS_SIZE_CLASS_DELTA); - return min_t(int, ZS_SIZE_CLASSES - 1, idx); + return min_t(int, pool->num_size_classes - 1, idx); } /* type can be of enum type class_stat_type or fullness_group */ @@ -591,7 +585,7 @@ static int zs_stats_size_show(struct seq_file *s, void *v) "obj_allocated", "obj_used", "pages_used", "pages_per_zspage", "freeable"); - for (i = 0; i < ZS_SIZE_CLASSES; i++) { + for (i = 0; i < pool->num_size_classes; i++) { class = pool->size_class[i]; if (class->index != i) @@ -777,13 +771,13 @@ static enum fullness_group fix_fullness_group(struct size_class *class, * link together 3 PAGE_SIZE sized pages to form a zspage * since then we can perfectly fit in 8 such objects. */ -static int get_pages_per_zspage(int class_size) +static int get_pages_per_zspage(u32 class_size, u32 num_pages) { int i, max_usedpc = 0; /* zspage order which gives maximum used size per KB */ int max_usedpc_order = 1; - for (i = 1; i <= ZS_MAX_PAGES_PER_ZSPAGE; i++) { + for (i = 1; i <= num_pages; i++) { int zspage_size; int waste, usedpc; @@ -1220,7 +1214,7 @@ unsigned int zs_lookup_class_index(struct zs_pool *pool, unsigned int size) { struct size_class *class; - class = pool->size_class[get_size_class_index(size)]; + class = pool->size_class[get_size_class_index(pool, size)]; return class->index; } @@ -1431,7 +1425,7 @@ unsigned long zs_malloc(struct zs_pool *pool, size_t size, gfp_t gfp) /* extra space in chunk to keep the handle */ size += ZS_HANDLE_SIZE; - class = pool->size_class[get_size_class_index(size)]; + class = pool->size_class[get_size_class_index(pool, size)]; /* class->lock effectively protects the zpage migration */ spin_lock(&class->lock); @@ -1980,7 +1974,7 @@ static void async_free_zspage(struct work_struct *work) struct zs_pool *pool = container_of(work, struct zs_pool, free_work); - for (i = 0; i < ZS_SIZE_CLASSES; i++) { + for (i = 0; i < pool->num_size_classes; i++) { class = pool->size_class[i]; if (class->index != i) continue; @@ -2129,7 +2123,7 @@ unsigned long zs_compact(struct zs_pool *pool) struct size_class *class; unsigned long pages_freed = 0; - for (i = ZS_SIZE_CLASSES - 1; i >= 0; i--) { + for (i = pool->num_size_classes - 1; i >= 0; i--) { class = pool->size_class[i]; if (class->index != i) continue; @@ -2173,7 +2167,7 @@ static unsigned long zs_shrinker_count(struct shrinker *shrinker, struct zs_pool *pool = container_of(shrinker, struct zs_pool, shrinker); - for (i = ZS_SIZE_CLASSES - 1; i >= 0; i--) { + for (i = pool->num_size_classes - 1; i >= 0; i--) { class = pool->size_class[i]; if (class->index != i) continue; @@ -2215,11 +2209,27 @@ struct zs_pool *zs_create_pool(const char *name) int i; struct zs_pool *pool; struct size_class *prev_class = NULL; + unsigned long num_pages; pool = kzalloc(sizeof(*pool), GFP_KERNEL); if (!pool) return NULL; + num_pages = 1UL << ZS_DEFAULT_PAGE_ORDER; + /* min_alloc_size must be multiple of ZS_ALIGN */ + pool->min_alloc_size = num_pages << PAGE_SHIFT >> OBJ_INDEX_BITS; + pool->min_alloc_size = max(pool->min_alloc_size, ZS_MIN_ALLOC_SIZE); + + pool->num_size_classes = + DIV_ROUND_UP(ZS_MAX_ALLOC_SIZE - pool->min_alloc_size, + ZS_SIZE_CLASS_DELTA) + 1; + + pool->size_class = kmalloc_array(pool->num_size_classes, + sizeof(struct size_class *), + GFP_KERNEL | __GFP_ZERO); + if (!pool->size_class) + goto err; + init_deferred_free(pool); rwlock_init(&pool->migrate_lock); @@ -2234,17 +2244,17 @@ struct zs_pool *zs_create_pool(const char *name) * Iterate reversely, because, size of size_class that we want to use * for merging should be larger or equal to current size. */ - for (i = ZS_SIZE_CLASSES - 1; i >= 0; i--) { + for (i = pool->num_size_classes - 1; i >= 0; i--) { int size; int pages_per_zspage; int objs_per_zspage; struct size_class *class; int fullness = 0; - size = ZS_MIN_ALLOC_SIZE + i * ZS_SIZE_CLASS_DELTA; + size = pool->min_alloc_size + i * ZS_SIZE_CLASS_DELTA; if (size > ZS_MAX_ALLOC_SIZE) size = ZS_MAX_ALLOC_SIZE; - pages_per_zspage = get_pages_per_zspage(size); + pages_per_zspage = get_pages_per_zspage(size, num_pages); objs_per_zspage = pages_per_zspage * PAGE_SIZE / size; /* @@ -2328,7 +2338,7 @@ void zs_destroy_pool(struct zs_pool *pool) zs_flush_migration(pool); zs_pool_stat_destroy(pool); - for (i = 0; i < ZS_SIZE_CLASSES; i++) { + for (i = 0; i < pool->num_size_classes; i++) { int fg; struct size_class *class = pool->size_class[i]; @@ -2348,6 +2358,7 @@ void zs_destroy_pool(struct zs_pool *pool) } destroy_cache(pool); + kfree(pool->size_class); kfree(pool->name); kfree(pool); } From patchwork Mon Oct 31 05:41:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13113 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp2127776wru; Sun, 30 Oct 2022 22:42:08 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5Dj0pbCr9pGrFKy13lRLDGxbn7xQwN/48bt31K1Gb8rycfzJBGSFB5wjHAyE9/LrcXV2oq X-Received: by 2002:a63:6f8a:0:b0:439:36bc:89f9 with SMTP id k132-20020a636f8a000000b0043936bc89f9mr10974954pgc.100.1667194927765; Sun, 30 Oct 2022 22:42:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667194927; cv=none; d=google.com; s=arc-20160816; b=NJ/K2JyKEKE0ZzRooz2W+PVIDJTHiOUhxM+IYhQrA6yvD61H6eSsIQcjBR/mUYbZNt zP4Bz4v6NBdDpOdOFJo6URJbhIHz8EN4dZ9sjSKaQnt9Jw3QkRstSSL1MUiyDS7EDBCD y2KuwX5SaO5DB9x45NQubPr7+TBOrw4r5Pz2IlV9fZDWSeC6mEVjMG5En+RSiLBCGEnn nXkCsmNGw5r3PFL+7GqPqUo+NkHEKq5iynYem6Ctj6EXAyhv9PV2OJGxs1B7AZPic4MV TwDJYNz3lVcagGFTkQdAztkgtucGgw5zGr4rwQD1eIAW7VnZGZ+dJMCv/d5ytQhHPAOj 3ScQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=q47a/PNtxAlJKL8yCX3+cfAOXZEGpDh+HXwifBH5lKc=; b=ZyiquEaZtv3SADig657q963UytcT2LpQVrOAmwnv8zBxiYCQZ+YHLcmlXTbKYHovn0 uLj2yMru7N95SaGFIliTNy0ZW5pvtFAt1BeeBHWn78cberiKSoMwfWENcl2rrukxbIdw 0yzK3E1nMRhZ+xN5xYVslpLV7QPqHvDyky1+KbQl+fq/9KKIyZHGKUZ62iHaFkr38C8b tV2iUYLJ5Cff4IjFX2z3TYcRTqeKIU9sjlcZZKr7Cf0tzR1wXfT6CKr1OjLuOLHc09Bv q3FYxemdzvwtBOfZU/a+XYfj+5ab9M8nU+QebXLOa4di5ZjxwtZArTXq0HRsvqLd9ExF 6oIQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=OYKyQePu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b5-20020a056a000cc500b0056bb101f472si8650873pfv.223.2022.10.30.22.41.55; Sun, 30 Oct 2022 22:42:07 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=OYKyQePu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229817AbiJaFlc (ORCPT + 99 others); Mon, 31 Oct 2022 01:41:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229561AbiJaFl1 (ORCPT ); Mon, 31 Oct 2022 01:41:27 -0400 Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 89343BC14 for ; Sun, 30 Oct 2022 22:41:25 -0700 (PDT) Received: by mail-pj1-x1035.google.com with SMTP id l22-20020a17090a3f1600b00212fbbcfb78so14940773pjc.3 for ; Sun, 30 Oct 2022 22:41:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=q47a/PNtxAlJKL8yCX3+cfAOXZEGpDh+HXwifBH5lKc=; b=OYKyQePuyQDoHhgCchBV4FMDx0nWgF7FRbwlJg+5H3agLccnKM3iywpXD9Ad6WlB9e 2hKoZcKk5G/boerau0bifRUyl66HD3wpSCWk/agkmOsrg0WbI+yuve3bQQODeG1tBhlr jW9CV4MIvC/C7cUB7KWXWdWNpbLLs4wlJYM+Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=q47a/PNtxAlJKL8yCX3+cfAOXZEGpDh+HXwifBH5lKc=; b=gUPEJf209kep/RKZT/gqNKHrI7nrQXYgMThJN6uYVpJYeF7B3LiT+lz0hE0RPewtU9 jKGoZVm0Il/D6Wi0km5b/jCFzwa0ExyjYEb0jhqTXlGQ4AVBbecvXkKoIe1eZCoycZcy Q4SSiu+AvktRJNBlRFn2H08k9v+4wXxZc1UBZwnpVxNMV7+GE9r8ihVvARE6BL33D527 pgz9n+zU/UP6T3asdgUt3zAjB8FnRWyRNOYGmnax7asxt0Gt+B91qVfk34l2Ck0x28t6 vvsAgNF4li3uE8UdrvOPeT6JbbHBRJDTpgYIk3xH5beWvJ37baj+KpyTZ6orDGDrXtKE Stqw== X-Gm-Message-State: ACrzQf2y0r3eyQ2xeZkghHJDEmrqm/nuHR0pkDliucmpE2f3txhisINo v178D6OrBzVoka2DspX5+lPhOw== X-Received: by 2002:a17:903:4d7:b0:178:8564:f754 with SMTP id jm23-20020a17090304d700b001788564f754mr12779632plb.60.1667194885017; Sun, 30 Oct 2022 22:41:25 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:7616:afe0:ba6c:96f4]) by smtp.gmail.com with ESMTPSA id w12-20020aa79a0c000000b0056befcd7958sm3573308pfj.84.2022.10.30.22.41.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 22:41:24 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv4 3/9] zsmalloc: move away from page order defines Date: Mon, 31 Oct 2022 14:41:02 +0900 Message-Id: <20221031054108.541190-4-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.1.273.g43a17bfeac-goog In-Reply-To: <20221031054108.541190-1-senozhatsky@chromium.org> References: <20221031054108.541190-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748180588463619581?= X-GMAIL-MSGID: =?utf-8?q?1748180588463619581?= There is no reason for us to require pages per-zspage to be a power of two. Rename macros and use plain limit numbers there instead of 2 ^ N values. This will let us to have more tunable limits. Signed-off-by: Sergey Senozhatsky --- include/linux/zsmalloc.h | 16 +++++++--------- mm/zsmalloc.c | 4 +--- 2 files changed, 8 insertions(+), 12 deletions(-) diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h index 6cd1d95b928a..b6b8654a2d45 100644 --- a/include/linux/zsmalloc.h +++ b/include/linux/zsmalloc.h @@ -33,17 +33,15 @@ enum zs_mapmode { */ }; -#define ZS_PAGE_ORDER_2 2 -#define ZS_PAGE_ORDER_4 4 - /* - * A single 'zspage' is composed of up to 2^N discontiguous 0-order (single) - * pages. ZS_MAX_PAGE_ORDER defines upper limit on N, ZS_MIN_PAGE_ORDER - * defines lower limit on N. ZS_DEFAULT_PAGE_ORDER is recommended value. + * A single 'zspage' is composed of up to N discontiguous 0-order + * (single) pages. ZS_MAX_PAGES_PER_ZSPAGE defines upper limit on N, + * ZS_MIN_PAGES_PER_ZSPAGE defines lower limit on N. + * ZS_DEFAULT_PAGES_PER_ZSPAGE is a recommended value. */ -#define ZS_MIN_PAGE_ORDER ZS_PAGE_ORDER_2 -#define ZS_MAX_PAGE_ORDER ZS_PAGE_ORDER_4 -#define ZS_DEFAULT_PAGE_ORDER ZS_PAGE_ORDER_2 +#define ZS_MIN_PAGES_PER_ZSPAGE 1 +#define ZS_MAX_PAGES_PER_ZSPAGE 16 +#define ZS_DEFAULT_PAGES_PER_ZSPAGE 4 struct zs_pool_stats { /* How many pages were migrated (freed) */ diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index a9773566f85b..5f79223e7bfe 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -74,8 +74,6 @@ */ #define ZS_ALIGN 8 -#define ZS_MAX_PAGES_PER_ZSPAGE (_AC(1, UL) << ZS_MAX_PAGE_ORDER) - #define ZS_HANDLE_SIZE (sizeof(unsigned long)) /* @@ -2215,7 +2213,7 @@ struct zs_pool *zs_create_pool(const char *name) if (!pool) return NULL; - num_pages = 1UL << ZS_DEFAULT_PAGE_ORDER; + num_pages = ZS_DEFAULT_PAGES_PER_ZSPAGE; /* min_alloc_size must be multiple of ZS_ALIGN */ pool->min_alloc_size = num_pages << PAGE_SHIFT >> OBJ_INDEX_BITS; pool->min_alloc_size = max(pool->min_alloc_size, ZS_MIN_ALLOC_SIZE); From patchwork Mon Oct 31 05:41:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13116 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp2127898wru; Sun, 30 Oct 2022 22:42:32 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5qzsn8eS9rSu0tFzMMbHNsblFi3gxOh2gqKd6YfabGXoeQvhFEO7FR8aHW8hACtsm9Gq2m X-Received: by 2002:a17:902:d486:b0:186:cf83:4ba8 with SMTP id c6-20020a170902d48600b00186cf834ba8mr12761900plg.154.1667194952033; Sun, 30 Oct 2022 22:42:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667194952; cv=none; d=google.com; s=arc-20160816; b=QKllFiQpiX8UHrIzLXpIgDJto7FbMVfbBdcxXPiuH/pfswkfF9WXPa7JMS7HzBMH2x iZnIOHcFxXl1IER+a87MSelBUm+FUB00saWlZSOS9NO2c1bMevSPiSk6C9qfUcoBVD7F 2nABrC4a5/ANxXxoS9X28axvhkw3D47b8VFsATXJHhbwd2GBBtMWWhx+VdCWTw3/qN1C 2KVyPUD/8Et3ucO1n5tghfgW2Rnvfm/NFjkuQLx3u6klKysXwvvIsemDQWAtU9FICVLX eFoaezFvVW4Jh57fsS6av/N3603MAMysiBTgaJcTmHZZhcTOTLG/uifHPZtYZTFwlUis E2hA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=xz2a6Gnxa0VzNk+aaf7V7ldsdawrXx9L2Y2yVCyV3wk=; b=ffcgT0RGQFmYkErQMf/ZpYgzU3/vdcyRtoSLN/5Xe89XIgMyg3KUGSdd+NSj15O6dO I/JAPzwxNtQjQKLHaQY2lh6hB9sekNZOn2ISXwsvKyFEG2Y2F9YzbPJRcI9DDtmgJiot JGKh0SZo6lea1aYET0f/6vId91sKBVMJbAYb3a5gC39ipaAb8vdqrmW0x84EtrER7G3a 2EaNoh7h8860CX1uQQ00z4vkRHq3YmMoEIhfnLrXdDiQbhxgCbR++i6mKF7hASzu5hfm GK2Uxs888fLNo4QPRkNLLFFBQtc7gah8kOuAv3Ws5fJiNvdL0EnBH5smlIB8Iy3epq0H EoxQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=hpb9n1tc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g10-20020a170902934a00b00186ada43088si6324010plp.517.2022.10.30.22.42.19; Sun, 30 Oct 2022 22:42:32 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=hpb9n1tc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229853AbiJaFlw (ORCPT + 99 others); Mon, 31 Oct 2022 01:41:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53818 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229842AbiJaFll (ORCPT ); Mon, 31 Oct 2022 01:41:41 -0400 Received: from mail-pl1-x62a.google.com (mail-pl1-x62a.google.com [IPv6:2607:f8b0:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5F8E4AE6F for ; Sun, 30 Oct 2022 22:41:28 -0700 (PDT) Received: by mail-pl1-x62a.google.com with SMTP id p3so9828968pld.10 for ; Sun, 30 Oct 2022 22:41:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xz2a6Gnxa0VzNk+aaf7V7ldsdawrXx9L2Y2yVCyV3wk=; b=hpb9n1tc4pH+/W4WEarHdEJTdKB8Ai2ziveapp1wFdmhGqQdaw2eIO47MVgU9X5z/x ksPR6wC6yaqkhc/7Gjw5j1+OxfdJ2fX7sdKnVGFSvnb2tH2Ewj3HO0DYDPUMqb3x5NBE XbPIXzjLvbXmstz37m4M+DFTtqlVUc/uSW7QM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xz2a6Gnxa0VzNk+aaf7V7ldsdawrXx9L2Y2yVCyV3wk=; b=ja5oo2UU877c+SUUtahOvfrx8R5J0jyrCBytNFpE4xTpM4QZo7I8mGvtOq0eDVAyY2 LcxOHuNNxwQl0WH2nmFMiS/12XlwIbKumZHYeyXlf5efiCX9WoB1PufGRDHqniBaQa6N qal9y/y4UleI8mhWQwnR+s0deZfmaNTj3vwVhsuARlnu+D+YMlCEDJUMF/qqTB1h4UdU 6JJAoHnJQ5tUDqOhWYljHZhidhXvMdKo0eD7y04OozJwFuWDKcpt+1oatpxfnlVCxku/ 5ZOcU2gPLKkldD0leN8rRWpWq7SbJbabX3iueOKtMW28rOua1J8atqonk/gyCKd3MiEu mMzg== X-Gm-Message-State: ACrzQf3mX4jcQ5uHaKsI+Wy0lj0ftldWAfvjuv5SpbpwCR8aXaNCSRUd j/EF5w3w/xBbUvHvNLygvk8K2XO7l/zTKQ== X-Received: by 2002:a17:903:1211:b0:178:9353:9e42 with SMTP id l17-20020a170903121100b0017893539e42mr12657945plh.45.1667194887696; Sun, 30 Oct 2022 22:41:27 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:7616:afe0:ba6c:96f4]) by smtp.gmail.com with ESMTPSA id w12-20020aa79a0c000000b0056befcd7958sm3573308pfj.84.2022.10.30.22.41.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 22:41:27 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv4 4/9] zsmalloc: make huge class watermark zs_pool member Date: Mon, 31 Oct 2022 14:41:03 +0900 Message-Id: <20221031054108.541190-5-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.1.273.g43a17bfeac-goog In-Reply-To: <20221031054108.541190-1-senozhatsky@chromium.org> References: <20221031054108.541190-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748180613921003311?= X-GMAIL-MSGID: =?utf-8?q?1748180613921003311?= We will permit per-pool configuration of pages per-zspage value, which changes characteristics of the classes and moves around huge class size watermark. Thus huge class size needs to be a per-pool variable. Signed-off-by: Sergey Senozhatsky --- mm/zsmalloc.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 5f79223e7bfe..d329bd673baa 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -178,7 +178,6 @@ static struct dentry *zs_stat_root; * (see: fix_fullness_group()) */ static const int fullness_threshold_frac = 4; -static size_t huge_class_size; struct size_class { spinlock_t lock; @@ -227,6 +226,7 @@ struct zs_pool { u32 num_size_classes; u32 min_alloc_size; + size_t huge_class_size; struct zs_pool_stats stats; @@ -1350,7 +1350,7 @@ EXPORT_SYMBOL_GPL(zs_unmap_object); */ size_t zs_huge_class_size(struct zs_pool *pool) { - return huge_class_size; + return pool->huge_class_size; } EXPORT_SYMBOL_GPL(zs_huge_class_size); @@ -2262,8 +2262,8 @@ struct zs_pool *zs_create_pool(const char *name) * endup in the huge class. */ if (pages_per_zspage != 1 && objs_per_zspage != 1 && - !huge_class_size) { - huge_class_size = size; + !pool->huge_class_size) { + pool->huge_class_size = size; /* * The object uses ZS_HANDLE_SIZE bytes to store the * handle. We need to subtract it, because zs_malloc() @@ -2273,7 +2273,7 @@ struct zs_pool *zs_create_pool(const char *name) * class because it grows by ZS_HANDLE_SIZE extra bytes * right before class lookup. */ - huge_class_size -= (ZS_HANDLE_SIZE - 1); + pool->huge_class_size -= (ZS_HANDLE_SIZE - 1); } /* From patchwork Mon Oct 31 05:41:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13114 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp2127837wru; Sun, 30 Oct 2022 22:42:21 -0700 (PDT) X-Google-Smtp-Source: AMsMyM76C6jjLN2UsQtnOlPncxiWeZtlqZQQ6KZCOlr1HGdMPhwZSYTLinVt5QZ/ql4+pHYl2quN X-Received: by 2002:a17:90b:1b05:b0:213:dc73:3084 with SMTP id nu5-20020a17090b1b0500b00213dc733084mr4506087pjb.13.1667194941276; Sun, 30 Oct 2022 22:42:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667194941; cv=none; d=google.com; s=arc-20160816; b=uw54DEh658osJ+jCFaIPbrukCtk0O4PLwMwLNSS+VolTJ1Iuwssu9WpYfu31wm6vSL 7wThoXGDYT54jmCKCHujky9HQx4G9iLEl3cDW1AbC1fD+gA6DALOX9qdnxOXeel1zgN2 2Z4SOz3hr6pcoVjmjTAxcbhf3IQEgSxWqfYfnH85UYBclKSNTa6UX9Cct1A4Nevu5RQb Ptd8jZ1DfAJfRJ4dBYF4uwiGro5N2VDGV9Pc6kzI8ktfne5DvixMo5rrQ5iNL8v9k7SJ f4aL3K5OXceDB8v3drILhGBzdPNlHTCyxR44DRBbCg1SghexBi6U1QLLygGxsSVdmNH1 NVKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=4A6bhxNpojaEfPLcsnRmMeXdGP9le8JW2bwOtY/TXkw=; b=lbb1RYll2FyyyDzT6OgGCs1L9YfutlIqQPI3skvOTyeZ8sGYL9dqdV6e+1rqNcHJxo WTqbrBkhnrxRUcFoI4qR/ado97KpxbjqOrrhXn6HyITJpTdbh/XEdX4+7WHH0zxhqLq+ Dk54MIzqXr+diJ+C8ZjFVKdHU4sZN5Y5qsanUzskS/UvcFy5sgCv0hTnumbC2c6Vkmbb HQtRuNt0DjuUt6oZWFw8ohJF9uDN/8Rig1pIPPHDEQxgY2xMW9bsS65xhDznPJgy24/A CSxFGWcvOjnX3Isor1jnQCSLd89QkkPe2eNlfZgiTg1Q+euE0hNLT1UJOMXS8DBfBUS0 V7oQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b="mVTml/Dy"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g14-20020a63564e000000b0043c9fcccbd9si8804824pgm.685.2022.10.30.22.42.08; Sun, 30 Oct 2022 22:42:21 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b="mVTml/Dy"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229868AbiJaFln (ORCPT + 99 others); Mon, 31 Oct 2022 01:41:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53560 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229823AbiJaFlf (ORCPT ); Mon, 31 Oct 2022 01:41:35 -0400 Received: from mail-pj1-x1036.google.com (mail-pj1-x1036.google.com [IPv6:2607:f8b0:4864:20::1036]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A6192BE2C for ; Sun, 30 Oct 2022 22:41:30 -0700 (PDT) Received: by mail-pj1-x1036.google.com with SMTP id r61-20020a17090a43c300b00212f4e9cccdso14920107pjg.5 for ; Sun, 30 Oct 2022 22:41:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4A6bhxNpojaEfPLcsnRmMeXdGP9le8JW2bwOtY/TXkw=; b=mVTml/Dy4+8p9+6rmaolR4vBzjYW7UCec/5EMJ+idSZbvLmxXWLwMeDUDy8NMfmSjx qH5MGu1oQ1A7u43SxQHdT0a+55WaYPhC5BtaEix7UtY1PUPttIA1tLy0w29dES+LR692 yWCf2+X19RD7iNSYK+TgQm6wZYQBsbJ2q97Gs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4A6bhxNpojaEfPLcsnRmMeXdGP9le8JW2bwOtY/TXkw=; b=tHSAu/5k66U4CfLIvwdEK5xl/9YG1m7tpA1qtbjoM815a2xdxl6fl3DmcFEqwhCwV1 4iAwpP0kerBBwc7xFZbplpIm0nDrt+MKIqx5Xm1pSCMzKeUEYOgdc0Z2NiI6103cmJqU 4zZgBO1xZPR+CUEjdbbgY1/tWHUEawAtjqoWhTwCm0twcN6GHSXdXqbtDWK3NvPMkWqO lAJ2z8cYTSgxSaN2MwpHSYNzbDpLvHEcLrWzJeKvlqzXppFK9MoE0tDvXgsHNHRxn7VJ p6kJ9kDch+eLkf3YsOuy5xQeA+197YSJAJqlYyVqWwXBaG9FBBlHrz2KBj3bX6K/DvV3 tumg== X-Gm-Message-State: ACrzQf0H5UZ0suseKWQddCdPsh1HOcNalY+a5BKtB9cst8O4JKAgfe7J Vpx4t+spQ9h+0bedDe9A8R6Zbw== X-Received: by 2002:a17:90b:1e43:b0:213:1efe:9815 with SMTP id pi3-20020a17090b1e4300b002131efe9815mr13311203pjb.164.1667194890154; Sun, 30 Oct 2022 22:41:30 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:7616:afe0:ba6c:96f4]) by smtp.gmail.com with ESMTPSA id w12-20020aa79a0c000000b0056befcd7958sm3573308pfj.84.2022.10.30.22.41.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 22:41:29 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv4 5/9] zram: huge size watermark cannot be global Date: Mon, 31 Oct 2022 14:41:04 +0900 Message-Id: <20221031054108.541190-6-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.1.273.g43a17bfeac-goog In-Reply-To: <20221031054108.541190-1-senozhatsky@chromium.org> References: <20221031054108.541190-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748180602637683358?= X-GMAIL-MSGID: =?utf-8?q?1748180602637683358?= ZRAM will pass pool specific limit on number of pages per-zspages which will affect pool's characteristics. Namely huge size class watermark value. Move huge_class_size to struct zram, because this value now can be unique to the pool (zram device). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 13 +++---------- drivers/block/zram/zram_drv.h | 5 +++++ 2 files changed, 8 insertions(+), 10 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index fd31beb6491a..90b0c66bbd5b 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -50,12 +50,6 @@ static const char *default_comp_algs[ZRAM_MAX_ZCOMPS] = { /* Module params (documentation at end) */ static unsigned int num_devices = 1; -/* - * Pages that compress to sizes equals or greater than this are stored - * uncompressed in memory. - */ -static size_t huge_class_size; - static const struct block_device_operations zram_devops; static void zram_free_page(struct zram *zram, size_t index); @@ -1259,8 +1253,7 @@ static bool zram_meta_alloc(struct zram *zram, u64 disksize) return false; } - if (!huge_class_size) - huge_class_size = zs_huge_class_size(zram->mem_pool); + zram->huge_class_size = zs_huge_class_size(zram->mem_pool); return true; } @@ -1488,7 +1481,7 @@ static int __zram_bvec_write(struct zram *zram, struct bio_vec *bvec, return ret; } - if (comp_len >= huge_class_size) + if (comp_len >= zram->huge_class_size) comp_len = PAGE_SIZE; /* * handle allocation has 2 paths: @@ -1665,7 +1658,7 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, * in a way that will save us memory. Mark the object so that we * don't attempt to re-compress it again (RECOMP_SKIP). */ - if (comp_len_next >= huge_class_size || + if (comp_len_next >= zram->huge_class_size || comp_len_next >= comp_len_prev || class_index_next >= class_index_prev || ret) { diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index 09b9ceb5dfa3..9d6fcfdf7aa7 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -120,6 +120,11 @@ struct zram { */ u64 disksize; /* bytes */ const char *comp_algs[ZRAM_MAX_ZCOMPS]; + /* + * Pages that compress to sizes equal or greater than this are stored + * uncompressed in memory. + */ + size_t huge_class_size; /* * zram is claimed so open request will be failed */ From patchwork Mon Oct 31 05:41:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13115 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp2127851wru; Sun, 30 Oct 2022 22:42:25 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6wXUEkwRTgDEUg9F/5PY3/WFu6vg72ex/ItJLJFzaCXsmGK6w2UMgtr9FbXUp0Ah4+eFx6 X-Received: by 2002:a05:6a00:80e:b0:56d:139f:e712 with SMTP id m14-20020a056a00080e00b0056d139fe712mr10904819pfk.36.1667194944882; Sun, 30 Oct 2022 22:42:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667194944; cv=none; d=google.com; s=arc-20160816; b=a/R4/Xfm6qwGMrZWsxeTFmebQ+qxVSIe84wJjlsRP/frdQwqaJD/fabesvudZZKZaD iugEYUszgQD7RGQefW5foQtdzOYjUH3qQCQ4UYzVR9no4qHBkxATTAHjjdJIiz0Jta3k 7l94pjXp0Te3n30l2lm+heDLUeijsi6V5IZfcxn0aXj9aMm/FeUQaonrHjQAmkPKH8LK Tz7hSf7O3WeWnqOqP+F0rWgSa8hjWGs79OsTwfITj/ObYidIqG2UKXaomcWWUc5QjnFn a6ic4cIvv/0wQnyOEhDJ82WP9POW9Rqmb/WnViZO2CmQyHB6051S1ITqN9IuMEtS4JW3 zt5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=VFR0j+YHx2uepPKlT+sIE6AyjijOgidhHUVwtVRcfQ8=; b=ZyKujRrZqRWVWVWk9v3qCjCHIFJJjl8aRGnePX5dPe4F1HopNd5vcnY8b+pGMWVxUb 5+OiEOop46Xj75V6khHcBaL/1wu8DIUlYBfu2VW74qF7+i+PW/XT7/7E97tJzPc47tBx LIIR4MezyfQXPZMNbpUDLUjHOSWSEGBzSjuQNT0P1fTx/o7qCs5MaDMFgngbzgU1Vh36 PkbdGobJom6PY9TwEgE9SrHAa8qKkknUpu2YH3nK73SI99iSED5oYqbPIArOUy+SdZTj ArXF0Dx+Sh/fO3cufbrj7P7nVZDx0LeD3OO2pwZtt7+pk8W7l6IQKv9HgZEioKMjDyBY YbKQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=OsAM7e9g; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g132-20020a636b8a000000b0045a43a568casi7927516pgc.717.2022.10.30.22.42.12; Sun, 30 Oct 2022 22:42:24 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=OsAM7e9g; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229885AbiJaFlq (ORCPT + 99 others); Mon, 31 Oct 2022 01:41:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53766 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229838AbiJaFll (ORCPT ); Mon, 31 Oct 2022 01:41:41 -0400 Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2CEA7BE3A for ; Sun, 30 Oct 2022 22:41:33 -0700 (PDT) Received: by mail-pl1-x632.google.com with SMTP id l2so9809504pld.13 for ; Sun, 30 Oct 2022 22:41:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VFR0j+YHx2uepPKlT+sIE6AyjijOgidhHUVwtVRcfQ8=; b=OsAM7e9gpky/PEpM6ussXnheiEIyAvoyhxEKv4YeN/y8428flisIoxqVDVQ/Z5NMre D5X/AKopD/9c9t4iyXFDie1MqC4kbMjkn2bPY7QhKi1M3jJRRwADtpxzFPYqbiKC4lsG L5Ihy/ujEO9g3F150Tblm3TeaA8a+A20XJX98= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VFR0j+YHx2uepPKlT+sIE6AyjijOgidhHUVwtVRcfQ8=; b=pH5l1rAcMIgs1aBTVE8gI71fhlyBHQiSa+iVDR50xkZ4cFnm2ORKBpm9v2dRRl+nkv 5oqSGv5vFpTXaCBP2dj50aQE8c3ivrKrCxuBs7Zi4AKd4iN9sYHR1+fiGYOJHpHRdpoc W54dMXy8S12kFkordIUU1rlzcA8ScrxCE9Kj1pqvQCW3FugHsdz/ipg/foqvf+U8a5J9 MEgg2+VUpLGSE7kZTdj17wgvWvtxduFo6pBrHYEVaFL6imny8JfiB0zOeCqzuBRrFD98 4sxhL95FeC+sylz8wZd6/GU8IaNHha2hwKMgmzyBmGnqKsFDAdmIUpB7GBE0WGV5M0BH pL4g== X-Gm-Message-State: ACrzQf1DwzgT1TXKEhH8PIrfVs6SyVokWOTf3zcBOirIYTvKB4viBVNz k0oaXSBQ4lZ3uCngpx4JeA2A1A== X-Received: by 2002:a17:90b:2705:b0:20a:b4fa:f624 with SMTP id px5-20020a17090b270500b0020ab4faf624mr13010028pjb.124.1667194892609; Sun, 30 Oct 2022 22:41:32 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:7616:afe0:ba6c:96f4]) by smtp.gmail.com with ESMTPSA id w12-20020aa79a0c000000b0056befcd7958sm3573308pfj.84.2022.10.30.22.41.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 22:41:32 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv4 6/9] zsmalloc: pass limit on pages per-zspage to zs_create_pool() Date: Mon, 31 Oct 2022 14:41:05 +0900 Message-Id: <20221031054108.541190-7-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.1.273.g43a17bfeac-goog In-Reply-To: <20221031054108.541190-1-senozhatsky@chromium.org> References: <20221031054108.541190-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748180606336329544?= X-GMAIL-MSGID: =?utf-8?q?1748180606336329544?= Allow zsmalloc pool owner to specify max number of pages per-zspage (during pool creation), so that different pools can have different characteristics. By default we pass ZS_DEFAULT_PAGES_PER_ZSPAGE which is 4 (matches the current order 2 zspages limit). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 3 ++- include/linux/zsmalloc.h | 2 +- mm/zsmalloc.c | 11 +++++++---- 3 files changed, 10 insertions(+), 6 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 90b0c66bbd5b..bec02f636bce 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1247,7 +1247,8 @@ static bool zram_meta_alloc(struct zram *zram, u64 disksize) if (!zram->table) return false; - zram->mem_pool = zs_create_pool(zram->disk->disk_name); + zram->mem_pool = zs_create_pool(zram->disk->disk_name, + ZS_DEFAULT_PAGES_PER_ZSPAGE); if (!zram->mem_pool) { vfree(zram->table); return false; diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h index b6b8654a2d45..28f2b9cb1c47 100644 --- a/include/linux/zsmalloc.h +++ b/include/linux/zsmalloc.h @@ -50,7 +50,7 @@ struct zs_pool_stats { struct zs_pool; -struct zs_pool *zs_create_pool(const char *name); +struct zs_pool *zs_create_pool(const char *name, unsigned long num_pages); void zs_destroy_pool(struct zs_pool *pool); unsigned long zs_malloc(struct zs_pool *pool, size_t size, gfp_t flags); diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index d329bd673baa..42987a913f45 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -366,7 +366,7 @@ static void *zs_zpool_create(const char *name, gfp_t gfp, * different contexts and its caller must provide a valid * gfp mask. */ - return zs_create_pool(name); + return zs_create_pool(name, ZS_DEFAULT_PAGES_PER_ZSPAGE); } static void zs_zpool_destroy(void *pool) @@ -2195,6 +2195,7 @@ static int zs_register_shrinker(struct zs_pool *pool) /** * zs_create_pool - Creates an allocation pool to work from. * @name: pool name to be created + * @num_pages: maximum number of pages per-zspage * * This function must be called before anything when using * the zsmalloc allocator. @@ -2202,18 +2203,20 @@ static int zs_register_shrinker(struct zs_pool *pool) * On success, a pointer to the newly created pool is returned, * otherwise NULL. */ -struct zs_pool *zs_create_pool(const char *name) +struct zs_pool *zs_create_pool(const char *name, unsigned long num_pages) { int i; struct zs_pool *pool; struct size_class *prev_class = NULL; - unsigned long num_pages; + + if (WARN_ON(num_pages < ZS_MIN_PAGES_PER_ZSPAGE || + num_pages > ZS_MAX_PAGES_PER_ZSPAGE)) + return NULL; pool = kzalloc(sizeof(*pool), GFP_KERNEL); if (!pool) return NULL; - num_pages = ZS_DEFAULT_PAGES_PER_ZSPAGE; /* min_alloc_size must be multiple of ZS_ALIGN */ pool->min_alloc_size = num_pages << PAGE_SHIFT >> OBJ_INDEX_BITS; pool->min_alloc_size = max(pool->min_alloc_size, ZS_MIN_ALLOC_SIZE); From patchwork Mon Oct 31 05:41:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13117 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp2128008wru; Sun, 30 Oct 2022 22:42:59 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4liU6fFG5girLdSpJLSQR5VdM+gzNMoZjxhW9P0RfglFDBfEADbW/DRP4XmQLSUj5yxB7e X-Received: by 2002:a17:902:c412:b0:186:a279:717f with SMTP id k18-20020a170902c41200b00186a279717fmr12684483plk.29.1667194979383; Sun, 30 Oct 2022 22:42:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667194979; cv=none; d=google.com; s=arc-20160816; b=0O/njAoV3Oy+pUyB19evhugJ4vOI1PwsJmcBXSxBrrCYbtuenCf+sfLLqbv+D97nQN J8uTa6r74iKFPHWo4dE6KefDcpu5rWrPC+wthDIfVsRkG5JiySiIBQmuBGH7Lz37qMiD S0I/P+TE1LiZMyKxWXGg4e/Y1EgY2BZOQZdVSucD4mYzbJUSkv7ylgTsQYzQgO2UFEyM /aSVUWZcGRklWpTYMaq+4twEmFMbYhLWTovp39Gt69UZFfPv+luuzbPGFYX+fCRFregC zFLxviQZm3JA5bJI5vtecjvtnCDLm76GIk5sw658Bx37UInUeyhjR/0OeYvehxjd3L15 5GQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=+Nf0+I2pfgnRZzXVFRfDlyfCUhznpUvJ315tu0OUOiI=; b=rcOIwGafzItHBDvePAt4F08dUbtNqgm4YMnd+kzC2PJPI4jlkhKNJgQGsznzY6FMec r3950W/tUk32Jdx5wMUo5gjCPzdcNnxBvPeTFxE+kq1NQ+eEIQh0qR/mIP7vtJiU52+g jD8NDvEPBmBs+SKA825qekLbuXIbxk/Af9EWYFUHOmZZAW5Ta0inuFSAHN314e11FO70 rjrqokP8ev9aR2Ph2RAvdXMHeD1zQEfLeKXO8d+0jky2nsdjDFPzJBQBHt4cpgU8NmAi xKFuWUIlqBjdv7GeV2o/rRTJDw/6AxEHFKJXFbIKLiCWHw5ObLQnHh24XNJaqbj8KKt9 lkCw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=MbpTgPIB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i16-20020a631310000000b0046eb96a9955si6518740pgl.576.2022.10.30.22.42.46; Sun, 30 Oct 2022 22:42:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=MbpTgPIB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229906AbiJaFmH (ORCPT + 99 others); Mon, 31 Oct 2022 01:42:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54170 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229895AbiJaFlw (ORCPT ); Mon, 31 Oct 2022 01:41:52 -0400 Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B16DFBF60 for ; Sun, 30 Oct 2022 22:41:35 -0700 (PDT) Received: by mail-pj1-x1034.google.com with SMTP id u8-20020a17090a5e4800b002106dcdd4a0so14961693pji.1 for ; Sun, 30 Oct 2022 22:41:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+Nf0+I2pfgnRZzXVFRfDlyfCUhznpUvJ315tu0OUOiI=; b=MbpTgPIBDUbHWkS+9YopOfHoRnV74v+JNz6BX6X//rBB/LmgPIElbp4GRlLTwjJzxc alebDGyAW4DJc/zlleAs/HRTGa9HptGICD/6ejNkUbdmk/vWUa3Ai/OhOrukv76eLsnd na8Fa3d+5bCIlVErli3M7VEvFks54n/u5Xz0I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+Nf0+I2pfgnRZzXVFRfDlyfCUhznpUvJ315tu0OUOiI=; b=Lkus3TW5xKpNSOeKIyA6vTZV3z4wN1OUee6JjV/gdRXM/CT93Q/O0n3/9WuxWz2px0 g67+yjO31HhUIiuc2tdxAjvFYAvFrDNqR3kGWyiNCtfdbHrrWVJlfoxY+pjAWfS83C8a t9WnE3yh4JYKDmg2RYc3wD8JuNXiHJxTvFmMvIhEhmXAe8XGvNgm3sihWd7ey/NamrPa kfKogAEyOnl8vrD+VAW8/6ynJiHJHVvGjcvT7Lsttk/B+HorU5ITvOikeOphOy+g7fYm YgDKBebRYdgpjgGYdkICUOKc+oDgKg1a1oDa6w6mqY6mFGeIzB54tPEeLN0DbDreoHuP 6stw== X-Gm-Message-State: ACrzQf3h47NYJRH4lXqhtri/dHj0pan/RIaQ3lQ19+Mxdxf76b4cLuJk ZpPKFlDIOR2H6x6mYDEiGGWZXg== X-Received: by 2002:a17:90b:1bca:b0:213:c9ce:dad4 with SMTP id oa10-20020a17090b1bca00b00213c9cedad4mr7600860pjb.205.1667194895001; Sun, 30 Oct 2022 22:41:35 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:7616:afe0:ba6c:96f4]) by smtp.gmail.com with ESMTPSA id w12-20020aa79a0c000000b0056befcd7958sm3573308pfj.84.2022.10.30.22.41.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 22:41:34 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv4 7/9] zram: add pages_per_pool_page device attribute Date: Mon, 31 Oct 2022 14:41:06 +0900 Message-Id: <20221031054108.541190-8-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.1.273.g43a17bfeac-goog In-Reply-To: <20221031054108.541190-1-senozhatsky@chromium.org> References: <20221031054108.541190-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748180642871378533?= X-GMAIL-MSGID: =?utf-8?q?1748180642871378533?= Add a new sysfs knob that allows user-space to set zsmalloc pages per-zspage limit value on per-device basis. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 44 ++++++++++++++++++++++++++++++++++- drivers/block/zram/zram_drv.h | 2 ++ 2 files changed, 45 insertions(+), 1 deletion(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index bec02f636bce..cf9d3474b80c 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1180,6 +1180,45 @@ static ssize_t mm_stat_show(struct device *dev, return ret; } +static ssize_t pages_per_pool_page_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + u32 val; + struct zram *zram = dev_to_zram(dev); + + down_read(&zram->init_lock); + val = zram->pages_per_pool_page; + up_read(&zram->init_lock); + + return scnprintf(buf, PAGE_SIZE, "%d\n", val); +} + +static ssize_t pages_per_pool_page_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct zram *zram = dev_to_zram(dev); + u32 val; + + if (kstrtou32(buf, 10, &val)) + return -EINVAL; + + if (val < ZS_MIN_PAGES_PER_ZSPAGE || val > ZS_MAX_PAGES_PER_ZSPAGE) + return -EINVAL; + + down_read(&zram->init_lock); + if (init_done(zram)) { + up_read(&zram->init_lock); + return -EINVAL; + } + + zram->pages_per_pool_page = val; + up_read(&zram->init_lock); + + return len; +} + #ifdef CONFIG_ZRAM_WRITEBACK #define FOUR_K(x) ((x) * (1 << (PAGE_SHIFT - 12))) static ssize_t bd_stat_show(struct device *dev, @@ -1248,7 +1287,7 @@ static bool zram_meta_alloc(struct zram *zram, u64 disksize) return false; zram->mem_pool = zs_create_pool(zram->disk->disk_name, - ZS_DEFAULT_PAGES_PER_ZSPAGE); + zram->pages_per_pool_page); if (!zram->mem_pool) { vfree(zram->table); return false; @@ -2174,6 +2213,7 @@ static DEVICE_ATTR_RW(writeback_limit_enable); static DEVICE_ATTR_RW(recomp_algorithm); static DEVICE_ATTR_WO(recompress); #endif +static DEVICE_ATTR_RW(pages_per_pool_page); static struct attribute *zram_disk_attrs[] = { &dev_attr_disksize.attr, @@ -2201,6 +2241,7 @@ static struct attribute *zram_disk_attrs[] = { &dev_attr_recomp_algorithm.attr, &dev_attr_recompress.attr, #endif + &dev_attr_pages_per_pool_page.attr, NULL, }; @@ -2238,6 +2279,7 @@ static int zram_add(void) goto out_free_idr; } + zram->pages_per_pool_page = ZS_DEFAULT_PAGES_PER_ZSPAGE; zram->disk->major = zram_major; zram->disk->first_minor = device_id; zram->disk->minors = 1; diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index 9d6fcfdf7aa7..bdfc9bf0bdd5 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -120,6 +120,8 @@ struct zram { */ u64 disksize; /* bytes */ const char *comp_algs[ZRAM_MAX_ZCOMPS]; + + u32 pages_per_pool_page; /* * Pages that compress to sizes equal or greater than this are stored * uncompressed in memory. From patchwork Mon Oct 31 05:41:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13118 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp2128057wru; Sun, 30 Oct 2022 22:43:09 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4QbNm9t7kSnvgzD0lWk3+sbTqVbGTlN3xJt2L85A7dhwrnBV9+F1uCDvkgxzhOQ5WQ4jic X-Received: by 2002:a63:1917:0:b0:43c:1471:52b7 with SMTP id z23-20020a631917000000b0043c147152b7mr10975543pgl.522.1667194989561; Sun, 30 Oct 2022 22:43:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667194989; cv=none; d=google.com; s=arc-20160816; b=le4dRpIiTByDL5kdakmaw7f3oo07nOKytpjzlzKxE3+rdd+Cs0r0nLViN2d2sN7Gld 2Pt1485h/IVWmy0ZgTehNzlTJoDl2aSGQURknOVeSJZeqauJYYIPTEnI74QuB7JP9uqc VaIvWuaGOuHy5bOCl+pYwvloSPFKMm4wxBgZrDtoqlPGnV0eor344P7H3EMFHR0atqge +cpgC7QD4Ym/Y4j38mXnjCqOESacAE3+IgoRx/EA/HLKq0mi4iTPAWBDftxbSq8C6xTL UgsBhvHta9mkvXaFZkivA4vKNzHWcPzcYKqxjEJTJx1TOV47HngUm2KRFOTvIXvLh+m5 a+ZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=qzo+UO6rKhGS8H3i9OK87Ngl18bf6HViCqWtuB03u/E=; b=rZJOAbveO4MNMHPtvwbN5JHfivVpq2HpIAYElGT/U2kykmf8LUsslENTIeyGUjE2wL GFZMDY0Qy19QCsKnMQTO4yHeRa0p5VNK2l5CiOweWH25sELgXWoLQ/95jJtYk+kVMwa/ DJkC2ig8HpyEk79n3diKkyzk2voJ7qsmOS+xTSdh7Atqx7RHr9XGgRPAIYQufqa7DFCK SM9wP/3mJGuFRui+6IzQDzaGuX6CNQO0+UJSuFfhy2ZEaWTNZAGelgxX6s84caWwfB3M 7tUIKw/m2mcO8BrqJLv8yoZldcyhVETr5PQZbe1tKZa4/ltgeIOtbZ1yLvnqABEG/ouB B/wA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=QRDjYTAa; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p5-20020a056a000b4500b0056368cb7d1asi8953915pfo.104.2022.10.30.22.42.57; Sun, 30 Oct 2022 22:43:09 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=QRDjYTAa; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229872AbiJaFmM (ORCPT + 99 others); Mon, 31 Oct 2022 01:42:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54398 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229917AbiJaFmD (ORCPT ); Mon, 31 Oct 2022 01:42:03 -0400 Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 748B4C75A for ; Sun, 30 Oct 2022 22:41:37 -0700 (PDT) Received: by mail-pl1-x636.google.com with SMTP id d24so9838499pls.4 for ; Sun, 30 Oct 2022 22:41:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qzo+UO6rKhGS8H3i9OK87Ngl18bf6HViCqWtuB03u/E=; b=QRDjYTAa+Msioavw2AlKW7FO+YjBklxX0gbCjIerT/q6sgi8l09+WqL5NncXeQx8HR PnpF5DB5BqOsiTGvYpsH7rM3Fl8J6lOQwfY44y+wCYfe8j//qW941wMUIO8QV/BUIVpj 6ckxz/ZqCZaUcEhZmAgLkZHCEIGZ391OlshRI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qzo+UO6rKhGS8H3i9OK87Ngl18bf6HViCqWtuB03u/E=; b=Egu83Qsq6Z/8CB5KtBC6Xsk3Z3ng9BIojrTegf+fexBTWzsYIEEH4sTxHcFaN4Q7Lq I1tAM3myCJg6RUKJhy++U/uAFjjwS0wEJ/zrMG+eWLPGi/siOQw9fCtr02vDDFcTtL6p LJQ0KStgTUxbyltr76nBTnAI8ZGig7YLFKsDfMhe3Wabuy9ZrxtPshTRgV2MFzhq69tE 9z0YzvepwDlL+UtyEaV5ACvHVACVnJHPyeFjYGrkpHCHs3USgEZOBhEyRN4jDCLA8a06 ueMN0bhorjwfWvSUGlkJmB8o4MZDfuAWFDMly7F45bpjS3Xs3whE+NqawRurwSHNj5SG chgA== X-Gm-Message-State: ACrzQf1NyY6FdgJNUcrN1rfzIBWz99lfpIKC8L2047v+fbTDt5Pn+U2T Odb6f68OhbsLTIFRQq3tOWU/69GCcuaQ/g== X-Received: by 2002:a17:90b:1c88:b0:203:8400:13a9 with SMTP id oo8-20020a17090b1c8800b00203840013a9mr13324117pjb.46.1667194897457; Sun, 30 Oct 2022 22:41:37 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:7616:afe0:ba6c:96f4]) by smtp.gmail.com with ESMTPSA id w12-20020aa79a0c000000b0056befcd7958sm3573308pfj.84.2022.10.30.22.41.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 22:41:37 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv4 8/9] Documentation: document zram pages_per_pool_page attribute Date: Mon, 31 Oct 2022 14:41:07 +0900 Message-Id: <20221031054108.541190-9-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.1.273.g43a17bfeac-goog In-Reply-To: <20221031054108.541190-1-senozhatsky@chromium.org> References: <20221031054108.541190-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748180653085087842?= X-GMAIL-MSGID: =?utf-8?q?1748180653085087842?= Provide a simple documentation for pages_per_pool_page ZRAM device attribute. Signed-off-by: Sergey Senozhatsky --- Documentation/admin-guide/blockdev/zram.rst | 38 ++++++++++++++++----- 1 file changed, 30 insertions(+), 8 deletions(-) diff --git a/Documentation/admin-guide/blockdev/zram.rst b/Documentation/admin-guide/blockdev/zram.rst index 010fb05a5999..4cb287520d45 100644 --- a/Documentation/admin-guide/blockdev/zram.rst +++ b/Documentation/admin-guide/blockdev/zram.rst @@ -112,7 +112,29 @@ to list all of them using, for instance, /proc/crypto or any other method. This, however, has an advantage of permitting the usage of custom crypto compression modules (implementing S/W or H/W compression). -4) Set Disksize +4) Set pages per-pool page limit: Optional +========================================== + +zsmalloc pages can consist of up to ZS_DEFAULT_PAGES_PER_ZSPAGE (single) +physical pages. The exact number is calculated for each zsmalloc size +class during zsmalloc pool creation. ZRAM provides pages_per_pool_page +device attribute that lets one adjust that limit (maximum possible value +is ZS_MAX_PAGES_PER_ZSPAGE). The default limit is considered to be good +enough, so tweak this value only when the changes in zsmalloc size classes +characteristics are beneficial for your data patterns. The limit on the +pages per zspages (currently) should be in [1,16] range; default value +is 4. + +Examples:: + + #show current zsmalloc pages per-pool page limit + cat /sys/block/zramX/pages_per_pool_page + 4 + + #set zsmalloc pages per-pool page limit + echo 8 > /sys/block/zramX/pages_per_pool_page + +5) Set Disksize =============== Set disk size by writing the value to sysfs node 'disksize'. @@ -132,7 +154,7 @@ There is little point creating a zram of greater than twice the size of memory since we expect a 2:1 compression ratio. Note that zram uses about 0.1% of the size of the disk when not in use so a huge zram is wasteful. -5) Set memory limit: Optional +6) Set memory limit: Optional ============================= Set memory limit by writing the value to sysfs node 'mem_limit'. @@ -151,7 +173,7 @@ Examples:: # To disable memory limit echo 0 > /sys/block/zram0/mem_limit -6) Activate +7) Activate =========== :: @@ -162,7 +184,7 @@ Examples:: mkfs.ext4 /dev/zram1 mount /dev/zram1 /tmp -7) Add/remove zram devices +8) Add/remove zram devices ========================== zram provides a control interface, which enables dynamic (on-demand) device @@ -182,7 +204,7 @@ execute:: echo X > /sys/class/zram-control/hot_remove -8) Stats +9) Stats ======== Per-device statistics are exported as various nodes under /sys/block/zram/ @@ -283,15 +305,15 @@ a single line of text and contains the following stats separated by whitespace: Unit: 4K bytes ============== ============================================================= -9) Deactivate -============= +10) Deactivate +============== :: swapoff /dev/zram0 umount /dev/zram1 -10) Reset +11) Reset ========= Write any positive value to 'reset' sysfs node:: From patchwork Mon Oct 31 05:41:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13119 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp2128147wru; Sun, 30 Oct 2022 22:43:28 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4QuXmgllau1yT2oYPQsVldEgg0L+RrlMJ5DD5o70ZWPRLeCIhsVHRR8hpb12OwEAWmGmpI X-Received: by 2002:a63:6a09:0:b0:43a:20d4:85fe with SMTP id f9-20020a636a09000000b0043a20d485femr11041848pgc.625.1667195007918; Sun, 30 Oct 2022 22:43:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667195007; cv=none; d=google.com; s=arc-20160816; b=jkA4Q97wK9J9k7sRoUPL29/wvunw5cmgDh5HajVNVcfl5RPVnVXxkpTmpWAyQQQLKJ 4mreL16A5/6rzAStd5WZm1bz1AxHpfx/+YqVE+vXV1rWKxzhDgAk/XGvRkW3CU4HIwGm etBHOX1AMRC2GhrKNBuwgOBen8HbLI7kgJAvBCtjL3qe4oBqghl5qd1K+0EgLpK0PH+Z Oqm+NuQr0rkT/NWDEylkFhFsX1iQMC7pYXqmslqV3L98QXQeZvQ7t6hB346lGQxUMtuT tF4Lkk6CUqnN0DojQunKCykR5gujOyChJkDdp9vRaWnyYD0/Bo67UTyK2xozexlOwQ+W B6pg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Y5bXC0243vzIlXmqJuaoPsZumfjC8+KlYBqtOKgOSLQ=; b=hBoaTEfvB+ShMKiPnfWkk3yIlGtBYPGrTPwxIH4CC8QJoYXN7IWC/xbC+CuHNNOXoV c+ZUec2IIVgI7ia6NdNKZgLkzO0C9pdoz7Ks1Q8MjLtEIzgjM2P9CxZamwTgEAjz50fz i0opYi/QX1LadSH9zmgxNWmGUNvNpy/x9I47f8KeEN5AlSrN/7DbVgsmA/+jyf389dD6 GzHmk3fr1cnU+nHaBg8FYwLg3Z9N9QByAJBxKGtd5XSWFdJ9m2/Jsx2k0deXixLVwIq3 T2JDueT1H26aEDMMIM3isyPx065PGgdETkpiMby1LPsaCQoumr9uES+c2oG6UsKKKCuk LuxA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=lFlf2oVx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h30-20020a63531e000000b0045f74df51e4si8264686pgb.803.2022.10.30.22.43.16; Sun, 30 Oct 2022 22:43:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=lFlf2oVx; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229930AbiJaFmT (ORCPT + 99 others); Mon, 31 Oct 2022 01:42:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54430 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229929AbiJaFmF (ORCPT ); Mon, 31 Oct 2022 01:42:05 -0400 Received: from mail-pj1-x1036.google.com (mail-pj1-x1036.google.com [IPv6:2607:f8b0:4864:20::1036]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0644CC76B for ; Sun, 30 Oct 2022 22:41:40 -0700 (PDT) Received: by mail-pj1-x1036.google.com with SMTP id r61-20020a17090a43c300b00212f4e9cccdso14920316pjg.5 for ; Sun, 30 Oct 2022 22:41:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Y5bXC0243vzIlXmqJuaoPsZumfjC8+KlYBqtOKgOSLQ=; b=lFlf2oVxRN2VsF9XNYteCD5RC9CGXPs9NIzyXxdrHmGL3x5ZbjCN4GvZDcPFRlnVBR daPvQtBhbBGUiA1cThKv+Xl9u3TMaERvo6NWER2YakMLm4rZdmtKCnSpTsLeGXVGwzvT mABpkqu6zE8My3T25QD1KwXgG5k6ux50h9GNw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Y5bXC0243vzIlXmqJuaoPsZumfjC8+KlYBqtOKgOSLQ=; b=c2XqMZaaf8cbucyyjxr5DQmlXLu67GkHv82GVQ0Z5C8/xmJlHXAqx8cj5dVhP7gCXk MEVYsw28wJILPa7prIrRIq0t+2cRdoggnwx1o4mNkReBSotSsCPUJjxzo+e/xoSl4amY m5Ga6Ej4x+TVNCmmoa+YjCbYmL8fAf8YGcC1yMtVs0UlqAxoyWL/iigcaHAPyk0LvQkB k3jGa1dlpUZ/J/llLuZpgNXqvGQdiCQENLDqsFTiehu2z+4WsAgWZBuR7cNmgZMeBawC AYirq/fmq5LwC77pjHy/4NyeFmZBED2KtXMjh3h7vE5BsW+JpXozWSQBpnFXi0iDkz3C cSLw== X-Gm-Message-State: ACrzQf25AU7bUj3ytYRP6OX0rgwd/dH7vqFU7nqtxYvu9grjo/yF0N98 YazaZLAx9vstzuuc+yKL1uym2w== X-Received: by 2002:a17:902:e402:b0:186:6f1d:b463 with SMTP id m2-20020a170902e40200b001866f1db463mr12461840ple.23.1667194900136; Sun, 30 Oct 2022 22:41:40 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:7616:afe0:ba6c:96f4]) by smtp.gmail.com with ESMTPSA id w12-20020aa79a0c000000b0056befcd7958sm3573308pfj.84.2022.10.30.22.41.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 30 Oct 2022 22:41:39 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: Nitin Gupta , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Sergey Senozhatsky Subject: [PATCHv4 9/9] zsmalloc: break out of loop when found perfect zspage order Date: Mon, 31 Oct 2022 14:41:08 +0900 Message-Id: <20221031054108.541190-10-senozhatsky@chromium.org> X-Mailer: git-send-email 2.38.1.273.g43a17bfeac-goog In-Reply-To: <20221031054108.541190-1-senozhatsky@chromium.org> References: <20221031054108.541190-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Spam-Status: No, score=-3.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748180672404887629?= X-GMAIL-MSGID: =?utf-8?q?1748180672404887629?= If we found zspage configuration that gives us perfect 100% used percentage (zero wasted space) then there is no point it trying any other configuration Signed-off-by: Sergey Senozhatsky --- mm/zsmalloc.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 42987a913f45..a40c548520d3 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -787,6 +787,9 @@ static int get_pages_per_zspage(u32 class_size, u32 num_pages) max_usedpc = usedpc; max_usedpc_order = i; } + + if (usedpc == 100) + break; } return max_usedpc_order;