From patchwork Mon Jun 5 08:54:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Domenico Cerasuolo X-Patchwork-Id: 103186 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2541626vqr; Mon, 5 Jun 2023 01:57:48 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6dh/UwuTsy/e27cYwJebJUnjYrez2OYQRp6hFdFqXWg6f9m5mlFRZq1FSOOuQ05VWDjsXk X-Received: by 2002:a05:622a:d5:b0:3f6:b1a4:5914 with SMTP id p21-20020a05622a00d500b003f6b1a45914mr5963163qtw.37.1685955467838; Mon, 05 Jun 2023 01:57:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685955467; cv=none; d=google.com; s=arc-20160816; b=JBXNaQTm3oB+Rv7Q+Ha+WmJUOi0wx2TSeeokTsXf0DYxgSFZiHbwWSVzhoDWSB9no6 YoDKADEL9Ylu8zRg3Uh7pIpwJR5Ev8/nUbUHAUIOlZiPNSSwgKW0JZVSToWxrYQi/8f4 9XqLgRs15Any9HX2Z4ktvGKz8jhgF4WoNvJclXaLn8hB2SQFlJTctnRAxLNfvyL7Shzy WmM7a2h58cKsw/FJN2vKMbCks5IqoA4BY+CcEfogFTzW6Fi+0N6jqsRMO8m5XvvrKUbX LsnePh5OQ4KC87qwzDOZaoTqfQN7MiFl1RPawt/R/W9HXm7/L+SbMQRHNeKfmtu4gHwG aDZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=h2DLNLAJGk9U4ajDcY71N5DEib8TdCX45Wj1IDbynZg=; b=JI3lZgDjRBcYdzBe6Zou2sLbIdxGD7B4n1uMJOVSQuYtsEBywnbynU3eFRp5Py/EHH Mt3TgoDWOOt79h8/flZrHOig3CBmt4srAi+r7nnSAHN5aN9Wkc1F2kfc2/Hq08C4AAJE zyZJIsc6lxm7pc+9b2VkzE4nsJzIFXdnU/dL/HKXUIz65jhfKvp0PlIDQm+X0w3xjGfP reuNyO9OQ1C7nLf453M3zneTZFkU8OOn7GcIv6MNbM0stW4CWGQF5MmvACt8G+8vl2eH MdzjZB41Cx5VJndHYcEjv4ZcaVPwoP/DCp8kyh86T5oOyghEkZwepG0ulEHqw2gjMXle L4TQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=eQiQ5afD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id v186-20020a6389c3000000b0053427b6841dsi5285735pgd.337.2023.06.05.01.57.35; Mon, 05 Jun 2023 01:57:47 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=eQiQ5afD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230348AbjFEIyj (ORCPT + 99 others); Mon, 5 Jun 2023 04:54:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39496 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230307AbjFEIyb (ORCPT ); Mon, 5 Jun 2023 04:54:31 -0400 Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com [IPv6:2a00:1450:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1DBC1FD for ; Mon, 5 Jun 2023 01:54:27 -0700 (PDT) Received: by mail-ej1-x62c.google.com with SMTP id a640c23a62f3a-97460240863so529386966b.0 for ; Mon, 05 Jun 2023 01:54:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685955265; x=1688547265; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=h2DLNLAJGk9U4ajDcY71N5DEib8TdCX45Wj1IDbynZg=; b=eQiQ5afD6YPcElylYEflOGi5aW/xEBHc41Rb9GZi6rSuBZaFGqTR32/nLMY5rz2Mqo ki14V4ucZgEzIqqzug/2Y08sQN7XzRMIxFSltWnLeYxmyJUiDLv44qd+4lT0hHzi+rV1 /W9vhFNcGRw9Y+g1qpX0l67xnC/H/+V5sqdwqGel1+VBuLZoce02j6lpkQfGC8NxMVLs jUoj/posgjUbCAXYceY84Ja3qU2Q34LMlvLE3Hehx4B13nx5PcdFKrIMWNW/l0FxUMpf E/V9e6IEChlFiKrvtMPf/MiU7Ly6e6YBTPy92LsKmUWl+N/LjPc0inFDcR3Gtfvjj5RP JBRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685955265; x=1688547265; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=h2DLNLAJGk9U4ajDcY71N5DEib8TdCX45Wj1IDbynZg=; b=kvOWNl5GFNbx7jm1qcKFdqBTaM1Aftrc3N21Nsf+c37FlLLMySQqnJE9LFhrnUR23K RQJbkhBv6M1BYlsn+V6JsgYiwkRfV49R7jyfIBijqcbyyJ321RDGUL/WhRwJ/atBPGb8 7fETydhZidXi4xVi0bYqLXybdZyucnDPb5zHlIIZ8u7Wl89XfoIjgswDi3vqbzToztZ0 BMLCXHJTNr5gsB2GxCzSj0wWOpvodLzUfSdBwgyYKFDfLdFQl0yQjqijfRgr3UID3p/o fIxX75UfnDrmX33k2BmnkY2z4BS3dfqX1N2WsVbIXk+QO0O4sZziU9yhY1qq7OQStX7d rc9A== X-Gm-Message-State: AC+VfDzrWJ51OAZDFcK1ddrHFL3w7XD3TtcipDKvSV1XN+Nsuc3fExXu AwqyQYsYtNXi8AuBgt1FOBo= X-Received: by 2002:a17:907:7e8e:b0:977:4b19:97a with SMTP id qb14-20020a1709077e8e00b009774b19097amr6311772ejc.73.1685955265373; Mon, 05 Jun 2023 01:54:25 -0700 (PDT) Received: from lelloman-5950.homenet.telecomitalia.it (host-82-53-8-153.retail.telecomitalia.it. [82.53.8.153]) by smtp.gmail.com with ESMTPSA id lx5-20020a170906af0500b00965c6c63ea3sm4048915ejb.35.2023.06.05.01.54.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jun 2023 01:54:25 -0700 (PDT) From: Domenico Cerasuolo To: vitaly.wool@konsulko.com, minchan@kernel.org, senozhatsky@chromium.org, yosryahmed@google.com, linux-mm@kvack.org Cc: ddstreet@ieee.org, sjenning@redhat.com, nphamcs@gmail.com, hannes@cmpxchg.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, kernel-team@meta.com, Domenico Cerasuolo Subject: [RFC PATCH 1/7] mm: zswap: add pool shrinking mechanism Date: Mon, 5 Jun 2023 10:54:13 +0200 Message-Id: <20230605085419.44383-2-cerasuolodomenico@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230605085419.44383-1-cerasuolodomenico@gmail.com> References: <20230605085419.44383-1-cerasuolodomenico@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767852440592103629?= X-GMAIL-MSGID: =?utf-8?q?1767852440592103629?= Each zpool driver (zbud, z3fold and zsmalloc) implements its own shrink function, which is called from zpool_shrink. However, with this commit, a unified shrink function is added to zswap. The ultimate goal is to eliminate the need for zpool_shrink once all zpool implementations have dropped their shrink code. To ensure the functionality of each commit, this change focuses solely on adding the mechanism itself. No modifications are made to the backends, meaning that functionally, there are no immediate changes. The zswap mechanism will only come into effect once the backends have removed their shrink code. The subsequent commits will address the modifications needed in the backends. Signed-off-by: Domenico Cerasuolo --- mm/zswap.c | 83 ++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 78 insertions(+), 5 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index bcb82e09eb64..80d7780bf066 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -159,6 +159,8 @@ struct zswap_pool { struct work_struct shrink_work; struct hlist_node node; char tfm_name[CRYPTO_MAX_ALG_NAME]; + struct list_head lru; + spinlock_t lock; }; /* @@ -176,10 +178,12 @@ struct zswap_pool { * be held while changing the refcount. Since the lock must * be held, there is no reason to also make refcount atomic. * length - the length in bytes of the compressed page data. Needed during - * decompression. For a same value filled page length is 0. + * decompression. For a same value filled page length is 0, and both + * pool and lru are invalid and must be ignored. * pool - the zswap_pool the entry's data is in * handle - zpool allocation handle that stores the compressed page data * value - value of the same-value filled pages which have same content + * lru - handle to the pool's lru used to evict pages. */ struct zswap_entry { struct rb_node rbnode; @@ -192,6 +196,7 @@ struct zswap_entry { unsigned long value; }; struct obj_cgroup *objcg; + struct list_head lru; }; struct zswap_header { @@ -364,6 +369,9 @@ static void zswap_free_entry(struct zswap_entry *entry) if (!entry->length) atomic_dec(&zswap_same_filled_pages); else { + spin_lock(&entry->pool->lock); + list_del_init(&entry->lru); + spin_unlock(&entry->pool->lock); zpool_free(entry->pool->zpool, entry->handle); zswap_pool_put(entry->pool); } @@ -584,14 +592,65 @@ static struct zswap_pool *zswap_pool_find_get(char *type, char *compressor) return NULL; } +static int zswap_shrink(struct zswap_pool *pool) +{ + struct zswap_entry *lru_entry, *tree_entry = NULL; + struct zswap_header *zhdr; + struct zswap_tree *tree; + swp_entry_t swpentry; + int ret; + + /* get a reclaimable entry from LRU */ + spin_lock(&pool->lock); + if (list_empty(&pool->lru)) { + spin_unlock(&pool->lock); + return -EINVAL; + } + lru_entry = list_last_entry(&pool->lru, struct zswap_entry, lru); + list_del_init(&lru_entry->lru); + zhdr = zpool_map_handle(pool->zpool, lru_entry->handle, ZPOOL_MM_RO); + tree = zswap_trees[swp_type(zhdr->swpentry)]; + zpool_unmap_handle(pool->zpool, lru_entry->handle); + swpentry = zhdr->swpentry; + spin_unlock(&pool->lock); + + /* hold a reference from tree so it won't be freed during writeback */ + spin_lock(&tree->lock); + tree_entry = zswap_entry_find_get(&tree->rbroot, swp_offset(swpentry)); + if (tree_entry != lru_entry) { + if (tree_entry) + zswap_entry_put(tree, tree_entry); + spin_unlock(&tree->lock); + return -EAGAIN; + } + spin_unlock(&tree->lock); + + ret = zswap_writeback_entry(pool->zpool, lru_entry->handle); + + spin_lock(&tree->lock); + if (ret) { + spin_lock(&pool->lock); + list_move(&lru_entry->lru, &pool->lru); + spin_unlock(&pool->lock); + } + zswap_entry_put(tree, tree_entry); + spin_unlock(&tree->lock); + + return ret ? -EAGAIN : 0; +} + static void shrink_worker(struct work_struct *w) { struct zswap_pool *pool = container_of(w, typeof(*pool), shrink_work); int ret, failures = 0; + /* zpool_evictable will be removed once all 3 backends have migrated*/ do { - ret = zpool_shrink(pool->zpool, 1, NULL); + if (zpool_evictable(pool->zpool)) + ret = zpool_shrink(pool->zpool, 1, NULL); + else + ret = zswap_shrink(pool); if (ret) { zswap_reject_reclaim_fail++; if (ret != -EAGAIN) @@ -655,6 +714,8 @@ static struct zswap_pool *zswap_pool_create(char *type, char *compressor) */ kref_init(&pool->kref); INIT_LIST_HEAD(&pool->list); + INIT_LIST_HEAD(&pool->lru); + spin_lock_init(&pool->lock); INIT_WORK(&pool->shrink_work, shrink_worker); zswap_pool_debug("created", pool); @@ -1270,7 +1331,7 @@ static int zswap_frontswap_store(unsigned type, pgoff_t offset, } /* store */ - hlen = zpool_evictable(entry->pool->zpool) ? sizeof(zhdr) : 0; + hlen = sizeof(zhdr); gfp = __GFP_NORETRY | __GFP_NOWARN | __GFP_KSWAPD_RECLAIM; if (zpool_malloc_support_movable(entry->pool->zpool)) gfp |= __GFP_HIGHMEM | __GFP_MOVABLE; @@ -1313,6 +1374,13 @@ static int zswap_frontswap_store(unsigned type, pgoff_t offset, zswap_entry_put(tree, dupentry); } } while (ret == -EEXIST); + INIT_LIST_HEAD(&entry->lru); + /* zpool_evictable will be removed once all 3 backends have migrated*/ + if (entry->length && !zpool_evictable(entry->pool->zpool)) { + spin_lock(&entry->pool->lock); + list_add(&entry->lru, &entry->pool->lru); + spin_unlock(&entry->pool->lock); + } spin_unlock(&tree->lock); /* update stats */ @@ -1384,8 +1452,7 @@ static int zswap_frontswap_load(unsigned type, pgoff_t offset, /* decompress */ dlen = PAGE_SIZE; src = zpool_map_handle(entry->pool->zpool, entry->handle, ZPOOL_MM_RO); - if (zpool_evictable(entry->pool->zpool)) - src += sizeof(struct zswap_header); + src += sizeof(struct zswap_header); if (!zpool_can_sleep_mapped(entry->pool->zpool)) { memcpy(tmp, src, entry->length); @@ -1415,6 +1482,12 @@ static int zswap_frontswap_load(unsigned type, pgoff_t offset, freeentry: spin_lock(&tree->lock); zswap_entry_put(tree, entry); + /* zpool_evictable will be removed once all 3 backends have migrated*/ + if (entry->length && !zpool_evictable(entry->pool->zpool)) { + spin_lock(&entry->pool->lock); + list_move(&entry->lru, &entry->pool->lru); + spin_unlock(&entry->pool->lock); + } spin_unlock(&tree->lock); return ret; From patchwork Mon Jun 5 08:54:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Domenico Cerasuolo X-Patchwork-Id: 103191 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2550058vqr; Mon, 5 Jun 2023 02:16:55 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5R46EIaRoXawIFQ1Vdgxj0ATTmjQufRjtHWFDSYIWdKh17Jay6TzMPNDQzlIe3Zi4F50uR X-Received: by 2002:aa7:88c3:0:b0:64c:aa98:a69f with SMTP id k3-20020aa788c3000000b0064caa98a69fmr16081171pff.1.1685956615473; Mon, 05 Jun 2023 02:16:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685956615; cv=none; d=google.com; s=arc-20160816; b=MI1n6+v/mK8xRIqBq/+R/Nbb4uqDksWsh+u6HeWz4FWYc5YEb+nxolnzgDmXbYdEIU feOke+SkcjjOB1lrCk9DZFEjvZPwqriO9e2iGmF63JjtJsBwK/KzUhidswK9fp598zTL rmeiFClKma62CVR/IkquuoSz87m1W27MBJE+KtCckZ/LXt2wRNk6/sY+x2hF7iRoJ70w i2s8lNSzlGnheul8kvim4t/umzcAvbDZ+7KjgRWNpDYX9+qMfHg05wD7XBP7OYM26Atc pj9R0eQlY1rXzwEKjnwQf1jljIxuFa7TwcdCcXgpDhBByjum2dh6V2/TL5UwXU8KLczs KZAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=TN78WiIx7cKXEIt/YKhQKp1rR04fEU3vjXSMTSFKJB8=; b=Be0Op+8zGe4nNZtuXVSUNhQ96RuBhmX6b2LROb+MIC+LdW79RI1eAVyEbaf/sLfMPZ 7yPx5DREShNeJAnSNX5K4SSQ5/XZVPVgrakTZ5ryCM+Qr10E787FBTbOCfhEeFRRsykn hZ60fb1vIajJ9NzMp7w/ou4GsEcRdy0JumrWmKBrI94LTEW/Jrr8Rk4F8Ld2YF/m5/FN HymZqoUmhRUpYXRlzVT58q9wgwe+pZaQyJWjo7/DNBlb+qnsvUwQPnnLPhdq1973FVMA vVBDUQ2GXzTTnQktURQ6Zizod0DJu9MWga0yxvvIvKOYfvjEVvjd7ITk8nGUp0VYBTjf zNcA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=UWV12U6c; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k81-20020a628454000000b0065dfb27ee3fsi261175pfd.246.2023.06.05.02.16.33; Mon, 05 Jun 2023 02:16:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=UWV12U6c; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230374AbjFEIym (ORCPT + 99 others); Mon, 5 Jun 2023 04:54:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39490 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229806AbjFEIyd (ORCPT ); Mon, 5 Jun 2023 04:54:33 -0400 Received: from mail-lj1-x22f.google.com (mail-lj1-x22f.google.com [IPv6:2a00:1450:4864:20::22f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B358FE9 for ; Mon, 5 Jun 2023 01:54:28 -0700 (PDT) Received: by mail-lj1-x22f.google.com with SMTP id 38308e7fff4ca-2b1b3836392so33292111fa.0 for ; Mon, 05 Jun 2023 01:54:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685955267; x=1688547267; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TN78WiIx7cKXEIt/YKhQKp1rR04fEU3vjXSMTSFKJB8=; b=UWV12U6c8huHFRDsSzfwHOrNWPM6lbwWeq0HFxKoyUB/+azTg5F3uh59/s64hax2vX NuiEstX02JUkFIc5aLWsrMnbkBJOZ5aIMVYRWMmOHb6XOPjz9WiAH0psHiQe7rkDapbn wEazo0J8Jb8aSbuPHmljoF9TZ3MtZ+e0lof7t5ujmz+el2GMiZhZuEQL2waB1bufhCVq BUpXy2rsuS66MEgOxXfiet5GJdfP/+PznOQAEySCDTd7pZlVCTx9FmqQdkw15Pc9h9ud zVbTg1eYYm7IUa1dERUQ+8Zsiv6/RohVbtOx4urbdqT/fo84rEUHIEEHXwxxA11zwk5u EAnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685955267; x=1688547267; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TN78WiIx7cKXEIt/YKhQKp1rR04fEU3vjXSMTSFKJB8=; b=Ux6iUYv/MXHsxeLMCTUexP9vlH5r1iABwTCbveEEuwxyHXlk7mPPIR6tC2TfeLmm6U Cn+PSpNl9HLknHYoc2Kks9jFPIYQB6Lw1A9CcV5SrRYoXiuylTnPrUsIyIPV5x63rj3k 4wgj0Zu3cVmmRDbp0Fl0GxqIcRl+998wu08D74BQYxWDmITmx4uPfnmHCKcjD6k6t30h 5cXbBBqFODYf+F7JEW9pAtuYapZRGE67UZi4St2MXmSrCSQSIkE+kEXlJfcD3uhd5hU6 GQq3wcyFmN86tHk8s1O0oA1iBS4yZNwbqDDxT0v5pI3/IUUur8DF+qKVN5Qf4UnDWgzd WAwQ== X-Gm-Message-State: AC+VfDy9fdPwlev62C1f7I2Iqh8DcIcSBF10Q5BE342rv9Xy5t3TLyjL CJ7RsFzQ+9+1pcSgNAm6/rk= X-Received: by 2002:a2e:9a8b:0:b0:2b1:b814:c112 with SMTP id p11-20020a2e9a8b000000b002b1b814c112mr3128462lji.49.1685955266690; Mon, 05 Jun 2023 01:54:26 -0700 (PDT) Received: from lelloman-5950.homenet.telecomitalia.it (host-82-53-8-153.retail.telecomitalia.it. [82.53.8.153]) by smtp.gmail.com with ESMTPSA id lx5-20020a170906af0500b00965c6c63ea3sm4048915ejb.35.2023.06.05.01.54.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jun 2023 01:54:26 -0700 (PDT) From: Domenico Cerasuolo To: vitaly.wool@konsulko.com, minchan@kernel.org, senozhatsky@chromium.org, yosryahmed@google.com, linux-mm@kvack.org Cc: ddstreet@ieee.org, sjenning@redhat.com, nphamcs@gmail.com, hannes@cmpxchg.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, kernel-team@meta.com, Domenico Cerasuolo Subject: [RFC PATCH 2/7] mm: zswap: remove page reclaim logic from zbud Date: Mon, 5 Jun 2023 10:54:14 +0200 Message-Id: <20230605085419.44383-3-cerasuolodomenico@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230605085419.44383-1-cerasuolodomenico@gmail.com> References: <20230605085419.44383-1-cerasuolodomenico@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767853643875039229?= X-GMAIL-MSGID: =?utf-8?q?1767853643875039229?= With the recent enhancement to zswap enabling direct page writeback, the need for the shrink code in zbud has become obsolete. As a result, this commit removes the page reclaim logic from zbud entirely. Signed-off-by: Domenico Cerasuolo --- mm/zbud.c | 164 +----------------------------------------------------- 1 file changed, 2 insertions(+), 162 deletions(-) diff --git a/mm/zbud.c b/mm/zbud.c index 3acd26193920..19bc662ef5e9 100644 --- a/mm/zbud.c +++ b/mm/zbud.c @@ -83,11 +83,7 @@ struct zbud_pool; * its free region. * @buddied: list tracking the zbud pages that contain two buddies; * these zbud pages are full - * @lru: list tracking the zbud pages in LRU order by most recently - * added buddy. * @pages_nr: number of zbud pages in the pool. - * @zpool: zpool driver - * @zpool_ops: zpool operations structure with an evict callback * * This structure is allocated at pool creation time and maintains metadata * pertaining to a particular zbud pool. @@ -102,26 +98,20 @@ struct zbud_pool { struct list_head buddied; struct list_head unbuddied[NCHUNKS]; }; - struct list_head lru; u64 pages_nr; - struct zpool *zpool; - const struct zpool_ops *zpool_ops; }; /* * struct zbud_header - zbud page metadata occupying the first chunk of each * zbud page. * @buddy: links the zbud page into the unbuddied/buddied lists in the pool - * @lru: links the zbud page into the lru list in the pool * @first_chunks: the size of the first buddy in chunks, 0 if free * @last_chunks: the size of the last buddy in chunks, 0 if free */ struct zbud_header { struct list_head buddy; - struct list_head lru; unsigned int first_chunks; unsigned int last_chunks; - bool under_reclaim; }; /***************** @@ -149,8 +139,6 @@ static struct zbud_header *init_zbud_page(struct page *page) zhdr->first_chunks = 0; zhdr->last_chunks = 0; INIT_LIST_HEAD(&zhdr->buddy); - INIT_LIST_HEAD(&zhdr->lru); - zhdr->under_reclaim = false; return zhdr; } @@ -221,7 +209,6 @@ static struct zbud_pool *zbud_create_pool(gfp_t gfp) for_each_unbuddied_list(i, 0) INIT_LIST_HEAD(&pool->unbuddied[i]); INIT_LIST_HEAD(&pool->buddied); - INIT_LIST_HEAD(&pool->lru); pool->pages_nr = 0; return pool; } @@ -310,11 +297,6 @@ static int zbud_alloc(struct zbud_pool *pool, size_t size, gfp_t gfp, list_add(&zhdr->buddy, &pool->buddied); } - /* Add/move zbud page to beginning of LRU */ - if (!list_empty(&zhdr->lru)) - list_del(&zhdr->lru); - list_add(&zhdr->lru, &pool->lru); - *handle = encode_handle(zhdr, bud); spin_unlock(&pool->lock); @@ -325,11 +307,6 @@ static int zbud_alloc(struct zbud_pool *pool, size_t size, gfp_t gfp, * zbud_free() - frees the allocation associated with the given handle * @pool: pool in which the allocation resided * @handle: handle associated with the allocation returned by zbud_alloc() - * - * In the case that the zbud page in which the allocation resides is under - * reclaim, as indicated by the PG_reclaim flag being set, this function - * only sets the first|last_chunks to 0. The page is actually freed - * once both buddies are evicted (see zbud_reclaim_page() below). */ static void zbud_free(struct zbud_pool *pool, unsigned long handle) { @@ -345,18 +322,11 @@ static void zbud_free(struct zbud_pool *pool, unsigned long handle) else zhdr->first_chunks = 0; - if (zhdr->under_reclaim) { - /* zbud page is under reclaim, reclaim will free */ - spin_unlock(&pool->lock); - return; - } - /* Remove from existing buddy list */ list_del(&zhdr->buddy); if (zhdr->first_chunks == 0 && zhdr->last_chunks == 0) { /* zbud page is empty, free */ - list_del(&zhdr->lru); free_zbud_page(zhdr); pool->pages_nr--; } else { @@ -368,110 +338,6 @@ static void zbud_free(struct zbud_pool *pool, unsigned long handle) spin_unlock(&pool->lock); } -/** - * zbud_reclaim_page() - evicts allocations from a pool page and frees it - * @pool: pool from which a page will attempt to be evicted - * @retries: number of pages on the LRU list for which eviction will - * be attempted before failing - * - * zbud reclaim is different from normal system reclaim in that the reclaim is - * done from the bottom, up. This is because only the bottom layer, zbud, has - * information on how the allocations are organized within each zbud page. This - * has the potential to create interesting locking situations between zbud and - * the user, however. - * - * To avoid these, this is how zbud_reclaim_page() should be called: - * - * The user detects a page should be reclaimed and calls zbud_reclaim_page(). - * zbud_reclaim_page() will remove a zbud page from the pool LRU list and call - * the user-defined eviction handler with the pool and handle as arguments. - * - * If the handle can not be evicted, the eviction handler should return - * non-zero. zbud_reclaim_page() will add the zbud page back to the - * appropriate list and try the next zbud page on the LRU up to - * a user defined number of retries. - * - * If the handle is successfully evicted, the eviction handler should - * return 0 _and_ should have called zbud_free() on the handle. zbud_free() - * contains logic to delay freeing the page if the page is under reclaim, - * as indicated by the setting of the PG_reclaim flag on the underlying page. - * - * If all buddies in the zbud page are successfully evicted, then the - * zbud page can be freed. - * - * Returns: 0 if page is successfully freed, otherwise -EINVAL if there are - * no pages to evict or an eviction handler is not registered, -EAGAIN if - * the retry limit was hit. - */ -static int zbud_reclaim_page(struct zbud_pool *pool, unsigned int retries) -{ - int i, ret, freechunks; - struct zbud_header *zhdr; - unsigned long first_handle = 0, last_handle = 0; - - spin_lock(&pool->lock); - if (list_empty(&pool->lru)) { - spin_unlock(&pool->lock); - return -EINVAL; - } - for (i = 0; i < retries; i++) { - zhdr = list_last_entry(&pool->lru, struct zbud_header, lru); - list_del(&zhdr->lru); - list_del(&zhdr->buddy); - /* Protect zbud page against free */ - zhdr->under_reclaim = true; - /* - * We need encode the handles before unlocking, since we can - * race with free that will set (first|last)_chunks to 0 - */ - first_handle = 0; - last_handle = 0; - if (zhdr->first_chunks) - first_handle = encode_handle(zhdr, FIRST); - if (zhdr->last_chunks) - last_handle = encode_handle(zhdr, LAST); - spin_unlock(&pool->lock); - - /* Issue the eviction callback(s) */ - if (first_handle) { - ret = pool->zpool_ops->evict(pool->zpool, first_handle); - if (ret) - goto next; - } - if (last_handle) { - ret = pool->zpool_ops->evict(pool->zpool, last_handle); - if (ret) - goto next; - } -next: - spin_lock(&pool->lock); - zhdr->under_reclaim = false; - if (zhdr->first_chunks == 0 && zhdr->last_chunks == 0) { - /* - * Both buddies are now free, free the zbud page and - * return success. - */ - free_zbud_page(zhdr); - pool->pages_nr--; - spin_unlock(&pool->lock); - return 0; - } else if (zhdr->first_chunks == 0 || - zhdr->last_chunks == 0) { - /* add to unbuddied list */ - freechunks = num_free_chunks(zhdr); - list_add(&zhdr->buddy, &pool->unbuddied[freechunks]); - } else { - /* add to buddied list */ - list_add(&zhdr->buddy, &pool->buddied); - } - - /* add to beginning of LRU */ - list_add(&zhdr->lru, &pool->lru); - } - spin_unlock(&pool->lock); - return -EAGAIN; -} - /** * zbud_map() - maps the allocation associated with the given handle * @pool: pool in which the allocation resides @@ -518,14 +384,7 @@ static void *zbud_zpool_create(const char *name, gfp_t gfp, const struct zpool_ops *zpool_ops, struct zpool *zpool) { - struct zbud_pool *pool; - - pool = zbud_create_pool(gfp); - if (pool) { - pool->zpool = zpool; - pool->zpool_ops = zpool_ops; - } - return pool; + return zbud_create_pool(gfp); } static void zbud_zpool_destroy(void *pool) @@ -543,25 +402,6 @@ static void zbud_zpool_free(void *pool, unsigned long handle) zbud_free(pool, handle); } -static int zbud_zpool_shrink(void *pool, unsigned int pages, - unsigned int *reclaimed) -{ - unsigned int total = 0; - int ret = -EINVAL; - - while (total < pages) { - ret = zbud_reclaim_page(pool, 8); - if (ret < 0) - break; - total++; - } - - if (reclaimed) - *reclaimed = total; - - return ret; -} - static void *zbud_zpool_map(void *pool, unsigned long handle, enum zpool_mapmode mm) { @@ -585,7 +425,7 @@ static struct zpool_driver zbud_zpool_driver = { .destroy = zbud_zpool_destroy, .malloc = zbud_zpool_malloc, .free = zbud_zpool_free, - .shrink = zbud_zpool_shrink, + .shrink = NULL, .map = zbud_zpool_map, .unmap = zbud_zpool_unmap, .total_size = zbud_zpool_total_size, From patchwork Mon Jun 5 08:54:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Domenico Cerasuolo X-Patchwork-Id: 103193 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2550765vqr; Mon, 5 Jun 2023 02:18:35 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6iLI+HHLybH9eJnfplbAGakl9Vm29QWFbFCw4MAH6yik9eqv7nTX+uJSioA9uNpiz/LElZ X-Received: by 2002:a05:6a20:42a3:b0:114:6669:21f8 with SMTP id o35-20020a056a2042a300b00114666921f8mr2262532pzj.35.1685956714886; Mon, 05 Jun 2023 02:18:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685956714; cv=none; d=google.com; s=arc-20160816; b=T5BkXH6RDIMWy3TRSLV1aMY2iXdCbpvI4U6HMA36pihQeCgjo0lDii4O7VvrcOSg2f GTdB6C2y1Ter6Ae/J/ZfXjW8bPCPMLRw92gAOjkQO6HLfZr4ifMGTUOSIAoCp12MaGXI hE3lyv/X/hNqn/qkqFldvhkUlf/FLnkbjqyOpUxecsSWbf37UBLg3gbmuEHLt8Ixvjv7 6R+LpdRbVmgncRrwGRWbIGgEFG1uBdRqBiNLXYQI9vKeRK49oDYg6bo48NdcebD87Idx 13/sLQGsWfjaOuQvD6IU9+7/MmFvkMBcXLQD1y6DfARRlNygnH6G8bS2dKTplhthog34 lg8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=eUrq3DY+OGrIYm8UgHc2JyLm8lRK0sjzVcTatP/l8cE=; b=RdGyvtp1ba57UdHYIkkPXOM3Iawmh160VI3ewntkQ8ZIHVNePg3af+Y9bRQrt3NlTo QHTq/GFS5k+tivQPh1ZsXLcHa/Ki+t/eAbeYt4xQlPyqOPchWZpTGnpyuepw0dKunkIH zYrCkld/U/vPu65p6mhFlGNlOwW3/mpdA3efUM0uk4C7ytKbGzmNmADBYVWeODfnXHau 9TpxK/K3XabZp4crZ+FIE1y2eBkR3askQCP7+dqSlvnBB4jZxm+xTiflZv+7aNMt3123 8G2Kjw9WkyATowM6O/x7fhx0U2DVeF6bunJIP5Sa5jfXuVHxpug/Ht1KF68nCsvVkulW r/AQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=R61dQ2k0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h184-20020a6383c1000000b00543ab2f4912si1734008pge.758.2023.06.05.02.18.19; Mon, 05 Jun 2023 02:18:34 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=R61dQ2k0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230406AbjFEIyp (ORCPT + 99 others); Mon, 5 Jun 2023 04:54:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230310AbjFEIye (ORCPT ); Mon, 5 Jun 2023 04:54:34 -0400 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D975D103 for ; Mon, 5 Jun 2023 01:54:29 -0700 (PDT) Received: by mail-ej1-x62b.google.com with SMTP id a640c23a62f3a-97458c97333so544598766b.2 for ; Mon, 05 Jun 2023 01:54:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685955268; x=1688547268; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eUrq3DY+OGrIYm8UgHc2JyLm8lRK0sjzVcTatP/l8cE=; b=R61dQ2k0UX8+n+93KaLmN2tNn3v9z2vJLfojb9RVUMiawZ6qHTzlaY4r69dZZRPvil LPIBvqI17yOd8DgOEV/HDTdpDXeliv9rzHWGeA/LByuBhbGKyvAWCNjROlSl80xGybqY /WXDpNcSAL8RLKVeCTJXMVLTEQB3AgPwbEQlpPrkvyhxgNQr3bszB+cogk5dPVusopej bOw5l5JSWrYHTYdRxO3LMELXKlMMxsOtbX4tno5Zlfe5iHp2Xp3gRl7OPOCNYuHRX8FP ie7lj+TqdS53xiHZgcrmWrSV4nahd+S210cPSjs+bFZyZOYwKmZk+W+kLwgObJbUs3eq sbww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685955268; x=1688547268; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eUrq3DY+OGrIYm8UgHc2JyLm8lRK0sjzVcTatP/l8cE=; b=BpOUY+MemX+Na8WyRFzRGTDkYMo0nXO9j2RJP+FGWm/K/S9cntmEiVtVmQIzH4pdEg usbF+314ciE/+enaIQV0BRnbSJo76QpVmLkTLP26Zdha3zrRexo6TGAMDGHYviVwIoCs 8dAi3qUCxfBJ3L/jafaFx0AWz55OZCa6L9a1x9o6AU4wKXFvCm9PT3/8xIrlYXmnTTRB wDUdn15YMF2klORNyCukYEFGGAZvS6N2335tBlsnh55XdyaLDJqf9QASg/Dv5Fzfwp6V ZCEEVYgmCgoipfdc80XEA6x3gIkEyOzu5nqy53MjKxAmI26Q9731PGw5if7WAAnYhBVR J/JQ== X-Gm-Message-State: AC+VfDy7M8lNomT8BRqmL4wugMyICZMhhB4ApbhLuSyRXb0fgI1AW4ep rMeHxBUzT8TuydoiFh9P8ic= X-Received: by 2002:a17:907:1b1c:b0:974:1ef7:15be with SMTP id mp28-20020a1709071b1c00b009741ef715bemr5483387ejc.30.1685955268041; Mon, 05 Jun 2023 01:54:28 -0700 (PDT) Received: from lelloman-5950.homenet.telecomitalia.it (host-82-53-8-153.retail.telecomitalia.it. [82.53.8.153]) by smtp.gmail.com with ESMTPSA id lx5-20020a170906af0500b00965c6c63ea3sm4048915ejb.35.2023.06.05.01.54.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jun 2023 01:54:27 -0700 (PDT) From: Domenico Cerasuolo To: vitaly.wool@konsulko.com, minchan@kernel.org, senozhatsky@chromium.org, yosryahmed@google.com, linux-mm@kvack.org Cc: ddstreet@ieee.org, sjenning@redhat.com, nphamcs@gmail.com, hannes@cmpxchg.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, kernel-team@meta.com, Domenico Cerasuolo Subject: [RFC PATCH 3/7] mm: zswap: remove page reclaim logic from z3fold Date: Mon, 5 Jun 2023 10:54:15 +0200 Message-Id: <20230605085419.44383-4-cerasuolodomenico@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230605085419.44383-1-cerasuolodomenico@gmail.com> References: <20230605085419.44383-1-cerasuolodomenico@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767853748379350401?= X-GMAIL-MSGID: =?utf-8?q?1767853748379350401?= With the recent enhancement to zswap enabling direct page writeback, the need for the shrink code in z3fold has become obsolete. As a result, this commit removes the page reclaim logic from z3fold entirely. Signed-off-by: Domenico Cerasuolo --- mm/z3fold.c | 246 +--------------------------------------------------- 1 file changed, 3 insertions(+), 243 deletions(-) diff --git a/mm/z3fold.c b/mm/z3fold.c index 0cef845d397b..4af8741553ac 100644 --- a/mm/z3fold.c +++ b/mm/z3fold.c @@ -125,13 +125,11 @@ struct z3fold_header { /** * struct z3fold_pool - stores metadata for each z3fold pool * @name: pool name - * @lock: protects pool unbuddied/lru lists + * @lock: protects pool unbuddied lists * @stale_lock: protects pool stale page list * @unbuddied: per-cpu array of lists tracking z3fold pages that contain 2- * buddies; the list each z3fold page is added to depends on * the size of its free region. - * @lru: list tracking the z3fold pages in LRU order by most recently - * added buddy. * @stale: list of pages marked for freeing * @pages_nr: number of z3fold pages in the pool. * @c_handle: cache for z3fold_buddy_slots allocation @@ -149,12 +147,9 @@ struct z3fold_pool { spinlock_t lock; spinlock_t stale_lock; struct list_head *unbuddied; - struct list_head lru; struct list_head stale; atomic64_t pages_nr; struct kmem_cache *c_handle; - struct zpool *zpool; - const struct zpool_ops *zpool_ops; struct workqueue_struct *compact_wq; struct workqueue_struct *release_wq; struct work_struct work; @@ -329,7 +324,6 @@ static struct z3fold_header *init_z3fold_page(struct page *page, bool headless, struct z3fold_header *zhdr = page_address(page); struct z3fold_buddy_slots *slots; - INIT_LIST_HEAD(&page->lru); clear_bit(PAGE_HEADLESS, &page->private); clear_bit(MIDDLE_CHUNK_MAPPED, &page->private); clear_bit(NEEDS_COMPACTING, &page->private); @@ -451,8 +445,6 @@ static void __release_z3fold_page(struct z3fold_header *zhdr, bool locked) set_bit(PAGE_STALE, &page->private); clear_bit(NEEDS_COMPACTING, &page->private); spin_lock(&pool->lock); - if (!list_empty(&page->lru)) - list_del_init(&page->lru); spin_unlock(&pool->lock); if (locked) @@ -930,7 +922,6 @@ static struct z3fold_pool *z3fold_create_pool(const char *name, gfp_t gfp) for_each_unbuddied_list(i, 0) INIT_LIST_HEAD(&unbuddied[i]); } - INIT_LIST_HEAD(&pool->lru); INIT_LIST_HEAD(&pool->stale); atomic64_set(&pool->pages_nr, 0); pool->name = name; @@ -1073,12 +1064,6 @@ static int z3fold_alloc(struct z3fold_pool *pool, size_t size, gfp_t gfp, headless: spin_lock(&pool->lock); - /* Add/move z3fold page to beginning of LRU */ - if (!list_empty(&page->lru)) - list_del(&page->lru); - - list_add(&page->lru, &pool->lru); - *handle = encode_handle(zhdr, bud); spin_unlock(&pool->lock); if (bud != HEADLESS) @@ -1115,9 +1100,6 @@ static void z3fold_free(struct z3fold_pool *pool, unsigned long handle) * immediately so we don't care about its value any more. */ if (!page_claimed) { - spin_lock(&pool->lock); - list_del(&page->lru); - spin_unlock(&pool->lock); put_z3fold_header(zhdr); free_z3fold_page(page, true); atomic64_dec(&pool->pages_nr); @@ -1172,194 +1154,6 @@ static void z3fold_free(struct z3fold_pool *pool, unsigned long handle) put_z3fold_header(zhdr); } -/** - * z3fold_reclaim_page() - evicts allocations from a pool page and frees it - * @pool: pool from which a page will attempt to be evicted - * @retries: number of pages on the LRU list for which eviction will - * be attempted before failing - * - * z3fold reclaim is different from normal system reclaim in that it is done - * from the bottom, up. This is because only the bottom layer, z3fold, has - * information on how the allocations are organized within each z3fold page. - * This has the potential to create interesting locking situations between - * z3fold and the user, however. - * - * To avoid these, this is how z3fold_reclaim_page() should be called: - * - * The user detects a page should be reclaimed and calls z3fold_reclaim_page(). - * z3fold_reclaim_page() will remove a z3fold page from the pool LRU list and - * call the user-defined eviction handler with the pool and handle as - * arguments. - * - * If the handle can not be evicted, the eviction handler should return - * non-zero. z3fold_reclaim_page() will add the z3fold page back to the - * appropriate list and try the next z3fold page on the LRU up to - * a user defined number of retries. - * - * If the handle is successfully evicted, the eviction handler should - * return 0 _and_ should have called z3fold_free() on the handle. z3fold_free() - * contains logic to delay freeing the page if the page is under reclaim, - * as indicated by the setting of the PG_reclaim flag on the underlying page. - * - * If all buddies in the z3fold page are successfully evicted, then the - * z3fold page can be freed. - * - * Returns: 0 if page is successfully freed, otherwise -EINVAL if there are - * no pages to evict or an eviction handler is not registered, -EAGAIN if - * the retry limit was hit. - */ -static int z3fold_reclaim_page(struct z3fold_pool *pool, unsigned int retries) -{ - int i, ret = -1; - struct z3fold_header *zhdr = NULL; - struct page *page = NULL; - struct list_head *pos; - unsigned long first_handle = 0, middle_handle = 0, last_handle = 0; - struct z3fold_buddy_slots slots __attribute__((aligned(SLOTS_ALIGN))); - - rwlock_init(&slots.lock); - slots.pool = (unsigned long)pool | (1 << HANDLES_NOFREE); - - spin_lock(&pool->lock); - for (i = 0; i < retries; i++) { - if (list_empty(&pool->lru)) { - spin_unlock(&pool->lock); - return -EINVAL; - } - list_for_each_prev(pos, &pool->lru) { - page = list_entry(pos, struct page, lru); - - zhdr = page_address(page); - if (test_bit(PAGE_HEADLESS, &page->private)) { - /* - * For non-headless pages, we wait to do this - * until we have the page lock to avoid racing - * with __z3fold_alloc(). Headless pages don't - * have a lock (and __z3fold_alloc() will never - * see them), but we still need to test and set - * PAGE_CLAIMED to avoid racing with - * z3fold_free(), so just do it now before - * leaving the loop. - */ - if (test_and_set_bit(PAGE_CLAIMED, &page->private)) - continue; - - break; - } - - if (!z3fold_page_trylock(zhdr)) { - zhdr = NULL; - continue; /* can't evict at this point */ - } - - /* test_and_set_bit is of course atomic, but we still - * need to do it under page lock, otherwise checking - * that bit in __z3fold_alloc wouldn't make sense - */ - if (zhdr->foreign_handles || - test_and_set_bit(PAGE_CLAIMED, &page->private)) { - z3fold_page_unlock(zhdr); - zhdr = NULL; - continue; /* can't evict such page */ - } - list_del_init(&zhdr->buddy); - zhdr->cpu = -1; - /* See comment in __z3fold_alloc. */ - kref_get(&zhdr->refcount); - break; - } - - if (!zhdr) - break; - - list_del_init(&page->lru); - spin_unlock(&pool->lock); - - if (!test_bit(PAGE_HEADLESS, &page->private)) { - /* - * We need encode the handles before unlocking, and - * use our local slots structure because z3fold_free - * can zero out zhdr->slots and we can't do much - * about that - */ - first_handle = 0; - last_handle = 0; - middle_handle = 0; - memset(slots.slot, 0, sizeof(slots.slot)); - if (zhdr->first_chunks) - first_handle = __encode_handle(zhdr, &slots, - FIRST); - if (zhdr->middle_chunks) - middle_handle = __encode_handle(zhdr, &slots, - MIDDLE); - if (zhdr->last_chunks) - last_handle = __encode_handle(zhdr, &slots, - LAST); - /* - * it's safe to unlock here because we hold a - * reference to this page - */ - z3fold_page_unlock(zhdr); - } else { - first_handle = encode_handle(zhdr, HEADLESS); - last_handle = middle_handle = 0; - } - /* Issue the eviction callback(s) */ - if (middle_handle) { - ret = pool->zpool_ops->evict(pool->zpool, middle_handle); - if (ret) - goto next; - } - if (first_handle) { - ret = pool->zpool_ops->evict(pool->zpool, first_handle); - if (ret) - goto next; - } - if (last_handle) { - ret = pool->zpool_ops->evict(pool->zpool, last_handle); - if (ret) - goto next; - } -next: - if (test_bit(PAGE_HEADLESS, &page->private)) { - if (ret == 0) { - free_z3fold_page(page, true); - atomic64_dec(&pool->pages_nr); - return 0; - } - spin_lock(&pool->lock); - list_add(&page->lru, &pool->lru); - spin_unlock(&pool->lock); - clear_bit(PAGE_CLAIMED, &page->private); - } else { - struct z3fold_buddy_slots *slots = zhdr->slots; - z3fold_page_lock(zhdr); - if (kref_put(&zhdr->refcount, - release_z3fold_page_locked)) { - kmem_cache_free(pool->c_handle, slots); - return 0; - } - /* - * if we are here, the page is still not completely - * free. Take the global pool lock then to be able - * to add it back to the lru list - */ - spin_lock(&pool->lock); - list_add(&page->lru, &pool->lru); - spin_unlock(&pool->lock); - if (list_empty(&zhdr->buddy)) - add_to_unbuddied(pool, zhdr); - clear_bit(PAGE_CLAIMED, &page->private); - z3fold_page_unlock(zhdr); - } - - /* We started off locked to we need to lock the pool back */ - spin_lock(&pool->lock); - } - spin_unlock(&pool->lock); - return -EAGAIN; -} - /** * z3fold_map() - maps the allocation associated with the given handle * @pool: pool in which the allocation resides @@ -1470,8 +1264,6 @@ static bool z3fold_page_isolate(struct page *page, isolate_mode_t mode) spin_lock(&pool->lock); if (!list_empty(&zhdr->buddy)) list_del_init(&zhdr->buddy); - if (!list_empty(&page->lru)) - list_del_init(&page->lru); spin_unlock(&pool->lock); kref_get(&zhdr->refcount); @@ -1531,9 +1323,6 @@ static int z3fold_page_migrate(struct page *newpage, struct page *page, encode_handle(new_zhdr, MIDDLE); set_bit(NEEDS_COMPACTING, &newpage->private); new_zhdr->cpu = smp_processor_id(); - spin_lock(&pool->lock); - list_add(&newpage->lru, &pool->lru); - spin_unlock(&pool->lock); __SetPageMovable(newpage, &z3fold_mops); z3fold_page_unlock(new_zhdr); @@ -1559,9 +1348,6 @@ static void z3fold_page_putback(struct page *page) INIT_LIST_HEAD(&page->lru); if (kref_put(&zhdr->refcount, release_z3fold_page_locked)) return; - spin_lock(&pool->lock); - list_add(&page->lru, &pool->lru); - spin_unlock(&pool->lock); if (list_empty(&zhdr->buddy)) add_to_unbuddied(pool, zhdr); clear_bit(PAGE_CLAIMED, &page->private); @@ -1582,14 +1368,7 @@ static void *z3fold_zpool_create(const char *name, gfp_t gfp, const struct zpool_ops *zpool_ops, struct zpool *zpool) { - struct z3fold_pool *pool; - - pool = z3fold_create_pool(name, gfp); - if (pool) { - pool->zpool = zpool; - pool->zpool_ops = zpool_ops; - } - return pool; + return z3fold_create_pool(name, gfp); } static void z3fold_zpool_destroy(void *pool) @@ -1607,25 +1386,6 @@ static void z3fold_zpool_free(void *pool, unsigned long handle) z3fold_free(pool, handle); } -static int z3fold_zpool_shrink(void *pool, unsigned int pages, - unsigned int *reclaimed) -{ - unsigned int total = 0; - int ret = -EINVAL; - - while (total < pages) { - ret = z3fold_reclaim_page(pool, 8); - if (ret < 0) - break; - total++; - } - - if (reclaimed) - *reclaimed = total; - - return ret; -} - static void *z3fold_zpool_map(void *pool, unsigned long handle, enum zpool_mapmode mm) { @@ -1649,7 +1409,7 @@ static struct zpool_driver z3fold_zpool_driver = { .destroy = z3fold_zpool_destroy, .malloc = z3fold_zpool_malloc, .free = z3fold_zpool_free, - .shrink = z3fold_zpool_shrink, + .shrink = NULL, .map = z3fold_zpool_map, .unmap = z3fold_zpool_unmap, .total_size = z3fold_zpool_total_size, From patchwork Mon Jun 5 08:54:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Domenico Cerasuolo X-Patchwork-Id: 103187 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2542022vqr; Mon, 5 Jun 2023 01:59:06 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6fBvo7VIgkLLauBWGeATZ/GKdcuNCFr+PYvbY1N6q4hCiDHGH89efNmELZBp8LxizeC++E X-Received: by 2002:a17:902:ce88:b0:1af:f4f5:6fae with SMTP id f8-20020a170902ce8800b001aff4f56faemr6678217plg.54.1685955545821; Mon, 05 Jun 2023 01:59:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685955545; cv=none; d=google.com; s=arc-20160816; b=oYj+ou0xuZfHSZhxTOWDxG3dhFy3b0XF6LB03K0M+VwdCir3iRXAUGiB/txeUCGGUx evCQ7Qeq5Q/mm5caL7l1YiOXZ1aO0Yt3+7LEk44P/C93ThJdzTIAw8E1NvVh4j2gUuoh 6KrUPAZzfRZCLkNdrkipKJ01YQy7s4emksPH1j6chZEuYxjUBLSjFYA7J+57/q12IBpS CLTIIoXdMTYbH5iJDOhopn09wqljsvTJ1ylAnKEvw8teO/TqGJDEuMLGWER8CAGnNDZd UNLO2lrRjF5t5ao0vCFBpNS9jKbkZRk4ifNsl+V5SR7J8VRDUY22lx/kLCHC43R3DzS9 qbiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=SelvWHuRmB9qYK6uvKY8c3bmTfu/gV9/UpW9ng/IiW8=; b=HC1RIVJoP8libQhBPp2KkM8i9d4/NWOI/GarN5hPbmaki3XnbtWIlGv1oFbLDiNj8Z hOxKXJGDbv6ZEstoZFqFL8NZvsGk6Uyk43Qne8mozzP2EaH8zPHD7havqtRupu1JFjfa lVvD8Tr0bNV4QFLdI+GpAjMmCkcM7esD+qH4MBuO/8dYGYuDPETC/TezYyjYRK6Mj0Fk 198o5XxwUzeP2yVgNNE8/9eedn1LMOdsfjqmL6i1TXTAUvWtHjU8G72jZx8jKT/pFVGg VO6R5jwa282iMRIJKriVxyil0oCPBf+dw+lsgPmN8WoCIIa8W1ZbRKw8pEBdoGgjyod2 0INA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=cbgueKOQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j13-20020a170903024d00b001b1a2c14a48si5356223plh.141.2023.06.05.01.58.53; Mon, 05 Jun 2023 01:59:05 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=cbgueKOQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230370AbjFEIzF (ORCPT + 99 others); Mon, 5 Jun 2023 04:55:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39542 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230322AbjFEIyf (ORCPT ); Mon, 5 Jun 2023 04:54:35 -0400 Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [IPv6:2a00:1450:4864:20::634]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 31BF910C for ; Mon, 5 Jun 2023 01:54:31 -0700 (PDT) Received: by mail-ej1-x634.google.com with SMTP id a640c23a62f3a-977cf86aae5so188982666b.0 for ; Mon, 05 Jun 2023 01:54:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685955269; x=1688547269; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SelvWHuRmB9qYK6uvKY8c3bmTfu/gV9/UpW9ng/IiW8=; b=cbgueKOQbNxHoHuF4V6qQsV6CQHBqS9C0tyOae7dbeOgLSxe3DfFUf2VIy888iH8Rh +FiT3UpNXP5s27/wP8KWthItEpMfOqKo1ZxyjNNyHjPWzW6mFzeSaCyVsl3eOa1K5dPY v1FGubyAKqYcb2ynOje2NPYX0JQx13GZJA+HkAj3MZOhUYqlLq/C0dXG3VXCqpC8AaXy PBLxPVAuB05smiQixJ7R9fe7Zz/vfbp0AQBV97blG7LHRfsW/Qw8Tc2PncdkXQlu1qAE ViobkuPoU5hDZozI92OqjNW8F5rkfF+IMBPeMU2TAhV7ifABrgaNBMJxE+W3ET5nJyCl 5N3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685955269; x=1688547269; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SelvWHuRmB9qYK6uvKY8c3bmTfu/gV9/UpW9ng/IiW8=; b=c8AX2bv2QkcV8pyXLrcHXJeACcyWU5cTHevEsdkZXFc3tiJ2n5WlQtlzzoNyS4KbxE v+ewJRZRrfkpabfcatEt+5WyYPpEPZcZk4vURJUZpRV5Zpk9Z/Q8Bmhk1MmrrfUGjpwk Uvw/DmDz7Rd4eR1OCJfFMR9xCrfVFkGi6RzRnd/eFGOroQvUsRvreoP8cVPj8KMEqiWk tbjFUPUrZymqgjM8zmCpVvG6l7hvf+Z7uhfMaz+qz+jNiMMOBxuYjOtrkzafIL/AMwTx mthkAVFD0+WvrIi0sj8rcJfK0SAPI78wCt55+T6HpsqHPZOdqsOie+htH0cKMa5X3rMV jSjw== X-Gm-Message-State: AC+VfDyXXRp1MmWyxio91PDfQiRPgPBbwoWWjPOnjonRkvVbqehsL4Zi ICfj9Kk/tlwojaxCpRLcx4I= X-Received: by 2002:a17:907:969f:b0:973:940e:a018 with SMTP id hd31-20020a170907969f00b00973940ea018mr4860860ejc.34.1685955269494; Mon, 05 Jun 2023 01:54:29 -0700 (PDT) Received: from lelloman-5950.homenet.telecomitalia.it (host-82-53-8-153.retail.telecomitalia.it. [82.53.8.153]) by smtp.gmail.com with ESMTPSA id lx5-20020a170906af0500b00965c6c63ea3sm4048915ejb.35.2023.06.05.01.54.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jun 2023 01:54:29 -0700 (PDT) From: Domenico Cerasuolo To: vitaly.wool@konsulko.com, minchan@kernel.org, senozhatsky@chromium.org, yosryahmed@google.com, linux-mm@kvack.org Cc: ddstreet@ieee.org, sjenning@redhat.com, nphamcs@gmail.com, hannes@cmpxchg.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, kernel-team@meta.com, Domenico Cerasuolo Subject: [RFC PATCH 4/7] mm: zswap: remove page reclaim logic from zsmalloc Date: Mon, 5 Jun 2023 10:54:16 +0200 Message-Id: <20230605085419.44383-5-cerasuolodomenico@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230605085419.44383-1-cerasuolodomenico@gmail.com> References: <20230605085419.44383-1-cerasuolodomenico@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767852522293185799?= X-GMAIL-MSGID: =?utf-8?q?1767852522293185799?= With the recent enhancement to zswap enabling direct page writeback, the need for the shrink code in zsmalloc has become obsolete. As a result, this commit removes the page reclaim logic from zsmalloc entirely. Signed-off-by: Domenico Cerasuolo --- mm/zsmalloc.c | 291 +------------------------------------------------- 1 file changed, 2 insertions(+), 289 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 02f7f414aade..c87a60514f21 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -250,13 +250,6 @@ struct zs_pool { /* Compact classes */ struct shrinker shrinker; -#ifdef CONFIG_ZPOOL - /* List tracking the zspages in LRU order by most recently added object */ - struct list_head lru; - struct zpool *zpool; - const struct zpool_ops *zpool_ops; -#endif - #ifdef CONFIG_ZSMALLOC_STAT struct dentry *stat_dentry; #endif @@ -279,13 +272,6 @@ struct zspage { unsigned int freeobj; struct page *first_page; struct list_head list; /* fullness list */ - -#ifdef CONFIG_ZPOOL - /* links the zspage to the lru list in the pool */ - struct list_head lru; - bool under_reclaim; -#endif - struct zs_pool *pool; rwlock_t lock; }; @@ -393,14 +379,7 @@ static void *zs_zpool_create(const char *name, gfp_t gfp, * different contexts and its caller must provide a valid * gfp mask. */ - struct zs_pool *pool = zs_create_pool(name); - - if (pool) { - pool->zpool = zpool; - pool->zpool_ops = zpool_ops; - } - - return pool; + return zs_create_pool(name); } static void zs_zpool_destroy(void *pool) @@ -422,27 +401,6 @@ static void zs_zpool_free(void *pool, unsigned long handle) zs_free(pool, handle); } -static int zs_reclaim_page(struct zs_pool *pool, unsigned int retries); - -static int zs_zpool_shrink(void *pool, unsigned int pages, - unsigned int *reclaimed) -{ - unsigned int total = 0; - int ret = -EINVAL; - - while (total < pages) { - ret = zs_reclaim_page(pool, 8); - if (ret < 0) - break; - total++; - } - - if (reclaimed) - *reclaimed = total; - - return ret; -} - static void *zs_zpool_map(void *pool, unsigned long handle, enum zpool_mapmode mm) { @@ -481,7 +439,7 @@ static struct zpool_driver zs_zpool_driver = { .malloc_support_movable = true, .malloc = zs_zpool_malloc, .free = zs_zpool_free, - .shrink = zs_zpool_shrink, + .shrink = NULL, .map = zs_zpool_map, .unmap = zs_zpool_unmap, .total_size = zs_zpool_total_size, @@ -884,14 +842,6 @@ static inline bool obj_allocated(struct page *page, void *obj, unsigned long *ph return obj_tagged(page, obj, phandle, OBJ_ALLOCATED_TAG); } -#ifdef CONFIG_ZPOOL -static bool obj_stores_deferred_handle(struct page *page, void *obj, - unsigned long *phandle) -{ - return obj_tagged(page, obj, phandle, OBJ_DEFERRED_HANDLE_TAG); -} -#endif - static void reset_page(struct page *page) { __ClearPageMovable(page); @@ -1006,9 +956,6 @@ static void free_zspage(struct zs_pool *pool, struct size_class *class, } remove_zspage(class, zspage, ZS_INUSE_RATIO_0); -#ifdef CONFIG_ZPOOL - list_del(&zspage->lru); -#endif __free_zspage(pool, class, zspage); } @@ -1054,11 +1001,6 @@ static void init_zspage(struct size_class *class, struct zspage *zspage) off %= PAGE_SIZE; } -#ifdef CONFIG_ZPOOL - INIT_LIST_HEAD(&zspage->lru); - zspage->under_reclaim = false; -#endif - set_freeobj(zspage, 0); } @@ -1525,13 +1467,6 @@ unsigned long zs_malloc(struct zs_pool *pool, size_t size, gfp_t gfp) /* We completely set up zspage so mark them as movable */ SetZsPageMovable(pool, zspage); out: -#ifdef CONFIG_ZPOOL - /* Add/move zspage to beginning of LRU */ - if (!list_empty(&zspage->lru)) - list_del(&zspage->lru); - list_add(&zspage->lru, &pool->lru); -#endif - spin_unlock(&pool->lock); return handle; @@ -1600,20 +1535,6 @@ void zs_free(struct zs_pool *pool, unsigned long handle) class = zspage_class(pool, zspage); class_stat_dec(class, ZS_OBJS_INUSE, 1); - -#ifdef CONFIG_ZPOOL - if (zspage->under_reclaim) { - /* - * Reclaim needs the handles during writeback. It'll free - * them along with the zspage when it's done with them. - * - * Record current deferred handle in the object's header. - */ - obj_free(class->size, obj, &handle); - spin_unlock(&pool->lock); - return; - } -#endif obj_free(class->size, obj, NULL); fullness = fix_fullness_group(class, zspage); @@ -1890,23 +1811,6 @@ static void lock_zspage(struct zspage *zspage) } #endif /* defined(CONFIG_ZPOOL) || defined(CONFIG_COMPACTION) */ -#ifdef CONFIG_ZPOOL -/* - * Unlocks all the pages of the zspage. - * - * pool->lock must be held before this function is called - * to prevent the underlying pages from migrating. - */ -static void unlock_zspage(struct zspage *zspage) -{ - struct page *page = get_first_page(zspage); - - do { - unlock_page(page); - } while ((page = get_next_page(page)) != NULL); -} -#endif /* CONFIG_ZPOOL */ - static void migrate_lock_init(struct zspage *zspage) { rwlock_init(&zspage->lock); @@ -2126,9 +2030,6 @@ static void async_free_zspage(struct work_struct *work) VM_BUG_ON(fullness != ZS_INUSE_RATIO_0); class = pool->size_class[class_idx]; spin_lock(&pool->lock); -#ifdef CONFIG_ZPOOL - list_del(&zspage->lru); -#endif __free_zspage(pool, class, zspage); spin_unlock(&pool->lock); } @@ -2474,10 +2375,6 @@ struct zs_pool *zs_create_pool(const char *name) */ zs_register_shrinker(pool); -#ifdef CONFIG_ZPOOL - INIT_LIST_HEAD(&pool->lru); -#endif - return pool; err: @@ -2520,190 +2417,6 @@ void zs_destroy_pool(struct zs_pool *pool) } EXPORT_SYMBOL_GPL(zs_destroy_pool); -#ifdef CONFIG_ZPOOL -static void restore_freelist(struct zs_pool *pool, struct size_class *class, - struct zspage *zspage) -{ - unsigned int obj_idx = 0; - unsigned long handle, off = 0; /* off is within-page offset */ - struct page *page = get_first_page(zspage); - struct link_free *prev_free = NULL; - void *prev_page_vaddr = NULL; - - /* in case no free object found */ - set_freeobj(zspage, (unsigned int)(-1UL)); - - while (page) { - void *vaddr = kmap_atomic(page); - struct page *next_page; - - while (off < PAGE_SIZE) { - void *obj_addr = vaddr + off; - - /* skip allocated object */ - if (obj_allocated(page, obj_addr, &handle)) { - obj_idx++; - off += class->size; - continue; - } - - /* free deferred handle from reclaim attempt */ - if (obj_stores_deferred_handle(page, obj_addr, &handle)) - cache_free_handle(pool, handle); - - if (prev_free) - prev_free->next = obj_idx << OBJ_TAG_BITS; - else /* first free object found */ - set_freeobj(zspage, obj_idx); - - prev_free = (struct link_free *)vaddr + off / sizeof(*prev_free); - /* if last free object in a previous page, need to unmap */ - if (prev_page_vaddr) { - kunmap_atomic(prev_page_vaddr); - prev_page_vaddr = NULL; - } - - obj_idx++; - off += class->size; - } - - /* - * Handle the last (full or partial) object on this page. - */ - next_page = get_next_page(page); - if (next_page) { - if (!prev_free || prev_page_vaddr) { - /* - * There is no free object in this page, so we can safely - * unmap it. - */ - kunmap_atomic(vaddr); - } else { - /* update prev_page_vaddr since prev_free is on this page */ - prev_page_vaddr = vaddr; - } - } else { /* this is the last page */ - if (prev_free) { - /* - * Reset OBJ_TAG_BITS bit to last link to tell - * whether it's allocated object or not. - */ - prev_free->next = -1UL << OBJ_TAG_BITS; - } - - /* unmap previous page (if not done yet) */ - if (prev_page_vaddr) { - kunmap_atomic(prev_page_vaddr); - prev_page_vaddr = NULL; - } - - kunmap_atomic(vaddr); - } - - page = next_page; - off %= PAGE_SIZE; - } -} - -static int zs_reclaim_page(struct zs_pool *pool, unsigned int retries) -{ - int i, obj_idx, ret = 0; - unsigned long handle; - struct zspage *zspage; - struct page *page; - int fullness; - - /* Lock LRU and fullness list */ - spin_lock(&pool->lock); - if (list_empty(&pool->lru)) { - spin_unlock(&pool->lock); - return -EINVAL; - } - - for (i = 0; i < retries; i++) { - struct size_class *class; - - zspage = list_last_entry(&pool->lru, struct zspage, lru); - list_del(&zspage->lru); - - /* zs_free may free objects, but not the zspage and handles */ - zspage->under_reclaim = true; - - class = zspage_class(pool, zspage); - fullness = get_fullness_group(class, zspage); - - /* Lock out object allocations and object compaction */ - remove_zspage(class, zspage, fullness); - - spin_unlock(&pool->lock); - cond_resched(); - - /* Lock backing pages into place */ - lock_zspage(zspage); - - obj_idx = 0; - page = get_first_page(zspage); - while (1) { - handle = find_alloced_obj(class, page, &obj_idx); - if (!handle) { - page = get_next_page(page); - if (!page) - break; - obj_idx = 0; - continue; - } - - /* - * This will write the object and call zs_free. - * - * zs_free will free the object, but the - * under_reclaim flag prevents it from freeing - * the zspage altogether. This is necessary so - * that we can continue working with the - * zspage potentially after the last object - * has been freed. - */ - ret = pool->zpool_ops->evict(pool->zpool, handle); - if (ret) - goto next; - - obj_idx++; - } - -next: - /* For freeing the zspage, or putting it back in the pool and LRU list. */ - spin_lock(&pool->lock); - zspage->under_reclaim = false; - - if (!get_zspage_inuse(zspage)) { - /* - * Fullness went stale as zs_free() won't touch it - * while the page is removed from the pool. Fix it - * up for the check in __free_zspage(). - */ - zspage->fullness = ZS_INUSE_RATIO_0; - - __free_zspage(pool, class, zspage); - spin_unlock(&pool->lock); - return 0; - } - - /* - * Eviction fails on one of the handles, so we need to restore zspage. - * We need to rebuild its freelist (and free stored deferred handles), - * put it back to the correct size class, and add it to the LRU list. - */ - restore_freelist(pool, class, zspage); - putback_zspage(class, zspage); - list_add(&zspage->lru, &pool->lru); - unlock_zspage(zspage); - } - - spin_unlock(&pool->lock); - return -EAGAIN; -} -#endif /* CONFIG_ZPOOL */ - static int __init zs_init(void) { int ret; From patchwork Mon Jun 5 08:54:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Domenico Cerasuolo X-Patchwork-Id: 103188 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2545643vqr; Mon, 5 Jun 2023 02:06:15 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7nSI8daKc32Wj/DXHBzdVU5BmR4ZW3N0En95uzQEGYUlqbg84zEuQMe7GMj/nDe6GLPuEQ X-Received: by 2002:a17:902:7c97:b0:1aa:f6e9:4cb3 with SMTP id y23-20020a1709027c9700b001aaf6e94cb3mr1825713pll.52.1685955974858; Mon, 05 Jun 2023 02:06:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685955974; cv=none; d=google.com; s=arc-20160816; b=X+HxDMMV6V4gDPo+TcPW6LhOo3wEH6W8JS0fNJxOCjmhlreG3ssuIftdDCUqKfIk7A oKKTB0MLG2HfkW4I10pPNFcPODifOwEfjb0RwvZDH57Rjsxaz9cfFW5qAXnacqDQrdqz 2YPXreXftvFTaJtpMavzjOg9eI0ofIVUneo0rf3fSYOi64s6G1BzMPEwzyw2Ix9qQBj8 hKOpnQ82qrF3N74UtXD/iiKDMl9/BlMV2P7Zec2GDmwy0uNhpL0DfPvb2eZAULryBnD3 kN2sPmNHZkWEGZM1bBKBBFfc3KmIcry8S/SmKmAfBF8KaSpzHK3LGNa+igsFCa1vrFqL buYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=VEQ6QPB4mJhVwoMuepvYuRemR1X9+w0A2PDCNfia9oI=; b=U70GwVJWHQfdjCQg1dkjwLqanKrBWi+6iKTiNz2tIVoTMHFOoHIqoSN1UzQOtiLQ2v YhPU4ynf8yKjvDV8nlxGa4IGDNC8w/ERSnSZ2XjMz0m8hrYjw5sf+IvxPG4E9m9YVEFu Yg6EY9i3x1djOl/ntMHTVHRI2BaWghQbsoc1aWvlN2nAhlxeFwJG0jZ0/hkwApjq7uOB krvfULBKXYerLWje/jMW9HhkZYs61wD1+SlpuwQf2CbHpegHfFxIh3n6HkAJBFNvXA9u MSlmWu525ItAhBPYT3R2/wE/73bs+0rW6mvVfLSmL9nTM9cLSUbauI8CiYyt/PGK51tX hR2g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=BwOdiyaS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i6-20020a17090332c600b001ae42afbf93si5293866plr.450.2023.06.05.02.05.57; Mon, 05 Jun 2023 02:06:14 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=BwOdiyaS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230169AbjFEIzI (ORCPT + 99 others); Mon, 5 Jun 2023 04:55:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39556 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230342AbjFEIyg (ORCPT ); Mon, 5 Jun 2023 04:54:36 -0400 Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 94ADBF4 for ; Mon, 5 Jun 2023 01:54:32 -0700 (PDT) Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-973f8dbcc02so765517366b.3 for ; Mon, 05 Jun 2023 01:54:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685955271; x=1688547271; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VEQ6QPB4mJhVwoMuepvYuRemR1X9+w0A2PDCNfia9oI=; b=BwOdiyaSZUHGVJGs/vjaFGq1KR8hK2/oGwJggh+ERhIKrheBPRdSrsSw1SR7vHSiY+ w93Ajuzl5hnY0RLzrF4L0k43s5v0/3i1iUVXGxjLzjp+TkHZUUDAXDXKO+JaUlGKA+xb go7LP0UckICUvJRZH2VgL/hMxgKro3pF9g07ylLXavvz+JCU32gYJQgLDZQWom0ieelm i5Zr4v6Cc9DEZBnuHIDfMZrBRZBnA5CA0DLQDhAAhodg6A7y++wAdZSo6/r23nucx6uO JkGM+kEN+Qg3dhs6zuuL/VZed4mNeRN8x9gRzs668XeJmS3CobO6OngIC4DNBUHQFxi3 nlBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685955271; x=1688547271; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VEQ6QPB4mJhVwoMuepvYuRemR1X9+w0A2PDCNfia9oI=; b=L8QYxEC+/3oKGguXHqCYVtk5XKV7sKYxSIHC5DEpTZ6Jj1m4C9pI3mHAYgAn5ilrYv PJ9UvAVYOKDIoyDFm74xR0RkdQnVnppmNhuHP8dOjgRFEP+l2LrDIUHTPRMZQE4Tw0pH +j/w4biWyWiYiKyennJbHJuUhCB4KgB+kd8N9UQO0tkP0rDdBjo3fb3lO8YP1mVYTUV6 p2QWswPPJq3xmKNnK6qWCaQqoNTQOOc2UnuPiDsUTqSG+5U/ujEzbWCB6XrdCONNdXsc GrBR/UeetqQAGhw1Oe0DeVrEnIZ0sX9UN01kLNxJ1bwfEG+2X0gkZsv8hpWNXo5JMspA Pf4A== X-Gm-Message-State: AC+VfDyjDOEHWVum3HxFcSiZVWpHytSd/aD+bvMh69uYurrmEyU+AiIM 9mc/32keypjBHkXHWjuhjfA= X-Received: by 2002:a17:907:268c:b0:974:76:dcdd with SMTP id bn12-20020a170907268c00b009740076dcddmr6165926ejc.55.1685955270926; Mon, 05 Jun 2023 01:54:30 -0700 (PDT) Received: from lelloman-5950.homenet.telecomitalia.it (host-82-53-8-153.retail.telecomitalia.it. [82.53.8.153]) by smtp.gmail.com with ESMTPSA id lx5-20020a170906af0500b00965c6c63ea3sm4048915ejb.35.2023.06.05.01.54.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jun 2023 01:54:30 -0700 (PDT) From: Domenico Cerasuolo To: vitaly.wool@konsulko.com, minchan@kernel.org, senozhatsky@chromium.org, yosryahmed@google.com, linux-mm@kvack.org Cc: ddstreet@ieee.org, sjenning@redhat.com, nphamcs@gmail.com, hannes@cmpxchg.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, kernel-team@meta.com, Domenico Cerasuolo Subject: [RFC PATCH 5/7] mm: zswap: remove shrink from zpool interface Date: Mon, 5 Jun 2023 10:54:17 +0200 Message-Id: <20230605085419.44383-6-cerasuolodomenico@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230605085419.44383-1-cerasuolodomenico@gmail.com> References: <20230605085419.44383-1-cerasuolodomenico@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767852972613591502?= X-GMAIL-MSGID: =?utf-8?q?1767852972613591502?= Now that all three zswap backends have removed their shrink code, it is no longer necessary for the zpool interface to include shrink/writeback endpoints. Signed-off-by: Domenico Cerasuolo --- include/linux/zpool.h | 19 ++--------------- mm/z3fold.c | 5 +---- mm/zbud.c | 5 +---- mm/zpool.c | 48 ++----------------------------------------- mm/zsmalloc.c | 5 +---- mm/zswap.c | 18 ++++------------ 6 files changed, 11 insertions(+), 89 deletions(-) diff --git a/include/linux/zpool.h b/include/linux/zpool.h index e8997010612a..6b15a4213de5 100644 --- a/include/linux/zpool.h +++ b/include/linux/zpool.h @@ -14,10 +14,6 @@ struct zpool; -struct zpool_ops { - int (*evict)(struct zpool *pool, unsigned long handle); -}; - /* * Control how a handle is mapped. It will be ignored if the * implementation does not support it. Its use is optional. @@ -40,7 +36,7 @@ enum zpool_mapmode { bool zpool_has_pool(char *type); struct zpool *zpool_create_pool(const char *type, const char *name, - gfp_t gfp, const struct zpool_ops *ops); + gfp_t gfp); const char *zpool_get_type(struct zpool *pool); @@ -53,9 +49,6 @@ int zpool_malloc(struct zpool *pool, size_t size, gfp_t gfp, void zpool_free(struct zpool *pool, unsigned long handle); -int zpool_shrink(struct zpool *pool, unsigned int pages, - unsigned int *reclaimed); - void *zpool_map_handle(struct zpool *pool, unsigned long handle, enum zpool_mapmode mm); @@ -72,7 +65,6 @@ u64 zpool_get_total_size(struct zpool *pool); * @destroy: destroy a pool. * @malloc: allocate mem from a pool. * @free: free mem from a pool. - * @shrink: shrink the pool. * @sleep_mapped: whether zpool driver can sleep during map. * @map: map a handle. * @unmap: unmap a handle. @@ -87,10 +79,7 @@ struct zpool_driver { atomic_t refcount; struct list_head list; - void *(*create)(const char *name, - gfp_t gfp, - const struct zpool_ops *ops, - struct zpool *zpool); + void *(*create)(const char *name, gfp_t gfp); void (*destroy)(void *pool); bool malloc_support_movable; @@ -98,9 +87,6 @@ struct zpool_driver { unsigned long *handle); void (*free)(void *pool, unsigned long handle); - int (*shrink)(void *pool, unsigned int pages, - unsigned int *reclaimed); - bool sleep_mapped; void *(*map)(void *pool, unsigned long handle, enum zpool_mapmode mm); @@ -113,7 +99,6 @@ void zpool_register_driver(struct zpool_driver *driver); int zpool_unregister_driver(struct zpool_driver *driver); -bool zpool_evictable(struct zpool *pool); bool zpool_can_sleep_mapped(struct zpool *pool); #endif diff --git a/mm/z3fold.c b/mm/z3fold.c index 4af8741553ac..e84de91ecccb 100644 --- a/mm/z3fold.c +++ b/mm/z3fold.c @@ -1364,9 +1364,7 @@ static const struct movable_operations z3fold_mops = { * zpool ****************/ -static void *z3fold_zpool_create(const char *name, gfp_t gfp, - const struct zpool_ops *zpool_ops, - struct zpool *zpool) +static void *z3fold_zpool_create(const char *name, gfp_t gfp) { return z3fold_create_pool(name, gfp); } @@ -1409,7 +1407,6 @@ static struct zpool_driver z3fold_zpool_driver = { .destroy = z3fold_zpool_destroy, .malloc = z3fold_zpool_malloc, .free = z3fold_zpool_free, - .shrink = NULL, .map = z3fold_zpool_map, .unmap = z3fold_zpool_unmap, .total_size = z3fold_zpool_total_size, diff --git a/mm/zbud.c b/mm/zbud.c index 19bc662ef5e9..2190cc1f37b3 100644 --- a/mm/zbud.c +++ b/mm/zbud.c @@ -380,9 +380,7 @@ static u64 zbud_get_pool_size(struct zbud_pool *pool) * zpool ****************/ -static void *zbud_zpool_create(const char *name, gfp_t gfp, - const struct zpool_ops *zpool_ops, - struct zpool *zpool) +static void *zbud_zpool_create(const char *name, gfp_t gfp) { return zbud_create_pool(gfp); } @@ -425,7 +423,6 @@ static struct zpool_driver zbud_zpool_driver = { .destroy = zbud_zpool_destroy, .malloc = zbud_zpool_malloc, .free = zbud_zpool_free, - .shrink = NULL, .map = zbud_zpool_map, .unmap = zbud_zpool_unmap, .total_size = zbud_zpool_total_size, diff --git a/mm/zpool.c b/mm/zpool.c index 6a19c4a58f77..846410479c2f 100644 --- a/mm/zpool.c +++ b/mm/zpool.c @@ -133,7 +133,6 @@ EXPORT_SYMBOL(zpool_has_pool); * @type: The type of the zpool to create (e.g. zbud, zsmalloc) * @name: The name of the zpool (e.g. zram0, zswap) * @gfp: The GFP flags to use when allocating the pool. - * @ops: The optional ops callback. * * This creates a new zpool of the specified type. The gfp flags will be * used when allocating memory, if the implementation supports it. If the @@ -145,8 +144,7 @@ EXPORT_SYMBOL(zpool_has_pool); * * Returns: New zpool on success, NULL on failure. */ -struct zpool *zpool_create_pool(const char *type, const char *name, gfp_t gfp, - const struct zpool_ops *ops) +struct zpool *zpool_create_pool(const char *type, const char *name, gfp_t gfp) { struct zpool_driver *driver; struct zpool *zpool; @@ -173,7 +171,7 @@ struct zpool *zpool_create_pool(const char *type, const char *name, gfp_t gfp, } zpool->driver = driver; - zpool->pool = driver->create(name, gfp, ops, zpool); + zpool->pool = driver->create(name, gfp); if (!zpool->pool) { pr_err("couldn't create %s pool\n", type); @@ -279,30 +277,6 @@ void zpool_free(struct zpool *zpool, unsigned long handle) zpool->driver->free(zpool->pool, handle); } -/** - * zpool_shrink() - Shrink the pool size - * @zpool: The zpool to shrink. - * @pages: The number of pages to shrink the pool. - * @reclaimed: The number of pages successfully evicted. - * - * This attempts to shrink the actual memory size of the pool - * by evicting currently used handle(s). If the pool was - * created with no zpool_ops, or the evict call fails for any - * of the handles, this will fail. If non-NULL, the @reclaimed - * parameter will be set to the number of pages reclaimed, - * which may be more than the number of pages requested. - * - * Implementations must guarantee this to be thread-safe. - * - * Returns: 0 on success, negative value on error/failure. - */ -int zpool_shrink(struct zpool *zpool, unsigned int pages, - unsigned int *reclaimed) -{ - return zpool->driver->shrink ? - zpool->driver->shrink(zpool->pool, pages, reclaimed) : -EINVAL; -} - /** * zpool_map_handle() - Map a previously allocated handle into memory * @zpool: The zpool that the handle was allocated from @@ -359,24 +333,6 @@ u64 zpool_get_total_size(struct zpool *zpool) return zpool->driver->total_size(zpool->pool); } -/** - * zpool_evictable() - Test if zpool is potentially evictable - * @zpool: The zpool to test - * - * Zpool is only potentially evictable when it's created with struct - * zpool_ops.evict and its driver implements struct zpool_driver.shrink. - * - * However, it doesn't necessarily mean driver will use zpool_ops.evict - * in its implementation of zpool_driver.shrink. It could do internal - * defragmentation instead. - * - * Returns: true if potentially evictable; false otherwise. - */ -bool zpool_evictable(struct zpool *zpool) -{ - return zpool->driver->shrink; -} - /** * zpool_can_sleep_mapped - Test if zpool can sleep when do mapped. * @zpool: The zpool to test diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index c87a60514f21..316eb2784643 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -370,9 +370,7 @@ static void record_obj(unsigned long handle, unsigned long obj) #ifdef CONFIG_ZPOOL -static void *zs_zpool_create(const char *name, gfp_t gfp, - const struct zpool_ops *zpool_ops, - struct zpool *zpool) +static void *zs_zpool_create(const char *name, gfp_t gfp) { /* * Ignore global gfp flags: zs_malloc() may be invoked from @@ -439,7 +437,6 @@ static struct zpool_driver zs_zpool_driver = { .malloc_support_movable = true, .malloc = zs_zpool_malloc, .free = zs_zpool_free, - .shrink = NULL, .map = zs_zpool_map, .unmap = zs_zpool_unmap, .total_size = zs_zpool_total_size, diff --git a/mm/zswap.c b/mm/zswap.c index 80d7780bf066..f5fcb07a181d 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -248,10 +248,6 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle); static int zswap_pool_get(struct zswap_pool *pool); static void zswap_pool_put(struct zswap_pool *pool); -static const struct zpool_ops zswap_zpool_ops = { - .evict = zswap_writeback_entry -}; - static bool zswap_is_full(void) { return totalram_pages() * zswap_max_pool_percent / 100 < @@ -645,12 +641,8 @@ static void shrink_worker(struct work_struct *w) shrink_work); int ret, failures = 0; - /* zpool_evictable will be removed once all 3 backends have migrated*/ do { - if (zpool_evictable(pool->zpool)) - ret = zpool_shrink(pool->zpool, 1, NULL); - else - ret = zswap_shrink(pool); + ret = zswap_shrink(pool); if (ret) { zswap_reject_reclaim_fail++; if (ret != -EAGAIN) @@ -688,7 +680,7 @@ static struct zswap_pool *zswap_pool_create(char *type, char *compressor) /* unique name for each pool specifically required by zsmalloc */ snprintf(name, 38, "zswap%x", atomic_inc_return(&zswap_pools_count)); - pool->zpool = zpool_create_pool(type, name, gfp, &zswap_zpool_ops); + pool->zpool = zpool_create_pool(type, name, gfp); if (!pool->zpool) { pr_err("%s zpool not available\n", type); goto error; @@ -1375,8 +1367,7 @@ static int zswap_frontswap_store(unsigned type, pgoff_t offset, } } while (ret == -EEXIST); INIT_LIST_HEAD(&entry->lru); - /* zpool_evictable will be removed once all 3 backends have migrated*/ - if (entry->length && !zpool_evictable(entry->pool->zpool)) { + if (entry->length) { spin_lock(&entry->pool->lock); list_add(&entry->lru, &entry->pool->lru); spin_unlock(&entry->pool->lock); @@ -1482,8 +1473,7 @@ static int zswap_frontswap_load(unsigned type, pgoff_t offset, freeentry: spin_lock(&tree->lock); zswap_entry_put(tree, entry); - /* zpool_evictable will be removed once all 3 backends have migrated*/ - if (entry->length && !zpool_evictable(entry->pool->zpool)) { + if (entry->length) { spin_lock(&entry->pool->lock); list_move(&entry->lru, &entry->pool->lru); spin_unlock(&entry->pool->lock); From patchwork Mon Jun 5 08:54:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Domenico Cerasuolo X-Patchwork-Id: 103190 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2548476vqr; Mon, 5 Jun 2023 02:12:52 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4w4JXTJyM/eSxfT5dmMlJpcQsIUr2D6qj8ACPtgvTNaq5nW/chxY2dcAn/YBB+Y1i4aUXW X-Received: by 2002:a17:90a:192:b0:256:2fd4:e239 with SMTP id 18-20020a17090a019200b002562fd4e239mr2543107pjc.23.1685956371900; Mon, 05 Jun 2023 02:12:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685956371; cv=none; d=google.com; s=arc-20160816; b=QfjEnZbcBXk+uPaMCBtSYlPAgW1QYvq1hKPDuLwhYqnoTXxe5NWd8Mav55nLNhhd2z AIOJgU07Rezl+F+fG2K9pv41AaPTfIkPUnA+kZPVeEDiRAJtD+utbzYfoOvJ05tSytiH /ZatA0r8IFiQ6WveFoEIcRatPysXR21MIIS7LdREoRtxSHTn2HoIplUr0Ye7MaezIZ6B 8l2eNJQ5BHUv1WdQNwMGRET6mnBoAlsgXSf+HyakQfNSf3kd0jApwp4Kxh9I7Dbzglsz 9fJktKH0BKBh5s+6jwO3qQnGloZ962+2reveWenVe7y5zDSxbQYUfNDaC3Pe08SPCE7l i6LA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=quwnb7o3aU6jP4gVPkDbqqMh/7gvdqRQsVfv3CnuXWc=; b=g/20fRUO2CaFYF8ksk5g2K1oUMd8Dpc7SqoBS/XAYTX8T80zXdzqkcjKuVwVexmH0L AiwHV4tQAwyq56gQ8tgS+Q8JlsSoo2S0yJZZQcUkcK+vZgVcNhejO0NuVGxjUEhLByGd KAoqVYxXugi8j8oN0nef1gfl7Rn9pVHM4YYRycZLjzfL7EACM6MoZKo+ehDQF6NTdESC tMgVC4z8j0XRR3hhdN64BlTbReS+QCufBoXBLrRRmC4xWIN46Fuky2GzdDZpVp+er20d ME1+FZnvjT9JmgPepcDqGtitoAOHpOLyQO93ErESt0WxtwS1Shiq0OMnKCfyFPcoovad k9sA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=lIdeY+JO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x69-20020a638648000000b0054201a3f08dsi5172728pgd.592.2023.06.05.02.12.37; Mon, 05 Jun 2023 02:12:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=lIdeY+JO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230336AbjFEIzL (ORCPT + 99 others); Mon, 5 Jun 2023 04:55:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230352AbjFEIyh (ORCPT ); Mon, 5 Jun 2023 04:54:37 -0400 Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D15B4E3 for ; Mon, 5 Jun 2023 01:54:33 -0700 (PDT) Received: by mail-ej1-x62e.google.com with SMTP id a640c23a62f3a-97458c97333so544607666b.2 for ; Mon, 05 Jun 2023 01:54:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685955272; x=1688547272; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=quwnb7o3aU6jP4gVPkDbqqMh/7gvdqRQsVfv3CnuXWc=; b=lIdeY+JOrMPJQm2NIXIo1MYmyMtIRWijY0IeEfWTJqO6LXtLTk2YMpxE5cTdaGqYqY ZbAy3O2zvhlTKmMZWnBW0PyCqRzrTfDAfuSvBDd6UbI3CJC1cKQtAYUfFAv7Pme/FE75 46RG1OvtTDIDR5ZOoXRbOqVj2y6VexDO+4b/f5SM6LmP2fgpnVSsnSAt++H0j8rtAklc lMxAwEmgOQXoLTYL9VcZU2z52Wv9cC/XvttYf72xG4xXNdyhDTXGUs4yu4ekBauZbHFV Jb7x+YN4r/CX2cWq6YzrgVorap/HskToNeFwDwlIKDNC0nDOye/Fo4iGMPUz/mVgQqnI 79xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685955272; x=1688547272; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=quwnb7o3aU6jP4gVPkDbqqMh/7gvdqRQsVfv3CnuXWc=; b=FB3RV9it2L7thprlx+z6f+pPs6LpplQS5lNfqasx98JSoLGJ4h83tOF7j2DqierLKL s+5GBqs08MClPiujUFwOPYZhQRlhep67Z/oHpJal7xlklak9yMbQzRMkTLGLKk74rmMr QHzvj3GVIXLli1SnH3a9Mh3A6yZKzGWNewQTFQZ2S8t3mVz9WMHI2eCmGBdRwsySHaDc O8l0Z1q0cBhDfc/F+rkmvis5STEZ2SY83dDdTDs7vWSSdPBVcQgbuFBuw87PxmJr+yqT msnAAFbpv/8kdXcJZoKwFhKaWW0alFbCZLt0bmwO7ZrAhnQ1rVu0/VQutYYI3ZAIEJdg Ii/w== X-Gm-Message-State: AC+VfDzir/BoBPMTWS0NGqc3nCXiaF6ZmgOFkHSSk8H4Ht0M2hlAg3ul IR55yEoyuQ1CNOG14T/msus= X-Received: by 2002:a17:907:6089:b0:94e:afa6:299f with SMTP id ht9-20020a170907608900b0094eafa6299fmr6692778ejc.22.1685955272281; Mon, 05 Jun 2023 01:54:32 -0700 (PDT) Received: from lelloman-5950.homenet.telecomitalia.it (host-82-53-8-153.retail.telecomitalia.it. [82.53.8.153]) by smtp.gmail.com with ESMTPSA id lx5-20020a170906af0500b00965c6c63ea3sm4048915ejb.35.2023.06.05.01.54.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jun 2023 01:54:31 -0700 (PDT) From: Domenico Cerasuolo To: vitaly.wool@konsulko.com, minchan@kernel.org, senozhatsky@chromium.org, yosryahmed@google.com, linux-mm@kvack.org Cc: ddstreet@ieee.org, sjenning@redhat.com, nphamcs@gmail.com, hannes@cmpxchg.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, kernel-team@meta.com, Domenico Cerasuolo Subject: [RFC PATCH 6/7] mm: zswap: simplify writeback function Date: Mon, 5 Jun 2023 10:54:18 +0200 Message-Id: <20230605085419.44383-7-cerasuolodomenico@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230605085419.44383-1-cerasuolodomenico@gmail.com> References: <20230605085419.44383-1-cerasuolodomenico@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TVD_PH_BODY_ACCOUNTS_PRE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767853388451850687?= X-GMAIL-MSGID: =?utf-8?q?1767853388451850687?= Previously, in zswap, the writeback function was passed to zpool drivers for their usage in calling the writeback operation. However, since the drivers did not possess references to entries and the function was specifically designed to work with handles, the writeback process has been modified to occur directly within zswap. Consequently, this change allows for some simplification of the writeback function, taking into account the updated workflow. Signed-off-by: Domenico Cerasuolo --- mm/zswap.c | 50 +++++++++++--------------------------------------- 1 file changed, 11 insertions(+), 39 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index f5fcb07a181d..8ee30a989508 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -244,7 +244,8 @@ static bool zswap_has_pool; pr_debug("%s pool %s/%s\n", msg, (p)->tfm_name, \ zpool_get_type((p)->zpool)) -static int zswap_writeback_entry(struct zpool *pool, unsigned long handle); +static int zswap_writeback_entry(struct zswap_entry *entry, struct zswap_header *zhdr, + struct zswap_tree *tree); static int zswap_pool_get(struct zswap_pool *pool); static void zswap_pool_put(struct zswap_pool *pool); @@ -621,7 +622,7 @@ static int zswap_shrink(struct zswap_pool *pool) } spin_unlock(&tree->lock); - ret = zswap_writeback_entry(pool->zpool, lru_entry->handle); + ret = zswap_writeback_entry(lru_entry, zhdr, tree); spin_lock(&tree->lock); if (ret) { @@ -1028,16 +1029,14 @@ static int zswap_get_swap_cache_page(swp_entry_t entry, * the swap cache, the compressed version stored by zswap can be * freed. */ -static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) +static int zswap_writeback_entry(struct zswap_entry *entry, struct zswap_header *zhdr, + struct zswap_tree *tree) { - struct zswap_header *zhdr; - swp_entry_t swpentry; - struct zswap_tree *tree; - pgoff_t offset; - struct zswap_entry *entry; + swp_entry_t swpentry = zhdr->swpentry; struct page *page; struct scatterlist input, output; struct crypto_acomp_ctx *acomp_ctx; + struct zpool *pool = entry->pool->zpool; u8 *src, *tmp = NULL; unsigned int dlen; @@ -1052,25 +1051,6 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) return -ENOMEM; } - /* extract swpentry from data */ - zhdr = zpool_map_handle(pool, handle, ZPOOL_MM_RO); - swpentry = zhdr->swpentry; /* here */ - tree = zswap_trees[swp_type(swpentry)]; - offset = swp_offset(swpentry); - zpool_unmap_handle(pool, handle); - - /* find and ref zswap entry */ - spin_lock(&tree->lock); - entry = zswap_entry_find_get(&tree->rbroot, offset); - if (!entry) { - /* entry was invalidated */ - spin_unlock(&tree->lock); - kfree(tmp); - return 0; - } - spin_unlock(&tree->lock); - BUG_ON(offset != entry->offset); - /* try to allocate swap cache page */ switch (zswap_get_swap_cache_page(swpentry, &page)) { case ZSWAP_SWAPCACHE_FAIL: /* no memory or invalidate happened */ @@ -1104,12 +1084,12 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); dlen = PAGE_SIZE; - zhdr = zpool_map_handle(pool, handle, ZPOOL_MM_RO); + zhdr = zpool_map_handle(pool, entry->handle, ZPOOL_MM_RO); src = (u8 *)zhdr + sizeof(struct zswap_header); if (!zpool_can_sleep_mapped(pool)) { memcpy(tmp, src, entry->length); src = tmp; - zpool_unmap_handle(pool, handle); + zpool_unmap_handle(pool, entry->handle); } mutex_lock(acomp_ctx->mutex); @@ -1124,7 +1104,7 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) if (!zpool_can_sleep_mapped(pool)) kfree(tmp); else - zpool_unmap_handle(pool, handle); + zpool_unmap_handle(pool, entry->handle); BUG_ON(ret); BUG_ON(dlen != PAGE_SIZE); @@ -1142,9 +1122,6 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) zswap_written_back_pages++; spin_lock(&tree->lock); - /* drop local reference */ - zswap_entry_put(tree, entry); - /* * There are two possible situations for entry here: * (1) refcount is 1(normal case), entry is valid and on the tree @@ -1152,7 +1129,7 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) * because invalidate happened during writeback * search the tree and free the entry if find entry */ - if (entry == zswap_rb_search(&tree->rbroot, offset)) + if (entry == zswap_rb_search(&tree->rbroot, swp_offset(swpentry))) zswap_entry_put(tree, entry); spin_unlock(&tree->lock); @@ -1166,13 +1143,8 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) * if we get here due to ZSWAP_SWAPCACHE_EXIST * a load may be happening concurrently. * it is safe and okay to not free the entry. - * if we free the entry in the following put * it is also okay to return !0 */ - spin_lock(&tree->lock); - zswap_entry_put(tree, entry); - spin_unlock(&tree->lock); - return ret; } From patchwork Mon Jun 5 08:54:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Domenico Cerasuolo X-Patchwork-Id: 103192 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2550229vqr; Mon, 5 Jun 2023 02:17:22 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4UmuSw4mfmEU564gpxUjSrBTidTL5FeSOms1pjU5kHdofFxrpvlQuCWxtnNY6DrkkUcZiP X-Received: by 2002:a05:6a21:9993:b0:10a:a0e1:96fa with SMTP id ve19-20020a056a21999300b0010aa0e196famr2067505pzb.22.1685956641735; Mon, 05 Jun 2023 02:17:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685956641; cv=none; d=google.com; s=arc-20160816; b=ijv1apjZlJpeTdmYwbR90jDVUbyPYv5wRrfqBBFAqL7zp6JhmWSCe7k1NRQBEXoTce NbY0lgy/LoA1HslYxoM15ac4DrWQYkjvEtAwaWn08c6V3OzA76q1lGQ5tFzH9dG2y2AU AyU0gcgpLpGWz3paexLCdly+mxL50zZO7kKanwmijFNOBV0zXtxD3TFKWKwTtmSep/c5 bFsjTEGk4wmNKbPQBynqVymZutlzqu3+/dUFOldQeU6Z/6ZEgVz/sQMhC0Z7UefoBUVI UYyKOmmmFUcgXaWHbsOgGn95a8qOeAnXd8xLmiz9OuswTLNoFccCJlw1KgbQ+4PUSXv5 kCyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=SVmMhDTQ10trTXW64TT8b/Jb6v8KkbgopQstkowo3kg=; b=hwYZKoeQgQoNndvU27y8JzYkQn/fZZ3wNx8TXv+MVf7JDAmzl85Pf0MaEec7Xj8DfK ZYE9GTNuEokzcZHJBh3bQBwAVz2i68ugK2o6xt151k61Mm3qUGihcZdiTf6dql2XeETh mUjw9FC7zGSrBccdxBOpPFc0IYxHur9+JwbAtw5OrSXzxqYgMFCXeY+ARlCOgbY/uSha k5NPRrLeToygH2aWC0Y82A3nrvgBmV4oDCfNU1xMs9kjF6lBvFteBqiMWEGqfOV2+wu3 5ouF5W9KGQFEhXd0P0rIebMaKvKu9YjwM5qBVdqkBgH/CGMY7hmYMLu3Ny48ryAr06QN Us3A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=S2CBCc7S; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i2-20020a170902eb4200b001ab13c7505asi5071338pli.125.2023.06.05.02.17.09; Mon, 05 Jun 2023 02:17:21 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=S2CBCc7S; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229772AbjFEIzP (ORCPT + 99 others); Mon, 5 Jun 2023 04:55:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39572 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230358AbjFEIyh (ORCPT ); Mon, 5 Jun 2023 04:54:37 -0400 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 42F42F9 for ; Mon, 5 Jun 2023 01:54:35 -0700 (PDT) Received: by mail-ej1-x62b.google.com with SMTP id a640c23a62f3a-973f78329e3so727200166b.3 for ; Mon, 05 Jun 2023 01:54:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685955273; x=1688547273; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SVmMhDTQ10trTXW64TT8b/Jb6v8KkbgopQstkowo3kg=; b=S2CBCc7S1D5TyINJ4n6MaFi+ueeEVJ8MTbgRBpyzA+R0AbwW9KkfEixJ2MhYOpcYDc EAzBemMbxXn4g5200Ys+TM3sRyeU1du0s8LzJG3OgUmM9dcf1tP843V4g7u8V4TaB8dT BGB1V5Hxgysph/WHqi40ZrmvS1cLQz8mIClJ0uSU2hM07vA6YGgwW3TGcwgXgcYovTYI Q7zhl4VQcEztJemlxc839J3aRhDXbvc9F+kl9FHIwzD307hlwdFuGJWPBz6beiL6RsuA LPCO9roEfxcRcTYXrSaZvZ9jYL/X9BuxyjJdvY8YvaY69dIIJWnEmc/I33T1hFbTROyM tRvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685955273; x=1688547273; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SVmMhDTQ10trTXW64TT8b/Jb6v8KkbgopQstkowo3kg=; b=T9vsb5qiW5juslvb5jGZgu8bPWndTlLytle4FBqjNSzqOcCPmAiKqYl26Y4y2BUgo6 YYfjY9rhw48od1CwXzNn2G+cr9SuAAqhDD2Y1UlCI1nN5097tBrjH+cxc2ox5/n9yZEI hYPRnbiCNrNk7CblALK3/s5YvVK8eHLJ+7a84SOi2Z3yz5kD7+TCx+tA1b2F58ToS3bz wh1U2EOxa7PNTBWd+0xCJWoPp+aGzXO+MDpVR3+V5Puozu9K2GxmlbK16Mpj/wUQLRpb CfbPzkQTBWQqkHGmRDxq29a55xGjX0pULdltC51dB/t9k+nq4fh0uR90xT6zkGEPDRlF Sq7Q== X-Gm-Message-State: AC+VfDwxjdqs7cOK/wtPiLMgT+Eyj7xoEOG79pfKCYp4n88nGM903JGR Yz84jySyOmuJKwz9HnUTe98= X-Received: by 2002:a17:907:2cc4:b0:94f:36a0:da45 with SMTP id hg4-20020a1709072cc400b0094f36a0da45mr6012980ejc.29.1685955273472; Mon, 05 Jun 2023 01:54:33 -0700 (PDT) Received: from lelloman-5950.homenet.telecomitalia.it (host-82-53-8-153.retail.telecomitalia.it. [82.53.8.153]) by smtp.gmail.com with ESMTPSA id lx5-20020a170906af0500b00965c6c63ea3sm4048915ejb.35.2023.06.05.01.54.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jun 2023 01:54:33 -0700 (PDT) From: Domenico Cerasuolo To: vitaly.wool@konsulko.com, minchan@kernel.org, senozhatsky@chromium.org, yosryahmed@google.com, linux-mm@kvack.org Cc: ddstreet@ieee.org, sjenning@redhat.com, nphamcs@gmail.com, hannes@cmpxchg.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, kernel-team@meta.com, Domenico Cerasuolo Subject: [RFC PATCH 7/7] mm: zswap: remove zswap_header Date: Mon, 5 Jun 2023 10:54:19 +0200 Message-Id: <20230605085419.44383-8-cerasuolodomenico@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230605085419.44383-1-cerasuolodomenico@gmail.com> References: <20230605085419.44383-1-cerasuolodomenico@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767853671288266626?= X-GMAIL-MSGID: =?utf-8?q?1767853671288266626?= Previously, zswap_header served the purpose of storing the swpentry within zpool pages. This allowed zpool implementations to pass relevant information to the writeback function. However, with the current implementation, writeback is directly handled within zswap. Consequently, there is no longer a necessity for zswap_header, as the swp_entry_t can be stored directly in zswap_entry. Suggested-by: Yosry Ahmed Signed-off-by: Domenico Cerasuolo --- mm/zswap.c | 52 ++++++++++++++++++++++------------------------------ 1 file changed, 22 insertions(+), 30 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 8ee30a989508..093beaeaaebc 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -187,7 +187,7 @@ struct zswap_pool { */ struct zswap_entry { struct rb_node rbnode; - pgoff_t offset; + swp_entry_t swpentry; int refcount; unsigned int length; struct zswap_pool *pool; @@ -199,10 +199,6 @@ struct zswap_entry { struct list_head lru; }; -struct zswap_header { - swp_entry_t swpentry; -}; - /* * The tree lock in the zswap_tree struct protects a few things: * - the rbtree @@ -244,7 +240,7 @@ static bool zswap_has_pool; pr_debug("%s pool %s/%s\n", msg, (p)->tfm_name, \ zpool_get_type((p)->zpool)) -static int zswap_writeback_entry(struct zswap_entry *entry, struct zswap_header *zhdr, +static int zswap_writeback_entry(struct zswap_entry *entry, struct zswap_tree *tree); static int zswap_pool_get(struct zswap_pool *pool); static void zswap_pool_put(struct zswap_pool *pool); @@ -305,12 +301,14 @@ static struct zswap_entry *zswap_rb_search(struct rb_root *root, pgoff_t offset) { struct rb_node *node = root->rb_node; struct zswap_entry *entry; + pgoff_t entry_offset; while (node) { entry = rb_entry(node, struct zswap_entry, rbnode); - if (entry->offset > offset) + entry_offset = swp_offset(entry->swpentry); + if (entry_offset > offset) node = node->rb_left; - else if (entry->offset < offset) + else if (entry_offset < offset) node = node->rb_right; else return entry; @@ -327,13 +325,15 @@ static int zswap_rb_insert(struct rb_root *root, struct zswap_entry *entry, { struct rb_node **link = &root->rb_node, *parent = NULL; struct zswap_entry *myentry; + pgoff_t myentry_offset, entry_offset = swp_offset(entry->swpentry); while (*link) { parent = *link; myentry = rb_entry(parent, struct zswap_entry, rbnode); - if (myentry->offset > entry->offset) + myentry_offset = swp_offset(myentry->swpentry); + if (myentry_offset > entry_offset) link = &(*link)->rb_left; - else if (myentry->offset < entry->offset) + else if (myentry_offset < entry_offset) link = &(*link)->rb_right; else { *dupentry = myentry; @@ -592,7 +592,6 @@ static struct zswap_pool *zswap_pool_find_get(char *type, char *compressor) static int zswap_shrink(struct zswap_pool *pool) { struct zswap_entry *lru_entry, *tree_entry = NULL; - struct zswap_header *zhdr; struct zswap_tree *tree; swp_entry_t swpentry; int ret; @@ -605,10 +604,8 @@ static int zswap_shrink(struct zswap_pool *pool) } lru_entry = list_last_entry(&pool->lru, struct zswap_entry, lru); list_del_init(&lru_entry->lru); - zhdr = zpool_map_handle(pool->zpool, lru_entry->handle, ZPOOL_MM_RO); - tree = zswap_trees[swp_type(zhdr->swpentry)]; - zpool_unmap_handle(pool->zpool, lru_entry->handle); - swpentry = zhdr->swpentry; + swpentry = lru_entry->swpentry; + tree = zswap_trees[swp_type(lru_entry->swpentry)]; spin_unlock(&pool->lock); /* hold a reference from tree so it won't be freed during writeback */ @@ -622,7 +619,7 @@ static int zswap_shrink(struct zswap_pool *pool) } spin_unlock(&tree->lock); - ret = zswap_writeback_entry(lru_entry, zhdr, tree); + ret = zswap_writeback_entry(lru_entry, tree); spin_lock(&tree->lock); if (ret) { @@ -1029,10 +1026,10 @@ static int zswap_get_swap_cache_page(swp_entry_t entry, * the swap cache, the compressed version stored by zswap can be * freed. */ -static int zswap_writeback_entry(struct zswap_entry *entry, struct zswap_header *zhdr, +static int zswap_writeback_entry(struct zswap_entry *entry, struct zswap_tree *tree) { - swp_entry_t swpentry = zhdr->swpentry; + swp_entry_t swpentry = entry->swpentry; struct page *page; struct scatterlist input, output; struct crypto_acomp_ctx *acomp_ctx; @@ -1072,7 +1069,7 @@ static int zswap_writeback_entry(struct zswap_entry *entry, struct zswap_header * writing. */ spin_lock(&tree->lock); - if (zswap_rb_search(&tree->rbroot, entry->offset) != entry) { + if (zswap_rb_search(&tree->rbroot, swp_offset(entry->swpentry)) != entry) { spin_unlock(&tree->lock); delete_from_swap_cache(page_folio(page)); ret = -ENOMEM; @@ -1084,8 +1081,7 @@ static int zswap_writeback_entry(struct zswap_entry *entry, struct zswap_header acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); dlen = PAGE_SIZE; - zhdr = zpool_map_handle(pool, entry->handle, ZPOOL_MM_RO); - src = (u8 *)zhdr + sizeof(struct zswap_header); + src = zpool_map_handle(pool, entry->handle, ZPOOL_MM_RO); if (!zpool_can_sleep_mapped(pool)) { memcpy(tmp, src, entry->length); src = tmp; @@ -1192,11 +1188,10 @@ static int zswap_frontswap_store(unsigned type, pgoff_t offset, struct obj_cgroup *objcg = NULL; struct zswap_pool *pool; int ret; - unsigned int hlen, dlen = PAGE_SIZE; + unsigned int dlen = PAGE_SIZE; unsigned long handle, value; char *buf; u8 *src, *dst; - struct zswap_header zhdr = { .swpentry = swp_entry(type, offset) }; gfp_t gfp; /* THP isn't supported */ @@ -1241,7 +1236,7 @@ static int zswap_frontswap_store(unsigned type, pgoff_t offset, src = kmap_atomic(page); if (zswap_is_page_same_filled(src, &value)) { kunmap_atomic(src); - entry->offset = offset; + entry->swpentry = swp_entry(type, offset); entry->length = 0; entry->value = value; atomic_inc(&zswap_same_filled_pages); @@ -1295,11 +1290,10 @@ static int zswap_frontswap_store(unsigned type, pgoff_t offset, } /* store */ - hlen = sizeof(zhdr); gfp = __GFP_NORETRY | __GFP_NOWARN | __GFP_KSWAPD_RECLAIM; if (zpool_malloc_support_movable(entry->pool->zpool)) gfp |= __GFP_HIGHMEM | __GFP_MOVABLE; - ret = zpool_malloc(entry->pool->zpool, hlen + dlen, gfp, &handle); + ret = zpool_malloc(entry->pool->zpool, dlen, gfp, &handle); if (ret == -ENOSPC) { zswap_reject_compress_poor++; goto put_dstmem; @@ -1309,13 +1303,12 @@ static int zswap_frontswap_store(unsigned type, pgoff_t offset, goto put_dstmem; } buf = zpool_map_handle(entry->pool->zpool, handle, ZPOOL_MM_WO); - memcpy(buf, &zhdr, hlen); - memcpy(buf + hlen, dst, dlen); + memcpy(buf, dst, dlen); zpool_unmap_handle(entry->pool->zpool, handle); mutex_unlock(acomp_ctx->mutex); /* populate entry */ - entry->offset = offset; + entry->swpentry = swp_entry(type, offset); entry->handle = handle; entry->length = dlen; @@ -1415,7 +1408,6 @@ static int zswap_frontswap_load(unsigned type, pgoff_t offset, /* decompress */ dlen = PAGE_SIZE; src = zpool_map_handle(entry->pool->zpool, entry->handle, ZPOOL_MM_RO); - src += sizeof(struct zswap_header); if (!zpool_can_sleep_mapped(entry->pool->zpool)) { memcpy(tmp, src, entry->length);