From patchwork Tue Jan 30 01:36:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 193781 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp942810dyb; Mon, 29 Jan 2024 17:45:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IGc1YQtYQZv12KQzZaiTFq5reC+GgrkC2JFcwjSQDgJRbWx8e1eDO+gNdDhejugww2GT8TN X-Received: by 2002:a05:620a:28ca:b0:784:a63:6566 with SMTP id l10-20020a05620a28ca00b007840a636566mr824081qkp.10.1706579134718; Mon, 29 Jan 2024 17:45:34 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706579134; cv=pass; d=google.com; s=arc-20160816; b=vuRxibxZkA/hnnJluSXELHQIN6fxg8bF1QL9qEMcWpRHSP7hIQku+DYKJqvZdEc3ny yzeVIQgjVqw6O5kAZbiqZ3eMzcJ47B2cImfsQuITU4G+qFUQzl9kdiz/s+Z4SMZCpyxc Y3B0d7vOp3OEh0mVhzmQEeCqv0tuC5Cs0Ddg+qHKQW+MTZ13lpCRwNcX1sGQ2OQ0h27g kQvYfEvVYoYUK33zupSYQY7rTmJWg/Z9NfrY8kaFRp3o5ObFedAK3LaJiqWPv1J9uemp sk/1t0tHkQCFJ8PAYYalsmNXm5wfGfISIcqeTHUAXfBb8AU19Qvhtu4yrg6+txMxhaU3 ne7g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=6SQ/aPKt2RcQwCRNk0SkwceZy4++62pAYToaUxPVif8=; fh=pEdtrChUTDmvuLILGSavwskaPZjBcM3e1cB9O4e7o4U=; b=OBs99+dH1WQMarO+xRGFNTkq+pnNAs9u6+X+Fj9lM5c2dsr6W2XeYDtPapxmLZH4e/ bsj8OOmcekKDC7sQjGx20cqp39nB0FhCqE1+nQa8MAo6/NK3g5doYFxDGLq1CgW4vVLs +69Asc8HScxeCiJLvaK4abWN4jPqg7glQ16Hr+P0FruJsVk3CbuEAb0uxNEeQ9Lr+dlj lhvkQk3DoeuCVh0lH7MsAAxmpn41/BeMQoAEwQZlbXsvfwwvJiAHpLFglXx7Jxj1N3e5 trIZasY3gCo/TGxWJOtwicb5dgJoCKSbjNonnd5FGnckzKSzZ6A9HyordgUm9H16963t Xm/g== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=z3i79o8n; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43751-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43751-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id p4-20020a05620a056400b00783de6351b4si7786335qkp.41.2024.01.29.17.45.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:45:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43751-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=z3i79o8n; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43751-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43751-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 67F881C22153 for ; Tue, 30 Jan 2024 01:45:34 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 95DA553802; Tue, 30 Jan 2024 01:42:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b="z3i79o8n" Received: from mail-qk1-f178.google.com (mail-qk1-f178.google.com [209.85.222.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 753FB383B1 for ; Tue, 30 Jan 2024 01:42:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578951; cv=none; b=REhr7txdjstltC/4GowZNPj7bUdYCSvLfmu1UvjaWALz0FKtSGe1sALapdP4X83QR2wz+eUgeZDToJxwFM36JQi6xix5Vhd/xrNBWvABIF3CEWqCZAqAm4KoB/gNcX1Yo+EfsD1ZqTy6lIM6s+xT8/3xNjHYgI2/v4KSUnPs06I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578951; c=relaxed/simple; bh=Buqaio9vCxFs4jxwDb/cl7MtdsrmjbRE2WvaIIBaHdo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rPuCNpyO/we+3UaFKKF9C+KeYCshCSZUyn5q4bmu6Uq5iFXiXuCP+vr9u/FukglXXbgV8EbEJH8lijhY8FxswCHCjKZEGzyXJ/1Hml95q2gWc59RQZTxK+2r/dZTXs4yVZ9H6V2iKRMPxOy6yA+8NTyu09Cks7P5cgenj+e1HKQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b=z3i79o8n; arc=none smtp.client-ip=209.85.222.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Received: by mail-qk1-f178.google.com with SMTP id af79cd13be357-783f27300b1so177391485a.1 for ; Mon, 29 Jan 2024 17:42:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578948; x=1707183748; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6SQ/aPKt2RcQwCRNk0SkwceZy4++62pAYToaUxPVif8=; b=z3i79o8nEJUB6l4W4nAwTvXxdxY32XVqrTRRbBxjXfrIA3vU7LpxTlc5BPPJYF+9f1 F2gangf9C+CkScZi0XH+iaoNnXUAMcye/6n4k2JpYQO3xH+qHug3HyABV6J+cDPplu8c xIQiJfEPic9Vvel8CaNFXe4C92Z8F3PvNNHhHe0GNqiwLHB2/invZRpfRQ38tKvDhQPP 4xz3s70UmzvgVmyGZZsNR+xIHh7hcnswn//i6mJy14M3iIiU6EsnHxUU/GaU1m0/gOlk QQalqxkyfVRV31bD3+3g1118sxQ+YtxeP2WbLDt4vMJVR7NO4gEnEgNgOo5+zrKk34mJ kuCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578948; x=1707183748; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6SQ/aPKt2RcQwCRNk0SkwceZy4++62pAYToaUxPVif8=; b=QlGVRB4Z4isW7727lOf7Q3Njb/n3qTFpzBrPgvy4F7O+bsfKZYT18jiUkZopoVtncX P1kRM2e8RYtW9ICs8zspmvtP5Et3TAN/mSvhFnDNTVtnO4OWMEjEm8oYfVaSEoCjaJZP VeZC+l2G4kNLHUo4EPaZJqz8wC6cjPmWBoNRi0Dr/gqteluAmdfCjiCsnOEh6I7+NdIu KbOXP6fTtor0G9ulN2xfY9k49J7bUdQVvPc/WQlXYG2YiZElKBJaAWkv14NJcLEXIap9 59ZGXJBF3ZcSguGceHF8EexQeoMbbfAf32D7y6ASmzEe4GukRaMT0c/OwatHKiWPU0tZ PDXA== X-Gm-Message-State: AOJu0Yz6kVCGVRyz4BKRerNMzqhLmf+4u6OYjK7K6mSQ4RcC1OSTu7rr 5EPWtSkRCuqKDT/vZm+eHexFY+Rejf/IYbqfE3TGG57qaf/osHeR78Yl0hm7DkQynk4his5iI+P B X-Received: by 2002:ac8:5994:0:b0:42a:b152:c0fe with SMTP id e20-20020ac85994000000b0042ab152c0femr1228897qte.31.1706578948436; Mon, 29 Jan 2024 17:42:28 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id ie7-20020a05622a698700b00427fabefe3bsm4023363qtb.52.2024.01.29.17.42.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:28 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 12/20] mm: zswap: function ordering: zswap_pools Date: Mon, 29 Jan 2024 20:36:48 -0500 Message-ID: <20240130014208.565554-13-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789477922841438088 X-GMAIL-MSGID: 1789477922841438088 Move the operations against the global zswap_pools list (current pool, last, find) to the pool section. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 150 ++++++++++++++++++++++++++--------------------------- 1 file changed, 73 insertions(+), 77 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 33775f2224b7..168afd6767b3 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -517,6 +517,79 @@ static void zswap_pool_put(struct zswap_pool *pool) kref_put(&pool->kref, __zswap_pool_empty); } +static struct zswap_pool *__zswap_pool_current(void) +{ + struct zswap_pool *pool; + + pool = list_first_or_null_rcu(&zswap_pools, typeof(*pool), list); + WARN_ONCE(!pool && zswap_has_pool, + "%s: no page storage pool!\n", __func__); + + return pool; +} + +static struct zswap_pool *zswap_pool_current(void) +{ + assert_spin_locked(&zswap_pools_lock); + + return __zswap_pool_current(); +} + +static struct zswap_pool *zswap_pool_current_get(void) +{ + struct zswap_pool *pool; + + rcu_read_lock(); + + pool = __zswap_pool_current(); + if (!zswap_pool_get(pool)) + pool = NULL; + + rcu_read_unlock(); + + return pool; +} + +static struct zswap_pool *zswap_pool_last_get(void) +{ + struct zswap_pool *pool, *last = NULL; + + rcu_read_lock(); + + list_for_each_entry_rcu(pool, &zswap_pools, list) + last = pool; + WARN_ONCE(!last && zswap_has_pool, + "%s: no page storage pool!\n", __func__); + if (!zswap_pool_get(last)) + last = NULL; + + rcu_read_unlock(); + + return last; +} + +/* type and compressor must be null-terminated */ +static struct zswap_pool *zswap_pool_find_get(char *type, char *compressor) +{ + struct zswap_pool *pool; + + assert_spin_locked(&zswap_pools_lock); + + list_for_each_entry_rcu(pool, &zswap_pools, list) { + if (strcmp(pool->tfm_name, compressor)) + continue; + /* all zpools share the same type */ + if (strcmp(zpool_get_type(pool->zpools[0]), type)) + continue; + /* if we can't get it, it's about to be destroyed */ + if (!zswap_pool_get(pool)) + continue; + return pool; + } + + return NULL; +} + /* should be called under RCU */ #ifdef CONFIG_MEMCG static inline struct mem_cgroup *mem_cgroup_from_entry(struct zswap_entry *entry) @@ -938,83 +1011,6 @@ static int zswap_cpu_comp_dead(unsigned int cpu, struct hlist_node *node) return 0; } -/********************************* -* pool functions -**********************************/ - -static struct zswap_pool *__zswap_pool_current(void) -{ - struct zswap_pool *pool; - - pool = list_first_or_null_rcu(&zswap_pools, typeof(*pool), list); - WARN_ONCE(!pool && zswap_has_pool, - "%s: no page storage pool!\n", __func__); - - return pool; -} - -static struct zswap_pool *zswap_pool_current(void) -{ - assert_spin_locked(&zswap_pools_lock); - - return __zswap_pool_current(); -} - -static struct zswap_pool *zswap_pool_current_get(void) -{ - struct zswap_pool *pool; - - rcu_read_lock(); - - pool = __zswap_pool_current(); - if (!zswap_pool_get(pool)) - pool = NULL; - - rcu_read_unlock(); - - return pool; -} - -static struct zswap_pool *zswap_pool_last_get(void) -{ - struct zswap_pool *pool, *last = NULL; - - rcu_read_lock(); - - list_for_each_entry_rcu(pool, &zswap_pools, list) - last = pool; - WARN_ONCE(!last && zswap_has_pool, - "%s: no page storage pool!\n", __func__); - if (!zswap_pool_get(last)) - last = NULL; - - rcu_read_unlock(); - - return last; -} - -/* type and compressor must be null-terminated */ -static struct zswap_pool *zswap_pool_find_get(char *type, char *compressor) -{ - struct zswap_pool *pool; - - assert_spin_locked(&zswap_pools_lock); - - list_for_each_entry_rcu(pool, &zswap_pools, list) { - if (strcmp(pool->tfm_name, compressor)) - continue; - /* all zpools share the same type */ - if (strcmp(zpool_get_type(pool->zpools[0]), type)) - continue; - /* if we can't get it, it's about to be destroyed */ - if (!zswap_pool_get(pool)) - continue; - return pool; - } - - return NULL; -} - static enum lru_status shrink_memcg_cb(struct list_head *item, struct list_lru_one *l, spinlock_t *lock, void *arg) {