From patchwork Sat Nov 19 00:15:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 23153 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp500206wrr; Fri, 18 Nov 2022 17:20:04 -0800 (PST) X-Google-Smtp-Source: AA0mqf7LGUy7NkylLMjGEIoyRpDdICh5Y6OMMcgUrTvYw5NC+BfJwEXKDgkeVYq2eW3wWEIjcP0J X-Received: by 2002:a17:906:dcc:b0:7ad:69eb:90c3 with SMTP id p12-20020a1709060dcc00b007ad69eb90c3mr8237165eji.285.1668820804303; Fri, 18 Nov 2022 17:20:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668820804; cv=none; d=google.com; s=arc-20160816; b=fp3M/0eJ8OcVs1TA3tYChQku372FQugI8KyBaitjIBhyvfNDgFIWjqBZz/gmo4LLWH rJJE2kifEt7Nq1cKArnoXYwVFj3PWBQkNuANzYm4ekGU74hwxdERKyybGp4zJ/0hyRVV z8CaS2jOTd3SsvmlsrLi7tETiyMQ+5WrcyCqw+ekxc/HQCr+pjHMTv1xKYGw5fWS5sYM MghH2zIxwt+x4p+FMnaqe3aS0OtKQ1YK5IJ3hQKtLL2a2Wzak/khQpjFMIRRPw78c6en SMYVEkTAKIqVw2uyWqldsq/hDejv2Xe+8GNe3HlrhJws4dfrYFtIvAfsTmXy5cUn6YqY tWyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=uhF53/xh3Hyv0X6zFlismYLwseOE8BPL7QAyLJFmpTSf8IuO/fFKgTY1stcJpxyVjh CMB3PJwaUmSRy+gCptamFjvbeNt4o6rrwfU2OhpdgeFqafwExDiSYfNhKOhFa8Zy+RVQ t1p3V+EqyaOamflKM6F5VP6pYVznXc0TKpKR0R9P3RgQn+67C/TqTb3pWtqEQNa2Z2QL kbaN3Goa9y3DBuU5y5I89SnNHMWwts0yz83gJmBTcRYcWFhXNzwkQNSNfyvmgqtLNr14 6ruBuZiP3juIp6IsfF224spdYAN4ZhmRsUJFM6FZvJvbREgcCQ44RC7erdS+omTnonKy lGGQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=NHkIkh8R; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id et19-20020a170907295300b0078db6f5f4b2si905612ejc.546.2022.11.18.17.19.40; Fri, 18 Nov 2022 17:20:04 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=NHkIkh8R; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229862AbiKSBSj (ORCPT + 99 others); Fri, 18 Nov 2022 20:18:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49788 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231645AbiKSBSM (ORCPT ); Fri, 18 Nov 2022 20:18:12 -0500 Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BCF61169C19 for ; Fri, 18 Nov 2022 16:15:39 -0800 (PST) Received: by mail-pj1-x1030.google.com with SMTP id v4-20020a17090a088400b00212cb0ed97eso6500902pjc.5 for ; Fri, 18 Nov 2022 16:15:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=NHkIkh8RaR25rj7+Wlw0rbe1vzjooV8Fsu/f07L3zCG5lUFzcdTGGBtND54rMrvFn2 1nztmuMnmKeDkLPZsShRcJCn8KfZDSS2xoMxkJ+8eWbfouubZv8PETaQAHxCEEQyuRcP 7dtpmnLSJFtricXu29dRABoMD0uTm0Xv+9F3R8Z0zOZfzFHEh/jZfd89vph1ur+jnvoi qe4O0YDtzONH4Dze1Yr/u8DHfwdnbozJEOewCLH0RyGcNrdx/Q/rLMIXB5yDuzL+iHp+ RKiSsuMYn6PCIoOSsyr2vmZRFTqbXCde8fgnZhLJQEkH9ydslMP7E4Ww+Oh0mh53K7e2 lK5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=G5R/8R7YXRr8brXvoLTTrFp6NL8YOyaF9a3oOeBBVQesgcjrHbvBd06hEDdKntOA85 PqaxQkBshUbcCwNnC756/KHa+oOLas/kvw0wBgPHuUDTZMHxq18NcWuZdslxP1gVPoBP qetMNhRozlBedAVIifm5p7mvgG2cTDBjHgWkmfH/SFHRNaRmSdDPkZERo8Lo581c++Qr y+hbtPmQvlVdMNja+L8Yh9qCJkSyp8KqTZepd2UMM6QHWv6Y+24scRW4SJ/2uKz2d3ih x74wIYldEZkfHLvZTmYUy0K3H5I3LwHujwNGf8Mg7XUN+OB7iQMl5EuS86YenVzKtugr SUgw== X-Gm-Message-State: ANoB5pmD5A42mbvOeA23Wr5wq7b+bLsKiBUIQufB6fqefL7K3FxfHArz GnkuTwbRcDBE0uZXHUp5aCI= X-Received: by 2002:a17:902:9a03:b0:186:9f20:e7e2 with SMTP id v3-20020a1709029a0300b001869f20e7e2mr1578912plp.174.1668816939044; Fri, 18 Nov 2022 16:15:39 -0800 (PST) Received: from localhost (fwdproxy-prn-010.fbsv.net. [2a03:2880:ff:a::face:b00c]) by smtp.gmail.com with ESMTPSA id x22-20020a170902821600b0017f48a9e2d6sm4202332pln.292.2022.11.18.16.15.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 16:15:38 -0800 (PST) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, minchan@kernel.org, ngupta@vflare.org, senozhatsky@chromium.org, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com Subject: [PATCH v6 1/6] zswap: fix writeback lock ordering for zsmalloc Date: Fri, 18 Nov 2022 16:15:31 -0800 Message-Id: <20221119001536.2086599-2-nphamcs@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221119001536.2086599-1-nphamcs@gmail.com> References: <20221119001536.2086599-1-nphamcs@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749885443750716068?= X-GMAIL-MSGID: =?utf-8?q?1749885443750716068?= From: Johannes Weiner zswap's customary lock order is tree->lock before pool->lock, because the tree->lock protects the entries' refcount, and the free callbacks in the backends acquire their respective pool locks to dispatch the backing object. zsmalloc's map callback takes the pool lock, so zswap must not grab the tree->lock while a handle is mapped. This currently only happens during writeback, which isn't implemented for zsmalloc. In preparation for it, move the tree->lock section out of the mapped entry section Signed-off-by: Johannes Weiner Signed-off-by: Nhat Pham --- mm/zswap.c | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-) -- 2.30.2 diff --git a/mm/zswap.c b/mm/zswap.c index 2d48fd59cc7a..2d69c1d678fe 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -958,7 +958,7 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) }; if (!zpool_can_sleep_mapped(pool)) { - tmp = kmalloc(PAGE_SIZE, GFP_ATOMIC); + tmp = kmalloc(PAGE_SIZE, GFP_KERNEL); if (!tmp) return -ENOMEM; } @@ -968,6 +968,7 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) swpentry = zhdr->swpentry; /* here */ tree = zswap_trees[swp_type(swpentry)]; offset = swp_offset(swpentry); + zpool_unmap_handle(pool, handle); /* find and ref zswap entry */ spin_lock(&tree->lock); @@ -975,20 +976,12 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) if (!entry) { /* entry was invalidated */ spin_unlock(&tree->lock); - zpool_unmap_handle(pool, handle); kfree(tmp); return 0; } spin_unlock(&tree->lock); BUG_ON(offset != entry->offset); - src = (u8 *)zhdr + sizeof(struct zswap_header); - if (!zpool_can_sleep_mapped(pool)) { - memcpy(tmp, src, entry->length); - src = tmp; - zpool_unmap_handle(pool, handle); - } - /* try to allocate swap cache page */ switch (zswap_get_swap_cache_page(swpentry, &page)) { case ZSWAP_SWAPCACHE_FAIL: /* no memory or invalidate happened */ @@ -1006,6 +999,14 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); dlen = PAGE_SIZE; + zhdr = zpool_map_handle(pool, handle, ZPOOL_MM_RO); + src = (u8 *)zhdr + sizeof(struct zswap_header); + if (!zpool_can_sleep_mapped(pool)) { + memcpy(tmp, src, entry->length); + src = tmp; + zpool_unmap_handle(pool, handle); + } + mutex_lock(acomp_ctx->mutex); sg_init_one(&input, src, entry->length); sg_init_table(&output, 1); @@ -1015,6 +1016,11 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) dlen = acomp_ctx->req->dlen; mutex_unlock(acomp_ctx->mutex); + if (!zpool_can_sleep_mapped(pool)) + kfree(tmp); + else + zpool_unmap_handle(pool, handle); + BUG_ON(ret); BUG_ON(dlen != PAGE_SIZE); @@ -1045,7 +1051,11 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) zswap_entry_put(tree, entry); spin_unlock(&tree->lock); - goto end; + return ret; + +fail: + if (!zpool_can_sleep_mapped(pool)) + kfree(tmp); /* * if we get here due to ZSWAP_SWAPCACHE_EXIST @@ -1054,17 +1064,10 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) * if we free the entry in the following put * it is also okay to return !0 */ -fail: spin_lock(&tree->lock); zswap_entry_put(tree, entry); spin_unlock(&tree->lock); -end: - if (zpool_can_sleep_mapped(pool)) - zpool_unmap_handle(pool, handle); - else - kfree(tmp); - return ret; }