From patchwork Thu Jul 27 16:22:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 127052 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a985:0:b0:3e4:2afc:c1 with SMTP id t5csp1219752vqo; Thu, 27 Jul 2023 09:31:46 -0700 (PDT) X-Google-Smtp-Source: APBJJlGxxucTk8LtnK6NyHHs4crNTLO6TQicmLN7W3bwJsS15qI88rWRti1YaU86B4WPVfYAKult X-Received: by 2002:a17:903:1112:b0:1bb:b832:4bed with SMTP id n18-20020a170903111200b001bbb8324bedmr3567210plh.20.1690475506555; Thu, 27 Jul 2023 09:31:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690475506; cv=none; d=google.com; s=arc-20160816; b=A/X8/dmyr7CE0piG6/zlPcVqsX+pp7lfL6flcbzqf3UiyQPTXGG6EIx8iRuG/GgD5G BdT1CBr1iplehWDgecYkpCAUcH9Mj6ZtJOs7EkfngSKJJeLhlDAdj2+R22gNkMjYOG5J TNT/kUcBxLiOaI78RXHCCahFvDPeDbQ3W1756ZaT+eF85gPKjMOokZFN9pYXofE9tA0Z WWGi97MB/wkLzy4yKZhS7GoqgXiLBXRxA4jQ+6qiK3kpe/ovEHeFqPg0y83aHx0rrxk7 8kZU8lJWKDHBWlmIiMF5tw3lzwOnfHI8EpHXFFybujVOik42QDO9LFobXC9OmyV7VM4+ 7Xsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=V4WqUpKSmXvGR1M+fIFBD/JXbqB10oByq1yE4svjvKk=; fh=DoWEzf7Q9nX9ab/egPFph1ANaSldApIC0eBRHr6e2qs=; b=DLg5ApuKLFwvINYSBsbY7d5xkVlZ+OFPLLqvb8hpDdiM9BifN4fhfrf29TgBENSCOl PBsUKAtNmDmDR5x7fBuGjJUXNbDZMZQV610X/TNxuEfke/hs3NdkpdHkPnFfumtKRBwo 6XQCl/S6gp5fZ3uUgrjOf8tnMpWyrvgyCZ742Jn9W1/e87D1E1/EIPypDO9pnKbxkBjT CogfcAPm8KTzIqucsRKWm3rtx0Qs/XZaVl0tUtCqXXWB8B7TuPHKqEESsuQvVKa3Zbew RnB+uST41beSfgcMlUnkX9s3Mon7Db1ogcT3ueZyqUQQ8OvJlYXhH0ERHRWuodKW6FlS Vvtg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=qNto7iFR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x20-20020a17090300d400b001b8b4a31a07si1429156plc.407.2023.07.27.09.31.33; Thu, 27 Jul 2023 09:31:46 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=qNto7iFR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234143AbjG0QXz (ORCPT + 99 others); Thu, 27 Jul 2023 12:23:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40420 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233922AbjG0QXv (ORCPT ); Thu, 27 Jul 2023 12:23:51 -0400 Received: from mail-qv1-xf36.google.com (mail-qv1-xf36.google.com [IPv6:2607:f8b0:4864:20::f36]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 45B5C2D4B for ; Thu, 27 Jul 2023 09:23:48 -0700 (PDT) Received: by mail-qv1-xf36.google.com with SMTP id 6a1803df08f44-63cf96c37beso5142926d6.0 for ; Thu, 27 Jul 2023 09:23:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20221208.gappssmtp.com; s=20221208; t=1690475027; x=1691079827; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=V4WqUpKSmXvGR1M+fIFBD/JXbqB10oByq1yE4svjvKk=; b=qNto7iFRrzOxESe9fv4KXfCHzvEpqtjq8zKP6G+FHbxknRdRQyruOVS/C6JV+mmMvY +/rWcuLtgsi9NpP8YgxSITdR8dLpz4uKoeqElIiF8b0B/yCc6cd24rpY8lAY+hhsYZCj nuB6QswJUQX/5foZILCFhUOhAhxdSuiL48uJ9//IKXR5DiNmxdHTFCXSUib5GYajdUMC BZX4uCMOPsz4wm4GIKg0pa4I8kmWdNTYMyl1/gbCDWOOuOnTP6xcrq/91YCR699FRai3 uD/9XMZLJawLy/Dvhly73UgklITjc1pFzK3nbx3xqKttOiuUSz1YJoFeSUFYn+hyS7zH wtww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690475027; x=1691079827; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=V4WqUpKSmXvGR1M+fIFBD/JXbqB10oByq1yE4svjvKk=; b=IzwlHxanvXS0f8sD+SMJGQh8RMXbBArVrUW9Xda/VY/w4nh7+1Br7ZmOd0Yr9w1aPh KZ+suPIpp68DQ+TN46QVNQSMATUF4QpiqLzpik8LSjp0lacfwUc95YO93fMwadsWSzcx R8AQdtr8Ly/hKhjmMDCh7LB7/5CFmlag15KnePE87nxaUT/ey92gAQ5N86jTr9bIbjPl CKQO9VlHVWLDQhPwb+4AReg5+Nox9dSF5qkvDLg86rVHTSk0NCiwI2/bEWDq2vfmikod 8XqVb5NyZkmYVEohrnifWnjnsSmAnisq9gThLYy8CSVZYxjnQmTYWJmUjOheSxhkaXxe OMpw== X-Gm-Message-State: ABy/qLZ9RnKyp1z3JSslzjpoG6Q9zUcGDtb6R3YSqq0HiK2QDzVawk/q LXx0xVLAOd0u8eNFTt6tjNVKJaD+3/6foktG00IaIg== X-Received: by 2002:a05:6214:d6a:b0:635:dbab:a588 with SMTP id 10-20020a0562140d6a00b00635dbaba588mr850652qvs.16.1690475027432; Thu, 27 Jul 2023 09:23:47 -0700 (PDT) Received: from localhost ([2620:10d:c091:400::5:ad06]) by smtp.gmail.com with ESMTPSA id s18-20020a0cb312000000b0063762ab2f90sm527074qve.83.2023.07.27.09.23.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Jul 2023 09:23:47 -0700 (PDT) From: Johannes Weiner To: Andrew Morton Cc: Yosry Ahmed , Nhat Pham , Domenico Cerasuolo , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/3] mm: zswap: use zswap_invalidate_entry() for duplicates Date: Thu, 27 Jul 2023 12:22:23 -0400 Message-ID: <20230727162343.1415598-2-hannes@cmpxchg.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230727162343.1415598-1-hannes@cmpxchg.org> References: <20230727162343.1415598-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772592044552459170 X-GMAIL-MSGID: 1772592044552459170 Minor cleanup. Instead of open-coding the tree deletion and the put, use the zswap_invalidate_entry() convenience helper. Suggested-by: Yosry Ahmed Signed-off-by: Johannes Weiner Reviewed-by: Yosry Ahmed --- mm/zswap.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 583ef7b84dc3..e123b1c7981c 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1344,9 +1344,7 @@ bool zswap_store(struct page *page) spin_lock(&tree->lock); while (zswap_rb_insert(&tree->rbroot, entry, &dupentry) == -EEXIST) { zswap_duplicate_entry++; - /* remove from rbtree */ - zswap_rb_erase(&tree->rbroot, dupentry); - zswap_entry_put(tree, dupentry); + zswap_invalidate_entry(tree, dupentry); } if (entry->length) { spin_lock(&entry->pool->lru_lock); From patchwork Thu Jul 27 16:22:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 127058 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a985:0:b0:3e4:2afc:c1 with SMTP id t5csp1225894vqo; Thu, 27 Jul 2023 09:42:37 -0700 (PDT) X-Google-Smtp-Source: APBJJlENeXmYQETcUiKUxapKT29aCbNsGRpi/dO2/tsADP7RVZgydlBUmtYRUqp6mTQvl7SGzE6q X-Received: by 2002:a17:906:77c3:b0:99b:d915:3c45 with SMTP id m3-20020a17090677c300b0099bd9153c45mr1849876ejn.39.1690476157050; Thu, 27 Jul 2023 09:42:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690476157; cv=none; d=google.com; s=arc-20160816; b=rvWskB3fatqdy/WfeU0pTT3l4Dat55WhCzjVlecW8b9PBkwgIziHU1irsctuuqt1Qa rw2Jaj1uSiBQZ5CqDnqEsvVhqrs9yaCyFg0BqYbhvHsT00W1OzHQJPOZ7FPPOgVUPfZl CwtAQD35hf3PaA8ioolPgSAta8sdZX/yoc8eqAXNJLfVhiuxq7JLTomYdJ64L/yu0syo boq+Qxhofwde7MfHyVcgOVnOofX1cB9hm/owf5GNlP4OVK02lbipSWngbb6PLE+9gIHJ She5ct272UO/D7SvMF9g7GlAA4bdJKwAFYwFQrz0rsp4GF4N/kB3AyjEBPRXKIiiQqAM xg5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=g6xeXNZ7MmGxs9MCe+qdgeE1O4rbj13SYexzHrnrzbc=; fh=DoWEzf7Q9nX9ab/egPFph1ANaSldApIC0eBRHr6e2qs=; b=OPLK+NdYbDwlpa7Ay5QRqv4IcKPg9dWMnhV+aJBegq42AyKfPU+fblKxJZrQrtQlyS 9KpRWhN3BBeNdBp7FdG3v4D0opONVBnVh64wcQCIhvC3/aE2Ov18QXVAPYrSQTJA0ijJ UP+4cZm6agB7oY2yBBG5T0oTs3JNgObsngVeMj+8OXfznPtXoiVlIhlq6HdgNz9aJU5m Jdi3QNixQQDDF+YgnW2oG2JAyndk/CXOwrdWRBHw5+I7I4zBr2erKnMrKhAXWmXQjVrD 3uY8UjnGfJthq2Gs9J9rSkzu2P59bdkUTZpOeaxLgcwQdpgOlNPFkYIyT32cBhXcs74e nEOA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=t1KwP89h; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k13-20020a170906970d00b0099ba3438550si1187823ejx.10.2023.07.27.09.42.11; Thu, 27 Jul 2023 09:42:37 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=t1KwP89h; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234286AbjG0QX7 (ORCPT + 99 others); Thu, 27 Jul 2023 12:23:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40396 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234003AbjG0QXv (ORCPT ); Thu, 27 Jul 2023 12:23:51 -0400 Received: from mail-qk1-x729.google.com (mail-qk1-x729.google.com [IPv6:2607:f8b0:4864:20::729]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D9E7D2D64 for ; Thu, 27 Jul 2023 09:23:49 -0700 (PDT) Received: by mail-qk1-x729.google.com with SMTP id af79cd13be357-768054797f7so101768685a.2 for ; Thu, 27 Jul 2023 09:23:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20221208.gappssmtp.com; s=20221208; t=1690475028; x=1691079828; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=g6xeXNZ7MmGxs9MCe+qdgeE1O4rbj13SYexzHrnrzbc=; b=t1KwP89hpnwg0NYhi1k1hMQB/3gQRSOWAy1zvJ7pueqsOL6lMhlnGs29bj9q3R7ed2 5TGX5AbXfBOKFrqHxQDMSSGFph1MqS/IIURzJrYtuyq5DJDNQSlPgyx6HKbsJCFrgyMs etJArP1Qo7jM62rDTvet0BjBWc9wIdYpICMtw4B6tf9ouYzlHD0s6flS1M2mtpzmsp3P Tg1utgnTtVxz7Ck+6yn7HuwnCpOeXrrLAxZP/B7IVeVZfU45bb/Ad1ZLt/8jOMuY4acP s9Ljx+sG57pZI23evKzyyIa5rmfHvAJu7nn7DIh1eF/R8VZWmTH5gwWvPzEBnh8xunF7 LIIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690475028; x=1691079828; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=g6xeXNZ7MmGxs9MCe+qdgeE1O4rbj13SYexzHrnrzbc=; b=NtSpgW7h4tLR1CtowqtNzfZVgxDUc2ptcNdn8nHCiDANe/vlmePDpaOjDWXMbhI9UH QGXWoY9uWcARYLoNLDoU5Imbr28ERcpEGHx5sMritglpLc5/JBtyaLUf0FXC/J5a32K4 PzpqxqWO6ldiEaOVuInPqW+YV5jCofGuC5Evof7zg91CH/TImOm3XxFEwpHekVW8xrT7 MLxIDbFMyQTo8K2Dh50hjbfVpXdSJPgv0/Trytc2sJmyD46JvdLAJIjRG3KMbj13PVg0 IylhqNKhBI/okHK1zRC8PLB012ZEebOzfKsxnbgqsQJldDk+E+IXork5d9NIT3gOERHH OQ4A== X-Gm-Message-State: ABy/qLZUwhfqzB54oYBno9DXP6qALgN2s/14eVj6mVWr+OtVK2+ksWaQ 2vB/HSA5LZ9EXI5indJgl4LMFA== X-Received: by 2002:a05:620a:ccf:b0:768:b0c:e4c1 with SMTP id b15-20020a05620a0ccf00b007680b0ce4c1mr434494qkj.27.1690475028555; Thu, 27 Jul 2023 09:23:48 -0700 (PDT) Received: from localhost ([2620:10d:c091:400::5:ad06]) by smtp.gmail.com with ESMTPSA id p13-20020ae9f30d000000b0076816153dcdsm505827qkg.106.2023.07.27.09.23.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Jul 2023 09:23:48 -0700 (PDT) From: Johannes Weiner To: Andrew Morton Cc: Yosry Ahmed , Nhat Pham , Domenico Cerasuolo , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/3] mm: zswap: tighten up entry invalidation Date: Thu, 27 Jul 2023 12:22:24 -0400 Message-ID: <20230727162343.1415598-3-hannes@cmpxchg.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230727162343.1415598-1-hannes@cmpxchg.org> References: <20230727162343.1415598-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772592727099563122 X-GMAIL-MSGID: 1772592727099563122 Removing a zswap entry from the tree is tied to an explicit operation that's supposed to drop the base reference: swap invalidation, exclusive load, duplicate store. Don't silently remove the entry on final put, but instead warn if an entry is in tree without reference. While in that diff context, convert a BUG_ON to a WARN_ON_ONCE. No need to crash on a refcount underflow. Signed-off-by: Johannes Weiner Reviewed-by: Yosry Ahmed --- mm/zswap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index e123b1c7981c..e34ac89e6098 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -402,9 +402,9 @@ static void zswap_entry_put(struct zswap_tree *tree, { int refcount = --entry->refcount; - BUG_ON(refcount < 0); + WARN_ON_ONCE(refcount < 0); if (refcount == 0) { - zswap_rb_erase(&tree->rbroot, entry); + WARN_ON_ONCE(!RB_EMPTY_NODE(&entry->rbnode)); zswap_free_entry(entry); } } From patchwork Thu Jul 27 16:22:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 127078 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a985:0:b0:3e4:2afc:c1 with SMTP id t5csp1249472vqo; Thu, 27 Jul 2023 10:21:57 -0700 (PDT) X-Google-Smtp-Source: APBJJlH5tMo9GHPqW97wG2ER4WV6vNw7r4CikYEUIzleNWBfbkFjHiG6Cw+Y2d7291mTbItIyvC+ X-Received: by 2002:a05:6a00:180d:b0:681:142f:e8e3 with SMTP id y13-20020a056a00180d00b00681142fe8e3mr7560330pfa.14.1690478517090; Thu, 27 Jul 2023 10:21:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690478517; cv=none; d=google.com; s=arc-20160816; b=rhdSHGOCIV8Tqc6jn4vY4EyVaVJhD3VScnisM0+tWJSu+LiAJzssHA24P50r1sPxOQ jcdFHy+f2FSQyFU7NT6T/v3cuuuZGYn1oQro77Bo4T+a4J+EYqIX8x4pychWZwjl4ZMc D3YvQ0vSVGKo84alEgfXLa+YdkpxugElGt6DGxaf/0S0Sei1COnUMKTjp675DyYENzID 8cWWJmwUQ3l+cLFIB339Yt/JA3zrA7mGKbUvUh4ZnJddICNjE38ls0ZabfGcGqw6hzBV +Ra/JcgKx3FnpmyuH3nh1BsKpiSg349qEjR7LDG9HMfGCV/fGemvHghu+8KqmXA5HUQp QG9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=9xKvUZAC/PcwLq4Mn3vj+fKTV8KIinkTpyaNVs2EYco=; fh=DoWEzf7Q9nX9ab/egPFph1ANaSldApIC0eBRHr6e2qs=; b=TrO+zRNEIE9MJ/FsSpXOzXlnpzl0jIyNl+VVXYnCXudXot/cC/nghWYoqULvSzDclG qPPjKx2NAPh/Rg47Mg4P2HdpP1oCOVzQR1cmd48jP0FTp9CobPt2TZKFobbxiPOpKc9/ 8fAuJjqqa9sv+tjNGlrBI076M/LG+gs46uM00rIbJS5a/dQgaQP0TwVi3uH+CK5kJyHD MTYpbwXL61h6bul7qT78MAlb1CFEisV+xQ4SSK4uZRrTkLzqlpPTM8UzwJER6Et8C0qD 60ngUXF054z0jBeDjmmiA+4o7BtrhGlQ2cR3E5qbWku951KY4MEjSSdmygtbFHelTxS+ QYhw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=ZM6siFmc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bt9-20020a056a00438900b00687080947e0si314489pfb.42.2023.07.27.10.21.43; Thu, 27 Jul 2023 10:21:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=ZM6siFmc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233954AbjG0QYC (ORCPT + 99 others); Thu, 27 Jul 2023 12:24:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40420 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234088AbjG0QXw (ORCPT ); Thu, 27 Jul 2023 12:23:52 -0400 Received: from mail-oa1-x2f.google.com (mail-oa1-x2f.google.com [IPv6:2001:4860:4864:20::2f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1C93B30ED for ; Thu, 27 Jul 2023 09:23:51 -0700 (PDT) Received: by mail-oa1-x2f.google.com with SMTP id 586e51a60fabf-1bb5ce75e43so808470fac.3 for ; Thu, 27 Jul 2023 09:23:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20221208.gappssmtp.com; s=20221208; t=1690475030; x=1691079830; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9xKvUZAC/PcwLq4Mn3vj+fKTV8KIinkTpyaNVs2EYco=; b=ZM6siFmcEpT22TCKTBC20dzc2m3blvxe+fV7xBWQ5V/W+B8u2sR51MOrUVn0oe9H9r p3SLhh1gZWJjZ41HzedjMK816QkKQbpYiYtUKKn82b4C9atpUvnedHQf+0jDaBf3j77Q tJOmmfcGqfxttLoXBrJCFjVGF8sW0Q572CZ16Ht4fa8chGgIb2Jw6/K1cT/xLpr/JkM5 r3iilsyakH1SBZfuqYO5hwJQP1WMnOayvcj1j6HQ6bVFu643j76ZwjEXMY25LUTF4q1y UHe6Ly/6TR285I3CJYLC6dr76PLwRPjFwQFb634smNKQW1NJjIxSKuDdZhMfZRorHggK kV/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690475030; x=1691079830; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9xKvUZAC/PcwLq4Mn3vj+fKTV8KIinkTpyaNVs2EYco=; b=Fjn64tsHhutAsxaz4eOb9uixzUsGm48tj1b4kdBW3Xs9cjs1AtDU/yTwJ9eXQB/DWr k59sdv2klcfHXPQguigtBbHdpRuE65X2Q9LtUr+QIq0VBYKSk9JeMpyKGRUQ94BOOUTh xxOMJ3JDNtqLCwpn9BWH2IGSCweO+IsV3Sbb0N85AQ3ApPaR2ar5CQhj0NR72M2H+MJP Iz5z26J2Lv2E86qfsjYLom+zrrzTWfZzqngco5CXswWX/4VHzK9oP+wONHO4XpU52f4K puk9z3+WKWQxlqYKvHXmDoITBgNThkNK0Y3C/6QCCYZFSJblPbQZbRm/W1AUqG6y340m nOlQ== X-Gm-Message-State: ABy/qLb6qqEYzYjNjk+GWzyjPuw1TQW4WGGn/3hP91wi/9kAil+RaTKJ 6z18JA+XkN1eIwIFkL8yp7uzAQ== X-Received: by 2002:a05:6871:593:b0:1bb:5ef8:d606 with SMTP id u19-20020a056871059300b001bb5ef8d606mr3430099oan.41.1690475029900; Thu, 27 Jul 2023 09:23:49 -0700 (PDT) Received: from localhost ([2620:10d:c091:400::5:ad06]) by smtp.gmail.com with ESMTPSA id x19-20020ac87ed3000000b004055106ee80sm519047qtj.44.2023.07.27.09.23.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Jul 2023 09:23:49 -0700 (PDT) From: Johannes Weiner To: Andrew Morton Cc: Yosry Ahmed , Nhat Pham , Domenico Cerasuolo , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/3] mm: zswap: kill zswap_get_swap_cache_page() Date: Thu, 27 Jul 2023 12:22:25 -0400 Message-ID: <20230727162343.1415598-4-hannes@cmpxchg.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230727162343.1415598-1-hannes@cmpxchg.org> References: <20230727162343.1415598-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772595201564489130 X-GMAIL-MSGID: 1772595201564489130 The __read_swap_cache_async() interface isn't more difficult to understand than what the helper abstracts. Save the indirection and a level of indentation for the primary work of the writeback func. Signed-off-by: Johannes Weiner Reviewed-by: Yosry Ahmed --- mm/zswap.c | 142 ++++++++++++++++++++--------------------------------- 1 file changed, 53 insertions(+), 89 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index e34ac89e6098..bba4ead672be 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1016,43 +1016,6 @@ static int zswap_enabled_param_set(const char *val, /********************************* * writeback code **********************************/ -/* return enum for zswap_get_swap_cache_page */ -enum zswap_get_swap_ret { - ZSWAP_SWAPCACHE_NEW, - ZSWAP_SWAPCACHE_EXIST, - ZSWAP_SWAPCACHE_FAIL, -}; - -/* - * zswap_get_swap_cache_page - * - * This is an adaption of read_swap_cache_async() - * - * This function tries to find a page with the given swap entry - * in the swapper_space address space (the swap cache). If the page - * is found, it is returned in retpage. Otherwise, a page is allocated, - * added to the swap cache, and returned in retpage. - * - * If success, the swap cache page is returned in retpage - * Returns ZSWAP_SWAPCACHE_EXIST if page was already in the swap cache - * Returns ZSWAP_SWAPCACHE_NEW if the new page needs to be populated, - * the new page is added to swapcache and locked - * Returns ZSWAP_SWAPCACHE_FAIL on error - */ -static int zswap_get_swap_cache_page(swp_entry_t entry, - struct page **retpage) -{ - bool page_was_allocated; - - *retpage = __read_swap_cache_async(entry, GFP_KERNEL, - NULL, 0, &page_was_allocated); - if (page_was_allocated) - return ZSWAP_SWAPCACHE_NEW; - if (!*retpage) - return ZSWAP_SWAPCACHE_FAIL; - return ZSWAP_SWAPCACHE_EXIST; -} - /* * Attempts to free an entry by adding a page to the swap cache, * decompressing the entry data into the page, and issuing a @@ -1073,7 +1036,7 @@ static int zswap_writeback_entry(struct zswap_entry *entry, struct scatterlist input, output; struct crypto_acomp_ctx *acomp_ctx; struct zpool *pool = entry->pool->zpool; - + bool page_was_allocated; u8 *src, *tmp = NULL; unsigned int dlen; int ret; @@ -1088,65 +1051,66 @@ static int zswap_writeback_entry(struct zswap_entry *entry, } /* try to allocate swap cache page */ - switch (zswap_get_swap_cache_page(swpentry, &page)) { - case ZSWAP_SWAPCACHE_FAIL: /* no memory or invalidate happened */ + page = __read_swap_cache_async(swpentry, GFP_KERNEL, NULL, 0, + &page_was_allocated); + if (!page) { ret = -ENOMEM; goto fail; + } - case ZSWAP_SWAPCACHE_EXIST: - /* page is already in the swap cache, ignore for now */ + /* Found an existing page, we raced with load/swapin */ + if (!page_was_allocated) { put_page(page); ret = -EEXIST; goto fail; + } - case ZSWAP_SWAPCACHE_NEW: /* page is locked */ - /* - * Having a local reference to the zswap entry doesn't exclude - * swapping from invalidating and recycling the swap slot. Once - * the swapcache is secured against concurrent swapping to and - * from the slot, recheck that the entry is still current before - * writing. - */ - spin_lock(&tree->lock); - if (zswap_rb_search(&tree->rbroot, swp_offset(entry->swpentry)) != entry) { - spin_unlock(&tree->lock); - delete_from_swap_cache(page_folio(page)); - ret = -ENOMEM; - goto fail; - } + /* + * Page is locked, and the swapcache is now secured against + * concurrent swapping to and from the slot. Verify that the + * swap entry hasn't been invalidated and recycled behind our + * backs (our zswap_entry reference doesn't prevent that), to + * avoid overwriting a new swap page with old compressed data. + */ + spin_lock(&tree->lock); + if (zswap_rb_search(&tree->rbroot, swp_offset(entry->swpentry)) != entry) { spin_unlock(&tree->lock); + delete_from_swap_cache(page_folio(page)); + ret = -ENOMEM; + goto fail; + } + spin_unlock(&tree->lock); - /* decompress */ - acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); - dlen = PAGE_SIZE; + /* decompress */ + acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); + dlen = PAGE_SIZE; - src = zpool_map_handle(pool, entry->handle, ZPOOL_MM_RO); - if (!zpool_can_sleep_mapped(pool)) { - memcpy(tmp, src, entry->length); - src = tmp; - zpool_unmap_handle(pool, entry->handle); - } + src = zpool_map_handle(pool, entry->handle, ZPOOL_MM_RO); + if (!zpool_can_sleep_mapped(pool)) { + memcpy(tmp, src, entry->length); + src = tmp; + zpool_unmap_handle(pool, entry->handle); + } - mutex_lock(acomp_ctx->mutex); - sg_init_one(&input, src, entry->length); - sg_init_table(&output, 1); - sg_set_page(&output, page, PAGE_SIZE, 0); - acomp_request_set_params(acomp_ctx->req, &input, &output, entry->length, dlen); - ret = crypto_wait_req(crypto_acomp_decompress(acomp_ctx->req), &acomp_ctx->wait); - dlen = acomp_ctx->req->dlen; - mutex_unlock(acomp_ctx->mutex); - - if (!zpool_can_sleep_mapped(pool)) - kfree(tmp); - else - zpool_unmap_handle(pool, entry->handle); + mutex_lock(acomp_ctx->mutex); + sg_init_one(&input, src, entry->length); + sg_init_table(&output, 1); + sg_set_page(&output, page, PAGE_SIZE, 0); + acomp_request_set_params(acomp_ctx->req, &input, &output, entry->length, dlen); + ret = crypto_wait_req(crypto_acomp_decompress(acomp_ctx->req), &acomp_ctx->wait); + dlen = acomp_ctx->req->dlen; + mutex_unlock(acomp_ctx->mutex); + + if (!zpool_can_sleep_mapped(pool)) + kfree(tmp); + else + zpool_unmap_handle(pool, entry->handle); - BUG_ON(ret); - BUG_ON(dlen != PAGE_SIZE); + BUG_ON(ret); + BUG_ON(dlen != PAGE_SIZE); - /* page is up to date */ - SetPageUptodate(page); - } + /* page is up to date */ + SetPageUptodate(page); /* move it to the tail of the inactive list after end_writeback */ SetPageReclaim(page); @@ -1157,16 +1121,16 @@ static int zswap_writeback_entry(struct zswap_entry *entry, zswap_written_back_pages++; return ret; + fail: if (!zpool_can_sleep_mapped(pool)) kfree(tmp); /* - * if we get here due to ZSWAP_SWAPCACHE_EXIST - * a load may be happening concurrently. - * it is safe and okay to not free the entry. - * it is also okay to return !0 - */ + * If we get here because the page is already in swapcache, a + * load may be happening concurrently. It is safe and okay to + * not free the entry. It is also okay to return !0. + */ return ret; }