From patchwork Tue Jan 30 01:36:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 193774 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp941947dyb; Mon, 29 Jan 2024 17:43:02 -0800 (PST) X-Google-Smtp-Source: AGHT+IFf4G7sbOAnwu//xeF3yTwN/u5I536T7qbjuWTUnu+o4hGKSOTaWG6ML7C+UcVIo6eySLxW X-Received: by 2002:a17:906:13d9:b0:a35:fdf9:e7e4 with SMTP id g25-20020a17090613d900b00a35fdf9e7e4mr1315882ejc.20.1706578982280; Mon, 29 Jan 2024 17:43:02 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706578982; cv=pass; d=google.com; s=arc-20160816; b=npMF/nGHBZIOUPw83ZKLesL9yUHxuBfebfn0X1RjcNmAXxqBbfrkYMXT7VikqogRNk AXS+US8UYw7kf7Tnep51/aoLL+/GoBfeJuB81EfLa/1rPs/CbQfUrYxLASbzZqLgjQLF mfLmhpI5Y6UE3IZeQEyNbfafE0zzg5doHw5uAUGjBkKL2eW7uxMbMrdtM0DKcMkb9fHk mqQgmyDrlV6je62OpbkY+Lki9bXKOY8JxETtucJlsyfl7otyS3fbJ9/VYa7H34GqeIDw 0puFg00R+5WUWp/XRIHUFKxWIgLIAXuK/gZeRbie2/dibz2U60IOjfi/lUwwOdiHLW+Q KY0w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=shLTg8CbwWEN2YIGT7dz17BRQIDu5G+tg/uQhre6QiQ=; fh=pEdtrChUTDmvuLILGSavwskaPZjBcM3e1cB9O4e7o4U=; b=KKmKy56GUZ52Ft2IlmNuky386hgn8L8wtCmpBjScceh1y0hBQyzDLRR0d4mnqK2mwU vxHAWQ3BJp6mOsjSR6Fxhst7ls7MmIM9XicUqMidPGqCNkHtF4nNpFozVbugUnbCEDm5 hz8N483RXo/fkK8RQxa4CAUzQJDFh8Ez8YzjYlzIc+YnD2LNBlB8T0kdGw+7hiEkdQNO NegCHVtC5LfIN+zvkYuaEs3MXXudsJQ3zQQ0CQatIbhkClInVSOS7wLBAN5EJl8v15Fa OknfzFrqKhJOTZgdHdPeR5sCm54LeHGJxGhjcBilph8k8TJRT6wdKG6vF0XOqjTcgnDy GMfQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=uqwkN1v1; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43740-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43740-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id u9-20020a17090617c900b00a35645ca34bsi2462010eje.1043.2024.01.29.17.43.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:43:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43740-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=uqwkN1v1; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43740-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43740-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id B7A311F281F2 for ; Tue, 30 Jan 2024 01:43:01 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7636436B1E; Tue, 30 Jan 2024 01:42:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b="uqwkN1v1" Received: from mail-qk1-f181.google.com (mail-qk1-f181.google.com [209.85.222.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 712DB36119 for ; Tue, 30 Jan 2024 01:42:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578938; cv=none; b=HFTz2lj4e5sxqaT+YtzO5bD1a6w6c5boL4Bg3Nyhr4pZTCVMLOejgJJ6dZW9RnA2lUaKKyclRig1YsuQBOa/sBpAe1eoBio33U6AK2p6rij1P/YG5CcJ1qJtdbCzeBuJXTzX8HSOAjplw0CyZFnaV1FiW5yZh3MwXCjdAUnch4w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578938; c=relaxed/simple; bh=T3SlR01LXPCA/d3uGBn94fFn0fTJP74UII1LlEvglvk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XnOTX6zO3qyBX2iwv+mlWeQrh8S/u3t7q6+fbT6V9X4R8+PUrlKWhH0Pp5anxeoid4FNFDIQk3kTn5TCpinuhdaEkkjT6h/rJazV8SKUo3AwgZnnFLwl+cHGduqkN6jyZq4tP21EeEeXsgb9mXG/rB5M4cfPQAQT+OWO7jyaoFc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b=uqwkN1v1; arc=none smtp.client-ip=209.85.222.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Received: by mail-qk1-f181.google.com with SMTP id af79cd13be357-783045e88a6so349529385a.0 for ; Mon, 29 Jan 2024 17:42:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578935; x=1707183735; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=shLTg8CbwWEN2YIGT7dz17BRQIDu5G+tg/uQhre6QiQ=; b=uqwkN1v1f/8NSvrP3loCd23fQEdQvj6q8b3PNfpd6P8CXL6t+K3KsWDj/mvpEeKorG N+9zOED8eaRXbhcLh72HC3Srw1r+ovOlDi7mmEpzvDKR0aNKINej2PFD58ii9hCLlI0s nZrpYfljr4u2nKLZ1rn+zWHgQ9QlcB8PpA4ysZzM0KVUHdwMGwo8YanGUGgeBYykL1PF fhB1oP5EIQ+8EXo2LiKdsWIjiZwaENB9sx7jtn69KQkfjTXh10DU/Dp5b08BK3ZJzkz0 WBojLNgo4VHAXZtUXjLcqiTmVThUr4suzMhK7+Q+kdjDEOGqJPEg2qcYAbMxG+pFfevY FJtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578935; x=1707183735; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=shLTg8CbwWEN2YIGT7dz17BRQIDu5G+tg/uQhre6QiQ=; b=EzcrHkPQF7sEYrRc1eyhgMRMlMGKGGNJqks/if3IhlCBhQ21RKMGSOG/boS4hIgEWY xgIsqeT53TQMifs3y4898eicjcXnjMjLFCQ1T74cu/HzTG/aue9S6YSLGEebV3KlrgLa c1oS4sc23ka37PIG/uY+jajgL0F+RZiWdsf+YUQ9Bb5atczDTikzZi//fydUfpd5b8X1 FlVRFBb4gvAz1txVRHlMNvzaIwL5zcbF8uXdUOVkK7RfCAq2A6CpAMTejGprsgPuMRN2 YLLup2vJmi4KDCWaNicfBjF5ybwd/osm05rGvGNCRyjj+Mxthz6cgneMBl4LGaxf9Tk0 YiPQ== X-Gm-Message-State: AOJu0YyVQA3U+2yHeqUojFSObCwOWW5xq+QQmcTA6AqEa9AyrrPHpiJx VsYx0TBAByZxeKk/INi3xipGzPKM7gqvdTa+hpxjIHssClzfeNk5BtqKVOBhZfTA7Cs9xcLio0u c X-Received: by 2002:a05:620a:15ae:b0:783:8a70:f64 with SMTP id f14-20020a05620a15ae00b007838a700f64mr6881421qkk.65.1706578935257; Mon, 29 Jan 2024 17:42:15 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id o26-20020a05620a111a00b00784046b7868sm794053qkk.90.2024.01.29.17.42.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:15 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 01/20] mm: zswap: rename zswap_free_entry to zswap_entry_free Date: Mon, 29 Jan 2024 20:36:37 -0500 Message-ID: <20240130014208.565554-2-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789477762873675592 X-GMAIL-MSGID: 1789477762873675592 There is a zswap_entry_ namespace with multiple functions already. Signed-off-by: Johannes Weiner Reviewed-by: Chengming Zhou Acked-by: Yosry Ahmed Acked-by: Nhat Pham --- mm/zswap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 7f88b3a77e4a..173f2e6657de 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -520,7 +520,7 @@ static struct zpool *zswap_find_zpool(struct zswap_entry *entry) * Carries out the common pattern of freeing and entry's zpool allocation, * freeing the entry itself, and decrementing the number of stored pages. */ -static void zswap_free_entry(struct zswap_entry *entry) +static void zswap_entry_free(struct zswap_entry *entry) { if (!entry->length) atomic_dec(&zswap_same_filled_pages); @@ -555,7 +555,7 @@ static void zswap_entry_put(struct zswap_entry *entry) WARN_ON_ONCE(refcount < 0); if (refcount == 0) { WARN_ON_ONCE(!RB_EMPTY_NODE(&entry->rbnode)); - zswap_free_entry(entry); + zswap_entry_free(entry); } } From patchwork Tue Jan 30 01:36:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 193775 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp942007dyb; Mon, 29 Jan 2024 17:43:14 -0800 (PST) X-Google-Smtp-Source: AGHT+IHUQOfMCBmFfpyMWoVS1XCVEqOxutVDT0SSWtqh/1q+ws6EkK+LpuYp05LF1N+ppeYtV+5j X-Received: by 2002:a05:6358:4403:b0:175:9073:d6c with SMTP id z3-20020a056358440300b0017590730d6cmr8057193rwc.20.1706578994355; Mon, 29 Jan 2024 17:43:14 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706578994; cv=pass; d=google.com; s=arc-20160816; b=fyP5DUGwfR4vf0DoAApwDpiWLo1n4taxyDyNi+q8/MdQ5b/mXryI6CS0eGKlEa3s1s RY7oWiGsPDaMIBYvD+IJbcZp/CA7QB3FU+YIDbRoRpkthoT2OPGYDblIX5IgVGWLKzY0 rg3BR9G9B++HMEm20N7zLYq+akrLlm7l4c3CDs30GUa63jocx09S3wTI4UnFO3ovpcWn TBvIiXxcGTHepy3vZYTHFRYD45ZH4ZA/6N6rYbyICJZ7pj516x940fzJVfRZ6/IFdlcQ wlyDcxgcAW/fyLqSWRAtfICNVUuYGdiWvJyLVn4MAsvDm7CFozVcQ7mAKOaPlHTQ08pk upkA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=XjDB9WMZenyNFfJ6FneP+eMAzd2uIdtYL+Q7hbWifG0=; fh=pEdtrChUTDmvuLILGSavwskaPZjBcM3e1cB9O4e7o4U=; b=u5zor0bYe36kpJVs/Xaxt/Re4S+Hz/0kGuTMQBOa0PdLNeTjO4LpkaAWX6R9XqDPxg xeTWAcw+zi++lvW1q/aIJi+9m0aPVztrpH7o3KRNzhwQNQzK6K+Ca/Hd2yH4moBT0qE5 DX5djGFFO+t6dgdgN2V15YgAOZi0mPykGLHzN2gZYzcdqgd5MzV7Go94PgsvXCX+XSBU bpR3O6anqHHxy9qkVkywW7r0fwKeyjkmQ6AOqgqdaxWxbwdXkjGAcydobzuh/rkJ8LCW ZROLLvxxBAdcVp/25RpWt6OijdoSJTGw1pi7IrSWAwYHX7QyN0w10td7pCHOQ51t53b4 6Xeg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=AHQnVhmf; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43741-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43741-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id az16-20020a17090b029000b0028e8527b20bsi7400931pjb.154.2024.01.29.17.43.14 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:43:14 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43741-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=AHQnVhmf; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43741-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43741-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 1FBEB28351D for ; Tue, 30 Jan 2024 01:43:14 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 60C8A374C9; Tue, 30 Jan 2024 01:42:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b="AHQnVhmf" Received: from mail-qk1-f172.google.com (mail-qk1-f172.google.com [209.85.222.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 69AF63611F for ; Tue, 30 Jan 2024 01:42:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578939; cv=none; b=LSCL1VQFe9DzhQ33GvyJt//i8gLebVo739elpmm+42OEPmYAV2c8ACf4jl/NSNnvj5plHBolYXcCcMOZ19KtoiBeEhHanN3hifRxqRTp1RYeaGjvdZbbd/oB1kjiehV1mNTdC8gRxFYZvBKbc29SadOlanxhOgwbrEeZ3KzOTDA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578939; c=relaxed/simple; bh=noTUjFASFtVL4nlWHNtBmau/aPj35kRvuT1b3oVbdds=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ticA+TNKMA4HOSB1dmjjnFbGBmBC+CjvpBVxYqRF+tBvcDSM0Q8ZHmsRMoAfhk/u1geYg9GkNoZF1LOqg9mxPIxMVf2QOLlbhVzP8GlXyI9OCvfcfH7++pJrN5HmgUOia2C1dPKjdwGIvrM468mjI9BpkQiRUZoY0kR//zR3G2U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b=AHQnVhmf; arc=none smtp.client-ip=209.85.222.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Received: by mail-qk1-f172.google.com with SMTP id af79cd13be357-78405c9a152so63648485a.1 for ; Mon, 29 Jan 2024 17:42:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578936; x=1707183736; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XjDB9WMZenyNFfJ6FneP+eMAzd2uIdtYL+Q7hbWifG0=; b=AHQnVhmfY/h2MSmZfnwrS6AYxkZk5EaAGvdfE38xjFK8XKBZlDE+qovV+t0qQzPL9s /vDe7m7wjXt709NpxVHlYnWfXQvvkfBct0TKRyRaDdYW7oMuFOgPUIdj+ZhlIoPW/7+h jMva+3EKv4QkIij+RymXA7615Y95jZiXt/ZW0UotZSLSa9uDWI7eRv+tgFzYwbiWmWEY kUwSvHqZW17L6kK43DwfUzdUmhb41hI4WCmqpp6RsZTqmB7eFoZ0grAnbm20sEszWqxA GsosJoiiBqZwQUe6e+L4t+XuQnJ0k0VS86IW0BTETz18/ZIKpr+vtn/nS0IIuU+JEz+B NtnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578936; x=1707183736; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XjDB9WMZenyNFfJ6FneP+eMAzd2uIdtYL+Q7hbWifG0=; b=masuGMnK3+F55a5AVXBLMF7R/lMUx/J4WlJsAIxrtTD9YVdtIaumzRNCF/gV1ZJkM1 Yp0Qrm6oQVfge9N6I0wxFc73JWCs1/K5SsHkIAAuByx9q24rXun02JNxAuvGXBGcAVDm v5kIdwMiXXyhkoddECAYu68Kn0Q9iBOgAxNZs55iRJW6iFqhjtR1ocB7uS3GgxHNTlyy CrXqnD755vG5dZ7XjyXDwv8tNFomGSHBqWH1KD40vafx/QBFvjekVodPIl5ftFyDw6GY /z1Ye/iCvMhHn/F+EZRaJClBY0czptKtDVRrj3w6wVHEGJepJUp3+oVxwLke4HPc/qDX nL7A== X-Gm-Message-State: AOJu0Yw76JVSBQdjd+GKSRRKAZX68Ymecf0yfnOspzwZolabwuakmmyF CmCQ03Vb/wWRr+tZ7fjuCirvSaoVu3JgKC2cGryim2KjmFEOlHxBx9clZPoJgPg= X-Received: by 2002:ad4:5c4d:0:b0:686:ac5b:fe7d with SMTP id a13-20020ad45c4d000000b00686ac5bfe7dmr9546558qva.125.1706578936382; Mon, 29 Jan 2024 17:42:16 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id u12-20020a0cea4c000000b0068c43ac393asm2518373qvp.146.2024.01.29.17.42.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:16 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 02/20] mm: zswap: inline and remove zswap_entry_find_get() Date: Mon, 29 Jan 2024 20:36:38 -0500 Message-ID: <20240130014208.565554-3-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789477775677196511 X-GMAIL-MSGID: 1789477775677196511 There is only one caller and the function is trivial. Inline it. Signed-off-by: Johannes Weiner Reviewed-by: Chengming Zhou Acked-by: Yosry Ahmed Reviewed-by: Nhat Pham --- mm/zswap.c | 17 ++--------------- 1 file changed, 2 insertions(+), 15 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 173f2e6657de..cf864aaa214d 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -559,19 +559,6 @@ static void zswap_entry_put(struct zswap_entry *entry) } } -/* caller must hold the tree lock */ -static struct zswap_entry *zswap_entry_find_get(struct rb_root *root, - pgoff_t offset) -{ - struct zswap_entry *entry; - - entry = zswap_rb_search(root, offset); - if (entry) - zswap_entry_get(entry); - - return entry; -} - /********************************* * shrinker functions **********************************/ @@ -1708,13 +1695,13 @@ bool zswap_load(struct folio *folio) VM_WARN_ON_ONCE(!folio_test_locked(folio)); - /* find */ spin_lock(&tree->lock); - entry = zswap_entry_find_get(&tree->rbroot, offset); + entry = zswap_rb_search(&tree->rbroot, offset); if (!entry) { spin_unlock(&tree->lock); return false; } + zswap_entry_get(entry); spin_unlock(&tree->lock); if (entry->length) From patchwork Tue Jan 30 01:36:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 193776 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp942056dyb; Mon, 29 Jan 2024 17:43:25 -0800 (PST) X-Google-Smtp-Source: AGHT+IHQ/HlBStZe2zyisWZlC+SJW+Bi2Ue8jcwwRgUIsHEulwhPMFG62TTTDZfTAepRvP9BACuB X-Received: by 2002:a17:906:fa9a:b0:a35:e22a:9d15 with SMTP id lt26-20020a170906fa9a00b00a35e22a9d15mr2135850ejb.8.1706579004968; Mon, 29 Jan 2024 17:43:24 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706579004; cv=pass; d=google.com; s=arc-20160816; b=g5I7Ub8FliKmhCuxK1URH8NIAQgXR6irrxLS18NgwwC9NFJBegkE1TeZVD1rQHqOsH YoIGumjOgBfEOIYZ6BC4w9LT4TTmPXEvPMHLV/JJY7XKFYB4vcjLYZiwG6tMvJggkC+6 JLCU66K1Gh+pg8P9wvhjiP8yYfIu3AVqHWVg3xbnx00B/wdpD+xpRf7sdU4+Qy45I9+B zhHazzWgRUiyz5od2kHyyr3Pc9VKs+4bynAMH5qOefjweAFyFpBKYivHw0HolpzA2mzg D51/6cUe/3xIKel9R+4eB4ILghNIv/WxA1ZBuaN8IqJcCCLEEtP96onTiKu/8GtHrK8t h4hw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=2V/6lwbRMGjS57LQdA40055rq8F7L2NGNcAA5/p/KKk=; fh=pEdtrChUTDmvuLILGSavwskaPZjBcM3e1cB9O4e7o4U=; b=A0XbzzW6na4RtLLizU3o8CYccGQGWOXou/dhlL3zVJERCeQKqVKVW/yS1mXglhR4gB 05bU6r6kKq7+SgZo320W4pHsR11yjicXn9ucfANb0/fXb/6ppUUESPneqInGmdSbQq2U zZTIzoc4JTFKIiLBRrsyFxj3ofEKBz28lykjf/lmdgo3o7vjXnIgAvuoUV8JxCmB6BVH DUht49UVnp9POs5tL5tZSBvlMSI2ZEDKJTDwjruhDj47Rfp6IeNJCjFvdZA0pOWtHik8 RDabvPKWj4nFAD/fyNYOxdC8Ijq/aYZUDZMifvi3DwRL9OVSjXNcbMn9CpGvQyIwsLx1 SWDw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b="1v/W6LFA"; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43742-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43742-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id z21-20020a170906241500b00a35e5bef438si925645eja.126.2024.01.29.17.43.24 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:43:24 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43742-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b="1v/W6LFA"; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43742-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43742-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 68B751F282E0 for ; Tue, 30 Jan 2024 01:43:24 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 03613374F7; Tue, 30 Jan 2024 01:42:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b="1v/W6LFA" Received: from mail-qv1-f51.google.com (mail-qv1-f51.google.com [209.85.219.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 96A0E2E40E for ; Tue, 30 Jan 2024 01:42:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578940; cv=none; b=E7aZpF/XxsU6BEFXpSgVW7L9wzs/DCoVQ40KtXFJd5FEy0hcg1M2LbosCpbgR0vLZHA2bcjrRlKnkUyrZFrQCaRtClfC0OkgaXOI7tG58tWdOKAeqrjVA/4dYNWgnpUGv8cqFc5xL0rmCcAdzIHlMlFNXnO3oa35PJGvAJGwfls= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578940; c=relaxed/simple; bh=afmGsTSHwwdohMS3QuarqOP8HJNNtAFIbuu/jsQLzdA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LjR7Lnpi9MyMrNi3dqhJaeuq2N/vad8INtpYRJIe2dkKVHYy+TKOROweNlDAFRgYf85ZIWHe9BO9mh6LL7Y89vy5Uus2Frn1Iwxbe/cO+dwvwSWPIQA/WehzQMpNRId33/K/aIC2cnHhue6X6XUEjrtB6LEZCtaSkWPAbPbbE7g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b=1v/W6LFA; arc=none smtp.client-ip=209.85.219.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Received: by mail-qv1-f51.google.com with SMTP id 6a1803df08f44-68c495ba558so13027686d6.3 for ; Mon, 29 Jan 2024 17:42:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578937; x=1707183737; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2V/6lwbRMGjS57LQdA40055rq8F7L2NGNcAA5/p/KKk=; b=1v/W6LFAo2HF9vrO2b7+cNDfYX51YQGsbS7IRT0rpoOf/NVcusSA/fQcbQxGlsCFLs FMDY+x/Kehmjn3akRLktAqny/oQuNdIzhU2vLNmZmaYmAUbM9xPOgyFFYO3cSQvmDJ++ r/lQ8X2uikRQeCFJ3eZLrQwQ69950oyq87m+5sKHKV4veTRX0wg0t+JPXaxFKMr82kAC 4RNA1/iHFvMw/h5clPmMvBCDw0C/T3b/11/71RyUsztQH5XR4WCUd4XQTcDV9OzpVz1U AkQa3NLsPSnhghYcA+SiaYaUKbldR8kRMhykstIo1LWvdIZHlwr67QZkRVO+D7UqXIEe m5qQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578937; x=1707183737; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2V/6lwbRMGjS57LQdA40055rq8F7L2NGNcAA5/p/KKk=; b=W1zPyxCNluIOEcTFMqGY4qbCx3p18+rCwEu6CtSWedDr/Vu0a2elHgcsbqT6cEfVHv xQEBK1xASfHF/YcE/07WFe88EPA2wm6RTMpbYAxkWYQHqymkiPIuzvtdcEQxEI78GWRH cEvuhH4rOb5+4L/1MT1fVv0jDlX25WIFXhP4vNNB/D4XuCVmnMp38gnFANODM3qUTI4p Ep/bcydZhQ0O+XSvSRBbb7grWjiNG4pZA6CYGdHWstKens7AqgC4JpeRjeYNsyofFk7f m70Qhy43/RX8npK9vOiRePJLnFUs0yJkroroA/I5yttaze44ml4afHeH/UjvaxAn0HBg 6+Wg== X-Gm-Message-State: AOJu0Yw4MFD71iuEBGok29+pcrdf8dIC4BXkgKXxUlipjovbJBwCkywp uprSsoGQk/IbMNIXXZaQ0a0jNnYYZiv5IOnKpiA5vZha2GWX2DTIIruCkHEofTkR6xpZ2phRNUO V X-Received: by 2002:ad4:4ea7:0:b0:680:815a:44d8 with SMTP id ed7-20020ad44ea7000000b00680815a44d8mr7696476qvb.130.1706578937523; Mon, 29 Jan 2024 17:42:17 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id 11-20020a0562140dcb00b0068c5116af11sm1223091qvt.142.2024.01.29.17.42.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:17 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 03/20] mm: zswap: move zswap_invalidate_entry() to related functions Date: Mon, 29 Jan 2024 20:36:39 -0500 Message-ID: <20240130014208.565554-4-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789477787086377095 X-GMAIL-MSGID: 1789477787086377095 Move it up to the other tree and refcounting functions. Signed-off-by: Johannes Weiner Reviewed-by: Chengming Zhou Acked-by: Yosry Ahmed Reviewed-by: Nhat Pham --- mm/zswap.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index cf864aaa214d..9f05282efe3c 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -559,6 +559,18 @@ static void zswap_entry_put(struct zswap_entry *entry) } } +/* + * If the entry is still valid in the tree, drop the initial ref and remove it + * from the tree. This function must be called with an additional ref held, + * otherwise it may race with another invalidation freeing the entry. + */ +static void zswap_invalidate_entry(struct zswap_tree *tree, + struct zswap_entry *entry) +{ + if (zswap_rb_erase(&tree->rbroot, entry)) + zswap_entry_put(entry); +} + /********************************* * shrinker functions **********************************/ @@ -809,18 +821,6 @@ static struct zswap_pool *zswap_pool_find_get(char *type, char *compressor) return NULL; } -/* - * If the entry is still valid in the tree, drop the initial ref and remove it - * from the tree. This function must be called with an additional ref held, - * otherwise it may race with another invalidation freeing the entry. - */ -static void zswap_invalidate_entry(struct zswap_tree *tree, - struct zswap_entry *entry) -{ - if (zswap_rb_erase(&tree->rbroot, entry)) - zswap_entry_put(entry); -} - static enum lru_status shrink_memcg_cb(struct list_head *item, struct list_lru_one *l, spinlock_t *lock, void *arg) { From patchwork Tue Jan 30 01:36:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 193777 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp942151dyb; Mon, 29 Jan 2024 17:43:41 -0800 (PST) X-Google-Smtp-Source: AGHT+IEEEf3ZpSfayhAQkGSYrjXdS0NexZgDPINm5ghOijyHn1apXfWzCXms3VBAstv8aURJEQfv X-Received: by 2002:a05:620a:14ad:b0:783:dfe5:3393 with SMTP id x13-20020a05620a14ad00b00783dfe53393mr5703724qkj.55.1706579021022; Mon, 29 Jan 2024 17:43:41 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706579021; cv=pass; d=google.com; s=arc-20160816; b=DeESz0FlWN34YUt6qfHjDcqcHSgXgX296Tq3ehkn2uaN60zVrbGlqm+zGm6kmVGux6 elzNFhnDt92NROkpuAqmK84GBCyMPRg+OdvYEf15s/k7/vWaUjnGFBv7Rci2VxscC/48 tWm/5Mr+KWMmrev7YokXSb1QSYL2wHeZeHs1M+tWAt+VoMSrzOnFFkHsr9Jaoo1LCXH3 sZDJA7r5E8eoh0cIOz2/aVuvPwGG9tmo32ZE4X7+niPieY7vIJ2T/f7zZUuOEei8n6bA cHpBXpb49vnmoG9mOt+zGShMG0bSZiPKoZpgKhmM0MtypARulILYY66TtqUihWol7J2o Gb1g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=sMNhWdXKrsGXbbTndUjwna9Z9KC3QLUt3pRgP5ry3e4=; fh=pEdtrChUTDmvuLILGSavwskaPZjBcM3e1cB9O4e7o4U=; b=WRprhfBboMwdIu/ZUEcQyZyYgs5wuIVLCdHAO72tPv16i9I3Ss1ORxvljrx/iYdkVW noQ3rWM/hVsZMqKVyFAmv/g1SnfzgQ6hG0p4DeiNJDpCfMASwb5WnDs/uZFvfSpNIfJP 2Pqhp4BqcOM58gWhDuRSZIHI0UFRX9tY3wCFuzrsWItQUTVHSemZ/Q37cc/HjSs+zRQH s9CRwCOm3N7q1OTocPUypI6PMK1ken79FdKXgKzkwywy/a/ZuQJlFv4/S+JEDNDtdpr5 2Cq3hmpEyIvrDhQ5yWU9I8uxJTVa5zwVs49Z7IzBQKo4pYuSa1ta4I5o6O1Hcxj2iDM6 yoOQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=Qj2APR78; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43743-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43743-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id b16-20020a05620a04f000b0078403a5e00csi2092166qkh.252.2024.01.29.17.43.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:43:41 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43743-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=Qj2APR78; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43743-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43743-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id C62C21C230A2 for ; Tue, 30 Jan 2024 01:43:40 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 243BE37702; Tue, 30 Jan 2024 01:42:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b="Qj2APR78" Received: from mail-qt1-f175.google.com (mail-qt1-f175.google.com [209.85.160.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B5DA7364AB for ; Tue, 30 Jan 2024 01:42:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578941; cv=none; b=Ql32flTg9L4cUrt8DhyOH3+4m9RyJVwpxE31afw5idBVOBeni6eeVBAUSpG0xOx+4hkYZ5mkBrUDc2eU12g8Y3c2KRLIyG7QWJ3A027+f2pSC2Pfd0aP9YMzCXrCO8N8c4TIgL3E1cY5IJwhSKX9UFaQ+TM7R84K6FofZI1+3Zg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578941; c=relaxed/simple; bh=B/uwPD5Q03PavcbhkfXKr63FB/8V2Nu3ZADVd0o9EXg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ihYlk3hS9AgsRgcph8AC/PO9/vYgWUNVnRjWqwKxqGxm1HV0YIBOZ8BnwegoWreGNopsj8U/Mcu2GoVUizUZSWQJ/TlAPfGZpVhA2yr+Ha98KtyEbM0Rp5Ip3VWER6qRGlfKSpSAwH053/oc1OA3qn/0grhRSEAL9iSvLs5m/10= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b=Qj2APR78; arc=none smtp.client-ip=209.85.160.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Received: by mail-qt1-f175.google.com with SMTP id d75a77b69052e-42aa4a9d984so17707211cf.1 for ; Mon, 29 Jan 2024 17:42:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578938; x=1707183738; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sMNhWdXKrsGXbbTndUjwna9Z9KC3QLUt3pRgP5ry3e4=; b=Qj2APR7869JDVbAwY87Hbsq0yccZ48zJzMFRA7aBMPTH/WxACmuLobcpJ3nlmnpOC5 2Bm1hZAdV6IZHJjH+abZtMC+9F+EQBVzWmzZXTZxIG3B/Bv1TYXeK5gpKiFeIGn7z/pY XhhTRdOM9CiQw59DrIp5adMqNoRsMbXhEI6G+8fFOrsaKjloVBaZS2HNgWH74Vml+ygw 8osBB959psK4Zj9ThIYJG3ir2iVJsWvRxtOeeslfWqczBfhyWGXI9goxsZnokGGeB+iK Gad8LFLVgh/eFrslMVP0nU5d6GjAVAmO/gkE9xcSTHjEJmIDWD0nVlcgvPOGvmFqNhet Cteg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578938; x=1707183738; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sMNhWdXKrsGXbbTndUjwna9Z9KC3QLUt3pRgP5ry3e4=; b=fotArQiWn6lsGJUjjyAv8fIK75ROolFvZajDZVn2kQkTcG8qnoq/IY20g9E1p4mHy/ v9bEEqgTQrf8OhLYTHj/dCQPCtpVeYOqjAWkloUwNA6vMdQyJDKoUxogvMXaMCQEW58O FQm+yr+qg9QVmmIt6cNfPvg+NXlirlWn1sEGUo86VKfZete/DK7Hg7t2VUD1eQ9r4oXy DEKFCkTgRp8qILG78BuBxG1aq/EkE6cD4VpInCzjewxmk1tRR7uVHsJ18R0PNSGQWH5N iiqi9eKBaenmmDOoollTnXAgxqlg7bn3D2cVC7uVIw+1dUjetp2fe16rVhA8xL+4TK3r tvbg== X-Gm-Message-State: AOJu0YwdxIXhPXFIWGzJSPM5DsCabtqFhdmsWuiVxZpD/zaYK+pYjNQu zakbtxvqurATgOZRCYzJhcF0eA/uQrNo4u9wUVCYVzc3H+GalxtR8utKkjNDlWE= X-Received: by 2002:ac8:7f0f:0:b0:42a:aaba:73cb with SMTP id f15-20020ac87f0f000000b0042aaaba73cbmr2427380qtk.74.1706578938767; Mon, 29 Jan 2024 17:42:18 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id y19-20020a05622a005300b004181e5a724csm3357547qtw.88.2024.01.29.17.42.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:18 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 04/20] mm: zswap: warn when referencing a dead entry Date: Mon, 29 Jan 2024 20:36:40 -0500 Message-ID: <20240130014208.565554-5-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789477803683353260 X-GMAIL-MSGID: 1789477803683353260 Put a standard sanity check on zswap_entry_get() for UAF scenario. Signed-off-by: Johannes Weiner Reviewed-by: Chengming Zhou Acked-by: Yosry Ahmed Reviewed-by: Nhat Pham --- mm/zswap.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/zswap.c b/mm/zswap.c index 9f05282efe3c..0c6adaf2fdb6 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -542,6 +542,7 @@ static void zswap_entry_free(struct zswap_entry *entry) /* caller must hold the tree lock */ static void zswap_entry_get(struct zswap_entry *entry) { + WARN_ON_ONCE(!entry->refcount); entry->refcount++; } From patchwork Tue Jan 30 01:36:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 193779 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp942669dyb; Mon, 29 Jan 2024 17:45:10 -0800 (PST) X-Google-Smtp-Source: AGHT+IGV10F2RcqpW/AW44w/anxnK3MI8i53vLKRlNul9kP/GtuMpNlwXClhtf9G6M87nEddIBsG X-Received: by 2002:a05:6358:ed13:b0:175:cb7d:74ef with SMTP id hy19-20020a056358ed1300b00175cb7d74efmr8189722rwb.25.1706579109980; Mon, 29 Jan 2024 17:45:09 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706579109; cv=pass; d=google.com; s=arc-20160816; b=JSbIW9dNRm17RDdaPMcMMz+g98vTGlFGZm3CE1ISaM74aGF9JTuQvIveIQuoz7WitN 0tWOMPbUAvorTSRUR2oXYTLvIAQ26mvnRF9RgDHpyl0foLlqNLiYg6detJCMqzy+mftG 4p3yQPJHKRcwm/mmBa1k/byxlb6ayXwVNlKdCvRIV1mZMbVVqdU/JoIlgIsGb3Ew3G19 sCAk/XuXi7B7ynycx/CQ1Dk2h1dvHNX7ztJhSZu97t5whqTEaMF6rbG+LBxYWGDnnef3 Tjjs/pStReQrSoizl3Ur03+PbImITEyyGW/NDHXvqAq73YI2MZuWGWHsw/HOc3iwFtIT zQIA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=3noBpEoNcYkgCi0cr1quPQy/gMKhnK1FE9Og99etBLE=; fh=pEdtrChUTDmvuLILGSavwskaPZjBcM3e1cB9O4e7o4U=; b=i9cDNhPBbb8QhmndIntrCRb4woz49ShLsTDuchTNuJ9Cxt8j4pHutLXPookMKO7KpB oeXzGdWSh6NsxHwMAKxo00VOPeT3e/EPZbz5wcLGyAE4+h40UPcpWN9bHSp3bo1z1X9j SSVuIXusLdmgEUYHkdLqsQyEcXo9Tso5EAguUynH+CifYJV4YwOLkNNL8KvfT777kJVu HhAedieJ5y7Zj5t17hW+9qDt82JON9WF+KpLzUhsWSlD19t3sQtf5p2S6zgKz0zOcrRU UtPwwDwzfgh5UPr73OVAalwU/wS863vanyhfKDcL4izpY/zAHNBxN6XYPyzyts19dr1Z DvNA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=hvd1WYna; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43744-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43744-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id h189-20020a6383c6000000b005ceb19326e3si6624600pge.251.2024.01.29.17.45.09 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:45:09 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43744-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=hvd1WYna; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43744-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43744-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 53260B23099 for ; Tue, 30 Jan 2024 01:43:54 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 49073374FB; Tue, 30 Jan 2024 01:42:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b="hvd1WYna" Received: from mail-qk1-f171.google.com (mail-qk1-f171.google.com [209.85.222.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0C0C836AFE for ; Tue, 30 Jan 2024 01:42:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578942; cv=none; b=kYE5fTdh5Fihh5nZX1kZaQ5aMO5YH6mwOvouqO1aplbnDw5x22BzNy5wSfAqeLfeNf+/+oKG1wkUw1AkahlArfuFRHOHQr1DheSehGcEQiB0LRLrKawYymH6H2Lu1dKuxfuXaRrvtE17iz2fPiU94WtLL7/D+7460B4RczHDdHY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578942; c=relaxed/simple; bh=GyovliTv6mCdRtvV87ncUhqOka1TJA4wbPaRsBeIVbE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=J0m5tFpJX5wci18KSbWHbdxj6yB8lKp7W72FNuqz3H3FPDAEP9RSANDi1IeZwpx/rFEsesjjTDaL6a7QF6cAKSwqjcbf9nvp/7sC+Cln9aSzVDn6HeIYZi5A5Q6Uucee69wQGq6Jlib7BkLUaj11aNxIfcWwhrqFnK0JRZebVC0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b=hvd1WYna; arc=none smtp.client-ip=209.85.222.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Received: by mail-qk1-f171.google.com with SMTP id af79cd13be357-783ced0216aso251321285a.1 for ; Mon, 29 Jan 2024 17:42:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578940; x=1707183740; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3noBpEoNcYkgCi0cr1quPQy/gMKhnK1FE9Og99etBLE=; b=hvd1WYnaT5BNguVj30FAaYd7ZgH0IAOpDrabH5bOi0DCgK5QnS7tEcgdqB7mPGGs9G YaR9cEb7S35B5OKpSImWU9BD9aByxPo1/VcSKzbswgyJG4Esr9xCQgE718NlURNMbYum s0Lbg67mv3AHmw/nehdQ1wUb8A+5vjcgjomXxzcqd2jWAc0cS8u4v6eJTqN7utQvfZdW 65BS2FYDOQQSDtn2bhX2spCuv87NdnIA1Y4Dm7o31hMwsiViQr+GUTqI3Nty4Aqbd5Rj H+zEbvKk5idmNDW7rtF1qc9OA7pdaHRJZcRLGEXLbbuRG+W/nMDHsyKLzjHrMGp1HYfw gLhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578940; x=1707183740; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3noBpEoNcYkgCi0cr1quPQy/gMKhnK1FE9Og99etBLE=; b=p8dyk7aV83RJxnxaI6gsR+oBN2w9gRhxOyPD3HEjolv+l5wxzgcJE5pT6/FkHe3IWO g+2mz5946MmaIxzhQNi4Uvvh++ICQ/Qon6u/RAD1lxkwKtYLL2DofBkr/SMQM+Q4xpW2 XsOKI0FBRC5+UN+faMN+my3E05Bg5jjxikKpsnN7rDCmIsxnKO5w8kDxDajSFaLKy+op lP6cM5AzbgazKvjkVYD2iAObBCzF4fNPmJh6BEsAWj3C+ba39qDb2DW7NyRiZuyu3oaF wGhdk6hxEI/2zBn1sbHbp2uoMBuCIF4GZSJrNkyM87VqHlLf4vFj57Jb/+m5phW9Eh7R W2+A== X-Gm-Message-State: AOJu0YyCqpMOKd8/Dpyb3t4ll2xUw/norbetf/HzV8KhPpLIr9tvqNOD Yw5KAvOXrM2iy0bmPkARw4uodd91TjFLOM/LNNiolxiIu9YsvsKnim/6u29b5WRriiJ3loF9MM0 B X-Received: by 2002:a05:620a:28d4:b0:784:79a:ce59 with SMTP id l20-20020a05620a28d400b00784079ace59mr1212077qkp.26.1706578939943; Mon, 29 Jan 2024 17:42:19 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id d23-20020a05620a205700b00783f77b968fsm1879593qka.109.2024.01.29.17.42.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:19 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 05/20] mm: zswap: clean up zswap_entry_put() Date: Mon, 29 Jan 2024 20:36:41 -0500 Message-ID: <20240130014208.565554-6-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789477896932072497 X-GMAIL-MSGID: 1789477896932072497 Remove stale comment and unnecessary local variable. Signed-off-by: Johannes Weiner Reviewed-by: Chengming Zhou Acked-by: Yosry Ahmed Reviewed-by: Nhat Pham --- mm/zswap.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 0c6adaf2fdb6..7a7e8da2b4f8 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -546,15 +546,11 @@ static void zswap_entry_get(struct zswap_entry *entry) entry->refcount++; } -/* caller must hold the tree lock -* remove from the tree and free it, if nobody reference the entry -*/ +/* caller must hold the tree lock */ static void zswap_entry_put(struct zswap_entry *entry) { - int refcount = --entry->refcount; - - WARN_ON_ONCE(refcount < 0); - if (refcount == 0) { + WARN_ON_ONCE(!entry->refcount); + if (--entry->refcount == 0) { WARN_ON_ONCE(!RB_EMPTY_NODE(&entry->rbnode)); zswap_entry_free(entry); } From patchwork Tue Jan 30 01:36:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 193790 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp944344dyb; Mon, 29 Jan 2024 17:50:22 -0800 (PST) X-Google-Smtp-Source: AGHT+IEvjF3QF9SRwPOl7IBSIuOqWqLCrEu/ZNUvckNRPJHIrV8T1nnCN7P5LDCZ6QJgdcAHobRR X-Received: by 2002:ad4:5103:0:b0:68c:3dbb:b33c with SMTP id g3-20020ad45103000000b0068c3dbbb33cmr5479304qvp.48.1706579422160; Mon, 29 Jan 2024 17:50:22 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706579422; cv=pass; d=google.com; s=arc-20160816; b=rSX0ZecZWaqkYXyiUlhRvKLdfSTUfAGeuZDbsJcMW4a+P33a9iE8o5S1ZcMINWSMce e6goGGVXKshibULS8kFfcQqr+3S59sOdOcAR362uVC2uuymSPX0GPjYm6luY0xPrL4/J QfgAQcjt247gFo4U+KevM64lNua2f2is/66CtE+thhlXE3IDYLsmjdZ1pFjvY/frpxDM YN6ZTY62W8Evl3idsyC8tB3FdfTzxfxQi20Jkp+TDefbUaKB1TQ9Zr5fnfph5I0YoPO2 zjtHEATjlAkmYj35F975vZRMwGZ88NsMv2xt/aJ2pJWGtamM/d9yOTTAj4/pftBuyLmm 1XVg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=mpwOGEgV3dvwZoaILLJtRsR55d6E5hoMpUKMvxq3d70=; fh=pEdtrChUTDmvuLILGSavwskaPZjBcM3e1cB9O4e7o4U=; b=RwbbkF1AcM/QpvsN4DRWI/0Eu0D7PAgn9yTpT2IvctzsHwZyDRsAHPsyRQKaaoSZ36 k5RD5zmz8bquS9caaqC2BnEHXrS0mBxngJlzYjLRaVkR8vlqMW/tMsN2DB094Fz37TQd ywFDoU2j6LqhR8W6E8UXXVQclqkuLrZFfMP5JUOZtZybS94KRqsHA4KoI35oH4KTii0X es4YMntzfNnn+S2BVX/GxM8d4f+RaZhzzlQTDju9NatwnyXA0tZl7CiPs6r2bLRS+sLG lQk9zqRNHM9CROEZ+mxR0hYvg5bQmuFgy25vyo9MJ0G2pzw8MwMel79Sqqadnr9fyuXR NluQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=ZT9hiui7; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43746-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43746-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id a3-20020a056214062300b006818c2de31dsi8653432qvx.502.2024.01.29.17.50.22 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:50:22 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43746-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=ZT9hiui7; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43746-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43746-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 798E91C23DA2 for ; Tue, 30 Jan 2024 01:44:34 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1AF5A383A6; Tue, 30 Jan 2024 01:42:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b="ZT9hiui7" Received: from mail-qk1-f169.google.com (mail-qk1-f169.google.com [209.85.222.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D7C6376EE for ; Tue, 30 Jan 2024 01:42:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578945; cv=none; b=e5crN1qSU+ER227sZkNFG+VbzW8rPPsfWDq9CHOODnzWcUerrxobibRGbenG4xJpyR4/vCc+AdNqJXX4RFlMhh9ANjaodYoyIGIE23JjJUJrnbLN7387J3RudGD99BVlb5RWQtxn3C6fdmxjY1vPdZm8rjKX5+Lj/53ubwmSG6c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578945; c=relaxed/simple; bh=tQ+eAqiC1Z1SgLLxIS6/zBi2cNhu06izlT87O1n52ho=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pFelBvNTPoE3hTHTWxSgdnv4K5ZcX33kds4wzIemjQCa2nRYrwgE8zxAjVEH/0TNHZerMplsuQttun0VLEv7lSa1Hq+k1stBOm2fMI//WeXzFLljUl/4SRoXCE2MK6Ibvur7EW2GyMsL5GGElVV2yczkwqU02Eo/FKL8tJmLT6c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b=ZT9hiui7; arc=none smtp.client-ip=209.85.222.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Received: by mail-qk1-f169.google.com with SMTP id af79cd13be357-783f0994eeeso136071485a.0 for ; Mon, 29 Jan 2024 17:42:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578942; x=1707183742; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mpwOGEgV3dvwZoaILLJtRsR55d6E5hoMpUKMvxq3d70=; b=ZT9hiui7CId8rNigQrVflUA6iIVZXtMVAKxeg4jvdbucWZ/gxzr1EIL39p3zeEI0zO bTs+5Hfckt+BuuQRP2qx3G5qyWjhWXEOYGorzYRWegBeyuwyXW1G2tZN/7JubdJaulVu skFNA5HN165VSQEourAlusdNB9EdW6uR8LTUxLj90UCGubo4MH+4aHBVuYADszh08cfX MsLcJj1l+QsxLYOBJRsmjxMUVzHNdGSBVRIbXlcSFQ998/kjopInqHITlVM629drvTuD HGEEW/D/nsUA/ocMevGzNDX6RyLkvvlK3wCyPE+yupQjd+Vgb1JD7cYP3eYBUHjh0g9h dLig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578942; x=1707183742; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mpwOGEgV3dvwZoaILLJtRsR55d6E5hoMpUKMvxq3d70=; b=M/2xihAc9NS1hfyUNoJgZk2qeGv31Rxcx7WGdoIAJztmXJbbxU97tMV9H1wCyXkrk8 ahcF+CxIViIgwnNmkz7ipMgLxw4N7ZZAclaxjT3DcJRVk0kWsmeElBLbIs48oLfMQGQB UM8GRgd5Oq245QcPUYmJZ8xNheiAfuxR/tcCRy40taHdRBe0zVognnr9ONssR+lW3uOb 2lyWiva78cIrkUJkAzBbOcH6z9bLocHwHe0vFjp1WBr+n/kH0nlCarQpkzexl4fDyld3 sn8VrSVfQ7NM+qzE2FBvxyQxivAM59OZa0MWUoDmHw3R+XRZPrVaghdc9r3B2m+X3UrH bpbA== X-Gm-Message-State: AOJu0Yy/H3awU0Bse+pLNkipq3L1C6MO5f5MlbYeed69FKwjsqc+9tX5 ael/o9vSoaH4wxJSgibD6jc6n5AGE17ToObPYWYhpxpnunLJLTDZ/rphOVCNHtA= X-Received: by 2002:a05:620a:b11:b0:783:d096:d761 with SMTP id t17-20020a05620a0b1100b00783d096d761mr5637186qkg.31.1706578942391; Mon, 29 Jan 2024 17:42:22 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id po30-20020a05620a385e00b00783e3b030e7sm2669196qkn.33.2024.01.29.17.42.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:21 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 07/20] mm: zswap: break out zwap_compress() Date: Mon, 29 Jan 2024 20:36:43 -0500 Message-ID: <20240130014208.565554-8-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789478224215161813 X-GMAIL-MSGID: 1789478224215161813 zswap_store() is long and mixes work at the zswap layer with work at the backend and compression layer. Move compression & backend work to zswap_compress(), mirroring zswap_decompress(). Signed-off-by: Johannes Weiner Reviewed-by: Chengming Zhou Acked-by: Yosry Ahmed Reviewed-by: Nhat Pham --- mm/zswap.c | 145 ++++++++++++++++++++++++++++------------------------- 1 file changed, 77 insertions(+), 68 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index bdc9f82fe4b9..f9b9494156ba 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1316,6 +1316,79 @@ static int zswap_enabled_param_set(const char *val, return ret; } +static bool zswap_compress(struct folio *folio, struct zswap_entry *entry) +{ + struct crypto_acomp_ctx *acomp_ctx; + struct scatterlist input, output; + unsigned int dlen = PAGE_SIZE; + unsigned long handle; + struct zpool *zpool; + char *buf; + gfp_t gfp; + int ret; + u8 *dst; + + acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); + + mutex_lock(&acomp_ctx->mutex); + + dst = acomp_ctx->buffer; + sg_init_table(&input, 1); + sg_set_page(&input, &folio->page, PAGE_SIZE, 0); + + /* + * We need PAGE_SIZE * 2 here since there maybe over-compression case, + * and hardware-accelerators may won't check the dst buffer size, so + * giving the dst buffer with enough length to avoid buffer overflow. + */ + sg_init_one(&output, dst, PAGE_SIZE * 2); + acomp_request_set_params(acomp_ctx->req, &input, &output, PAGE_SIZE, dlen); + + /* + * it maybe looks a little bit silly that we send an asynchronous request, + * then wait for its completion synchronously. This makes the process look + * synchronous in fact. + * Theoretically, acomp supports users send multiple acomp requests in one + * acomp instance, then get those requests done simultaneously. but in this + * case, zswap actually does store and load page by page, there is no + * existing method to send the second page before the first page is done + * in one thread doing zwap. + * but in different threads running on different cpu, we have different + * acomp instance, so multiple threads can do (de)compression in parallel. + */ + ret = crypto_wait_req(crypto_acomp_compress(acomp_ctx->req), &acomp_ctx->wait); + dlen = acomp_ctx->req->dlen; + if (ret) { + zswap_reject_compress_fail++; + goto unlock; + } + + zpool = zswap_find_zpool(entry); + gfp = __GFP_NORETRY | __GFP_NOWARN | __GFP_KSWAPD_RECLAIM; + if (zpool_malloc_support_movable(zpool)) + gfp |= __GFP_HIGHMEM | __GFP_MOVABLE; + ret = zpool_malloc(zpool, dlen, gfp, &handle); + if (ret == -ENOSPC) { + zswap_reject_compress_poor++; + goto unlock; + } + if (ret) { + zswap_reject_alloc_fail++; + goto unlock; + } + + buf = zpool_map_handle(zpool, handle, ZPOOL_MM_WO); + memcpy(buf, dst, dlen); + zpool_unmap_handle(zpool, handle); + + entry->handle = handle; + entry->length = dlen; + +unlock: + mutex_unlock(&acomp_ctx->mutex); + return ret == 0; +} + static void zswap_decompress(struct zswap_entry *entry, struct page *page) { struct zpool *zpool = zswap_find_zpool(entry); @@ -1472,18 +1545,11 @@ bool zswap_store(struct folio *folio) struct page *page = &folio->page; struct zswap_tree *tree = swap_zswap_tree(swp); struct zswap_entry *entry, *dupentry; - struct scatterlist input, output; - struct crypto_acomp_ctx *acomp_ctx; struct obj_cgroup *objcg = NULL; struct mem_cgroup *memcg = NULL; struct zswap_pool *pool; - struct zpool *zpool; - unsigned int dlen = PAGE_SIZE; - unsigned long handle, value; - char *buf; - u8 *src, *dst; - gfp_t gfp; - int ret; + unsigned long value; + u8 *src; VM_WARN_ON_ONCE(!folio_test_locked(folio)); VM_WARN_ON_ONCE(!folio_test_swapcache(folio)); @@ -1568,65 +1634,10 @@ bool zswap_store(struct folio *folio) mem_cgroup_put(memcg); } - /* compress */ - acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); - - mutex_lock(&acomp_ctx->mutex); - - dst = acomp_ctx->buffer; - sg_init_table(&input, 1); - sg_set_page(&input, &folio->page, PAGE_SIZE, 0); + if (!zswap_compress(folio, entry)) + goto put_pool; - /* - * We need PAGE_SIZE * 2 here since there maybe over-compression case, - * and hardware-accelerators may won't check the dst buffer size, so - * giving the dst buffer with enough length to avoid buffer overflow. - */ - sg_init_one(&output, dst, PAGE_SIZE * 2); - acomp_request_set_params(acomp_ctx->req, &input, &output, PAGE_SIZE, dlen); - /* - * it maybe looks a little bit silly that we send an asynchronous request, - * then wait for its completion synchronously. This makes the process look - * synchronous in fact. - * Theoretically, acomp supports users send multiple acomp requests in one - * acomp instance, then get those requests done simultaneously. but in this - * case, zswap actually does store and load page by page, there is no - * existing method to send the second page before the first page is done - * in one thread doing zwap. - * but in different threads running on different cpu, we have different - * acomp instance, so multiple threads can do (de)compression in parallel. - */ - ret = crypto_wait_req(crypto_acomp_compress(acomp_ctx->req), &acomp_ctx->wait); - dlen = acomp_ctx->req->dlen; - - if (ret) { - zswap_reject_compress_fail++; - goto put_dstmem; - } - - /* store */ - zpool = zswap_find_zpool(entry); - gfp = __GFP_NORETRY | __GFP_NOWARN | __GFP_KSWAPD_RECLAIM; - if (zpool_malloc_support_movable(zpool)) - gfp |= __GFP_HIGHMEM | __GFP_MOVABLE; - ret = zpool_malloc(zpool, dlen, gfp, &handle); - if (ret == -ENOSPC) { - zswap_reject_compress_poor++; - goto put_dstmem; - } - if (ret) { - zswap_reject_alloc_fail++; - goto put_dstmem; - } - buf = zpool_map_handle(zpool, handle, ZPOOL_MM_WO); - memcpy(buf, dst, dlen); - zpool_unmap_handle(zpool, handle); - mutex_unlock(&acomp_ctx->mutex); - - /* populate entry */ entry->swpentry = swp; - entry->handle = handle; - entry->length = dlen; insert_entry: entry->objcg = objcg; @@ -1663,8 +1674,6 @@ bool zswap_store(struct folio *folio) return true; -put_dstmem: - mutex_unlock(&acomp_ctx->mutex); put_pool: zswap_pool_put(entry->pool); freepage: From patchwork Tue Jan 30 01:36:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 193795 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp947316dyb; Mon, 29 Jan 2024 18:00:06 -0800 (PST) X-Google-Smtp-Source: AGHT+IFnJ9uwPVU/ERteHn/UEY/g0KzUdaj+Yf8uiefOwnjgwYWMH2qDT+Q8U9iYO5mPXxCcMnQV X-Received: by 2002:a05:622a:7189:b0:42a:a348:7cad with SMTP id jd9-20020a05622a718900b0042aa3487cadmr432463qtb.65.1706580005866; Mon, 29 Jan 2024 18:00:05 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706580005; cv=pass; d=google.com; s=arc-20160816; b=koj483aP5Z5mNnJh70M7zw70AaWFdLdg2nqU1hk121gnT/eIxOReZeRnFPDnbtuvh0 Pd4rK4rUFcqMDKxB7v/UESOzkneRrLes5eJf/fZGQ7u5VmZIyixnMSEaUhof89g+x9Hy ESQSa1TT9skWu9xODPbxXxtbiH4AAP5c1CeQj0Vm/z/5NfAS9AmK/hYlXLe4DtdPuSEM ivs2qEpw2WB8RtWIDfiqxH7AnNseWzlDDLjTxLGpfeS1tKkHq7nkqdkWvD38lrGQHEaw zwj/4xE0D6An2Dx5cT8j5BwqsAxCXQ82DgxZjcHlgF+8tMvPuWc2Yj8ZotrKXtGXoGoy DlHQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=mALgLaCiaIzpkji8oSjATmvxJWj+cS9YMUqWO4u+ZHc=; fh=pEdtrChUTDmvuLILGSavwskaPZjBcM3e1cB9O4e7o4U=; b=dxNOO9e5q5TnrfQ+JxKO6sZqGA8ROvv+PPBTEFQlOmZ1VYOiQJwZWEAmIlei3QYjfw 6I96/vqCHaHjy6ztXnlzLgfr9Z1U2cJkzn43NPBRCQ/V8ghttkpTKIi5d1tcqtWa7/6Y qIS/66PylcrxYGCQlGrhf9sqpm4KUjHLQ7nO/xZhrYkKWLRSejrjz5bgKiNTWDV+q1Ee xD2ZErE8C/NAloEi1CLfOUjQL83rbTRuSTfMx0hTXuhhos/nDFxTgeCz7SH0DKHB6Tld A3VYy2Li+OhwxlqngGTcdPjyH39AlodyAkKckKoGfuMpW56Li+guEfYFbBWCLR8/7l61 hT5A== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=Lwj5HmQz; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43747-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43747-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id e17-20020ac85dd1000000b0042a1c4da40asi8639715qtx.301.2024.01.29.18.00.05 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 18:00:05 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43747-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=Lwj5HmQz; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43747-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43747-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 912601C23D95 for ; Tue, 30 Jan 2024 01:44:37 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5BC4F383AE; Tue, 30 Jan 2024 01:42:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b="Lwj5HmQz" Received: from mail-qv1-f52.google.com (mail-qv1-f52.google.com [209.85.219.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 937CF3770E for ; Tue, 30 Jan 2024 01:42:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578946; cv=none; b=UVtxNSAnBqHTaMp6KU1GNNun0xmZj5KpQaP5H2tnYsAhojm4wA0yxhyMIKfpApNwONe/bdE3qWY3DNIYVFjp0t6puy71g4wgHbYTBY0yWo4pyvMTeDKt4id1PPDJZomz7/ThClgkESCHRIymRSn9t3sBNOW+UHWMXZQNt+Q8kpg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578946; c=relaxed/simple; bh=meDiOpiwOmc5O23emXHPLEn8CVIgwW9U83t0CK4mvi4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HUwhXmceSLlUgojuS9UMpYyZkFJTC3GmZ+Z4rIJzurlVM28Q01Z7h6Zum6QLCNcdhxHv7UAgNWpLtOFDIanIBZwLJqOscQ/2gwrZEXah4INmPqWSTc7Llg+40R9utQ61yQbvCMmyY1ivrCWsQUM0ogUOeDxPp4q0HPUhC3CWpN0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b=Lwj5HmQz; arc=none smtp.client-ip=209.85.219.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Received: by mail-qv1-f52.google.com with SMTP id 6a1803df08f44-680b1335af6so40905566d6.1 for ; Mon, 29 Jan 2024 17:42:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578943; x=1707183743; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mALgLaCiaIzpkji8oSjATmvxJWj+cS9YMUqWO4u+ZHc=; b=Lwj5HmQzaNZV1GAT993BnYTbUhU3xPr8WSOTpH3j3s4SprsujIy1uT5omNbOtEsNz5 Gmb6u0AoqasnDz8CGBb60mDBicPeQeZ5mGcroAFhXNe6K6ayjekIDdoRbiP4+G1SFGhP jiGSH0oNl0bWtEuL3aXR59hjxSAOegTbqpv1HM91+sDZUMFiv5dN67kYoLdtlePb3kWc V+CWO7CJ5Mn7R0iHy36csu+O14L6wFB1D5AbHMlt8o+xVLhPaoiGyPZ6NFQjNlbLnL19 tk1tptLt/0cp+elRpyTjhMwONJuV1YldB3cJfcDjl/ZdhyVoq0fM8JMpsU6PHVZA3EsY Nebg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578943; x=1707183743; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mALgLaCiaIzpkji8oSjATmvxJWj+cS9YMUqWO4u+ZHc=; b=HkV2a0dxKyRGGG5GrT47NlUWOfVAf6JsNsuRdCVueUNO5bBxgNOF47ktC62Wp+PRrJ u6wt4HSNo2Iw0l4t+1vAJN66NA6+zCvaUgticaBaNIEdWXhk3851hkLUHWJ/qOcx0LIl yFxyZ8TvZcXiPUoaWUOkih4NTokh4kML/orviL05/4dHtrseOJeNDDxanDHC/7wirxsd HI36AB9tDoMIpI7TxSUNMs+Ad2mFTgQRGYKqlKD3skFFD0g1ZEy9xhPZAoGYopoYdGdG 2uxbo2uyULCHrWa720Lfu9RftmVIYpl57Lc7o8ubOg4tLaklyPzaJXq3/0fldDVGTvdN V+dw== X-Gm-Message-State: AOJu0Yz94PsfzjfzRVdUDgMw0VYe7qG1zd2EvildiNUyxJooyZJKla86 5wpeGUUXMjixtPbtp3PU8TDFMMpdkIYsGEGxsXNWQADC06RrZRCiPrCdYmbcBfo= X-Received: by 2002:ad4:5aa5:0:b0:68c:5c8a:dc7b with SMTP id u5-20020ad45aa5000000b0068c5c8adc7bmr396936qvg.13.1706578943526; Mon, 29 Jan 2024 17:42:23 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id pc2-20020a056214488200b0068c4aefa96bsm1889828qvb.69.2024.01.29.17.42.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:23 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 08/20] mm: zswap: further cleanup zswap_store() Date: Mon, 29 Jan 2024 20:36:44 -0500 Message-ID: <20240130014208.565554-9-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789478835759880370 X-GMAIL-MSGID: 1789478835759880370 - Remove dupentry, reusing entry works just fine. - Rename pool to shrink_pool, as this one actually is confusing. - Remove page, use folio_nid() and kmap_local_folio() directly. - Set entry->swpentry in a common path. - Move value and src to local scope of use. Signed-off-by: Johannes Weiner Reviewed-by: Chengming Zhou Acked-by: Yosry Ahmed Reviewed-by: Nhat Pham --- mm/zswap.c | 28 +++++++++++++--------------- 1 file changed, 13 insertions(+), 15 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index f9b9494156ba..cde309c539b3 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1542,14 +1542,11 @@ bool zswap_store(struct folio *folio) { swp_entry_t swp = folio->swap; pgoff_t offset = swp_offset(swp); - struct page *page = &folio->page; struct zswap_tree *tree = swap_zswap_tree(swp); struct zswap_entry *entry, *dupentry; struct obj_cgroup *objcg = NULL; struct mem_cgroup *memcg = NULL; - struct zswap_pool *pool; - unsigned long value; - u8 *src; + struct zswap_pool *shrink_pool; VM_WARN_ON_ONCE(!folio_test_locked(folio)); VM_WARN_ON_ONCE(!folio_test_swapcache(folio)); @@ -1567,10 +1564,10 @@ bool zswap_store(struct folio *folio) * the tree, and it might be written back overriding the new data. */ spin_lock(&tree->lock); - dupentry = zswap_rb_search(&tree->rbroot, offset); - if (dupentry) { + entry = zswap_rb_search(&tree->rbroot, offset); + if (entry) { + zswap_invalidate_entry(tree, entry); zswap_duplicate_entry++; - zswap_invalidate_entry(tree, dupentry); } spin_unlock(&tree->lock); objcg = get_obj_cgroup_from_folio(folio); @@ -1598,17 +1595,19 @@ bool zswap_store(struct folio *folio) } /* allocate entry */ - entry = zswap_entry_cache_alloc(GFP_KERNEL, page_to_nid(page)); + entry = zswap_entry_cache_alloc(GFP_KERNEL, folio_nid(folio)); if (!entry) { zswap_reject_kmemcache_fail++; goto reject; } if (zswap_same_filled_pages_enabled) { - src = kmap_local_page(page); + unsigned long value; + u8 *src; + + src = kmap_local_folio(folio, 0); if (zswap_is_page_same_filled(src, &value)) { kunmap_local(src); - entry->swpentry = swp; entry->length = 0; entry->value = value; atomic_inc(&zswap_same_filled_pages); @@ -1637,9 +1636,8 @@ bool zswap_store(struct folio *folio) if (!zswap_compress(folio, entry)) goto put_pool; - entry->swpentry = swp; - insert_entry: + entry->swpentry = swp; entry->objcg = objcg; if (objcg) { obj_cgroup_charge_zswap(objcg, entry->length); @@ -1684,9 +1682,9 @@ bool zswap_store(struct folio *folio) return false; shrink: - pool = zswap_pool_last_get(); - if (pool && !queue_work(shrink_wq, &pool->shrink_work)) - zswap_pool_put(pool); + shrink_pool = zswap_pool_last_get(); + if (shrink_pool && !queue_work(shrink_wq, &shrink_pool->shrink_work)) + zswap_pool_put(shrink_pool); goto reject; } From patchwork Tue Jan 30 01:36:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 193778 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp942533dyb; Mon, 29 Jan 2024 17:44:48 -0800 (PST) X-Google-Smtp-Source: AGHT+IHJJhUckMs/OTG3WK1mO7nzEXQT0MlYWBNsmVJoLGjqxzd8VG+qMUJVXs6ZHwGctCsQYuSf X-Received: by 2002:aa7:da0d:0:b0:55f:ed:27e8 with SMTP id r13-20020aa7da0d000000b0055f00ed27e8mr2735520eds.30.1706579087889; Mon, 29 Jan 2024 17:44:47 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706579087; cv=pass; d=google.com; s=arc-20160816; b=eLF5dGXvvHPY1HzF6m5bZNMq6NfB4wwegr4xoVjAcxNrgpx5z3bJ9ADlvPX3/n17Sa VIdkTR+uoyZlKcnxBaIVg124fTHGCq9govJa6a9h4OBlau4yia4tHQB2M7liPxbgW2jY b5l6hsjD9fBTZNSWT9tWBgczVeRtLK6yGrXuJoV1v2HsX0heYK4rGIj3mjV3fkoyVVr1 re7B0HZvvt+ULQ9gGOi3m2+rpRpQbhuhceShTE0Z6agXjAYyLBw++2uz3B4Efqqie82g miBerkw5iEkORVqGebvhfx/gIi34g55jX+UX4qlCWuTPHNzf8ZzqI0+/ayBUW/tfi7i7 MOCw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=YF/Bz5keaw8EKm3mlmWlxcJJmelaMJ52iqxIOq55K14=; fh=pEdtrChUTDmvuLILGSavwskaPZjBcM3e1cB9O4e7o4U=; b=DWiCdrlGZw3QAzikUd838WnrMKoI5IFjb8MgI9Nplzs8x7MXzsUieICDlMRSEaSvdA fHZovnOm9E0hL0iPfX5Yj6d2ZrrnZeVEENtIoE7fZewvHrUaMXlvw6gZF9m5zLeKYr6b lAKpGTt++cUUkpqR5OEkx+BD+bV5pxhlSlduYmBOhh55oSIGbg+Jgb7ZuLVXitr8dHfm BRMCkKGwbpYznDUl6sYQEsm+hwdbuxCuvGn8NM59lIPTTS8OOIZqJ9fHzl7kuQeF46mG oxGEcJPMOHd3TTQ/BQqHS/f8mEFo9myYZNMZE5B+urNkIpWIOlyHS50dK4n2k3Mz7FFt Hg5A== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=DhOdvnCB; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43748-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43748-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id z5-20020a056402274500b0055ca92fb6c6si4423440edd.34.2024.01.29.17.44.47 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:44:47 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43748-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=DhOdvnCB; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43748-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43748-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 7FC651F28941 for ; Tue, 30 Jan 2024 01:44:47 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0700C38DE1; Tue, 30 Jan 2024 01:42:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b="DhOdvnCB" Received: from mail-qv1-f53.google.com (mail-qv1-f53.google.com [209.85.219.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B6F2C381AC for ; Tue, 30 Jan 2024 01:42:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578947; cv=none; b=dZ2MtH8wSOuyOQQhon8cEURM2MMimcwgC2/TQ7zsLmzt3LtG+67u/y2DO6ZyaTkD/1ssx8BQxMh+cU/R/bHYbEZ9upc73xGuV6THTD9mX/4ES6QysVLttn1uvy94C5sl3/m0MxZ8HG1SlKkKGezEAmsLBh8yVnnG74tn5Doaqxg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578947; c=relaxed/simple; bh=NSf7/7XE+Ua256/ji15QEBMmIEdQMUCKLsK1Z44ZjCo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=r/vFRtOgl9amIAJyn3GjG54FE4hmAGYYb++qAzs5+sSXSQnsxbVJS1W1HUX0HWzBichdKno7lO6AqMWJ3fc+r114gUcxEuv/rAkGjbGSsfo3GzsiZYPHxCZ/rR7HOR0sqoX0XtXSqZlUVpDpZ1TFVFbSW+G77C2ZJxDEksnLjz4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b=DhOdvnCB; arc=none smtp.client-ip=209.85.219.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Received: by mail-qv1-f53.google.com with SMTP id 6a1803df08f44-68c495ba558so13028036d6.3 for ; Mon, 29 Jan 2024 17:42:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578944; x=1707183744; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YF/Bz5keaw8EKm3mlmWlxcJJmelaMJ52iqxIOq55K14=; b=DhOdvnCBSRNAf53Ma4noBLU2yvWiveDDh04yo/cRMvwHODciaBPksZq2Um17ckQaBW 0Hq/LTjG8OCgdTAaVLJMan+5TSM/zVx3GYd88SehecPaB/dMtgFz16XEPOkyC7aggE5M rTz0WgWmFlQbokM4jmBd03rZPyCFNvZ8xkPXKP9+zNTHKc/BfFJZPe5cERZGUVS5fF9M kHqCh7U2z3EN06jjKsbtQrS61ytqBPo5CcyxMA+IlMyOPBGzj8XF8bTrqJEKX3jXcWTW 33/y8xyjQCz8C2rCFxYsRjzyt9PrS/ocBoiZ+t/DyuarHKcykpDOUwoXDQ4cS4IyDjAk XW6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578944; x=1707183744; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YF/Bz5keaw8EKm3mlmWlxcJJmelaMJ52iqxIOq55K14=; b=VxZUgJdCHPXLOVt9IERw4w9zP0wZPa5yA07kaunP6Rt4B1yFG5Q7T8MVIuDu4bGbi9 Am9GgL7o2nTZKZpa2QVvfSQ4c/5QFyryNWs/mjBCcdo9INZegG6wHtOHyg6MviPiOySc PGNaChibAhxKjHVG26oUD308oUxXNjAzVYtbNs3FgpFZRcOgnmPgul+doC6An0h+kqEa c4Vx8eGvsQFSbbDHMEuXlDLTC2Cgx+PXVfCpIMF/xd/GO98GqUUQ8jekLy3PZBzh7TDk wJWVB1FmjqKQ8lfRjAXLIrJu5DmCu0NWS6hj9abcYVxVnjoJpFyGFJBKzxetc5mIJGcF 1gOw== X-Gm-Message-State: AOJu0YyFEUdG5rj392Tdt1qd7MwgWbMFzlul8M8+Cb0RHLXpetvKynNB 03afODnVpnkpnmwRULJcrTh/LoZj/8X0kRHcsRnf7T0m8zF1BIzoruWwFvy96sc= X-Received: by 2002:a05:6214:1804:b0:68c:623d:13d3 with SMTP id o4-20020a056214180400b0068c623d13d3mr24326qvw.68.1706578944683; Mon, 29 Jan 2024 17:42:24 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id kf4-20020a056214524400b006837a012417sm3992019qvb.51.2024.01.29.17.42.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:24 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 09/20] mm: zswap: simplify zswap_invalidate() Date: Mon, 29 Jan 2024 20:36:45 -0500 Message-ID: <20240130014208.565554-10-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789477873830734561 X-GMAIL-MSGID: 1789477873830734561 The branching is awkward and duplicates code. The comment about writeback is also misleading: yes, the entry might have been written back. Or it might have never been stored in zswap to begin with due to a rejection - zswap_invalidate() is called on all exiting swap entries. Signed-off-by: Johannes Weiner Reviewed-by: Chengming Zhou Acked-by: Yosry Ahmed Reviewed-by: Nhat Pham --- mm/zswap.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index cde309c539b3..082d076a758d 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1739,15 +1739,10 @@ void zswap_invalidate(int type, pgoff_t offset) struct zswap_tree *tree = swap_zswap_tree(swp_entry(type, offset)); struct zswap_entry *entry; - /* find */ spin_lock(&tree->lock); entry = zswap_rb_search(&tree->rbroot, offset); - if (!entry) { - /* entry was written back */ - spin_unlock(&tree->lock); - return; - } - zswap_invalidate_entry(tree, entry); + if (entry) + zswap_invalidate_entry(tree, entry); spin_unlock(&tree->lock); } From patchwork Tue Jan 30 01:36:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 193780 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp942719dyb; Mon, 29 Jan 2024 17:45:18 -0800 (PST) X-Google-Smtp-Source: AGHT+IFbu78CkfFpO20SLcLp10NENmr2CGA0LD2Mu3nD87TDq0AWQMtN07wqSuGuzfhV+H0SciwL X-Received: by 2002:a05:620a:372a:b0:783:f683:e17c with SMTP id de42-20020a05620a372a00b00783f683e17cmr4447940qkb.4.1706579117982; Mon, 29 Jan 2024 17:45:17 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706579117; cv=pass; d=google.com; s=arc-20160816; b=gj1v8v75r8IQs7oUO11Z5SQNTl/yI83NBPvhFDeVM4ss1dFcQviWUL4AX/v/G2GqBE Cu5z8CKkd+bkftYzQC2NR6hs6Huu3roUx0OgoLidfE+dn2jq48koHIjCyts7ExRe5XkD YUDje4qtro8THecHlpxAFhifEqMj9CJnqXahlIJyw0igSJrk+0bzIvWZCYgY8EirIqlh 7gInE5eFq/Io7SeWJkP6/6XtrthSSrYiaTW+6UAwRqHtjmL0NY7AHeT1FBdPUZbLjOQf dy5UUaZuEOAukBdQmUPwMS99aKEb9wv9kWXlkZjaGYiXfqVrAuTbh/t/wWocBqNt5iWF 1Otg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=nW+93M2Q9nWxSPgR21ybfgSivHJtX7f76WUEXvkAumg=; fh=pEdtrChUTDmvuLILGSavwskaPZjBcM3e1cB9O4e7o4U=; b=LZoGWTQn+fNpDoYbaARUfWcFt56On3IWM5YSUqJtAQCCDxjIO712bD2icUA4Ly+XXN uZuAXFkk6YysWveULq44i3qH9i38xdsJFymD3gNKVnMceeZeH/seukcP+ztWVa6NwpSx 4yYJVwxCIfyy/A3MBIJ43N0NMDx6pWfcGenSFmTCK45uF/FK9Ih8WvBSrds+KFty83TZ Z/Jm/83BH4ovI+AffU2u48dyCiynL/yHsWt9aJ1jQrp6ACivmtj43XKI7xtybstdQNm4 hxtGvY/OWdvJt6eXcSbj+06cKs6o1i3zhQolsu0SLQG8ZOC+o+U9FYus+jLf9KEuvSJM 0JAQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=0e5cKCip; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43750-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43750-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id p4-20020a05620a056400b00783de6351b4si7786335qkp.41.2024.01.29.17.45.17 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:45:17 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43750-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=0e5cKCip; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43750-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43750-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id B1CE31C223E5 for ; Tue, 30 Jan 2024 01:45:17 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5E1934500D; Tue, 30 Jan 2024 01:42:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b="0e5cKCip" Received: from mail-qv1-f50.google.com (mail-qv1-f50.google.com [209.85.219.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5510738393 for ; Tue, 30 Jan 2024 01:42:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578950; cv=none; b=unjdyYhAkgRuywN/p8AHdh0FrPJ4KZfNBOGGNr7KeR5xbn0M9OHVSVJk903QYmuIxivMv2hJtdqLIuCTQtBx2ipnMgVWe6EfC7Bb6XjQ/QRlZSKQ6MQ6CNy4hSCmj0akj+wcYN4BaXPt938T+Wd1nMy7IfTFR2HCPvwtl8vG1xk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578950; c=relaxed/simple; bh=9xw/W03ukpr/L6i7AIbF53T7wElpUaGJf2hRLuUR5ws=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GzOpBQoc9ArjqXpZeh4BRLC2qL3sCX9ULwGRuLCOERl9nn95VFraiE9N5xHkHuY8SRD0LIeZsbjkDVmfQZfLHejn1DseopksO9zLlm4hiwd8DfT1h+cR1SCOcY3RDUGJCuz/OVLwWRveQfhQMcECZ+FcpCj+ETzNrO7EuY8gQWw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b=0e5cKCip; arc=none smtp.client-ip=209.85.219.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Received: by mail-qv1-f50.google.com with SMTP id 6a1803df08f44-68c444f9272so14319546d6.3 for ; Mon, 29 Jan 2024 17:42:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578947; x=1707183747; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nW+93M2Q9nWxSPgR21ybfgSivHJtX7f76WUEXvkAumg=; b=0e5cKCipbmAbU9orMkEXr3CPe36KKB1C+HdJ0pnKiwUkkDGfvKsujIvPdVx4UIRCfp xN3glyGGe6wLu9Ww0AzCkrY+ebAEBEGTVcQG177fl/OoqIIJuQ4HB2zPQRrj+WLh/96s RQB0QwuNO5fbojE8HAdOousipxB6IZPBTvXeboIRhlFdYb4gsOKVugoiH+fmRwetgfFZ O7IFFSvIrlgvrUu6Gi9fTm4RbDNEmvbJb2zsE2r6B5i4Uiu4/SCrND5DZ2105HPfOoFm fLrJQOSh5pmr5NI6cvQ3TTSR1YXHMcqNgPd85v+SL8OEGV6PTLCOrb/QqSFX4s0CpVqv nsjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578947; x=1707183747; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nW+93M2Q9nWxSPgR21ybfgSivHJtX7f76WUEXvkAumg=; b=AM+L5tTCzS4LyYRikW55QS1fjsJVDC3EfSulBfY6IVQZLLe0pfbe/08nh610xr9OTj dCycLbZ8aaLra2HbxUG3Zsx6b/sj30snq9b0nWBJQi+21Eu9676TMAOa3jXsbrXwtauV sfyrk9H5hSxAWfl0HjPxOt/Qu8ZdlG9xoTmL7KjeoVpcRbLRHLnVN3IOy7MeGshtCteS PkR7lTQk88KLvCfOnaNRsaQzaSYE0zHvWWc3tjqoFeVeaFtuQJsfFO0pFCA/NhZbKaPG cI6PGAr1uKJA8vf7qbB3uyNcvm5CK6qBMagzrkIXpAW0TxDXFxcYO26fL0J+wYrWMTVU 1TDQ== X-Gm-Message-State: AOJu0Yx4r6VCDCYtgd6Q0MUltSWeESH/yLt2uty4icq3pIqg9xXEDtlZ TKfP/xQocnTBMJDAB5TJu8FuCagoU5s2Hbi+j+MFC/vJZKl/CRMlDFyjzs1V3II= X-Received: by 2002:a05:6214:cab:b0:686:9441:ac30 with SMTP id s11-20020a0562140cab00b006869441ac30mr8699046qvs.22.1706578947264; Mon, 29 Jan 2024 17:42:27 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id lr9-20020a0562145bc900b006819bb31533sm3993392qvb.99.2024.01.29.17.42.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:26 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 11/20] mm: zswap: function ordering: pool refcounting Date: Mon, 29 Jan 2024 20:36:47 -0500 Message-ID: <20240130014208.565554-12-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789477904851304997 X-GMAIL-MSGID: 1789477904851304997 Move pool refcounting functions into the pool section. First the destroy functions, then the get and put which uses them. __zswap_pool_empty() has an upward reference to the global zswap_pools, to sanity check it's not the currently active pool that's being freed. That gets the forward decl for zswap_pool_cuyrrent(). This puts the get and put function above all callers, so kill the forward decls as well. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 94 +++++++++++++++++++++++++++--------------------------- 1 file changed, 47 insertions(+), 47 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 805d9a35f633..33775f2224b7 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -278,8 +278,6 @@ static inline struct zswap_tree *swap_zswap_tree(swp_entry_t swp) static int zswap_writeback_entry(struct zswap_entry *entry, swp_entry_t swpentry); -static int zswap_pool_get(struct zswap_pool *pool); -static void zswap_pool_put(struct zswap_pool *pool); static bool zswap_is_full(void) { @@ -472,6 +470,53 @@ static void zswap_pool_destroy(struct zswap_pool *pool) kfree(pool); } +static void __zswap_pool_release(struct work_struct *work) +{ + struct zswap_pool *pool = container_of(work, typeof(*pool), + release_work); + + synchronize_rcu(); + + /* nobody should have been able to get a kref... */ + WARN_ON(kref_get_unless_zero(&pool->kref)); + + /* pool is now off zswap_pools list and has no references. */ + zswap_pool_destroy(pool); +} + +static struct zswap_pool *zswap_pool_current(void); + +static void __zswap_pool_empty(struct kref *kref) +{ + struct zswap_pool *pool; + + pool = container_of(kref, typeof(*pool), kref); + + spin_lock(&zswap_pools_lock); + + WARN_ON(pool == zswap_pool_current()); + + list_del_rcu(&pool->list); + + INIT_WORK(&pool->release_work, __zswap_pool_release); + schedule_work(&pool->release_work); + + spin_unlock(&zswap_pools_lock); +} + +static int __must_check zswap_pool_get(struct zswap_pool *pool) +{ + if (!pool) + return 0; + + return kref_get_unless_zero(&pool->kref); +} + +static void zswap_pool_put(struct zswap_pool *pool) +{ + kref_put(&pool->kref, __zswap_pool_empty); +} + /* should be called under RCU */ #ifdef CONFIG_MEMCG static inline struct mem_cgroup *mem_cgroup_from_entry(struct zswap_entry *entry) @@ -1122,51 +1167,6 @@ static void shrink_worker(struct work_struct *w) zswap_pool_put(pool); } -static int __must_check zswap_pool_get(struct zswap_pool *pool) -{ - if (!pool) - return 0; - - return kref_get_unless_zero(&pool->kref); -} - -static void __zswap_pool_release(struct work_struct *work) -{ - struct zswap_pool *pool = container_of(work, typeof(*pool), - release_work); - - synchronize_rcu(); - - /* nobody should have been able to get a kref... */ - WARN_ON(kref_get_unless_zero(&pool->kref)); - - /* pool is now off zswap_pools list and has no references. */ - zswap_pool_destroy(pool); -} - -static void __zswap_pool_empty(struct kref *kref) -{ - struct zswap_pool *pool; - - pool = container_of(kref, typeof(*pool), kref); - - spin_lock(&zswap_pools_lock); - - WARN_ON(pool == zswap_pool_current()); - - list_del_rcu(&pool->list); - - INIT_WORK(&pool->release_work, __zswap_pool_release); - schedule_work(&pool->release_work); - - spin_unlock(&zswap_pools_lock); -} - -static void zswap_pool_put(struct zswap_pool *pool) -{ - kref_put(&pool->kref, __zswap_pool_empty); -} - /********************************* * param callbacks **********************************/ From patchwork Tue Jan 30 01:36:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 193781 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp942810dyb; Mon, 29 Jan 2024 17:45:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IGc1YQtYQZv12KQzZaiTFq5reC+GgrkC2JFcwjSQDgJRbWx8e1eDO+gNdDhejugww2GT8TN X-Received: by 2002:a05:620a:28ca:b0:784:a63:6566 with SMTP id l10-20020a05620a28ca00b007840a636566mr824081qkp.10.1706579134718; Mon, 29 Jan 2024 17:45:34 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706579134; cv=pass; d=google.com; s=arc-20160816; b=vuRxibxZkA/hnnJluSXELHQIN6fxg8bF1QL9qEMcWpRHSP7hIQku+DYKJqvZdEc3ny yzeVIQgjVqw6O5kAZbiqZ3eMzcJ47B2cImfsQuITU4G+qFUQzl9kdiz/s+Z4SMZCpyxc Y3B0d7vOp3OEh0mVhzmQEeCqv0tuC5Cs0Ddg+qHKQW+MTZ13lpCRwNcX1sGQ2OQ0h27g kQvYfEvVYoYUK33zupSYQY7rTmJWg/Z9NfrY8kaFRp3o5ObFedAK3LaJiqWPv1J9uemp sk/1t0tHkQCFJ8PAYYalsmNXm5wfGfISIcqeTHUAXfBb8AU19Qvhtu4yrg6+txMxhaU3 ne7g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=6SQ/aPKt2RcQwCRNk0SkwceZy4++62pAYToaUxPVif8=; fh=pEdtrChUTDmvuLILGSavwskaPZjBcM3e1cB9O4e7o4U=; b=OBs99+dH1WQMarO+xRGFNTkq+pnNAs9u6+X+Fj9lM5c2dsr6W2XeYDtPapxmLZH4e/ bsj8OOmcekKDC7sQjGx20cqp39nB0FhCqE1+nQa8MAo6/NK3g5doYFxDGLq1CgW4vVLs +69Asc8HScxeCiJLvaK4abWN4jPqg7glQ16Hr+P0FruJsVk3CbuEAb0uxNEeQ9Lr+dlj lhvkQk3DoeuCVh0lH7MsAAxmpn41/BeMQoAEwQZlbXsvfwwvJiAHpLFglXx7Jxj1N3e5 trIZasY3gCo/TGxWJOtwicb5dgJoCKSbjNonnd5FGnckzKSzZ6A9HyordgUm9H16963t Xm/g== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=z3i79o8n; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43751-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43751-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id p4-20020a05620a056400b00783de6351b4si7786335qkp.41.2024.01.29.17.45.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:45:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43751-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=z3i79o8n; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43751-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43751-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 67F881C22153 for ; Tue, 30 Jan 2024 01:45:34 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 95DA553802; Tue, 30 Jan 2024 01:42:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b="z3i79o8n" Received: from mail-qk1-f178.google.com (mail-qk1-f178.google.com [209.85.222.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 753FB383B1 for ; Tue, 30 Jan 2024 01:42:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578951; cv=none; b=REhr7txdjstltC/4GowZNPj7bUdYCSvLfmu1UvjaWALz0FKtSGe1sALapdP4X83QR2wz+eUgeZDToJxwFM36JQi6xix5Vhd/xrNBWvABIF3CEWqCZAqAm4KoB/gNcX1Yo+EfsD1ZqTy6lIM6s+xT8/3xNjHYgI2/v4KSUnPs06I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578951; c=relaxed/simple; bh=Buqaio9vCxFs4jxwDb/cl7MtdsrmjbRE2WvaIIBaHdo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rPuCNpyO/we+3UaFKKF9C+KeYCshCSZUyn5q4bmu6Uq5iFXiXuCP+vr9u/FukglXXbgV8EbEJH8lijhY8FxswCHCjKZEGzyXJ/1Hml95q2gWc59RQZTxK+2r/dZTXs4yVZ9H6V2iKRMPxOy6yA+8NTyu09Cks7P5cgenj+e1HKQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b=z3i79o8n; arc=none smtp.client-ip=209.85.222.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Received: by mail-qk1-f178.google.com with SMTP id af79cd13be357-783f27300b1so177391485a.1 for ; Mon, 29 Jan 2024 17:42:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578948; x=1707183748; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6SQ/aPKt2RcQwCRNk0SkwceZy4++62pAYToaUxPVif8=; b=z3i79o8nEJUB6l4W4nAwTvXxdxY32XVqrTRRbBxjXfrIA3vU7LpxTlc5BPPJYF+9f1 F2gangf9C+CkScZi0XH+iaoNnXUAMcye/6n4k2JpYQO3xH+qHug3HyABV6J+cDPplu8c xIQiJfEPic9Vvel8CaNFXe4C92Z8F3PvNNHhHe0GNqiwLHB2/invZRpfRQ38tKvDhQPP 4xz3s70UmzvgVmyGZZsNR+xIHh7hcnswn//i6mJy14M3iIiU6EsnHxUU/GaU1m0/gOlk QQalqxkyfVRV31bD3+3g1118sxQ+YtxeP2WbLDt4vMJVR7NO4gEnEgNgOo5+zrKk34mJ kuCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578948; x=1707183748; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6SQ/aPKt2RcQwCRNk0SkwceZy4++62pAYToaUxPVif8=; b=QlGVRB4Z4isW7727lOf7Q3Njb/n3qTFpzBrPgvy4F7O+bsfKZYT18jiUkZopoVtncX P1kRM2e8RYtW9ICs8zspmvtP5Et3TAN/mSvhFnDNTVtnO4OWMEjEm8oYfVaSEoCjaJZP VeZC+l2G4kNLHUo4EPaZJqz8wC6cjPmWBoNRi0Dr/gqteluAmdfCjiCsnOEh6I7+NdIu KbOXP6fTtor0G9ulN2xfY9k49J7bUdQVvPc/WQlXYG2YiZElKBJaAWkv14NJcLEXIap9 59ZGXJBF3ZcSguGceHF8EexQeoMbbfAf32D7y6ASmzEe4GukRaMT0c/OwatHKiWPU0tZ PDXA== X-Gm-Message-State: AOJu0Yz6kVCGVRyz4BKRerNMzqhLmf+4u6OYjK7K6mSQ4RcC1OSTu7rr 5EPWtSkRCuqKDT/vZm+eHexFY+Rejf/IYbqfE3TGG57qaf/osHeR78Yl0hm7DkQynk4his5iI+P B X-Received: by 2002:ac8:5994:0:b0:42a:b152:c0fe with SMTP id e20-20020ac85994000000b0042ab152c0femr1228897qte.31.1706578948436; Mon, 29 Jan 2024 17:42:28 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id ie7-20020a05622a698700b00427fabefe3bsm4023363qtb.52.2024.01.29.17.42.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:28 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 12/20] mm: zswap: function ordering: zswap_pools Date: Mon, 29 Jan 2024 20:36:48 -0500 Message-ID: <20240130014208.565554-13-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789477922841438088 X-GMAIL-MSGID: 1789477922841438088 Move the operations against the global zswap_pools list (current pool, last, find) to the pool section. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 150 ++++++++++++++++++++++++++--------------------------- 1 file changed, 73 insertions(+), 77 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 33775f2224b7..168afd6767b3 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -517,6 +517,79 @@ static void zswap_pool_put(struct zswap_pool *pool) kref_put(&pool->kref, __zswap_pool_empty); } +static struct zswap_pool *__zswap_pool_current(void) +{ + struct zswap_pool *pool; + + pool = list_first_or_null_rcu(&zswap_pools, typeof(*pool), list); + WARN_ONCE(!pool && zswap_has_pool, + "%s: no page storage pool!\n", __func__); + + return pool; +} + +static struct zswap_pool *zswap_pool_current(void) +{ + assert_spin_locked(&zswap_pools_lock); + + return __zswap_pool_current(); +} + +static struct zswap_pool *zswap_pool_current_get(void) +{ + struct zswap_pool *pool; + + rcu_read_lock(); + + pool = __zswap_pool_current(); + if (!zswap_pool_get(pool)) + pool = NULL; + + rcu_read_unlock(); + + return pool; +} + +static struct zswap_pool *zswap_pool_last_get(void) +{ + struct zswap_pool *pool, *last = NULL; + + rcu_read_lock(); + + list_for_each_entry_rcu(pool, &zswap_pools, list) + last = pool; + WARN_ONCE(!last && zswap_has_pool, + "%s: no page storage pool!\n", __func__); + if (!zswap_pool_get(last)) + last = NULL; + + rcu_read_unlock(); + + return last; +} + +/* type and compressor must be null-terminated */ +static struct zswap_pool *zswap_pool_find_get(char *type, char *compressor) +{ + struct zswap_pool *pool; + + assert_spin_locked(&zswap_pools_lock); + + list_for_each_entry_rcu(pool, &zswap_pools, list) { + if (strcmp(pool->tfm_name, compressor)) + continue; + /* all zpools share the same type */ + if (strcmp(zpool_get_type(pool->zpools[0]), type)) + continue; + /* if we can't get it, it's about to be destroyed */ + if (!zswap_pool_get(pool)) + continue; + return pool; + } + + return NULL; +} + /* should be called under RCU */ #ifdef CONFIG_MEMCG static inline struct mem_cgroup *mem_cgroup_from_entry(struct zswap_entry *entry) @@ -938,83 +1011,6 @@ static int zswap_cpu_comp_dead(unsigned int cpu, struct hlist_node *node) return 0; } -/********************************* -* pool functions -**********************************/ - -static struct zswap_pool *__zswap_pool_current(void) -{ - struct zswap_pool *pool; - - pool = list_first_or_null_rcu(&zswap_pools, typeof(*pool), list); - WARN_ONCE(!pool && zswap_has_pool, - "%s: no page storage pool!\n", __func__); - - return pool; -} - -static struct zswap_pool *zswap_pool_current(void) -{ - assert_spin_locked(&zswap_pools_lock); - - return __zswap_pool_current(); -} - -static struct zswap_pool *zswap_pool_current_get(void) -{ - struct zswap_pool *pool; - - rcu_read_lock(); - - pool = __zswap_pool_current(); - if (!zswap_pool_get(pool)) - pool = NULL; - - rcu_read_unlock(); - - return pool; -} - -static struct zswap_pool *zswap_pool_last_get(void) -{ - struct zswap_pool *pool, *last = NULL; - - rcu_read_lock(); - - list_for_each_entry_rcu(pool, &zswap_pools, list) - last = pool; - WARN_ONCE(!last && zswap_has_pool, - "%s: no page storage pool!\n", __func__); - if (!zswap_pool_get(last)) - last = NULL; - - rcu_read_unlock(); - - return last; -} - -/* type and compressor must be null-terminated */ -static struct zswap_pool *zswap_pool_find_get(char *type, char *compressor) -{ - struct zswap_pool *pool; - - assert_spin_locked(&zswap_pools_lock); - - list_for_each_entry_rcu(pool, &zswap_pools, list) { - if (strcmp(pool->tfm_name, compressor)) - continue; - /* all zpools share the same type */ - if (strcmp(zpool_get_type(pool->zpools[0]), type)) - continue; - /* if we can't get it, it's about to be destroyed */ - if (!zswap_pool_get(pool)) - continue; - return pool; - } - - return NULL; -} - static enum lru_status shrink_memcg_cb(struct list_head *item, struct list_lru_one *l, spinlock_t *lock, void *arg) { From patchwork Tue Jan 30 01:36:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 193783 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp942955dyb; Mon, 29 Jan 2024 17:46:07 -0800 (PST) X-Google-Smtp-Source: AGHT+IEEP4XyGIW4EmB1Ocj6+lxqAmVJPVwfqoOygmUYff4og5PBOq4rymkNwfTp6PXPpulGyh0s X-Received: by 2002:a05:6402:1c8b:b0:55f:3463:2109 with SMTP id cy11-20020a0564021c8b00b0055f34632109mr247169edb.10.1706579166995; Mon, 29 Jan 2024 17:46:06 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706579166; cv=pass; d=google.com; s=arc-20160816; b=tFWDX39IzNxxCf9n48UtzwuL0N17YMNtIovGsCOmxTUEqj80/qv2QakELtzRnluq6X oJ24j6z+OlFeSGeGkRcxG7bOOdgzOaEFD1G78aaS6VXO1Y2rS2OaPdcqjS+6hScex/fS lf5RwjsZuTS/RtWTssZ1I4aEmHtppxT3FROmYLfMC6KB2TOpsf1uNU1FAYSXbUoCtdna VN9MLwjx5ay6sWq8Zig+j9GKsFBTTxejZxrEq/kKjspZFog4re14IUXwkcV/yNBF4+76 0OS8xrFRQsxo7qqLMlctL6ZFuXLWBb6jzmwAc208FbHA5AqAiQlmfL0r3fCu7ge4Adt9 UpYg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=ivn7Y9AQrDvvewOffcfUAdTab4Q3w5wTeKa9SOf1gRc=; fh=pEdtrChUTDmvuLILGSavwskaPZjBcM3e1cB9O4e7o4U=; b=IenjwjYjvRLbmHNVX1VxJ8V2ByZgjQRdw+eOq14FpnA9FWbArDwakjwawBjMFzCVHC fmTYUlY8+qkLqCAgdL7hz2//wzmqBtVuegn0jQLlnfV3RQP77mXIR0QG/yiVkZQZd7cO tWsz8HTghqFpNGDsUqZi+7nSLoL7KetHe83W2tONFC0vqqKIvZOsStAyrVQEoPFRW0ne ciESA6Gq4UeRj5ayIjojo88bSihUTc30HH2F+khE7kJYnVl5nAqzjBE+l7C/3Rma8bOr 1+xnUimZITttvg4oaFZHK0oQW7VeCfLkDI2dwohGy7puy0IrXDTBIx3RF+qKvdOcq7Re hVzg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=FqwZ7i5h; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43752-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43752-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id u29-20020a50c05d000000b0055f19a354c9si907604edd.537.2024.01.29.17.46.06 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:46:06 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43752-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=FqwZ7i5h; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43752-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43752-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 728F41F289B6 for ; Tue, 30 Jan 2024 01:46:06 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6D50E55E5C; Tue, 30 Jan 2024 01:42:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b="FqwZ7i5h" Received: from mail-qt1-f172.google.com (mail-qt1-f172.google.com [209.85.160.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0C71638F83 for ; Tue, 30 Jan 2024 01:42:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578952; cv=none; b=u7kzojg+nCPZoAnH30wy6dB9wa6QezHWeiHHWsMgnvfx/YJAnkSYh4t9ScoDzjZPn+Befg+wAWglQp5ut/1GO3ShfeTmm0Ep04kQ9KRgPy/zrLzw1UHgnttMRBIWz/HjyJa8tpmA1u4Ri0P06uuv111ltXPNKtQaU2et/NVpfsU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578952; c=relaxed/simple; bh=X5Tv+PeG/ZURIxGV7LTek+nHYXs1A+DSPnPjP2zYBUY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=K9xlZdxm1MlepREt1GO0HU6VbCedi0LBVgmFg91q52ozhP0wo+CZWBhaqIatDn5Zi5Yk61n71GNDVRBnaoaALg0Tqwg5pv8Q0iF/5kuCioWFtzUq9K+DD+0VE50N88qfaDtBgoeTYe2SkTYjA5BkTTNNcARpihM1BsdY6dXkBW4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b=FqwZ7i5h; arc=none smtp.client-ip=209.85.160.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Received: by mail-qt1-f172.google.com with SMTP id d75a77b69052e-42a8900bb93so35053391cf.1 for ; Mon, 29 Jan 2024 17:42:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578950; x=1707183750; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ivn7Y9AQrDvvewOffcfUAdTab4Q3w5wTeKa9SOf1gRc=; b=FqwZ7i5hQvSUvl4GUZurve89f9OYym5VNiWbj4hUPPqDusWlHg6CvIsnfjfRlYmg9C 948Slmbt0R8Lrv8CCHOLCMFuZeGOGXOgiL2yPwCgqj5AwRB+tv/EnbKC7vz+tCSVaQ1R 9QjNXeDjRrV8GVeSqkqikRrvHURyEOAXlyDsew1hvGFHRiRiTZaVRnDp4y/ZUcn6lqx2 sLuDXDMBOCCN0CokkL/NAroMPVOzsQkEjlXJ9No1mGXfMidkKjY2L6jKxUP5Fa406Gif i1a3PmiTxd91pLcRMz8YsLzd1h07EZIFBKW7pnG2Cv/5eWounpYXEe/BwaBmZWPumnVo CvNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578950; x=1707183750; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ivn7Y9AQrDvvewOffcfUAdTab4Q3w5wTeKa9SOf1gRc=; b=ZcC9ckXAp7BaIR37ewfesr1r5rMtYCLhr/RznbLvWzZ4LltizzGGS9ViJ7HMKhcqQC Vfc8ZZfmtqmpDi5+FqQdmNkCB/PeVq6pO7Q8CskfWdFF8NNUxi/CRGPijuWoCCOgGe5M u4MDFWTouUNysfOVTC0CSSYpapZ3M4zYURi4JL4MvmSHIb9W/4MWEbcdneJFsaodASGC m9kP3Hi22G3QFwYFU3Jg7IJsQg6GfQNL9LcztH/oZFK8m1CvYgbkeuhn7dU+425PFNLk ABrS3R+rXD0mIaQdwlDF0DSEMO6nZKhgxIyWnOLqwvb8+dOnrdPksgbujJ520du6g2vr UmTg== X-Gm-Message-State: AOJu0YwLAeqUCX88W2xMMqPErWigNyr8s17A25M9ZVstKtlo3t5/Ie/q BQGyZkWCYG2bWnl/fEK7YN07uWI6E4XGRdsMCDQ1q6+HFDqRb1FBwSf5mg9wcSNAQ7GEXJCsrt8 6 X-Received: by 2002:a05:622a:1e09:b0:42a:600c:2d7e with SMTP id br9-20020a05622a1e0900b0042a600c2d7emr9275227qtb.51.1706578949706; Mon, 29 Jan 2024 17:42:29 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id cp7-20020a05622a420700b0042a6859ff3esm3078179qtb.61.2024.01.29.17.42.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:29 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 13/20] mm: zswap: function ordering: pool params Date: Mon, 29 Jan 2024 20:36:49 -0500 Message-ID: <20240130014208.565554-14-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789477956813099766 X-GMAIL-MSGID: 1789477956813099766 The parameters primarily control pool attributes. Move those operations up to the pool section. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 312 ++++++++++++++++++++++++++--------------------------- 1 file changed, 156 insertions(+), 156 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 168afd6767b3..e650fc587116 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -590,6 +590,162 @@ static struct zswap_pool *zswap_pool_find_get(char *type, char *compressor) return NULL; } +/********************************* +* param callbacks +**********************************/ + +static bool zswap_pool_changed(const char *s, const struct kernel_param *kp) +{ + /* no change required */ + if (!strcmp(s, *(char **)kp->arg) && zswap_has_pool) + return false; + return true; +} + +/* val must be a null-terminated string */ +static int __zswap_param_set(const char *val, const struct kernel_param *kp, + char *type, char *compressor) +{ + struct zswap_pool *pool, *put_pool = NULL; + char *s = strstrip((char *)val); + int ret = 0; + bool new_pool = false; + + mutex_lock(&zswap_init_lock); + switch (zswap_init_state) { + case ZSWAP_UNINIT: + /* if this is load-time (pre-init) param setting, + * don't create a pool; that's done during init. + */ + ret = param_set_charp(s, kp); + break; + case ZSWAP_INIT_SUCCEED: + new_pool = zswap_pool_changed(s, kp); + break; + case ZSWAP_INIT_FAILED: + pr_err("can't set param, initialization failed\n"); + ret = -ENODEV; + } + mutex_unlock(&zswap_init_lock); + + /* no need to create a new pool, return directly */ + if (!new_pool) + return ret; + + if (!type) { + if (!zpool_has_pool(s)) { + pr_err("zpool %s not available\n", s); + return -ENOENT; + } + type = s; + } else if (!compressor) { + if (!crypto_has_acomp(s, 0, 0)) { + pr_err("compressor %s not available\n", s); + return -ENOENT; + } + compressor = s; + } else { + WARN_ON(1); + return -EINVAL; + } + + spin_lock(&zswap_pools_lock); + + pool = zswap_pool_find_get(type, compressor); + if (pool) { + zswap_pool_debug("using existing", pool); + WARN_ON(pool == zswap_pool_current()); + list_del_rcu(&pool->list); + } + + spin_unlock(&zswap_pools_lock); + + if (!pool) + pool = zswap_pool_create(type, compressor); + + if (pool) + ret = param_set_charp(s, kp); + else + ret = -EINVAL; + + spin_lock(&zswap_pools_lock); + + if (!ret) { + put_pool = zswap_pool_current(); + list_add_rcu(&pool->list, &zswap_pools); + zswap_has_pool = true; + } else if (pool) { + /* add the possibly pre-existing pool to the end of the pools + * list; if it's new (and empty) then it'll be removed and + * destroyed by the put after we drop the lock + */ + list_add_tail_rcu(&pool->list, &zswap_pools); + put_pool = pool; + } + + spin_unlock(&zswap_pools_lock); + + if (!zswap_has_pool && !pool) { + /* if initial pool creation failed, and this pool creation also + * failed, maybe both compressor and zpool params were bad. + * Allow changing this param, so pool creation will succeed + * when the other param is changed. We already verified this + * param is ok in the zpool_has_pool() or crypto_has_acomp() + * checks above. + */ + ret = param_set_charp(s, kp); + } + + /* drop the ref from either the old current pool, + * or the new pool we failed to add + */ + if (put_pool) + zswap_pool_put(put_pool); + + return ret; +} + +static int zswap_compressor_param_set(const char *val, + const struct kernel_param *kp) +{ + return __zswap_param_set(val, kp, zswap_zpool_type, NULL); +} + +static int zswap_zpool_param_set(const char *val, + const struct kernel_param *kp) +{ + return __zswap_param_set(val, kp, NULL, zswap_compressor); +} + +static int zswap_enabled_param_set(const char *val, + const struct kernel_param *kp) +{ + int ret = -ENODEV; + + /* if this is load-time (pre-init) param setting, only set param. */ + if (system_state != SYSTEM_RUNNING) + return param_set_bool(val, kp); + + mutex_lock(&zswap_init_lock); + switch (zswap_init_state) { + case ZSWAP_UNINIT: + if (zswap_setup()) + break; + fallthrough; + case ZSWAP_INIT_SUCCEED: + if (!zswap_has_pool) + pr_err("can't enable, no pool configured\n"); + else + ret = param_set_bool(val, kp); + break; + case ZSWAP_INIT_FAILED: + pr_err("can't enable, initialization failed\n"); + } + mutex_unlock(&zswap_init_lock); + + return ret; +} + /* should be called under RCU */ #ifdef CONFIG_MEMCG static inline struct mem_cgroup *mem_cgroup_from_entry(struct zswap_entry *entry) @@ -1163,162 +1319,6 @@ static void shrink_worker(struct work_struct *w) zswap_pool_put(pool); } -/********************************* -* param callbacks -**********************************/ - -static bool zswap_pool_changed(const char *s, const struct kernel_param *kp) -{ - /* no change required */ - if (!strcmp(s, *(char **)kp->arg) && zswap_has_pool) - return false; - return true; -} - -/* val must be a null-terminated string */ -static int __zswap_param_set(const char *val, const struct kernel_param *kp, - char *type, char *compressor) -{ - struct zswap_pool *pool, *put_pool = NULL; - char *s = strstrip((char *)val); - int ret = 0; - bool new_pool = false; - - mutex_lock(&zswap_init_lock); - switch (zswap_init_state) { - case ZSWAP_UNINIT: - /* if this is load-time (pre-init) param setting, - * don't create a pool; that's done during init. - */ - ret = param_set_charp(s, kp); - break; - case ZSWAP_INIT_SUCCEED: - new_pool = zswap_pool_changed(s, kp); - break; - case ZSWAP_INIT_FAILED: - pr_err("can't set param, initialization failed\n"); - ret = -ENODEV; - } - mutex_unlock(&zswap_init_lock); - - /* no need to create a new pool, return directly */ - if (!new_pool) - return ret; - - if (!type) { - if (!zpool_has_pool(s)) { - pr_err("zpool %s not available\n", s); - return -ENOENT; - } - type = s; - } else if (!compressor) { - if (!crypto_has_acomp(s, 0, 0)) { - pr_err("compressor %s not available\n", s); - return -ENOENT; - } - compressor = s; - } else { - WARN_ON(1); - return -EINVAL; - } - - spin_lock(&zswap_pools_lock); - - pool = zswap_pool_find_get(type, compressor); - if (pool) { - zswap_pool_debug("using existing", pool); - WARN_ON(pool == zswap_pool_current()); - list_del_rcu(&pool->list); - } - - spin_unlock(&zswap_pools_lock); - - if (!pool) - pool = zswap_pool_create(type, compressor); - - if (pool) - ret = param_set_charp(s, kp); - else - ret = -EINVAL; - - spin_lock(&zswap_pools_lock); - - if (!ret) { - put_pool = zswap_pool_current(); - list_add_rcu(&pool->list, &zswap_pools); - zswap_has_pool = true; - } else if (pool) { - /* add the possibly pre-existing pool to the end of the pools - * list; if it's new (and empty) then it'll be removed and - * destroyed by the put after we drop the lock - */ - list_add_tail_rcu(&pool->list, &zswap_pools); - put_pool = pool; - } - - spin_unlock(&zswap_pools_lock); - - if (!zswap_has_pool && !pool) { - /* if initial pool creation failed, and this pool creation also - * failed, maybe both compressor and zpool params were bad. - * Allow changing this param, so pool creation will succeed - * when the other param is changed. We already verified this - * param is ok in the zpool_has_pool() or crypto_has_acomp() - * checks above. - */ - ret = param_set_charp(s, kp); - } - - /* drop the ref from either the old current pool, - * or the new pool we failed to add - */ - if (put_pool) - zswap_pool_put(put_pool); - - return ret; -} - -static int zswap_compressor_param_set(const char *val, - const struct kernel_param *kp) -{ - return __zswap_param_set(val, kp, zswap_zpool_type, NULL); -} - -static int zswap_zpool_param_set(const char *val, - const struct kernel_param *kp) -{ - return __zswap_param_set(val, kp, NULL, zswap_compressor); -} - -static int zswap_enabled_param_set(const char *val, - const struct kernel_param *kp) -{ - int ret = -ENODEV; - - /* if this is load-time (pre-init) param setting, only set param. */ - if (system_state != SYSTEM_RUNNING) - return param_set_bool(val, kp); - - mutex_lock(&zswap_init_lock); - switch (zswap_init_state) { - case ZSWAP_UNINIT: - if (zswap_setup()) - break; - fallthrough; - case ZSWAP_INIT_SUCCEED: - if (!zswap_has_pool) - pr_err("can't enable, no pool configured\n"); - else - ret = param_set_bool(val, kp); - break; - case ZSWAP_INIT_FAILED: - pr_err("can't enable, initialization failed\n"); - } - mutex_unlock(&zswap_init_lock); - - return ret; -} - static bool zswap_compress(struct folio *folio, struct zswap_entry *entry) { struct crypto_acomp_ctx *acomp_ctx; From patchwork Tue Jan 30 01:36:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 193782 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp942916dyb; Mon, 29 Jan 2024 17:45:58 -0800 (PST) X-Google-Smtp-Source: AGHT+IGr2eIYl8X0mmrFo3+clk//txQyZuc7tZArxEdjgq0FniBVxu1IT89NDeb7hBlJf/kB6SLZ X-Received: by 2002:a05:620a:22a4:b0:784:c30:bd8c with SMTP id p4-20020a05620a22a400b007840c30bd8cmr233570qkh.66.1706579158470; Mon, 29 Jan 2024 17:45:58 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706579158; cv=pass; d=google.com; s=arc-20160816; b=mm18/ACUOxbEgQQl5a2WnJ78VRh64gipIJ5aROeDCRKOj1xDiM7IOfPDFvQKrU0Ttr viFCi45Xp3BBVlpDVCLDIvPHU09lOxCdPqmCA41GwjHW+Ko2nuMOVEM7tsFy6YT8EjYK Cj6x+a/uUd+Ad3s/H1a4yM15djcgxEyIStocNqvfpCcsu/i9FwZX22ibFVf/JxSaKpGA e0nuqvEfd87buSr2crAxtcFPhpLHwJFMd266ojw0X7RdMS0RYsIQA0o9rkLLnlIatS4E kQlmwrIHhXWT5K+9xKctYF8CjD/662Q+mYHLiI0CuxczZdFE8bzALxHyHA9LTDsXVwD2 pItA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=IOr65lcwE97/+IL040qC2nQXATF0TLQoSMjJ/TctotE=; fh=pEdtrChUTDmvuLILGSavwskaPZjBcM3e1cB9O4e7o4U=; b=0YkQVfX9CkVx+W5Z8+jWa3ImtTAiXrlelrMkz09PoaznVC8qzwZ4l/Fgb+U2uCimHm 3mqI7SdFrUz0nOwtxkBk9xgs5D6pgLHOlPSkOwlbxLMDOLaGx6W5dS7YG9GNDbUn8yOo KDOfuxz+yfu5gqE/+ZxifGm/aOyTYyvkmWA3y0auDGahDOmc5jXRIeA/aqXq3WvMicyK EHhyWOMbJPBxd90Kp+VEQhSzqY3v6qoWpLWn/4WkjDR/PpDFwxqOmycw00zXsja6P8wD sEo+XjI2kKyMAt/+A1LlwqIEkRLgN73Fa+jP4AJqbpU5dGCmoRD468AEs6pACfTF/HAQ G0gw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=CkD3Z2+B; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43753-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43753-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id g3-20020a05620a218300b00783e24807d1si6945594qka.280.2024.01.29.17.45.58 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:45:58 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43753-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=CkD3Z2+B; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43753-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43753-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 433421C208BB for ; Tue, 30 Jan 2024 01:45:58 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D6A3654FB1; Tue, 30 Jan 2024 01:42:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b="CkD3Z2+B" Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0FAE538DE0 for ; Tue, 30 Jan 2024 01:42:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578953; cv=none; b=MwGErnV2I6li5WT+v+Bpr98cOTc6tfXbab8ljt4ErJen7oTwTySaPnmUhNZM1uiHOzuPCTX6qCKS4LnYClFh/rchmcJpktYH4fJemKxXyH8qKb/M0B6D4ddZ9qgZ35qqtsy6zQt6yJhuagF4z+0plmNKJ7IDSnlEnLMEq8wFEII= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578953; c=relaxed/simple; bh=OAJsTuMgjZCEgvmJagS5ZBSB40PFf9Qscvr3W/cEgM0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nPoRAqa1KW9E3zxW6cRvv/cbdKa51ntAmV8bpwWXUYLwqP9CHYndY7TE+kuBVpj5hM3yf/BG73iA9STlcIzAvmS8CvfuO1ZwfaxGYiP+Kw78g4l9Ax3w6MfCGsh0lbQbFwvhvIxlTbLH29KodGLOyXKN6YegSgG/jIGO5OD4Gnw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b=CkD3Z2+B; arc=none smtp.client-ip=209.85.222.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Received: by mail-qk1-f176.google.com with SMTP id af79cd13be357-783f27300b1so177393885a.1 for ; Mon, 29 Jan 2024 17:42:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578951; x=1707183751; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IOr65lcwE97/+IL040qC2nQXATF0TLQoSMjJ/TctotE=; b=CkD3Z2+BJDaLaG9KMCbpYY8ywdwzFENO/jD+aVsycZshMsfXIywSlNCoWBLCqGD1l8 jLtecnaME4plphW72l7GjiyoJe0a8gmgCuNQak8Ujh2X1Z7J2aRQekbai0DGALmROvq3 RMISGk7ryF5/od/m2Jxyvo78D829hjgzwWJoCymTQQ6ze5+zX1d2lzb3/VPmTfJnLahe bWoDJzxTDR0UUzKv0rqLQdASYpTWhEyPAicvX7hBVUbE9y6Txfi1Aufgex6gD7yzTLnP bcDceeDZUL2yScqwDpmFm8FqsxHMtYjAtu3HEjicIWcUGSsjQr/7LFxfW6X61LEV4qv1 qTpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578951; x=1707183751; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IOr65lcwE97/+IL040qC2nQXATF0TLQoSMjJ/TctotE=; b=HoMd4HCmLHaGgKv+vOyx9GrJlA5/StXrDInGc83M/zCoKfArOUSnCD2NpqEoRmtI4h S3986t7a8s58mxKw2Z92JFeATYcSpAGlgfQEZm27Es/MFJXl/zvXkO8VuVvJxAfdOAsr uIfWuX28wkOx46nyEsk9VUxyyp9v5gL22gJ2yap/4OybVBljt4lDxwn9wHai/91Ughz4 94WV4qkXQGHPhGVGFZal/l12S83Wy8UsaNBdyvsS7Ny+97dQogZJVUcl3bkBV8X6twi0 P4cuLoMnEH26G8pbH3TeRwTKfjT84ti4vxV2u557JHQiQ3bCMydlEFA/pS11I4MlVqSs gBOg== X-Gm-Message-State: AOJu0YwuMvmA4f4JeX/it5MPc/NSq4c+nj/6XbRfELsa5BE5W0s/xj0k 7kvSDvYvVlDxOZPzYk3p9ASy6zlmMUXASyvkIrXjLnkZ0f2u+UP3ioH5OH1qljg= X-Received: by 2002:a05:6214:20ce:b0:67a:a721:b1b5 with SMTP id 14-20020a05621420ce00b0067aa721b1b5mr9104092qve.112.1706578951030; Mon, 29 Jan 2024 17:42:31 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id b6-20020a05620a0f8600b007832575779esm3580991qkn.52.2024.01.29.17.42.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:30 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 14/20] mm: zswap: function ordering: public lru api Date: Mon, 29 Jan 2024 20:36:50 -0500 Message-ID: <20240130014208.565554-15-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789477947431618855 X-GMAIL-MSGID: 1789477947431618855 The zswap entry section sits awkwardly in the middle of LRU-related functions. Group the external LRU API functions first. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 37 +++++++++++++++++++------------------ 1 file changed, 19 insertions(+), 18 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index e650fc587116..511bfafc1456 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -746,6 +746,10 @@ static int zswap_enabled_param_set(const char *val, return ret; } +/********************************* +* lru functions +**********************************/ + /* should be called under RCU */ #ifdef CONFIG_MEMCG static inline struct mem_cgroup *mem_cgroup_from_entry(struct zswap_entry *entry) @@ -764,6 +768,21 @@ static inline int entry_to_nid(struct zswap_entry *entry) return page_to_nid(virt_to_page(entry)); } +void zswap_lruvec_state_init(struct lruvec *lruvec) +{ + atomic_long_set(&lruvec->zswap_lruvec_state.nr_zswap_protected, 0); +} + +void zswap_folio_swapin(struct folio *folio) +{ + struct lruvec *lruvec; + + if (folio) { + lruvec = folio_lruvec(folio); + atomic_long_inc(&lruvec->zswap_lruvec_state.nr_zswap_protected); + } +} + void zswap_memcg_offline_cleanup(struct mem_cgroup *memcg) { struct zswap_pool *pool; @@ -798,24 +817,6 @@ static void zswap_entry_cache_free(struct zswap_entry *entry) kmem_cache_free(zswap_entry_cache, entry); } -/********************************* -* zswap lruvec functions -**********************************/ -void zswap_lruvec_state_init(struct lruvec *lruvec) -{ - atomic_long_set(&lruvec->zswap_lruvec_state.nr_zswap_protected, 0); -} - -void zswap_folio_swapin(struct folio *folio) -{ - struct lruvec *lruvec; - - if (folio) { - lruvec = folio_lruvec(folio); - atomic_long_inc(&lruvec->zswap_lruvec_state.nr_zswap_protected); - } -} - /********************************* * lru functions **********************************/ From patchwork Tue Jan 30 01:36:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 193793 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp945688dyb; Mon, 29 Jan 2024 17:54:49 -0800 (PST) X-Google-Smtp-Source: AGHT+IHTzzl6c9NRKWpSmvYirK/fwXchT+KwD2uXfGipL6kw+pYZcqRaaNMGuyaIyEDR17HA3Nni X-Received: by 2002:a05:6a20:8630:b0:19c:6dc3:a500 with SMTP id l48-20020a056a20863000b0019c6dc3a500mr2714835pze.19.1706579689624; Mon, 29 Jan 2024 17:54:49 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706579689; cv=pass; d=google.com; s=arc-20160816; b=V3WgW+ejds5ce5mE63UuyW9w1CfEtgppJzJ7C1Y3fdKoLr9oz/wlE2J03/aXkkwbFk /peGY2YOvYYg/mrZszw//3ctLRz+jpSdlgsoCzH0BhjwYRPFfQ3kPyLhUUJ6d+C845Dm cZbnVu30Jza5pQubHwwLtsnnD+0zNdI+lWBfq18XroR/J5DhpKIfDliSOB/cH3ERojo6 AK0LMhoTo/S/p1dRLhhbPspapTztgpyXRyN+pwvkOCZ8+b4YIRXdLP0zh79hUpJ19VKd mmkQxGAsm5RJWhnuHA2IoRjMFSEFkOqDatDUQewqhFavTcs/ivRw26xVUOXwArVP9RVI bndQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=F1iB6scB+xehMKrtYira4F0YAgxikUXsHI43Fnlp9WY=; fh=pEdtrChUTDmvuLILGSavwskaPZjBcM3e1cB9O4e7o4U=; b=WgAJHavpeaNf/rPtUKColT/Iy1nCpEPGmPEZmzQKbfnUbhzoX+ccyWJ7iV7vxz5F/G OAo9cwvnfnvnZx4FHBEgxgENPjc5eSMayJMSR35cxSFOqFdG3/CtK6om2VYKAO/2dvoX y2CPocJepi8dbr/a/xJM7cWE8OAuYPjIp6Toxj4+0HrsO1oD8yixvRRYbizf23+Ckf5e EMUzaYVnG8d1iHYcNIGxtscf2EbfHRtxjqi8xEweSaURgJEafLMmZf7JpiTQeN8gBsYC 1XInKRYv3YJumhVkdcsDASpFrBr0sjvhq5mtQVHAv0kovhxFJD5wLiAVngzUHr72vH28 OD6Q== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b="r/3KW+de"; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43756-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43756-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id q17-20020a17090311d100b001d5f261a528si3874954plh.398.2024.01.29.17.54.49 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:54:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43756-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b="r/3KW+de"; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43756-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43756-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id A909FB24287 for ; Tue, 30 Jan 2024 01:46:44 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B23211586E8; Tue, 30 Jan 2024 01:42:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b="r/3KW+de" Received: from mail-qv1-f50.google.com (mail-qv1-f50.google.com [209.85.219.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A61353E3D for ; Tue, 30 Jan 2024 01:42:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578957; cv=none; b=dvq8AD7u4zDdsaN0cX/JoBWOeBorPNv58tHHONjbCiGB/3P67me0mxga4Dd5CMwQOfKh9sLhI5RcsiXVk7206FNMtlndbfZmm4FUCYOXtaXzz17jL92Dbn8BB/CRPPIxIqX824jiemqKlACybpk0NzaCrifKUY4eniQiPXWN4VY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578957; c=relaxed/simple; bh=T6a005OxCxWw2FWJbDa32NRjtAb1L9KNEaMrj9iwn0s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rA01H2P1v1qXwvOx7O7VWyTde83p35wV+ycUGHDmhgIk3PbaOBBqGah/dvrCJFOZogvGh3n88XggdtJwaejRAd2N2UviHu+6ByAh2BkdsTUx1Q1MNUyzLeS9k+Qcz39YSHVVjBFZWM/mSl2Gx11YrFm9X+Fu+W2nZORZZ+L+T7o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b=r/3KW+de; arc=none smtp.client-ip=209.85.219.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Received: by mail-qv1-f50.google.com with SMTP id 6a1803df08f44-6818f3cf00aso30235016d6.0 for ; Mon, 29 Jan 2024 17:42:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578954; x=1707183754; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=F1iB6scB+xehMKrtYira4F0YAgxikUXsHI43Fnlp9WY=; b=r/3KW+dew/AJTme8z5hUhJMp8IwQbvCNOl5UzmwEUGTWXDk5GGFL6tddXHQYCVE7Uc uMa5+YM+yBgGqA6KS+S1aEPb1KGWDV4KfhZSQXWGqGiyM1VWlaBGCfW4b/t299a6u0r3 RoBIafI7p3CtX26lo7IGcSX0QB6CL5meKyGfxaSCBDFpSFKNTW5hketENFZQ2eTeX5S2 /rzhmuno1XiOwURQfQRAFlY84Rb/PV/prI7qWQbmLVQmBfb7wuJmHGCnlzdcbUzlHUxM BfQ9ZRqSg+uYUQ8EgVkR17WJSc08bZg5UfFDP71O8vYzm95KW7hu8j0J/nWmiJo2HKKg YIKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578954; x=1707183754; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=F1iB6scB+xehMKrtYira4F0YAgxikUXsHI43Fnlp9WY=; b=wRr30V27yOxAOcDCpBvIlRb3UVvIlUHdawJbBecPSSFdzF8xlGnnCBLsWj6bXFwyar XONVKV7qhRjPJuHuB2XqTySRWl4wFlDhVIL4p2wNtuU/XbcGhobh7HSccXA3YCFQjSX0 Q0ThjibtQGaqzOvRujq2EAESVBVzNtZURXBl9Ys51094mHAllhZAJOIRDuFtJ+LxAFsp Pnnf6d9159GhYXb/2a8t/yleqP/FUWAlGsJ1oZHfZfizSIrQnmVT8BSJwWkgFmJZDOoc opusx/5tS5Q3clmpPLHvsutm6/a5yNeQNbe1jlkAbHTZXT1ICIUWhMN5TQfnNC2IuGw5 d0RA== X-Gm-Message-State: AOJu0Yz/cLvkB5/Xta89YPfUw6UvfXQQQD8Ng4Lo1V2OZJltp4wD1t8H 3gZNfUQNmwxoTSXCC8pZ0GGP1qLC39hH149ANg5ITERIvvD8tscGYFSzCPzBsts= X-Received: by 2002:a05:6214:628:b0:681:5800:9f23 with SMTP id a8-20020a056214062800b0068158009f23mr8224751qvx.115.1706578954521; Mon, 29 Jan 2024 17:42:34 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id mw2-20020a05621433c200b006891e1f098csm3996652qvb.78.2024.01.29.17.42.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:34 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 17/20] mm: zswap: function ordering: compress & decompress functions Date: Mon, 29 Jan 2024 20:36:53 -0500 Message-ID: <20240130014208.565554-18-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789478504815971538 X-GMAIL-MSGID: 1789478504815971538 Writeback needs to decompress. Move the (de)compression API above what will be the consolidated shrinking/writeback code. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 207 +++++++++++++++++++++++++++-------------------------- 1 file changed, 105 insertions(+), 102 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 80adc2f7d1a2..17356b2e35c2 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -992,6 +992,111 @@ static void zswap_invalidate_entry(struct zswap_tree *tree, zswap_entry_put(entry); } +/********************************* +* compressed storage functions +**********************************/ +static bool zswap_compress(struct folio *folio, struct zswap_entry *entry) +{ + struct crypto_acomp_ctx *acomp_ctx; + struct scatterlist input, output; + unsigned int dlen = PAGE_SIZE; + unsigned long handle; + struct zpool *zpool; + char *buf; + gfp_t gfp; + int ret; + u8 *dst; + + acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); + + mutex_lock(&acomp_ctx->mutex); + + dst = acomp_ctx->buffer; + sg_init_table(&input, 1); + sg_set_page(&input, &folio->page, PAGE_SIZE, 0); + + /* + * We need PAGE_SIZE * 2 here since there maybe over-compression case, + * and hardware-accelerators may won't check the dst buffer size, so + * giving the dst buffer with enough length to avoid buffer overflow. + */ + sg_init_one(&output, dst, PAGE_SIZE * 2); + acomp_request_set_params(acomp_ctx->req, &input, &output, PAGE_SIZE, dlen); + + /* + * it maybe looks a little bit silly that we send an asynchronous request, + * then wait for its completion synchronously. This makes the process look + * synchronous in fact. + * Theoretically, acomp supports users send multiple acomp requests in one + * acomp instance, then get those requests done simultaneously. but in this + * case, zswap actually does store and load page by page, there is no + * existing method to send the second page before the first page is done + * in one thread doing zwap. + * but in different threads running on different cpu, we have different + * acomp instance, so multiple threads can do (de)compression in parallel. + */ + ret = crypto_wait_req(crypto_acomp_compress(acomp_ctx->req), &acomp_ctx->wait); + dlen = acomp_ctx->req->dlen; + if (ret) { + zswap_reject_compress_fail++; + goto unlock; + } + + zpool = zswap_find_zpool(entry); + gfp = __GFP_NORETRY | __GFP_NOWARN | __GFP_KSWAPD_RECLAIM; + if (zpool_malloc_support_movable(zpool)) + gfp |= __GFP_HIGHMEM | __GFP_MOVABLE; + ret = zpool_malloc(zpool, dlen, gfp, &handle); + if (ret == -ENOSPC) { + zswap_reject_compress_poor++; + goto unlock; + } + if (ret) { + zswap_reject_alloc_fail++; + goto unlock; + } + + buf = zpool_map_handle(zpool, handle, ZPOOL_MM_WO); + memcpy(buf, dst, dlen); + zpool_unmap_handle(zpool, handle); + + entry->handle = handle; + entry->length = dlen; + +unlock: + mutex_unlock(&acomp_ctx->mutex); + return ret == 0; +} + +static void zswap_decompress(struct zswap_entry *entry, struct page *page) +{ + struct zpool *zpool = zswap_find_zpool(entry); + struct scatterlist input, output; + struct crypto_acomp_ctx *acomp_ctx; + u8 *src; + + acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); + mutex_lock(&acomp_ctx->mutex); + + src = zpool_map_handle(zpool, entry->handle, ZPOOL_MM_RO); + if (!zpool_can_sleep_mapped(zpool)) { + memcpy(acomp_ctx->buffer, src, entry->length); + src = acomp_ctx->buffer; + zpool_unmap_handle(zpool, entry->handle); + } + + sg_init_one(&input, src, entry->length); + sg_init_table(&output, 1); + sg_set_page(&output, page, PAGE_SIZE, 0); + acomp_request_set_params(acomp_ctx->req, &input, &output, entry->length, PAGE_SIZE); + BUG_ON(crypto_wait_req(crypto_acomp_decompress(acomp_ctx->req), &acomp_ctx->wait)); + BUG_ON(acomp_ctx->req->dlen != PAGE_SIZE); + mutex_unlock(&acomp_ctx->mutex); + + if (zpool_can_sleep_mapped(zpool)) + zpool_unmap_handle(zpool, entry->handle); +} + /********************************* * shrinker functions **********************************/ @@ -1317,108 +1422,6 @@ static void shrink_worker(struct work_struct *w) zswap_pool_put(pool); } -static bool zswap_compress(struct folio *folio, struct zswap_entry *entry) -{ - struct crypto_acomp_ctx *acomp_ctx; - struct scatterlist input, output; - unsigned int dlen = PAGE_SIZE; - unsigned long handle; - struct zpool *zpool; - char *buf; - gfp_t gfp; - int ret; - u8 *dst; - - acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); - - mutex_lock(&acomp_ctx->mutex); - - dst = acomp_ctx->buffer; - sg_init_table(&input, 1); - sg_set_page(&input, &folio->page, PAGE_SIZE, 0); - - /* - * We need PAGE_SIZE * 2 here since there maybe over-compression case, - * and hardware-accelerators may won't check the dst buffer size, so - * giving the dst buffer with enough length to avoid buffer overflow. - */ - sg_init_one(&output, dst, PAGE_SIZE * 2); - acomp_request_set_params(acomp_ctx->req, &input, &output, PAGE_SIZE, dlen); - - /* - * it maybe looks a little bit silly that we send an asynchronous request, - * then wait for its completion synchronously. This makes the process look - * synchronous in fact. - * Theoretically, acomp supports users send multiple acomp requests in one - * acomp instance, then get those requests done simultaneously. but in this - * case, zswap actually does store and load page by page, there is no - * existing method to send the second page before the first page is done - * in one thread doing zwap. - * but in different threads running on different cpu, we have different - * acomp instance, so multiple threads can do (de)compression in parallel. - */ - ret = crypto_wait_req(crypto_acomp_compress(acomp_ctx->req), &acomp_ctx->wait); - dlen = acomp_ctx->req->dlen; - if (ret) { - zswap_reject_compress_fail++; - goto unlock; - } - - zpool = zswap_find_zpool(entry); - gfp = __GFP_NORETRY | __GFP_NOWARN | __GFP_KSWAPD_RECLAIM; - if (zpool_malloc_support_movable(zpool)) - gfp |= __GFP_HIGHMEM | __GFP_MOVABLE; - ret = zpool_malloc(zpool, dlen, gfp, &handle); - if (ret == -ENOSPC) { - zswap_reject_compress_poor++; - goto unlock; - } - if (ret) { - zswap_reject_alloc_fail++; - goto unlock; - } - - buf = zpool_map_handle(zpool, handle, ZPOOL_MM_WO); - memcpy(buf, dst, dlen); - zpool_unmap_handle(zpool, handle); - - entry->handle = handle; - entry->length = dlen; - -unlock: - mutex_unlock(&acomp_ctx->mutex); - return ret == 0; -} - -static void zswap_decompress(struct zswap_entry *entry, struct page *page) -{ - struct zpool *zpool = zswap_find_zpool(entry); - struct scatterlist input, output; - struct crypto_acomp_ctx *acomp_ctx; - u8 *src; - - acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); - mutex_lock(&acomp_ctx->mutex); - - src = zpool_map_handle(zpool, entry->handle, ZPOOL_MM_RO); - if (!zpool_can_sleep_mapped(zpool)) { - memcpy(acomp_ctx->buffer, src, entry->length); - src = acomp_ctx->buffer; - zpool_unmap_handle(zpool, entry->handle); - } - - sg_init_one(&input, src, entry->length); - sg_init_table(&output, 1); - sg_set_page(&output, page, PAGE_SIZE, 0); - acomp_request_set_params(acomp_ctx->req, &input, &output, entry->length, PAGE_SIZE); - BUG_ON(crypto_wait_req(crypto_acomp_decompress(acomp_ctx->req), &acomp_ctx->wait)); - BUG_ON(acomp_ctx->req->dlen != PAGE_SIZE); - mutex_unlock(&acomp_ctx->mutex); - - if (zpool_can_sleep_mapped(zpool)) - zpool_unmap_handle(zpool, entry->handle); -} - /********************************* * writeback code **********************************/ From patchwork Tue Jan 30 01:36:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 193792 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp945280dyb; Mon, 29 Jan 2024 17:53:24 -0800 (PST) X-Google-Smtp-Source: AGHT+IE3BckboD1hLaOswErwK/3V0ikfkS20VrlA86qyynwe+2CkbFrpTWBQnbqITH2MM/HY8YJG X-Received: by 2002:a17:903:32c8:b0:1d8:d38f:5476 with SMTP id i8-20020a17090332c800b001d8d38f5476mr3861356plr.64.1706579604240; Mon, 29 Jan 2024 17:53:24 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706579604; cv=pass; d=google.com; s=arc-20160816; b=Bi56UMI9jr7HfDe7Jp7BHn2BwG0UL7TEjPOuOask7IQKsX9ueVeKQc0OLGFtvqhAYr Bd1Kk3f7AlaoQQ+plCtj6HylVoiJuZSMqw2OA1PhRmmrjQNYcL9WVPpNB3myW6RyjuJ/ rAQNG4KKAcUAn/DFJD5FjbGskXvMPNU03lZ4DqFGTecEPON/Qi/I/z6lwcO9EOcxg0iy DDhLUZ5T8RbR2bn8K+EhAHj5mvDJ6ljfy8w15noCX7ht9VQs8WJ3QslxKPx8u0eJea7h /GmP0lqE1SaumHEqBO/x9GJDFFbhNpx3D+QOcCtJloFr2cKMP1FOjm4cprEywKYYlkvE 6l2g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=pWfrSpQCJ73gDMIVUCgMZppF172UOPzzTZlrxOiNavU=; fh=pEdtrChUTDmvuLILGSavwskaPZjBcM3e1cB9O4e7o4U=; b=CvCUFXWPlQAiUm35N4+KtsyBVFblMY++Enn7s1MoVlUWnNoaKmZi7CfCWnXOm4G42N AqsFAkz/4wGDMV7TB6ImCcL4IfC7P2o04GFivT9Q74B1/I07qII7dTk/6rGpr2qAXepD u7S+X7f1HPMkfwQMQznXOXP3XaFgwsV6oGLsoJzU1uBmUgdffAqNc/v5wTU9D98caLne DYUlyLr8/EixcosQwlvuNjQCNiOzI99m84GDY27wtUxURp+Ls7pTT3aHRXxdRseV1inE IMZLcnQv6R9mWJ/S/CSPi65QB/Zck2W6GVQ5+pvDCSsxzkc5qZDansrJ/pF+u9h4/48n QCFg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=LE+YoTTC; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43757-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43757-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id ja17-20020a170902efd100b001d8d052e99esi3330075plb.597.2024.01.29.17.53.24 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:53:24 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43757-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=LE+YoTTC; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43757-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43757-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 11A6B286E72 for ; Tue, 30 Jan 2024 01:46:49 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3DE3A364C7; Tue, 30 Jan 2024 01:42:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b="LE+YoTTC" Received: from mail-qv1-f52.google.com (mail-qv1-f52.google.com [209.85.219.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CBBD154F92 for ; Tue, 30 Jan 2024 01:42:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578958; cv=none; b=q+7YfqYFPNo0Q4bxUnYXgtslOAarXDd6FiMps4qFg+IfTP8kVx5y4biC9aL+JjeIZWH4XX6blhsPco/gX2ucpbzSNHPIXHFN1m8oPmAoVGJ30+nPffBbkClt3VB85F0PxoyegaNYAVmE8QFjxKAK4V///E0DBdjIH9TY+nAR7Ic= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578958; c=relaxed/simple; bh=enNppWOMz5X+Bo+gNc3K2O0BwrrF0fPHp/zDjikXOCM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dw8fWQqYU1vbFglxSXFGn4IoqjjAyIqqvpykCkXcwPl1w/0g0SCSDr4GGxCFVRllaTXdhcwCg8MAw2WM6Cfzq0OYiN5wEF0CxCkX2vHUBxUQOtfk5LCRqXOlHUlGqQgKG7rkUFSd2FWm3yPqvBM+heJ9iSPk4fk91jnZyWQ+if4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b=LE+YoTTC; arc=none smtp.client-ip=209.85.219.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Received: by mail-qv1-f52.google.com with SMTP id 6a1803df08f44-6818aa07d81so24778326d6.0 for ; Mon, 29 Jan 2024 17:42:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578956; x=1707183756; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pWfrSpQCJ73gDMIVUCgMZppF172UOPzzTZlrxOiNavU=; b=LE+YoTTCqqwfTT9UXfvlrhvq2ELx1LEgn7BWWqkcFObg0fsfHWTgGmclznOV8Kd5Ia 1qWE3YubkkMFotwRVcOfoBPIHqPt0vylT7LA6ITj7Mxa9Ylun23jE3M2yd5GiBnWVicG WvRADvZqSgQmYWmKH7QVl6rIApd0nm7vKi79UjyicD8cjXfbPgwTQ+z7mNyM6PMKpf8w 9x174L3k5ZCvYLkfPbe69pj9K4LQAIqHiCp3zo+DX5nEMmBPTaUhWPptK8ny4eRdQXpM 7xPN4KQcIjLzcs1W8W5yL4pzJcj5/RrQ7rp+FxC9+05EHCKpLWDx6tezU0UwX2ESJXoe MoOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578956; x=1707183756; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pWfrSpQCJ73gDMIVUCgMZppF172UOPzzTZlrxOiNavU=; b=WSu4WmLTzV2zPJ1hjeb/kQgq2MHD78xsDzIV/rw/4XI8H3rlQrR6evv9tHSehiSDVB selocOonMoDVWoAxK8zApZOyqXVTCsuyQQ//bCY2nEUZjIgijbIK4pKuEAB2/+tNa/OX UxwomwtaMHe3qj0RjhWQFEVcS0ClaOEh2sbob6UwmP1FuwxYH1NH27J8Me/NoAUaPiSy jTYIvHNYQQWpbXBQQdOyr7k4gn4U4eDFqZkv1WgVa1C6wQyLipbkTqFAsphsSQfXJFjx Yu1fxUP65qTEjfKcvghfFjyiCoC1SSp26t+a7y9yUvFADW04f0k7jO3ZjPEy6EyalCu5 l0XQ== X-Gm-Message-State: AOJu0YxQmLeuDmOiuUC0dc7SBl44gCBdZ63CDh9PeVDQk4At96nC2aRd SxjwcYrX0PMQ9Xklr782X0gbeaQWtGSgXYkcJ2dUEkq8CO887QTK8qTW87nkpyQ= X-Received: by 2002:a05:6214:400c:b0:68c:5ae6:de8d with SMTP id kd12-20020a056214400c00b0068c5ae6de8dmr1717725qvb.39.1706578955770; Mon, 29 Jan 2024 17:42:35 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id df12-20020a056214080c00b0068c4542eb0fsm2419516qvb.137.2024.01.29.17.42.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:35 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 18/20] mm: zswap: function ordering: per-cpu compression infra Date: Mon, 29 Jan 2024 20:36:54 -0500 Message-ID: <20240130014208.565554-19-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789478414846970340 X-GMAIL-MSGID: 1789478414846970340 The per-cpu compression init/exit callbacks are awkwardly in the middle of the shrinker code. Move them up to the compression section. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 135 ++++++++++++++++++++++++++--------------------------- 1 file changed, 66 insertions(+), 69 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 17356b2e35c2..acd7dcd1e0f2 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -995,6 +995,72 @@ static void zswap_invalidate_entry(struct zswap_tree *tree, /********************************* * compressed storage functions **********************************/ +static int zswap_cpu_comp_prepare(unsigned int cpu, struct hlist_node *node) +{ + struct zswap_pool *pool = hlist_entry(node, struct zswap_pool, node); + struct crypto_acomp_ctx *acomp_ctx = per_cpu_ptr(pool->acomp_ctx, cpu); + struct crypto_acomp *acomp; + struct acomp_req *req; + int ret; + + mutex_init(&acomp_ctx->mutex); + + acomp_ctx->buffer = kmalloc_node(PAGE_SIZE * 2, GFP_KERNEL, cpu_to_node(cpu)); + if (!acomp_ctx->buffer) + return -ENOMEM; + + acomp = crypto_alloc_acomp_node(pool->tfm_name, 0, 0, cpu_to_node(cpu)); + if (IS_ERR(acomp)) { + pr_err("could not alloc crypto acomp %s : %ld\n", + pool->tfm_name, PTR_ERR(acomp)); + ret = PTR_ERR(acomp); + goto acomp_fail; + } + acomp_ctx->acomp = acomp; + + req = acomp_request_alloc(acomp_ctx->acomp); + if (!req) { + pr_err("could not alloc crypto acomp_request %s\n", + pool->tfm_name); + ret = -ENOMEM; + goto req_fail; + } + acomp_ctx->req = req; + + crypto_init_wait(&acomp_ctx->wait); + /* + * if the backend of acomp is async zip, crypto_req_done() will wakeup + * crypto_wait_req(); if the backend of acomp is scomp, the callback + * won't be called, crypto_wait_req() will return without blocking. + */ + acomp_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG, + crypto_req_done, &acomp_ctx->wait); + + return 0; + +req_fail: + crypto_free_acomp(acomp_ctx->acomp); +acomp_fail: + kfree(acomp_ctx->buffer); + return ret; +} + +static int zswap_cpu_comp_dead(unsigned int cpu, struct hlist_node *node) +{ + struct zswap_pool *pool = hlist_entry(node, struct zswap_pool, node); + struct crypto_acomp_ctx *acomp_ctx = per_cpu_ptr(pool->acomp_ctx, cpu); + + if (!IS_ERR_OR_NULL(acomp_ctx)) { + if (!IS_ERR_OR_NULL(acomp_ctx->req)) + acomp_request_free(acomp_ctx->req); + if (!IS_ERR_OR_NULL(acomp_ctx->acomp)) + crypto_free_acomp(acomp_ctx->acomp); + kfree(acomp_ctx->buffer); + } + + return 0; +} + static bool zswap_compress(struct folio *folio, struct zswap_entry *entry) { struct crypto_acomp_ctx *acomp_ctx; @@ -1201,75 +1267,6 @@ static void zswap_alloc_shrinker(struct zswap_pool *pool) pool->shrinker->seeks = DEFAULT_SEEKS; } -/********************************* -* per-cpu code -**********************************/ -static int zswap_cpu_comp_prepare(unsigned int cpu, struct hlist_node *node) -{ - struct zswap_pool *pool = hlist_entry(node, struct zswap_pool, node); - struct crypto_acomp_ctx *acomp_ctx = per_cpu_ptr(pool->acomp_ctx, cpu); - struct crypto_acomp *acomp; - struct acomp_req *req; - int ret; - - mutex_init(&acomp_ctx->mutex); - - acomp_ctx->buffer = kmalloc_node(PAGE_SIZE * 2, GFP_KERNEL, cpu_to_node(cpu)); - if (!acomp_ctx->buffer) - return -ENOMEM; - - acomp = crypto_alloc_acomp_node(pool->tfm_name, 0, 0, cpu_to_node(cpu)); - if (IS_ERR(acomp)) { - pr_err("could not alloc crypto acomp %s : %ld\n", - pool->tfm_name, PTR_ERR(acomp)); - ret = PTR_ERR(acomp); - goto acomp_fail; - } - acomp_ctx->acomp = acomp; - - req = acomp_request_alloc(acomp_ctx->acomp); - if (!req) { - pr_err("could not alloc crypto acomp_request %s\n", - pool->tfm_name); - ret = -ENOMEM; - goto req_fail; - } - acomp_ctx->req = req; - - crypto_init_wait(&acomp_ctx->wait); - /* - * if the backend of acomp is async zip, crypto_req_done() will wakeup - * crypto_wait_req(); if the backend of acomp is scomp, the callback - * won't be called, crypto_wait_req() will return without blocking. - */ - acomp_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG, - crypto_req_done, &acomp_ctx->wait); - - return 0; - -req_fail: - crypto_free_acomp(acomp_ctx->acomp); -acomp_fail: - kfree(acomp_ctx->buffer); - return ret; -} - -static int zswap_cpu_comp_dead(unsigned int cpu, struct hlist_node *node) -{ - struct zswap_pool *pool = hlist_entry(node, struct zswap_pool, node); - struct crypto_acomp_ctx *acomp_ctx = per_cpu_ptr(pool->acomp_ctx, cpu); - - if (!IS_ERR_OR_NULL(acomp_ctx)) { - if (!IS_ERR_OR_NULL(acomp_ctx->req)) - acomp_request_free(acomp_ctx->req); - if (!IS_ERR_OR_NULL(acomp_ctx->acomp)) - crypto_free_acomp(acomp_ctx->acomp); - kfree(acomp_ctx->buffer); - } - - return 0; -} - static enum lru_status shrink_memcg_cb(struct list_head *item, struct list_lru_one *l, spinlock_t *lock, void *arg) { From patchwork Tue Jan 30 01:36:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 193785 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp943298dyb; Mon, 29 Jan 2024 17:47:04 -0800 (PST) X-Google-Smtp-Source: AGHT+IFxsYhYCQN3KSiiLKhswt25qYvDnWsBIieOXxdMMEYpGuQs8bSzS9hfVrSNUouhGKarRm+0 X-Received: by 2002:a05:6214:5016:b0:686:6cb:7dc0 with SMTP id jo22-20020a056214501600b0068606cb7dc0mr9583663qvb.106.1706579224728; Mon, 29 Jan 2024 17:47:04 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706579224; cv=pass; d=google.com; s=arc-20160816; b=j8/H7w7LVKUjCiYnb9A1pJWxEpHXzxSgnu5JUKeenKZmE9l7gCOvNxn+VrUHhse3VY /bV4mmX51RL67btnROCMfxRfZ1VdFry6pGGHmEPKKvIn6xYFIPxi3vjjAU18g36TcR8O eB/tTs4BlU1pJQnW1trGsnjwHECSN0Hu2gaaotik7FP4BU+1THinTYol+31+mtCL9re1 N9qto9fyIHyLtmvUL5GidO3TbuAZbXcboGAlTyKfHu7Oxutri6vOXpZ2ry9GBD10Kku4 AzuvPVULZC1d2c2LmKjWHyjnAk+I7WkuKoYmsX9bR2VAO7AVwoo44IwiGqeQUA3L4iJp MTwA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=nthkOb34nC29m9Z7rdKEYXbNdpH1xuatn2MKs2M9M7s=; fh=pEdtrChUTDmvuLILGSavwskaPZjBcM3e1cB9O4e7o4U=; b=MThXNiCaHH+IjihgG7ukpovi3AwTZ5KhBY6dgnKee1wVG7gFaWfEzDh3Cj0u/eknC/ nHAA55ch+945CdZjZY2KddrX+sXDr0KxHS7e2b8l3lHBqIfzlYrTuRF+IyeNpX5KeVDh v1W1DyJ1FKd+V9FnAOYP2xW4blDkD3eESVNX5FXgvELxpi2kbWJOQUxcb4XrWsJmcAVM 0hZFlSV4FL4UYMJgqApTgDG6APWdV9P2M8EtRw5EDx+TEKwPN5yJARgGOupfV2BEV5DC qX6zhOFwSAEFTgVBEBJCyK/0I+FM8G/dTNgOKoTn6ZfphmFq8u7LwOh8FmoLRMGyrRz9 e+nA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=pItiNqAP; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43758-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43758-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id ey7-20020a0562140b6700b006818638771asi9559480qvb.200.2024.01.29.17.47.04 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:47:04 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43758-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=pItiNqAP; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43758-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43758-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id B82601C241F8 for ; Tue, 30 Jan 2024 01:47:01 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D32A3364CD; Tue, 30 Jan 2024 01:42:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b="pItiNqAP" Received: from mail-qv1-f44.google.com (mail-qv1-f44.google.com [209.85.219.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E3566F074 for ; Tue, 30 Jan 2024 01:42:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578959; cv=none; b=uH1aL6qrMIGpW6y/L6CMiKTZ+aV3hYi3ohVTkKmoRtML+HCrUhD7YHgBMsB1qViJj/JOt7BF/z83p0O1bZzHjH+dLiRT76wl+Y43DIi2OtjtUg3GA8zt1lXXZtev9mvhQtRbcpDpx8imuqtL6t7eVOjnT/d/sNSvjOnfpZ9D3sc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578959; c=relaxed/simple; bh=THpur6kVnyXr5u+tiyxAdLgZj7BGAETob5/AxR3iXb4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=povsYM4ILWswB3bBm+QcG9S0TYkfnamxjxBSb5YWyU8tHFF7//kOu/wLNaoJ/Cf+hVG4RNY03p9jLRdB4tW7+Bd9qboKghiIkloAkUcv8cF3hLbFH3VHXjsDmAzZP8kOx/oWsH1k0uQe5KCT2JVrUihNlZq3sCg0tPiOYabJ4tI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b=pItiNqAP; arc=none smtp.client-ip=209.85.219.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Received: by mail-qv1-f44.google.com with SMTP id 6a1803df08f44-68c4fb9e7e0so11623886d6.2 for ; Mon, 29 Jan 2024 17:42:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578957; x=1707183757; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nthkOb34nC29m9Z7rdKEYXbNdpH1xuatn2MKs2M9M7s=; b=pItiNqAPDkTa03gPDOfnaXjMwOlLItkgSc7zNj4udIULGVH7fUgJV40inMJzQR42Xf sDGs6Duey3u0u8S5fnN0Ys0A6hDHRGnByZ6pzp5V7PvYc9MPEYjEuxjDu3kFcZkaSKsk LKt/1ttK9T/BioOL/mq/uTQwOf+qac8lYylXuFLR5ae/55vowcxx68u/2SNqqdVcHrAH +QoWxsOzOnFnNBkvbujJU1XX9vN4iKHoReSxBKCtvu8j0zikaLGuDbMMV453BfG2OKYo A7CqZsevxxzLkycQA4acQUco3KA/mw09SDRLXpZgyG5n6+98ipOXBwkhowDXkcPTUJOS qS6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578957; x=1707183757; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nthkOb34nC29m9Z7rdKEYXbNdpH1xuatn2MKs2M9M7s=; b=T09EtHAX46ulRX2X013O0RcmkZN/vEA2TzVX/YzTyIkwR7uUDFAsH+J6lgocCsCMWW vyNDaSMFxKMPOqijAaTg3p/g48LWGSlAygB2maEi95IiWI1b8jctnCpqPpePQYQgOtDG pPdXHIrQo/cLyeR4SUSoPooGTIUUrMRYBHg1JJ8qgUdBDpLhrf+OlTao8GshIXl33FQB +8lyJvrIqivIEJXHUnZRUuV/B9klcvZVjLXv8/uY1WUbeHZUD54IP20jLbaHEng/aa7S X1GFXeFxbjKi73AzHdANAFNXgUrW/a2OQxLzx74x4WKdOgzYIAO63l6Dze9yNVEu6qFK OnHA== X-Gm-Message-State: AOJu0YwPWjBhFjxuJAqzArPpBsmzZFjSX8KQa+9Vr8W7GUYAUNheRHC7 ZDZJUjhxj5gCUQD7+lIzUV7NrTUMJZxhlmnAH/mt28Yxx0e+Un3QrdIGtasBI9WfIqtXVIQAVyz o X-Received: by 2002:a05:6214:c68:b0:68c:498e:d09f with SMTP id t8-20020a0562140c6800b0068c498ed09fmr4765609qvj.54.1706578956903; Mon, 29 Jan 2024 17:42:36 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id p16-20020a0cc3d0000000b00686ac3226ccsm3935915qvi.114.2024.01.29.17.42.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:36 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 19/20] mm: zswap: function ordering: writeback Date: Mon, 29 Jan 2024 20:36:55 -0500 Message-ID: <20240130014208.565554-20-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789478016994890940 X-GMAIL-MSGID: 1789478016994890940 Shrinking needs writeback. Naturally, move the writeback code above the shrinking code. Delete the forward decl. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 183 ++++++++++++++++++++++++++--------------------------- 1 file changed, 90 insertions(+), 93 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index acd7dcd1e0f2..0cb3437d47eb 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -276,9 +276,6 @@ static inline struct zswap_tree *swap_zswap_tree(swp_entry_t swp) pr_debug("%s pool %s/%s\n", msg, (p)->tfm_name, \ zpool_get_type((p)->zpools[0])) -static int zswap_writeback_entry(struct zswap_entry *entry, - swp_entry_t swpentry); - static bool zswap_is_full(void) { return totalram_pages() * zswap_max_pool_percent / 100 < @@ -1163,6 +1160,96 @@ static void zswap_decompress(struct zswap_entry *entry, struct page *page) zpool_unmap_handle(zpool, entry->handle); } +/********************************* +* writeback code +**********************************/ +/* + * Attempts to free an entry by adding a folio to the swap cache, + * decompressing the entry data into the folio, and issuing a + * bio write to write the folio back to the swap device. + * + * This can be thought of as a "resumed writeback" of the folio + * to the swap device. We are basically resuming the same swap + * writeback path that was intercepted with the zswap_store() + * in the first place. After the folio has been decompressed into + * the swap cache, the compressed version stored by zswap can be + * freed. + */ +static int zswap_writeback_entry(struct zswap_entry *entry, + swp_entry_t swpentry) +{ + struct zswap_tree *tree; + struct folio *folio; + struct mempolicy *mpol; + bool folio_was_allocated; + struct writeback_control wbc = { + .sync_mode = WB_SYNC_NONE, + }; + + /* try to allocate swap cache folio */ + mpol = get_task_policy(current); + folio = __read_swap_cache_async(swpentry, GFP_KERNEL, mpol, + NO_INTERLEAVE_INDEX, &folio_was_allocated, true); + if (!folio) + return -ENOMEM; + + /* + * Found an existing folio, we raced with swapin or concurrent + * shrinker. We generally writeback cold folios from zswap, and + * swapin means the folio just became hot, so skip this folio. + * For unlikely concurrent shrinker case, it will be unlinked + * and freed when invalidated by the concurrent shrinker anyway. + */ + if (!folio_was_allocated) { + folio_put(folio); + return -EEXIST; + } + + /* + * folio is locked, and the swapcache is now secured against + * concurrent swapping to and from the slot. Verify that the + * swap entry hasn't been invalidated and recycled behind our + * backs (our zswap_entry reference doesn't prevent that), to + * avoid overwriting a new swap folio with old compressed data. + */ + tree = swap_zswap_tree(swpentry); + spin_lock(&tree->lock); + if (zswap_rb_search(&tree->rbroot, swp_offset(swpentry)) != entry) { + spin_unlock(&tree->lock); + delete_from_swap_cache(folio); + folio_unlock(folio); + folio_put(folio); + return -ENOMEM; + } + + /* Safe to deref entry after the entry is verified above. */ + zswap_entry_get(entry); + spin_unlock(&tree->lock); + + zswap_decompress(entry, &folio->page); + + count_vm_event(ZSWPWB); + if (entry->objcg) + count_objcg_event(entry->objcg, ZSWPWB); + + spin_lock(&tree->lock); + zswap_invalidate_entry(tree, entry); + zswap_entry_put(entry); + spin_unlock(&tree->lock); + + /* folio is up to date */ + folio_mark_uptodate(folio); + + /* move it to the tail of the inactive list after end_writeback */ + folio_set_reclaim(folio); + + /* start writeback */ + __swap_writepage(folio, &wbc); + folio_put(folio); + + return 0; +} + /********************************* * shrinker functions **********************************/ @@ -1419,96 +1506,6 @@ static void shrink_worker(struct work_struct *w) zswap_pool_put(pool); } -/********************************* -* writeback code -**********************************/ -/* - * Attempts to free an entry by adding a folio to the swap cache, - * decompressing the entry data into the folio, and issuing a - * bio write to write the folio back to the swap device. - * - * This can be thought of as a "resumed writeback" of the folio - * to the swap device. We are basically resuming the same swap - * writeback path that was intercepted with the zswap_store() - * in the first place. After the folio has been decompressed into - * the swap cache, the compressed version stored by zswap can be - * freed. - */ -static int zswap_writeback_entry(struct zswap_entry *entry, - swp_entry_t swpentry) -{ - struct zswap_tree *tree; - struct folio *folio; - struct mempolicy *mpol; - bool folio_was_allocated; - struct writeback_control wbc = { - .sync_mode = WB_SYNC_NONE, - }; - - /* try to allocate swap cache folio */ - mpol = get_task_policy(current); - folio = __read_swap_cache_async(swpentry, GFP_KERNEL, mpol, - NO_INTERLEAVE_INDEX, &folio_was_allocated, true); - if (!folio) - return -ENOMEM; - - /* - * Found an existing folio, we raced with swapin or concurrent - * shrinker. We generally writeback cold folios from zswap, and - * swapin means the folio just became hot, so skip this folio. - * For unlikely concurrent shrinker case, it will be unlinked - * and freed when invalidated by the concurrent shrinker anyway. - */ - if (!folio_was_allocated) { - folio_put(folio); - return -EEXIST; - } - - /* - * folio is locked, and the swapcache is now secured against - * concurrent swapping to and from the slot. Verify that the - * swap entry hasn't been invalidated and recycled behind our - * backs (our zswap_entry reference doesn't prevent that), to - * avoid overwriting a new swap folio with old compressed data. - */ - tree = swap_zswap_tree(swpentry); - spin_lock(&tree->lock); - if (zswap_rb_search(&tree->rbroot, swp_offset(swpentry)) != entry) { - spin_unlock(&tree->lock); - delete_from_swap_cache(folio); - folio_unlock(folio); - folio_put(folio); - return -ENOMEM; - } - - /* Safe to deref entry after the entry is verified above. */ - zswap_entry_get(entry); - spin_unlock(&tree->lock); - - zswap_decompress(entry, &folio->page); - - count_vm_event(ZSWPWB); - if (entry->objcg) - count_objcg_event(entry->objcg, ZSWPWB); - - spin_lock(&tree->lock); - zswap_invalidate_entry(tree, entry); - zswap_entry_put(entry); - spin_unlock(&tree->lock); - - /* folio is up to date */ - folio_mark_uptodate(folio); - - /* move it to the tail of the inactive list after end_writeback */ - folio_set_reclaim(folio); - - /* start writeback */ - __swap_writepage(folio, &wbc); - folio_put(folio); - - return 0; -} - static int zswap_is_page_same_filled(void *ptr, unsigned long *value) { unsigned long *page; From patchwork Tue Jan 30 01:36:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 193794 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp946595dyb; Mon, 29 Jan 2024 17:57:41 -0800 (PST) X-Google-Smtp-Source: AGHT+IHf2SQn8lxWheJkYSXdWinSIFyPAI5AhiJOxTFKngx0jPFgqSiZ/XYUYh2h0QqEw1PnQlwn X-Received: by 2002:a17:902:c94e:b0:1d7:78ed:4b7e with SMTP id i14-20020a170902c94e00b001d778ed4b7emr8503825pla.35.1706579861322; Mon, 29 Jan 2024 17:57:41 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706579861; cv=pass; d=google.com; s=arc-20160816; b=IAQ9Rls6IyJsuwKtbndM+VEGCpIlTqAxjuNdgMquXRGXIAQ7AYxZe1bEMyRf1J5C+Q ND+fVfgYuTYNXIZit6gxBzoCG7J9hty7qbCGoouVJf/VDq/i/eagRdJQQ8v1Q5aOwuZF CEpI4mn3KZP6BKRu3Vh/If9Xa/hKvcjujRHKQbOYHWKIeblF10sBB2ajdPg9HjXp5LKt DR6Upw6qQZKDil84GY5PcS9qXthp5SnJl93SLEtLxMnJrw8RI1Zirs2zHX3BV5udJxep IrLI5Zn9fEdusiAuw2mJg9mW+qM/SUjpnIAUlY4RNg1Vywpy6mc/+is6LCCF1ZxicRUx eoLQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=jHRUZ67yNEKZS9FrhnY4ruPkQ8T6m4mDy3O5Fk7spbc=; fh=pEdtrChUTDmvuLILGSavwskaPZjBcM3e1cB9O4e7o4U=; b=AwmkQHtnO9tQmzu65EAjVZC6lx6Oq4msh/80vbbcYW7ddA2uNFXdy9KuiOXvuCUEjN YNcjZIm0Bbh3uMHKhQAZKEGIFYhkoSBzt0Bk20B5VbGVhSgqNaA8eTS2WVcQgl6ujfBf 8mweaFMO/Xhpkd7skaeZuXsE4KYu2qyoW2q2PNK5yX5/yx15sfSiFjT262vADZijyPTP c7UHWrbIND7rK15bGs8voSCVg1juE5oUVqQmfB8I+YgrRfBxZ8lFl6UmMvx+n9U2R51+ pTibR5RpM6hauJRJX6ZQKz/h0WKVuAA2gy0O4LV4jIUFgqu96v9K3MlIfNFXhy4geSX1 JhVw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b="e2lXP/hr"; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43759-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43759-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id u6-20020a170903124600b001d6ed2e9c87si6616302plh.29.2024.01.29.17.57.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:57:41 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43759-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b="e2lXP/hr"; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43759-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43759-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 88D18287305 for ; Tue, 30 Jan 2024 01:47:04 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 799C515A4A7; Tue, 30 Jan 2024 01:42:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b="e2lXP/hr" Received: from mail-qt1-f169.google.com (mail-qt1-f169.google.com [209.85.160.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 28395151CE0 for ; Tue, 30 Jan 2024 01:42:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578960; cv=none; b=Fncrv29lSG8J/6oenEBpDPQQjhOiJCFtzndRvDsDQz+5QpESCvi7WTQlQlFiPovkwBhDhSfDyY1pZv00StlYEgglx8Cu4cfq+Ew6v2eyDIq+2nVFwa9a7Qqbj8fOMsj2CjfQEJxhxNpc4NjHXXCwLMJROsLff2TBhNUXC+MxYC4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578960; c=relaxed/simple; bh=cnPSFO5BO6ITGOvn3tMGN0G1eY3jniMhH/tSuYdVq+o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GtKUXd/L6X7EMe9fhgZguJlMIq1ZYsf7R/+AKuKUDX1PqbonCW9Z2IQxHUogpJ/Kz3fq0pY48QISAuTKkxJOhTdtkhP0Qhm94yyg05TXHiEJc4be1IYbPyYx/iXXAtnbfXrE2PXCPbixMTTSQFcdRyiWJzvQC5jifLZv0aYnQmg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b=e2lXP/hr; arc=none smtp.client-ip=209.85.160.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Received: by mail-qt1-f169.google.com with SMTP id d75a77b69052e-42aadddeaf8so7284781cf.3 for ; Mon, 29 Jan 2024 17:42:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578958; x=1707183758; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jHRUZ67yNEKZS9FrhnY4ruPkQ8T6m4mDy3O5Fk7spbc=; b=e2lXP/hryEhSICaRjELXgNd5vdKL174JP46OdavL1I2edc+3EpvP9ghIilnq/TRkSx 5UphqClq1HZoEZd9QBm4+jZYvifV/I/qHXAE10LL8HV/xWRiWrW6cjLk/ufrpP6sbnku GBSXHi1kS0aBio6/Hf37QZLjWBEuTDDM8pU+X3X+gO6YbXiWX4j5EZ1UplaMd/8W1I5m PUha7pcWNEnumFWDtRt/ejV0X0iSdfHMPGlPW9Wu1mmxCUBmzbTB+AhsxtHiDL0c0UZp wx1cyvWQt8KijkpRutTCNjzafYJhhmuO6fUqGlIvg2l0OVklUESnVXuS2fqXr7AyruDu 6w7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578958; x=1707183758; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jHRUZ67yNEKZS9FrhnY4ruPkQ8T6m4mDy3O5Fk7spbc=; b=tqBqwb0oQfsQKxdG09wQnjT/nZ6R/4zXUWoXnIreDjkPKW3YcUOprdm24vX/c+U5sn cvShbNfHPA/zNmGic3CjnbpKh6uxOzUXmTP/O3pIH/wnmWxdBIwaetPVh0pSPuzjCXr1 ry60y6FhyQBoXfNAayjk85NipNCv/J9+ZcET8k1JfhNt7ljJE8TVFMTmIocoI6BT0gAo QmhnidMCYHWzbFdnPI+mYiV8FRWTAZUh67b/0KjgUbqP1UL+0URbz7ImX0BgEMXmDHkA 8FTFssI6WwRuKJWdItcdkR0XPJKbe9NjZhj5xP8DyXlWjtUrar26nZWbe0oMstvBQU0h B/rw== X-Gm-Message-State: AOJu0Yyabqv8DPnC/xu7/GuJTovHr6pBvKxreJ9yLvYN4cBFj8331yxA pWOyq3SvnUcYrW3yIgZ2J0DUPoqeuJGZWu6oWptibvTdcmCyn4O+8pYz0LLGyf0= X-Received: by 2002:ac8:5bcc:0:b0:42a:9c39:9727 with SMTP id b12-20020ac85bcc000000b0042a9c399727mr5682189qtb.83.1706578958090; Mon, 29 Jan 2024 17:42:38 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id bl4-20020a05622a244400b004260b65b4f7sm1369468qtb.97.2024.01.29.17.42.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:37 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 20/20] mm: zswap: function ordering: shrink_memcg_cb Date: Mon, 29 Jan 2024 20:36:56 -0500 Message-ID: <20240130014208.565554-21-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789478684571015308 X-GMAIL-MSGID: 1789478684571015308 shrink_memcg_cb() is called by the shrinker and is based on zswap_writeback_entry(). Move it in between. Save one fwd decl. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 125 ++++++++++++++++++++++++++--------------------------- 1 file changed, 61 insertions(+), 64 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 0cb3437d47eb..4aea03285532 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1254,7 +1254,67 @@ static int zswap_writeback_entry(struct zswap_entry *entry, * shrinker functions **********************************/ static enum lru_status shrink_memcg_cb(struct list_head *item, struct list_lru_one *l, - spinlock_t *lock, void *arg); + spinlock_t *lock, void *arg) +{ + struct zswap_entry *entry = container_of(item, struct zswap_entry, lru); + bool *encountered_page_in_swapcache = (bool *)arg; + swp_entry_t swpentry; + enum lru_status ret = LRU_REMOVED_RETRY; + int writeback_result; + + /* + * Rotate the entry to the tail before unlocking the LRU, + * so that in case of an invalidation race concurrent + * reclaimers don't waste their time on it. + * + * If writeback succeeds, or failure is due to the entry + * being invalidated by the swap subsystem, the invalidation + * will unlink and free it. + * + * Temporary failures, where the same entry should be tried + * again immediately, almost never happen for this shrinker. + * We don't do any trylocking; -ENOMEM comes closest, + * but that's extremely rare and doesn't happen spuriously + * either. Don't bother distinguishing this case. + * + * But since they do exist in theory, the entry cannot just + * be unlinked, or we could leak it. Hence, rotate. + */ + list_move_tail(item, &l->list); + + /* + * Once the lru lock is dropped, the entry might get freed. The + * swpentry is copied to the stack, and entry isn't deref'd again + * until the entry is verified to still be alive in the tree. + */ + swpentry = entry->swpentry; + + /* + * It's safe to drop the lock here because we return either + * LRU_REMOVED_RETRY or LRU_RETRY. + */ + spin_unlock(lock); + + writeback_result = zswap_writeback_entry(entry, swpentry); + + if (writeback_result) { + zswap_reject_reclaim_fail++; + ret = LRU_RETRY; + + /* + * Encountering a page already in swap cache is a sign that we are shrinking + * into the warmer region. We should terminate shrinking (if we're in the dynamic + * shrinker context). + */ + if (writeback_result == -EEXIST && encountered_page_in_swapcache) + *encountered_page_in_swapcache = true; + } else { + zswap_written_back_pages++; + } + + spin_lock(lock); + return ret; +} static unsigned long zswap_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc) @@ -1354,69 +1414,6 @@ static void zswap_alloc_shrinker(struct zswap_pool *pool) pool->shrinker->seeks = DEFAULT_SEEKS; } -static enum lru_status shrink_memcg_cb(struct list_head *item, struct list_lru_one *l, - spinlock_t *lock, void *arg) -{ - struct zswap_entry *entry = container_of(item, struct zswap_entry, lru); - bool *encountered_page_in_swapcache = (bool *)arg; - swp_entry_t swpentry; - enum lru_status ret = LRU_REMOVED_RETRY; - int writeback_result; - - /* - * Rotate the entry to the tail before unlocking the LRU, - * so that in case of an invalidation race concurrent - * reclaimers don't waste their time on it. - * - * If writeback succeeds, or failure is due to the entry - * being invalidated by the swap subsystem, the invalidation - * will unlink and free it. - * - * Temporary failures, where the same entry should be tried - * again immediately, almost never happen for this shrinker. - * We don't do any trylocking; -ENOMEM comes closest, - * but that's extremely rare and doesn't happen spuriously - * either. Don't bother distinguishing this case. - * - * But since they do exist in theory, the entry cannot just - * be unlinked, or we could leak it. Hence, rotate. - */ - list_move_tail(item, &l->list); - - /* - * Once the lru lock is dropped, the entry might get freed. The - * swpentry is copied to the stack, and entry isn't deref'd again - * until the entry is verified to still be alive in the tree. - */ - swpentry = entry->swpentry; - - /* - * It's safe to drop the lock here because we return either - * LRU_REMOVED_RETRY or LRU_RETRY. - */ - spin_unlock(lock); - - writeback_result = zswap_writeback_entry(entry, swpentry); - - if (writeback_result) { - zswap_reject_reclaim_fail++; - ret = LRU_RETRY; - - /* - * Encountering a page already in swap cache is a sign that we are shrinking - * into the warmer region. We should terminate shrinking (if we're in the dynamic - * shrinker context). - */ - if (writeback_result == -EEXIST && encountered_page_in_swapcache) - *encountered_page_in_swapcache = true; - } else { - zswap_written_back_pages++; - } - - spin_lock(lock); - return ret; -} - static int shrink_memcg(struct mem_cgroup *memcg) { struct zswap_pool *pool;