From patchwork Sun Nov 19 19:47:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166861 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1809542vqn; Sun, 19 Nov 2023 11:48:19 -0800 (PST) X-Google-Smtp-Source: AGHT+IGTvRNbarNDUT3sKNMnceNdMGnhkMDxmmBBqLtSgen9+2I409g1C7p6hBnjMtK1xYi694sj X-Received: by 2002:a05:6a20:d420:b0:187:a581:cc4e with SMTP id il32-20020a056a20d42000b00187a581cc4emr3069084pzb.29.1700423299277; Sun, 19 Nov 2023 11:48:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423299; cv=none; d=google.com; s=arc-20160816; b=JMwOB1XgUsPT53TIBDM168F+FElzUbQ5vUrHwgytKUuA202asiAbgQcWLSq5xrte0e pEGOGQ8CFiUB+0kkklNm7SlA9BKZ45t1cHSAywJ0r5ZRMXVEBmeQg26fCXh5GrsS+V/a Oj0Nvcf3AuIvqPZ7UbDd16ji4kuE4Y+4qyvlVv8HL3bLNXbplZIAsDP2cIELYs8raxne tUVlkXyCL4hNEb99TzIXfZqX6AnugZod/9WCtP7uouvWxRp/MbEBUWoQz6CqvDzcLQRL Sdx7VePo6/O4jTFl0SFAVvopGT5sT4i5640jejYJjXnClMUWFFAn4gl580ophCuzDvCB dbww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=tM0GQjuVnFv7RufWXzCKq9li8NCfwru2okUxp/uvL+E=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=ge+tDEonR5AdmUi1edH2GzuQXyaTl443sZX1gF1v1Zjv5bZWcw2dx8MiQu9i+vuJaG s39lzBSRP8oYQx80el1N/WtomgnYfTFOm/3asQ0G6mZFESn8eZqyiaDMCHGsF43eHRsc Zqwyf53wVBJ+P1g1KoBpoCRNHNeRPqAcTQhrBj08niAABOqrmmudvngUjqFYS6VpvK75 UhiYAVQ5g5ZWfFkRqnFD3qSJx9QgxL391JYc66BiFtlQjLDXvTmT822h9BFYBnJ74Z1O IVJp35200N6tWg9NQUL8D1wd5AjTQUqVCzzphWf0uHlqSUMixkENuIY8VLydunIcuhFV Bu6Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=bQbQ3YTv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id f13-20020aa79d8d000000b00690d8c05582si6443655pfq.372.2023.11.19.11.48.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:48:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=bQbQ3YTv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 6592180A5FAB; Sun, 19 Nov 2023 11:48:18 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229847AbjKSTsK (ORCPT + 29 others); Sun, 19 Nov 2023 14:48:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42230 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229665AbjKSTsJ (ORCPT ); Sun, 19 Nov 2023 14:48:09 -0500 Received: from mail-il1-x133.google.com (mail-il1-x133.google.com [IPv6:2607:f8b0:4864:20::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E008211A for ; Sun, 19 Nov 2023 11:48:05 -0800 (PST) Received: by mail-il1-x133.google.com with SMTP id e9e14a558f8ab-357cc880bd8so14138705ab.0 for ; Sun, 19 Nov 2023 11:48:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423285; x=1701028085; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=tM0GQjuVnFv7RufWXzCKq9li8NCfwru2okUxp/uvL+E=; b=bQbQ3YTvLIP9LH2t1GMs9zqgejkdaGNPxoDVKKh73N5sqhbbjr8T6V8/JKDnQabOOh /qRdKJtbeBuVKz6qiLkgB1i/i+neW8S75xSXJ3l6Unn82Qj/mkeUawLjWCa6Jae8iE6Q d47bG2IU4g7wCrGZayiNRT+8QdmukBz5hAlQbze6YsyWinY/5XOIImih14QN9GLMcA6N yJpz0WDO4TpISDbkVkKip/X64Q2twM4B/o1Yz7Ep5t13eEoLwJoljNgQBOVvvFBUi2t0 oIzwf5HvzXOT4ZWxSHh+ggG3pSD624M+HRrNDoPEmrogKKVAPm3xTaQ4w6xNohGPbxvV n2LQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423285; x=1701028085; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=tM0GQjuVnFv7RufWXzCKq9li8NCfwru2okUxp/uvL+E=; b=DfHFiiH72e5TOK7Zq8n4zpmve9Q34oPdBKUOZuBIfId2Y2a+HWlXlgDSrWq6ZYnnIV mj9vsl2cYP2B0Miz6hN1eMewBmNXG63IPjuFpbz4UmeE3KmPO5rMiEStgioDox55xdq2 8faPFb7xR14lUllobk+WJGTI3wxmfa3SS/N5E3OzEcJKD62cyItlVh98v3gdvnY2UR00 qxcY1C0W4p3Td4+fdUMTULPBHCcy0uHvcDxsc62+2PskX18UZZFDbBqYtyPvlbYx/zUT rWo0uqzsT8tCfZV5LpcqYUKHK0mpJSmpUHMbTkDrURVCScq2qPxBctMVjPKws/5CfFAx uqfg== X-Gm-Message-State: AOJu0YxsnnAxhFbBWO2h8Rd2pgVIY1p03Tgv+pKpX5nFF7PpL6YR1yM2 RwsPqK7R9cTzGT5EHvaBh7I= X-Received: by 2002:a92:c26d:0:b0:35b:695:c3c8 with SMTP id h13-20020a92c26d000000b0035b0695c3c8mr353889ild.9.1700423285183; Sun, 19 Nov 2023 11:48:05 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.02 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:04 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 01/24] mm/swap: fix a potential undefined behavior issue Date: Mon, 20 Nov 2023 03:47:17 +0800 Message-ID: <20231119194740.94101-2-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:48:18 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023061325559218 X-GMAIL-MSGID: 1783023061325559218 From: Kairui Song When folio is NULL, taking the address of its struct member is an undefined behavior, the UB is caused by applying -> operator to a pointer not pointing to any object. Although in practice this won't lead to a real issue, still better to fix it, also makes the code less error-prone, when folio is NULL, page is also NULL, instead of a meanless offset value. Signed-off-by: Kairui Song --- mm/memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/memory.c b/mm/memory.c index e27e2e5beb3f..70ffa867b1be 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3861,7 +3861,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) /* skip swapcache */ folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, vma, vmf->address, false); - page = &folio->page; if (folio) { __folio_set_locked(folio); __folio_set_swapbacked(folio); @@ -3879,6 +3878,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) workingset_refault(folio, shadow); folio_add_lru(folio); + page = &folio->page; /* To provide entry to swap_readpage() */ folio->swap = entry; From patchwork Sun Nov 19 19:47:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166862 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1809556vqn; Sun, 19 Nov 2023 11:48:23 -0800 (PST) X-Google-Smtp-Source: AGHT+IF67qPfg59eMtUSwy3LoGs+IpJoj5XKDdgtouAsnGxktILMl2PqNxrziPAeOjhYOXhlZC5p X-Received: by 2002:a6b:3f86:0:b0:792:9416:9c7f with SMTP id m128-20020a6b3f86000000b0079294169c7fmr6528280ioa.9.1700423303007; Sun, 19 Nov 2023 11:48:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423302; cv=none; d=google.com; s=arc-20160816; b=kVDkABzPNxrheeFOTaUFFuD7IES/IkXileIHoUSiq+MHVeZM5BJwM88xnO6ZscAZIg bqCe+2SScPHOB5XNFpzkk+S4DRUFZFbGw6No5trpcJJ016LAgsGaBDhKLNSoaPpCxa/p dSPa32LpHMAoQR4DjJahGTQiYAYM0r0ido1QOLmhLMdt5SXhyPBXNtl5AugFwgTWShz6 8ToJBZPbTPHHl5Diz2LTGnHmkDvN6p4GNxCUKqr4vV9lqZzX39HZcKTh31r/JvHVWlgz ypjrfVAPDLmLt9VveU+4iy99YcYqlt2g87mLp6Z6R5Boh8qjzQinf4b0ohtM9jH/igL/ t9LA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=cduqWiCMIKVDQKzalcbjrj5u/UZrGRNgI9eE9WPHu3A=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=aIwq6O94jqbmZiUi3FG20jHWs6ciPXAm7L6zuGor7noavQdQfuvTYbteNeENU8S60A Yhh6io0kOnqpAyCOhFibbRZH39O0k/g3Jh986SN41F1FQWwjlsA1HIrJbWk3qO85tk6r vLXQjlEENMdgRODYR0+u+iOE9Mhd3B49Tw2Iam3GNO3Q1cYmYlEi3nhA7C5LexpxwIfE drYYpqTBZRzMNES6f5jhpDDp4/9WdK8c8MBYgL41KGuWkFKZtFVnTQwrScoOZ7Azzpto hbBL/OFSzG3sVzUXq/qf8rwa2bW8dlfINzB86xfnrlZLl0YO3o+1daNrLOgn9AYI0MAt KRAw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=Y9sAFqwP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id b28-20020a631b1c000000b005c21c4a46ddsi4190840pgb.356.2023.11.19.11.48.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:48:22 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=Y9sAFqwP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id A78CB80A5FAC; Sun, 19 Nov 2023 11:48:21 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230393AbjKSTsP (ORCPT + 29 others); Sun, 19 Nov 2023 14:48:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34146 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229665AbjKSTsM (ORCPT ); Sun, 19 Nov 2023 14:48:12 -0500 Received: from mail-yw1-x1136.google.com (mail-yw1-x1136.google.com [IPv6:2607:f8b0:4864:20::1136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3749111A for ; Sun, 19 Nov 2023 11:48:09 -0800 (PST) Received: by mail-yw1-x1136.google.com with SMTP id 00721157ae682-5c9d850a0dbso6658997b3.2 for ; Sun, 19 Nov 2023 11:48:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423288; x=1701028088; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=cduqWiCMIKVDQKzalcbjrj5u/UZrGRNgI9eE9WPHu3A=; b=Y9sAFqwPPdpUQXqbpYpZjJ/bJFuU5qndB3eaipJ94VTJIKWoYM50djbzA8SIXrJp0E IQB+DXITZVEmKmCwo/wFMp1N+qG6z0btZOGJVXKggy+SgcgidRpVGvdBG4xhV4FE7AkJ l4rixv+zfi2XOjZY8M7Z/7CO8AVI+XwvjCe6aBACRlKbXd72bHRyilUuKRc5lIYCDts1 8HNy+kHE3SV6ZJvDbF9sGSrswumW19xFIu7juY64z337qf3doIndP5V1a1OyCwmyU5sA PF5ITZ1VZkihNjhzdlLL6Df5ZWehP+u2rGd4pmS2P/flRVc9zWrd6aZlwiFW9dJQcnAQ +PUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423288; x=1701028088; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=cduqWiCMIKVDQKzalcbjrj5u/UZrGRNgI9eE9WPHu3A=; b=Yj3eNh5jUBAYPYvV47npF8G3aojkyElxcmw0nAUeXqjHu5dzbTX2PMrdkcx8dIHSSa DQ0RoMYLhigpSvwDTto6btWVqAQu5aF1DNESgT9vPsnP+70h2E4pdZH0CJgkBE2M0WN3 ovdRVzX20sXBeKJd/zgJ95aZbOXKf2ETuzlYrNyIW1Znf8BM2T4mMSNsKpDSNDDEy874 cUNjRuizQH3mwx4GLTgWf9TgFuWRJ2BNrTTKx1W/WgTWarnrKQ0/x//JHMqXhf1eqjeb X7BHIz/pejB41bTRlSi9uKHxb+Xr95lsA9x7pvN8eYOXUqJ04jlO8a5+BEsoMzxqdtq4 itSw== X-Gm-Message-State: AOJu0YzS44t4sk1xM3JMDaCRHfjvwppsUbVPIeIer9h9MyGrUF+pRknw eq7Qc1mj36D36QQzv3rlvv0= X-Received: by 2002:a81:4841:0:b0:5be:94a6:d919 with SMTP id v62-20020a814841000000b005be94a6d919mr6198288ywa.25.1700423288412; Sun, 19 Nov 2023 11:48:08 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.05 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:07 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 02/24] mm/swapfile.c: add back some comment Date: Mon, 20 Nov 2023 03:47:18 +0800 Message-ID: <20231119194740.94101-3-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:48:21 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023065284357869 X-GMAIL-MSGID: 1783023065284357869 From: Kairui Song Some useful comments were dropped in commit b56a2d8af914 ('mm: rid swapoff of quadratic complexity'), add them back. Signed-off-by: Kairui Song --- mm/swapfile.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/mm/swapfile.c b/mm/swapfile.c index 4bc70f459164..756104ebd585 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1879,6 +1879,17 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, folio = page_folio(page); } if (!folio) { + /* + * The entry could have been freed, and will not + * be reused since swapoff() already disabled + * allocation from here, or alloc_page() failed. + * + * We don't hold lock here, so the swap entry could be + * SWAP_MAP_BAD (when the cluster is discarding). + * Instead of fail out, We can just skip the swap + * entry because swapoff will wait for discarding + * finish anyway. + */ swp_count = READ_ONCE(si->swap_map[offset]); if (swp_count == 0 || swp_count == SWAP_MAP_BAD) continue; From patchwork Sun Nov 19 19:47:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166863 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1809589vqn; Sun, 19 Nov 2023 11:48:28 -0800 (PST) X-Google-Smtp-Source: AGHT+IEm2NjBsjmqy3goMwuEYhWgNM8oqATmttJgZ/KWckgresFtGDSZJmy/kmDXPgUiFrbyKcgu X-Received: by 2002:a17:902:7e83:b0:1cc:4e79:4b38 with SMTP id z3-20020a1709027e8300b001cc4e794b38mr5615322pla.3.1700423308206; Sun, 19 Nov 2023 11:48:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423308; cv=none; d=google.com; s=arc-20160816; b=kAOmxdZ/n4pkEvhLCI/KG46rhYKBKrZGyi01PjYRurWGtSGuOlLCt7jw4pxy2l99p8 Wvt+unXDQ14Ghg87E1mZfDED+N9OkFbkvufyjcykBEsXX2SdyE35qiTXG8EKboXvwPlo 0MJVr57btNzk2nj7dw/j1sxYcRQM6YeeBUu7OJOIOh4ma6vM47TZQrnR4Pkai5fD/lLA GvzQ9n9qoVXRNSeoS+2KZCd4JkNVUx7OR59u8gPH1R36g7iLd72GJhrmHQ4ad95ljdLA gJA1x47wowEJOgvMNNOZKrRBUfcWSW7viZGDOSZgcqgfi3hh6E7WvtKIfHw/hZu2lMg6 UdFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=NgugyapInVThiGa6OPAw9VMpbPbUYFH6tRNCt397cNY=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=coXjqXpK8S0UIv3f0h6ZAzI8F+qQiqFzdrGFtQ2G26xIvfsHy2FoRO+ySfrNY3A8Si vnafpJdsQLrB8HbhZc7EWHzQCrq/DPVEti19W4qVt1VGorCWB8NlncBAdNBrL5jWhX08 ErMlbJtYsnSmQIGQDA+x3wh741y7fzaWHitOMBISs2i0itCWXce1vauA48ZUzrO8XncC ciIFEQhqlnlCqRh0uASFscUlEgSB0RCUTgOhR6ouRTKEhbUz++GIKLpGppi8jHhrTwFT 3/0tYOrWXRQbtZElU8Taa5RSL9dyL5LOpe3d0KsmbqhOlkP3KT1DYb3kZhicEzeuSG2m oFgg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=GObJgcqd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id f8-20020a170902ce8800b001cc550c83d5si6913554plg.515.2023.11.19.11.48.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:48:28 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=GObJgcqd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 6233080A5F93; Sun, 19 Nov 2023 11:48:27 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231472AbjKSTsU (ORCPT + 29 others); Sun, 19 Nov 2023 14:48:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34256 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231493AbjKSTsS (ORCPT ); Sun, 19 Nov 2023 14:48:18 -0500 Received: from mail-pf1-x436.google.com (mail-pf1-x436.google.com [IPv6:2607:f8b0:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6AB84196 for ; Sun, 19 Nov 2023 11:48:12 -0800 (PST) Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-6bd32d1a040so3864878b3a.3 for ; Sun, 19 Nov 2023 11:48:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423292; x=1701028092; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=NgugyapInVThiGa6OPAw9VMpbPbUYFH6tRNCt397cNY=; b=GObJgcqdCADqRZUeGgzpzML598kfsRjsx1/ApIwBRkwyRZfv4JTJNzzVMRhEGnYOus AZIRet3CKOI5UCSlWU96VblTv2eoDprSaflA2IPFtEpQBS907win5ArQrJ2GtII08ELF 8Tf7VnsLtZNTyUJo4U0mYdFaQA7K11AgoSzwUjRIo6dq8XcT4Ny9Hn/E1IfhswM0AZ/y d5QLcXh8DgXNbdepUWhVDW+pNPyVPkVn3pYZdS4vySb6TYSwD1kN3nvKoTALAl3UT0fR Nf2IVyLuNlYmS33XSWQCC2cAr9yIrEPYBfB8dejDBsOqsSePzR6Js7Gk/KgfE4dJ6lvw qbcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423292; x=1701028092; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=NgugyapInVThiGa6OPAw9VMpbPbUYFH6tRNCt397cNY=; b=ns/c3LVNwQPPucnVHnC5CZD5Ik3PBZEoAfA/LI7JONYthWx9wmeQGmOLCI6D/0Ke0P yRvbURQuiWteqxuKROTOdsXMCIXWyba8YYjSaiQitW/UUvdTLEknMKib1QpSnSfcpZsn +kUObfviycudohqu881n9YXMTJIQChyXCIOfZVP2yrC0LOllPNYZhoTOAKEh0NYcsTTs shPTNW0KqUnyHpLoDS6p/tIk+9cPm13eYWpM4jzfUlbmk8wq93RyGR+6uGbvf9aC4ubv GfHrfailACG3KyDJdQt4+M/z60rPjxfknj1viPMOgyUr40E11oBKH0OBjSJJXMnn8Bq7 r9vA== X-Gm-Message-State: AOJu0YyCmEi39P8k0/lAmouc+Tr2deMjjpDWh4yXeGMDolWRjdJND91n JGdMP3aQkeq6gjIWfxTqFHgsFreWPs1kb56R X-Received: by 2002:a05:6a20:e104:b0:18a:181b:146b with SMTP id kr4-20020a056a20e10400b0018a181b146bmr5052162pzb.29.1700423291675; Sun, 19 Nov 2023 11:48:11 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.08 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:11 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 03/24] mm/swap: move no readahead swapin code to a stand alone helper Date: Mon, 20 Nov 2023 03:47:19 +0800 Message-ID: <20231119194740.94101-4-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:48:27 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023070747262067 X-GMAIL-MSGID: 1783023070747262067 From: Kairui Song No feature change, simply move the routine to a standalone function to be used later. The error path handling is copied from the "out_page" label, to make the code change minimized for easier reviewing. Signed-off-by: Kairui Song --- mm/memory.c | 33 +++++---------------------------- mm/swap.h | 2 ++ mm/swap_state.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 55 insertions(+), 28 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 70ffa867b1be..fba4a5229163 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3794,7 +3794,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) swp_entry_t entry; pte_t pte; vm_fault_t ret = 0; - void *shadow = NULL; if (!pte_unmap_same(vmf)) goto out; @@ -3858,33 +3857,11 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) if (!folio) { if (data_race(si->flags & SWP_SYNCHRONOUS_IO) && __swap_count(entry) == 1) { - /* skip swapcache */ - folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, - vma, vmf->address, false); - if (folio) { - __folio_set_locked(folio); - __folio_set_swapbacked(folio); - - if (mem_cgroup_swapin_charge_folio(folio, - vma->vm_mm, GFP_KERNEL, - entry)) { - ret = VM_FAULT_OOM; - goto out_page; - } - mem_cgroup_swapin_uncharge_swap(entry); - - shadow = get_shadow_from_swap_cache(entry); - if (shadow) - workingset_refault(folio, shadow); - - folio_add_lru(folio); - page = &folio->page; - - /* To provide entry to swap_readpage() */ - folio->swap = entry; - swap_readpage(page, true, NULL); - folio->private = NULL; - } + /* skip swapcache and readahead */ + page = swapin_no_readahead(entry, GFP_HIGHUSER_MOVABLE, + vmf); + if (page) + folio = page_folio(page); } else { page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, vmf); diff --git a/mm/swap.h b/mm/swap.h index 73c332ee4d91..ea4be4791394 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -56,6 +56,8 @@ struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t flag, struct mempolicy *mpol, pgoff_t ilx); struct page *swapin_readahead(swp_entry_t entry, gfp_t flag, struct vm_fault *vmf); +struct page *swapin_no_readahead(swp_entry_t entry, gfp_t flag, + struct vm_fault *vmf); static inline unsigned int folio_swap_flags(struct folio *folio) { diff --git a/mm/swap_state.c b/mm/swap_state.c index 85d9e5806a6a..ac4fa404eaa7 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -853,6 +853,54 @@ static struct page *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask, return page; } +/** + * swapin_no_readahead - swap in pages skipping swap cache and readahead + * @entry: swap entry of this memory + * @gfp_mask: memory allocation flags + * @vmf: fault information + * + * Returns the struct page for entry and addr after the swap entry is read + * in. + */ +struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, + struct vm_fault *vmf) +{ + struct vm_area_struct *vma = vmf->vma; + struct page *page = NULL; + struct folio *folio; + void *shadow = NULL; + + folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, + vma, vmf->address, false); + if (folio) { + __folio_set_locked(folio); + __folio_set_swapbacked(folio); + + if (mem_cgroup_swapin_charge_folio(folio, + vma->vm_mm, GFP_KERNEL, + entry)) { + folio_unlock(folio); + folio_put(folio); + return NULL; + } + mem_cgroup_swapin_uncharge_swap(entry); + + shadow = get_shadow_from_swap_cache(entry); + if (shadow) + workingset_refault(folio, shadow); + + folio_add_lru(folio); + + /* To provide entry to swap_readpage() */ + folio->swap = entry; + page = &folio->page; + swap_readpage(page, true, NULL); + folio->private = NULL; + } + + return page; +} + /** * swapin_readahead - swap in pages in hope we need them soon * @entry: swap entry of this memory From patchwork Sun Nov 19 19:47:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166864 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1809601vqn; Sun, 19 Nov 2023 11:48:31 -0800 (PST) X-Google-Smtp-Source: AGHT+IEoVAfxqyj98H/qF3T9AseRBdZLe2x8w1ezygswot6nPTOYSZ7oio2UTpsmz4dos0vIHmuF X-Received: by 2002:a17:90a:1690:b0:281:416e:1c3f with SMTP id o16-20020a17090a169000b00281416e1c3fmr3936343pja.28.1700423311332; Sun, 19 Nov 2023 11:48:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423311; cv=none; d=google.com; s=arc-20160816; b=XjwSqRWi3jtbU6gqt8sCrsOLb79Jw0bKv4zggJ8gKVD+LK7a7EP7KJDYPFK1CF3pF/ AhUqplHtXGhJ7obJWEdPtonIY3FOwKUOryaemVzImy1qr7Sy/WSf8AB80+gWEiYB5IvU YNzB1EpQOz7EwsZ28xT/ILTY7ews0rmShEzIOkRJUjTzKbKDqzJmrHWtaCtbhMdVID2j aSyVUYFqMxIF82FLU7s3acnJDTHzpNazj+fChBs9XkRFnZcb26w+qJcn0Ah28QWawZHs P5CGD4Wb5Kq9DJplkBSJpf6M4jedPITSi85ihdIFwk0/CK8pRecWXs93IfnuTEIyfZmn QjKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=xjzoOYVfaQZliXw2umUqNXWj2wEYGvE1fLa1+gbge5g=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=fpuBL2q2kzDlwym0RvFMbyowuZm2aWw4T+VjzvEAvOGFsqY0rOZDzDG7ufucguMrRS XeYBW4V2QkRdjB23P1WQNFoeMzpWReV+dr2lIQKIyr5yjpKrC4fb/MDKT6yV9kMB1whc GgyRRYoUa2l3mEhm5Im0Otx3ukS/y3yciKU+BQzg8CMkSyj8AQX+b6Hg7vvxn7wuIPr2 sR9zanOK5sO+LoRpekuD6YqWhdCbmKxnSwCV8VDZEl9qQ0392uRI9P5jj+pbVbQA4HY4 o3AR3dlifvK6bl7GPSigBzG/3RyNauOFe+icvg3uscqY7DufP9xHMGYcEjjYM5Tfhk8L s4Ug== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=IusQLIKD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id s7-20020a170902ea0700b001ce64a096dfsi5381323plg.502.2023.11.19.11.48.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:48:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=IusQLIKD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 75AC480A9900; Sun, 19 Nov 2023 11:48:30 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230053AbjKSTsX (ORCPT + 29 others); Sun, 19 Nov 2023 14:48:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34310 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230475AbjKSTsU (ORCPT ); Sun, 19 Nov 2023 14:48:20 -0500 Received: from mail-ot1-x335.google.com (mail-ot1-x335.google.com [IPv6:2607:f8b0:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E4EDED5D for ; Sun, 19 Nov 2023 11:48:15 -0800 (PST) Received: by mail-ot1-x335.google.com with SMTP id 46e09a7af769-6ce2cf67be2so2082085a34.2 for ; Sun, 19 Nov 2023 11:48:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423295; x=1701028095; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=xjzoOYVfaQZliXw2umUqNXWj2wEYGvE1fLa1+gbge5g=; b=IusQLIKDW1ybycdFgJM22Bx23qUm7uOyWwwZrWxeWOz54H0+166QH1cupFkrJ7z3qU PKVbvp8SDoKUR63g3SMFujLfAAB0PkeY7X/R0ugnhUve16dPomwhOgnL8eZjHl7nFOxo McGvs6UkJCeQ5zdZBZLBdIdu770XpqgpoqREtIcC77A/Q/A6an29HV7BEXk6J+/BLmTC VkH+s4IpSReoXwzBFlgK4wcThRzXOgmucaS/agfqEUVQxrvL29D8wJ7p4jiTRRy8ylNK 7VeOoXaKKnQiT2xHZpcgSbwxJvE69Ena15wcbOkK6o6vmoIDXJYe9vL7hxcczurKFa1+ yrew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423295; x=1701028095; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=xjzoOYVfaQZliXw2umUqNXWj2wEYGvE1fLa1+gbge5g=; b=wJgwmfXTB+WolAYtw0dtjmnvzSbO+Ua/M2C6WUk7NtFGmUUl5pk8z/YlzSCi7hSk6F EVYYqOOPTkkMNtIQvQINE2uVEHLx7UQ9JImgD7oHs6ve5VOxJr0waqhcXRRJSfEWcHwe fx2KBq5Jl58cKsRz+dTaBlhIf7EhZ2x6RiCx7p+AAc5lf/huaCAnu1XkIZ2fCMhY4IKl 0t52lkHTa5ZcQJJAdDe3qOgGu9QWGdC3QreuPxDJDOgpWUkDgGpBKRDTXpvI47CZIFPv rRO5KLjh5y3V6EynpcRVWLMM49RBh/mn7pcSYF7T1CWznuKBhIdOIY5Lu82eF83SikLZ 5eww== X-Gm-Message-State: AOJu0YyTrmwdZDSR+nShLug3Z+4TycM3VtBPe9nhXusLhw0MsEW7hXvm RdXm9Ft3YWdKh53ajIMRS3I= X-Received: by 2002:a05:6870:3b85:b0:1f9:36fe:fd0e with SMTP id gi5-20020a0568703b8500b001f936fefd0emr1616531oab.47.1700423294901; Sun, 19 Nov 2023 11:48:14 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.12 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:14 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 04/24] mm/swap: avoid setting page lock bit and doing extra unlock check Date: Mon, 20 Nov 2023 03:47:20 +0800 Message-ID: <20231119194740.94101-5-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:48:30 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023073626289089 X-GMAIL-MSGID: 1783023073626289089 From: Kairui Song When swapping in a page, mem_cgroup_swapin_charge_folio is called for new allocated folio, nothing else is referencing the folio so no need to set the lock bit. This avoided doing unlock check on error path. Signed-off-by: Kairui Song --- mm/swap_state.c | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/mm/swap_state.c b/mm/swap_state.c index ac4fa404eaa7..45dd8b7c195d 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -458,6 +458,8 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, mpol, ilx, numa_node_id()); if (!folio) goto fail_put_swap; + if (mem_cgroup_swapin_charge_folio(folio, NULL, gfp_mask, entry)) + goto fail_put_folio; /* * Swap entry may have been freed since our caller observed it. @@ -483,13 +485,9 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, /* * The swap entry is ours to swap in. Prepare the new page. */ - __folio_set_locked(folio); __folio_set_swapbacked(folio); - if (mem_cgroup_swapin_charge_folio(folio, NULL, gfp_mask, entry)) - goto fail_unlock; - /* May fail (-ENOMEM) if XArray node allocation failed. */ if (add_to_swap_cache(folio, entry, gfp_mask & GFP_RECLAIM_MASK, &shadow)) goto fail_unlock; @@ -510,6 +508,7 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, fail_unlock: put_swap_folio(folio, entry); folio_unlock(folio); +fail_put_folio: folio_put(folio); fail_put_swap: put_swap_device(si); @@ -873,16 +872,15 @@ struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, vma, vmf->address, false); if (folio) { - __folio_set_locked(folio); - __folio_set_swapbacked(folio); - - if (mem_cgroup_swapin_charge_folio(folio, - vma->vm_mm, GFP_KERNEL, - entry)) { - folio_unlock(folio); + if (mem_cgroup_swapin_charge_folio(folio, vma->vm_mm, + GFP_KERNEL, entry)) { folio_put(folio); return NULL; } + + __folio_set_locked(folio); + __folio_set_swapbacked(folio); + mem_cgroup_swapin_uncharge_swap(entry); shadow = get_shadow_from_swap_cache(entry); From patchwork Sun Nov 19 19:47:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166865 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1809674vqn; Sun, 19 Nov 2023 11:48:46 -0800 (PST) X-Google-Smtp-Source: AGHT+IHBKbGqdTlHbo1IgmLS1WLxwDz7JjTcmGIXJFy2Xqs+uFzrpi/Vgt7uYGuWR6oNmtRunLBC X-Received: by 2002:a05:6808:4493:b0:3af:75e2:4c34 with SMTP id eq19-20020a056808449300b003af75e24c34mr8406410oib.50.1700423326331; Sun, 19 Nov 2023 11:48:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423326; cv=none; d=google.com; s=arc-20160816; b=L29Yh8OxJzzSKY3QBEl0IaymBNL4t9tTJYbrB24j97znPPG4RF/OPOFDVkKXNGmt57 n+eRfJFmfVikxgbmqHfX56noXdFI3E4epRDrW1I5DMHHlZRuHoiC5vWEF/BiJx8A1yiq CK8N4Kq2ArSDFABhFczv+AxO8Fc6xbtZ8t6763qVQ40TOzpVI2K/BxaX0cnxB9rK9Dig DaomH1oY1U6k/ecrNoHIDLtpvTldB0knKfPC7ymbGFSDXPbxjwVGnca3HbpRjyXA5x79 tExtZupjlkGVwASDoj41B49rnLULl9qbXs4pwb+1jvWcGMNhygO7ylrQfACP8B1Xse6S LLgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ZPY5phyUcHn1HYB2k83NkwocYwAsatgYc16U9qxXR/Y=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=O4hDakd3A3VY3+MOfWCddUto3oXuiFlVI024pgYPU0x6RCW5tP2Wsggab9573EAKEs lQew1xbyUyczH98LSExpJhbyCRrFt1OPZnCAHBZXxDAtJKc8FMtR4+belyizbYwm4zND DqKr+oeRW5oImQZzG4yqT/4gTNq8HtYoMg9BYv2MfnGuow8Dbr4KhHmCYogW88fcTtqz UI81IT2X1BsuvKbnHfRVlqtiMbzut/DwQRCgBeoJKqWUp4AkFwb6lHfDKbcRZnd5pjBo GoV3IaeaLBk3wVIhhM/TiV9fzFzhjWof7JkdpLixR1xpnagIyq15i2JpVHmjU4+7SHiT hChQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=ThMc3rQj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id h31-20020a63f91f000000b005be029a66d1si6545631pgi.806.2023.11.19.11.48.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:48:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=ThMc3rQj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 8ADF280AB1FA; Sun, 19 Nov 2023 11:48:42 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231616AbjKSTse (ORCPT + 29 others); Sun, 19 Nov 2023 14:48:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231603AbjKSTs0 (ORCPT ); Sun, 19 Nov 2023 14:48:26 -0500 Received: from mail-pf1-x436.google.com (mail-pf1-x436.google.com [IPv6:2607:f8b0:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1A59FD4C for ; Sun, 19 Nov 2023 11:48:19 -0800 (PST) Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-6b87c1edfd5so2937275b3a.1 for ; Sun, 19 Nov 2023 11:48:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423298; x=1701028098; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=ZPY5phyUcHn1HYB2k83NkwocYwAsatgYc16U9qxXR/Y=; b=ThMc3rQjeBc3YRBPY3lazI6ZtrlWg6OFwSUsgJ98uViolM4G9UyMkniVR0nipcv976 jpksn9aq2OMWEsUIgW2yl+CLwRVRxrQY3aHDnL+IyOAFATJRTvmKKNCcnHJk/Iwu8Lnc GwILHIAPriFqtgjEMkhyV8Je9zTxiSBqJemUSDYfrYuHbOZBWNrkJfU+pNC4G27o9Jkh oq187xX7M4l7kjHkEKatUul8NPsbCtPlYqo8Lv1OJpebh31lAt0PJrh2RTh5lTcaE2Ia XCWQiNDJK3GOOL9S2pusKxL8g7qPCpZ0sXvBPYA40puwG+u7BLzvn5hbp5Q0Sj8SuL+k eVrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423298; x=1701028098; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ZPY5phyUcHn1HYB2k83NkwocYwAsatgYc16U9qxXR/Y=; b=b/TlLi+Clipcox/wIVm4zXDqlhdPf/YQCZza8Gv/8vCLSfsL9O23s/t9Winyoa8wjC kpxtmyMbrJF56tPHEldCfKrIoUgnd7r7cGiOwRFp0IGpGoMjwCDp8w+J2zP+nYAbVg99 UndSfCIaHUH9pDG+BxDR4k/zrP9eE2I9mvdyS8SJgEpFkkYLgBxydgc2zbzhdkf7cddU JVaol55SOrImIHp1flHNqpWL6ylRVe9nJtPNMN2u4F3UG79pPSG4zqC3tGIocIKKJBvv JqviRmldK2PpB9unYKiyWSpn1Z93xvQN3d/M9J3rA6G1LT82wNPjEjhbayBllZNfzwM3 jt7Q== X-Gm-Message-State: AOJu0Yy1cq9vhTUMcAsxELBL8sXZgT+obonFotBxgxrizh9zwMtI7CYX 417NIpT/DEuVTv2AyZweBk8= X-Received: by 2002:aa7:9ddd:0:b0:6c4:dc5b:5b2b with SMTP id g29-20020aa79ddd000000b006c4dc5b5b2bmr3806580pfq.20.1700423298129; Sun, 19 Nov 2023 11:48:18 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.15 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:17 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 05/24] mm/swap: move readahead policy checking into swapin_readahead Date: Mon, 20 Nov 2023 03:47:21 +0800 Message-ID: <20231119194740.94101-6-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:48:45 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023090036537985 X-GMAIL-MSGID: 1783023090036537985 From: Kairui Song This makes swapin_readahead a main entry for swapin pages, prepare for optimizations in later commits. This also makes swapoff able to make use of readahead checking based on entry. Swapping off a 10G ZRAM (lzo-rle) is faster: Before: time swapoff /dev/zram0 real 0m12.337s user 0m0.001s sys 0m12.329s After: time swapoff /dev/zram0 real 0m9.728s user 0m0.001s sys 0m9.719s And what's more, because now swapoff will also make use of no-readahead swapin helper, this also fixed a bug for no-readahead case (eg. ZRAM): when a process that swapped out some memory previously was moved to a new cgroup, and the original cgroup is dead, swapoff the swap device will make the swapped in pages accounted into the process doing the swapoff instead of the new cgroup the process was moved to. This can be easily reproduced by: - Setup a ramdisk (eg. ZRAM) swap. - Create memory cgroup A, B and C. - Spawn process P1 in cgroup A and make it swap out some pages. - Move process P1 to memory cgroup B. - Destroy cgroup A. - Do a swapoff in cgroup C. - Swapped in pages is accounted into cgroup C. This patch will fix it make the swapped in pages accounted in cgroup B. The same bug exists for readahead path too, we'll fix it in later commits. Signed-off-by: Kairui Song --- mm/memory.c | 22 +++++++--------------- mm/swap.h | 6 ++---- mm/swap_state.c | 33 ++++++++++++++++++++++++++------- mm/swapfile.c | 2 +- 4 files changed, 36 insertions(+), 27 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index fba4a5229163..f4237a2e3b93 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3792,6 +3792,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) rmap_t rmap_flags = RMAP_NONE; bool exclusive = false; swp_entry_t entry; + bool swapcached; pte_t pte; vm_fault_t ret = 0; @@ -3855,22 +3856,13 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) swapcache = folio; if (!folio) { - if (data_race(si->flags & SWP_SYNCHRONOUS_IO) && - __swap_count(entry) == 1) { - /* skip swapcache and readahead */ - page = swapin_no_readahead(entry, GFP_HIGHUSER_MOVABLE, - vmf); - if (page) - folio = page_folio(page); + page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, + vmf, &swapcached); + if (page) { + folio = page_folio(page); + if (swapcached) + swapcache = folio; } else { - page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, - vmf); - if (page) - folio = page_folio(page); - swapcache = folio; - } - - if (!folio) { /* * Back out if somebody else faulted in this pte * while we released the pte lock. diff --git a/mm/swap.h b/mm/swap.h index ea4be4791394..f82d43d7b52a 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -55,9 +55,7 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t flag, struct mempolicy *mpol, pgoff_t ilx); struct page *swapin_readahead(swp_entry_t entry, gfp_t flag, - struct vm_fault *vmf); -struct page *swapin_no_readahead(swp_entry_t entry, gfp_t flag, - struct vm_fault *vmf); + struct vm_fault *vmf, bool *swapcached); static inline unsigned int folio_swap_flags(struct folio *folio) { @@ -89,7 +87,7 @@ static inline struct page *swap_cluster_readahead(swp_entry_t entry, } static inline struct page *swapin_readahead(swp_entry_t swp, gfp_t gfp_mask, - struct vm_fault *vmf) + struct vm_fault *vmf, bool *swapcached) { return NULL; } diff --git a/mm/swap_state.c b/mm/swap_state.c index 45dd8b7c195d..fd0047ae324e 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -316,6 +316,11 @@ void free_pages_and_swap_cache(struct encoded_page **pages, int nr) release_pages(pages, nr); } +static inline bool swap_use_no_readahead(struct swap_info_struct *si, swp_entry_t entry) +{ + return data_race(si->flags & SWP_SYNCHRONOUS_IO) && __swap_count(entry) == 1; +} + static inline bool swap_use_vma_readahead(void) { return READ_ONCE(enable_vma_readahead) && !atomic_read(&nr_rotate_swap); @@ -861,8 +866,8 @@ static struct page *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask, * Returns the struct page for entry and addr after the swap entry is read * in. */ -struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, - struct vm_fault *vmf) +static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, + struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; struct page *page = NULL; @@ -904,6 +909,8 @@ struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, * @entry: swap entry of this memory * @gfp_mask: memory allocation flags * @vmf: fault information + * @swapcached: pointer to a bool used as indicator if the + * page is swapped in through swapcache. * * Returns the struct page for entry and addr, after queueing swapin. * @@ -912,17 +919,29 @@ struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, * or vma-based(ie, virtual address based on faulty address) readahead. */ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, - struct vm_fault *vmf) + struct vm_fault *vmf, bool *swapcached) { struct mempolicy *mpol; - pgoff_t ilx; struct page *page; + pgoff_t ilx; + bool cached; mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); - page = swap_use_vma_readahead() ? - swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf) : - swap_cluster_readahead(entry, gfp_mask, mpol, ilx); + if (swap_use_no_readahead(swp_swap_info(entry), entry)) { + page = swapin_no_readahead(entry, gfp_mask, vmf); + cached = false; + } else if (swap_use_vma_readahead()) { + page = swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf); + cached = true; + } else { + page = swap_cluster_readahead(entry, gfp_mask, mpol, ilx); + cached = true; + } mpol_cond_put(mpol); + + if (swapcached) + *swapcached = cached; + return page; } diff --git a/mm/swapfile.c b/mm/swapfile.c index 756104ebd585..0142bfc71b81 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1874,7 +1874,7 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, }; page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, - &vmf); + &vmf, NULL); if (page) folio = page_folio(page); } From patchwork Sun Nov 19 19:47:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166866 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1809687vqn; Sun, 19 Nov 2023 11:48:51 -0800 (PST) X-Google-Smtp-Source: AGHT+IGNZVoZM8cKfoIsKZ1DWgLzHxRcpqRCrMGMM+Aq38AxwPcgHsQ0T18Hb+8FRd/bI2ex9y8T X-Received: by 2002:a05:6a20:4401:b0:189:df1b:661a with SMTP id ce1-20020a056a20440100b00189df1b661amr3809430pzb.52.1700423330851; Sun, 19 Nov 2023 11:48:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423330; cv=none; d=google.com; s=arc-20160816; b=H4O3OOvN0tWJgAr3irQzk/FK2Q8x647BrQM4O9SCuQYQ25AX2yWF75UW8Lax0px6W1 s0MaO06HLQEOt0yEHp1suuip/U0iltI8G4LeFGXau9od952bBah6NNlwIz+ZwIjTMdyR oYYiw9E73lfS3xrVt5gRkcLtdOpUo4Nyc6IUpBfW3kRwdShlBj+QaLM3u8IrKMIIWcpH YQ3jCNmTYSRiIMqocwQEjM84qAPvIqg4jopOfE4NyBDVkU+1gFgNzWkUz3T868Z1QD7p 6Ew8trxppb48/oFz75DYZRE+Wl0+xBhD+/byBZbtjvLt1haKlR+M97h7RyTZwkrlI+lP VdiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=UpGTVr/h0gRLNFrOGvqL9Asvs1D6zvF8lBMhGssTB0o=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=Q9RrLeqqmZctB8QUbxliz+8cr7eP4HTbGBGQPObASfZhf7X+zxkqG8Q43QcqijWYbk sIINGH1GdIlK2N2QNYP0bF1dp8q4oU/6DLILj+I/mdEU0XwFLFyTzZT6X7WrDGukgORh rOl7GEWYlRIqXaBVUFMPUUZucGcD1mxC1Q6lcNTv4XXYrqAeXv17vgRUUNz4f9hJQury HSvfTMJ3jzGaeRgRQiRExuRBT2bbRNP4Iqj6pwR7NJtfBXN9iQiD0t+Z3P86+l0sfzQU 0QinWxRXWaVr4ju01z94AGPRRoL5HvO3dnOf5JWs+efEMK8lzpBpe2cx7rCczNgilQ+n h3GQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=XnNMl8Ga; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id o2-20020a635a02000000b005c2185da898si4608283pgb.832.2023.11.19.11.48.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:48:50 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=XnNMl8Ga; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 1550C80ABFCF; Sun, 19 Nov 2023 11:48:50 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231550AbjKSTsg (ORCPT + 29 others); Sun, 19 Nov 2023 14:48:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34252 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231624AbjKSTsa (ORCPT ); Sun, 19 Nov 2023 14:48:30 -0500 Received: from mail-pf1-x436.google.com (mail-pf1-x436.google.com [IPv6:2607:f8b0:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43D08D67 for ; Sun, 19 Nov 2023 11:48:22 -0800 (PST) Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-6c33ab26dddso3169833b3a.0 for ; Sun, 19 Nov 2023 11:48:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423301; x=1701028101; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=UpGTVr/h0gRLNFrOGvqL9Asvs1D6zvF8lBMhGssTB0o=; b=XnNMl8Ga2yZtHo/rmlT8dejCkZca0iXOSyw88mV4ZxNzqnt7obnsKqxL67gT9F7mP4 stNe9RLFHjpiZPchL4n3ojUiIQaXnbHn2Oo4tcAJPfvaozdUkhcMm5kVseBWYD4PatrX it/4SaVCte/NV+BqH0IOwZECAaDm2sw6al/DdYT7mLNPYv5eavbyWLFIAHD/yv1kIxWd zOmT4ECj5Z9JBfMDdRF4tKdF/ogC6yfZwab4FSsjOp/Ht4cZ3KC9T5CV445o3L0s5sVW 0Msu/X32pj7swmQbSUuzeJJeVh9uO1ytJtp/0dcE0k3QMT9q1DJF3xi+0/gnS4R/kATu BbUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423301; x=1701028101; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=UpGTVr/h0gRLNFrOGvqL9Asvs1D6zvF8lBMhGssTB0o=; b=nuJJ1y1Qm3xvE17ZW5aCs0+rt97ONmOzYjE16hHOrIFHsiubPCdfmSdNmbSrDrnHvT 8r4XZocPlYrut1stIvNVCZxzFzuz8m1kdNV8agyFj7dzCLzgk8NtZHY6TwuwNtZPiAcC tamCIWZ0sOoSj8XTOFqdn2VU1c/lo8ubtM/4ptn/t0PuJwUb13wBZmnyMM32SbP41s5q ieYzgz0kesnmxNZiHTqz1lP5XrVaWmEvd09/j6/mxBK1JYe++r7skTiXPA2tMU4ASchE 0KUPQyFiIcASeKeAC62AU/hvJJzjUFN8GxnUABMHC37Ke/h3teNcq9s0T12h5plowK06 5XsQ== X-Gm-Message-State: AOJu0YyzaIzwjM9x3vxbPNNVRFync1bFzFeiCqEMsO+lxFTMWAL4jL8+ 1LWtc0LCozmpbaehmdu7s1cTjUOUP/3EYKfH X-Received: by 2002:a05:6a20:b797:b0:187:8eca:8dc6 with SMTP id fh23-20020a056a20b79700b001878eca8dc6mr2490060pzb.34.1700423301416; Sun, 19 Nov 2023 11:48:21 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.18 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:20 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 06/24] swap: rework swapin_no_readahead arguments Date: Mon, 20 Nov 2023 03:47:22 +0800 Message-ID: <20231119194740.94101-7-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:48:50 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023094989102166 X-GMAIL-MSGID: 1783023094989102166 From: Kairui Song Make it use alloc_pages_mpol instead of vma_alloc_folio, and accept mm_struct directly as an argument instead of taking a vmf as argument. Make its arguments similar to swap_{cluster,vma}_readahead, to make the code more aligned. Also prepare for following commits which will skip vmf for certain swapin paths. Signed-off-by: Kairui Song --- mm/swap_state.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/mm/swap_state.c b/mm/swap_state.c index fd0047ae324e..ff6756f2e8e4 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -867,17 +867,17 @@ static struct page *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask, * in. */ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, - struct vm_fault *vmf) + struct mempolicy *mpol, pgoff_t ilx, + struct mm_struct *mm) { - struct vm_area_struct *vma = vmf->vma; - struct page *page = NULL; struct folio *folio; + struct page *page; void *shadow = NULL; - folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, - vma, vmf->address, false); + page = alloc_pages_mpol(gfp_mask, 0, mpol, ilx, numa_node_id()); + folio = (struct folio *)page; if (folio) { - if (mem_cgroup_swapin_charge_folio(folio, vma->vm_mm, + if (mem_cgroup_swapin_charge_folio(folio, mm, GFP_KERNEL, entry)) { folio_put(folio); return NULL; @@ -896,7 +896,6 @@ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, /* To provide entry to swap_readpage() */ folio->swap = entry; - page = &folio->page; swap_readpage(page, true, NULL); folio->private = NULL; } @@ -928,7 +927,7 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); if (swap_use_no_readahead(swp_swap_info(entry), entry)) { - page = swapin_no_readahead(entry, gfp_mask, vmf); + page = swapin_no_readahead(entry, gfp_mask, mpol, ilx, vmf->vma->vm_mm); cached = false; } else if (swap_use_vma_readahead()) { page = swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf); From patchwork Sun Nov 19 19:47:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166867 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1809695vqn; Sun, 19 Nov 2023 11:48:53 -0800 (PST) X-Google-Smtp-Source: AGHT+IF8fgi06KxuOuxoqwO/rEwMFeMlTEFP81oOGTdIr7DaKpN8JY1LuQjdtjdO+5M7kJU8ZeX0 X-Received: by 2002:a17:90a:e44e:b0:285:117a:deb2 with SMTP id jp14-20020a17090ae44e00b00285117adeb2mr5688702pjb.23.1700423333596; Sun, 19 Nov 2023 11:48:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423333; cv=none; d=google.com; s=arc-20160816; b=WVCWHdH7HO8FhlkLfx+eRtFs0xX3ugjwHZmXW2G4YI65v+Wre8zU7JJzhuoww10mxz W9hG4X0de1B1c1IyrEqKYEcj+lYTgS9AHLmSDY6Q/vHm5OGuHpzMDEKtuw/pzI6IKIDZ 0zcQp72FSFDsD3A6YYrchIHFsMuqrubpgXNGLYlI1uLuls2Md2MQfXbh1UWqlX+jAoSq CYEpQKNtB1cgmZdW8G4W4NnauXhhHSPEo8BEBQSAo+7EcIJwzAEixiRMq415X2PVTM8x iNlqeY9Cu4hKLM2rmX7A7ygd0YQblzCiOzPGlPhcfQY3yJP23aT3nwjTRkFdhzA/jBPI LNSg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=65vQj2xJm5p5mk5SbjAKMeJnwT21tT7eyXmu2H+Etu0=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=UGMGrzsCzarL1YjbCrwa58QKuvbgoZxgsSYjXlo6A15wwsHnNTXvgLuEbdOvInMfy+ CgLg/OBz/0iJqiFCY1WsLmWlrj4MPTkycjiAtCsw3pAsFP+do//qyVYwqjqKDPM9sQz7 HgxyHDUFl8+YHp+QI+nk3qMom6agyo3cmkro2rrl9BG2eSiSU0BjzqqvRA7btLeH8FO4 gtXvoai0/DKGpZV1KL3Lzo4KnwHrCqwEErsJ5fj2Ors9ZH8PWNo4mGg/qYv97/Y8J/F0 GqPjVC/Ztz0vrODv+9Pd1B44hnzJLPeQH/WFNotQLXFhsrpCLBBRklosSkfbnYo3wU9I BnYg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=B2qyu2tH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id cm11-20020a056a020a0b00b005bdfdf1c44dsi7089999pgb.126.2023.11.19.11.48.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:48:53 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=B2qyu2tH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id BB6A0804C57D; Sun, 19 Nov 2023 11:48:52 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231637AbjKSTss (ORCPT + 29 others); Sun, 19 Nov 2023 14:48:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231564AbjKSTsh (ORCPT ); Sun, 19 Nov 2023 14:48:37 -0500 Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1666810EC for ; Sun, 19 Nov 2023 11:48:25 -0800 (PST) Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-6cb66fbc63dso521503b3a.0 for ; Sun, 19 Nov 2023 11:48:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423304; x=1701028104; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=65vQj2xJm5p5mk5SbjAKMeJnwT21tT7eyXmu2H+Etu0=; b=B2qyu2tHHOjSYZHMRZ1A/7KsSk+HF7D9RY6pMMFaA3xA7VmeEeQXegg0xgu2IhPvJl uuw/J7qKy27Ew+eaLv9nZPNQKm+I2LoasjIwWE3NV/3YQHs8/tjWCnB0Vk7xJoPmkwSG wPmhJ8XfDAHUf0VG44SCngQRGazrHxflE2bVngV1ONwqQ/XcwawtdHCwjVNES5cdoQ0G pMjdgO6j76wbQI5M1n+5PeCPj1SAEjkWq63I4VR4NUMRfQIik/wA+LsabbkrlQgEhFdz M4U1IV6cbf4Bn5rZvZfv+huCUtl/P6USk35yCCoP7v6odo7ycHcaypk/rRWLMfl+9fX/ rcww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423304; x=1701028104; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=65vQj2xJm5p5mk5SbjAKMeJnwT21tT7eyXmu2H+Etu0=; b=rgWWBdtHKwDllUR8IOgH2vQ1LPTqjtIkIogYaY3pFkhHm32Jl2wsCm8JpUJj32bqv7 OHfmfd/mKe6qAWUwDkuA/xW/Ija72/u1CGWXKYFhDkCoGp4DFDCreql99fAyNpQKPj5s ZzauF492NA9nBCkNrQ2FJyqGw1atIUeu/hVsCYBGdJj4Q/cud4meXs5Qp0GAQtgwE4dJ FZ9RR9lUXhv7kZimFc7zUFSsissXlEIyZNTpcR6U6fTSRxS5lAx5qoIBHaTi+Vlcd1PE ZEvIvdJBhs1cgoBBvbyNSTuS5BcXRkiNoJjf/V8GEyP6FBiyOD4RGqh+X7Bjh9Iz541K PW7g== X-Gm-Message-State: AOJu0Yy4OYyNny2ENBUyAfMgK+3l0Kd7fY+6GlJhuj7uVcCaxgoydn75 c0aJ1IK3/ywsZRlSQFP0oGo= X-Received: by 2002:a05:6a00:1390:b0:6cb:8abd:39b5 with SMTP id t16-20020a056a00139000b006cb8abd39b5mr2493633pfg.1.1700423304666; Sun, 19 Nov 2023 11:48:24 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.21 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:24 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 07/24] mm/swap: move swap_count to header to be shared Date: Mon, 20 Nov 2023 03:47:23 +0800 Message-ID: <20231119194740.94101-8-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:48:52 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023097659861020 X-GMAIL-MSGID: 1783023097659861020 From: Kairui Song No feature change, prepare for later commits. Signed-off-by: Kairui Song --- mm/swap.h | 5 +++++ mm/swapfile.c | 5 ----- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/swap.h b/mm/swap.h index f82d43d7b52a..a9a654af791e 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -61,6 +61,11 @@ static inline unsigned int folio_swap_flags(struct folio *folio) { return page_swap_info(&folio->page)->flags; } + +static inline unsigned char swap_count(unsigned char ent) +{ + return ent & ~SWAP_HAS_CACHE; /* may include COUNT_CONTINUED flag */ +} #else /* CONFIG_SWAP */ struct swap_iocb; static inline void swap_readpage(struct page *page, bool do_poll, diff --git a/mm/swapfile.c b/mm/swapfile.c index 0142bfc71b81..a8ae472ed2b6 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -114,11 +114,6 @@ static struct swap_info_struct *swap_type_to_swap_info(int type) return READ_ONCE(swap_info[type]); /* rcu_dereference() */ } -static inline unsigned char swap_count(unsigned char ent) -{ - return ent & ~SWAP_HAS_CACHE; /* may include COUNT_CONTINUED flag */ -} - /* Reclaim the swap entry anyway if possible */ #define TTRS_ANYWAY 0x1 /* From patchwork Sun Nov 19 19:47:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166868 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1809720vqn; Sun, 19 Nov 2023 11:48:59 -0800 (PST) X-Google-Smtp-Source: AGHT+IEv2hfmFM8aGa11SqBMOEDa9nPgQ+kCRJ+KeYpSnePpO8S3jzr//d8ddiQ03JNJLvoyyuwa X-Received: by 2002:a05:6a20:748e:b0:187:fe09:272a with SMTP id p14-20020a056a20748e00b00187fe09272amr7090581pzd.49.1700423339641; Sun, 19 Nov 2023 11:48:59 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423339; cv=none; d=google.com; s=arc-20160816; b=mPVyg5HrN00fQefbuJ9UWoluBEainjEdVuDlCuPjH33jgrxTRhxjrOHXcp8yWFfhoW vl06SUhPYbn+KjDWaJbvBpMByPQtRrfKFO3NFrUTXqhJNB2eRUbBiS8Bpw13nuJVF3SV JE8aURv2+QWT79dw7cFkr95KwqChu41oOFcVh3PF4Op1nzDFBZ+9y9vBN3bbUd8OPkWc Ng6xeipfcMWCh5Sb3FEQDLnkJKpyg8Md9jWRZ5eydcUaLmgKrvbB62TvxW8j8pV9MgZF Y2IEYimrp7HLOIHr3aFQO2mdEwtI0AdFCGA+Mk8ZgC1r8PmMEV3JYP0ZpFiRKh5W1OMv iEIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=/Iv3a0RkKO4tJSHfzH299Wh8nqjqW5sZox/cVI//TM8=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=PNbNEvFabfk0nsqZh6OlPON1O4aCMP051gE94VhPUmIZf0s1TdTNSN5CFCCqezgAJY QV3fUdV491GiyS1quXYhg9qGf9eEdffE9LCyj9apRD9tkvvzzrfDzkjXfMSUPx335qSp JpRt7aYgPmxJzHdaS3HPjXx2GB/g+qA1IwpyeiVfWJWfT0YJ34zUdT5N5HXuCG07F5zP xcwP+XBVTdLKESba1GbiJ7/MBdMqsg9wGfnDq2EmB3aLlh/RGLsn78seOIsXkSblNEX2 39v8tQMMtBuEcbWasSCUizDzyDWdVGTu9AEGDTBds/NKIvJkU48WEd/epScd8otYbpRK CWbQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=TfQ3+1Ev; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id w2-20020a6556c2000000b005c2423377a8si662789pgs.759.2023.11.19.11.48.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:48:59 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=TfQ3+1Ev; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id B6701804ACEB; Sun, 19 Nov 2023 11:48:58 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231695AbjKSTsz (ORCPT + 29 others); Sun, 19 Nov 2023 14:48:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34488 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231600AbjKSTsn (ORCPT ); Sun, 19 Nov 2023 14:48:43 -0500 Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 80435D45 for ; Sun, 19 Nov 2023 11:48:28 -0800 (PST) Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-6b2018a11efso3881298b3a.0 for ; Sun, 19 Nov 2023 11:48:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423308; x=1701028108; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=/Iv3a0RkKO4tJSHfzH299Wh8nqjqW5sZox/cVI//TM8=; b=TfQ3+1EvjgS8cNADoaSCxk1RMNz27Q2YIcqO8OiMF+oW9o0ikJ4nXGGF17AzeDFaa6 +rWmHBxTqxHuzPbIb1SfZtZUkixsBU9pPq1b59jlXNSlLaoKb9ISAZp/30pcLNazFK0h Mzo9AuYMrTf3wUQY1N7urp2LElyKDZfpORxu8Sa5TmHgaAexu3qwL8NjKu5wIcZ4jUvL TatZNoDWnmYQgjj5DvaKgj8pyELvBLE+3S8LnH+ekrOZ6b7Cgm52gSovQnFyZzuB2C2C Dt2o/aLSbXnT6Hzg9gsLIOGDnbCfQtoiOe5IOTIY6VP3SlpOeRwgXV03oqLctxfRmkbM zX4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423308; x=1701028108; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=/Iv3a0RkKO4tJSHfzH299Wh8nqjqW5sZox/cVI//TM8=; b=L0zmYDNp9ECwyGOlZsqUalZlHeFv/vjnAwAnza7DmLpXKGYBlphM4gfGujoie5/Tmk zWwQl988sdHCocDK1iGCXQetjnkEdocewxexKgek6ftduSVthZy7G5YhXVq8k/FbWPOi IbgOCMKNYpXxU7Wbn5oi5ocSmHuy1QHMoyjqeo/LBip/HHDLtDUhyD0FjFdKiu68Dq30 j+9EUQa0vjFmyMBJwcTDE2QIZMK+ulzaNAHYLBtv8Ee0BrTPc0t64I11j8n2BW4ygtq5 HUSu3BAcZ1R1cFPDzrHZQwgq5auoTlsrV+jtEsrj35RLhQ/fOmu+JFIoOETFPskxJ64F lbTg== X-Gm-Message-State: AOJu0YzKjU+vKJ/uT5mvPnqbcEpxQy7zsjcJ+R1VqK7SKwqh34czLLlx 7/o3qmrVKV39/Vi8YU0F/3U= X-Received: by 2002:a05:6a20:3d84:b0:17b:426f:829 with SMTP id s4-20020a056a203d8400b0017b426f0829mr7488318pzi.37.1700423307883; Sun, 19 Nov 2023 11:48:27 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.25 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:27 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 08/24] mm/swap: check readahead policy per entry Date: Mon, 20 Nov 2023 03:47:24 +0800 Message-ID: <20231119194740.94101-9-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:48:58 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023103877767219 X-GMAIL-MSGID: 1783023103877767219 From: Kairui Song Currently VMA readahead is globally disabled when any rotate disk is used as swap backend. So multiple swap devices are enabled, if a slower hard disk is set as a low priority fallback, and a high performance SSD is used and high priority swap device, vma readahead is disabled globally. The SSD swap device performance will drop by a lot. Check readahead policy per entry to avoid such problem. Signed-off-by: Kairui Song --- mm/swap_state.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mm/swap_state.c b/mm/swap_state.c index ff6756f2e8e4..fb78f7f18ed7 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -321,9 +321,9 @@ static inline bool swap_use_no_readahead(struct swap_info_struct *si, swp_entry_ return data_race(si->flags & SWP_SYNCHRONOUS_IO) && __swap_count(entry) == 1; } -static inline bool swap_use_vma_readahead(void) +static inline bool swap_use_vma_readahead(struct swap_info_struct *si) { - return READ_ONCE(enable_vma_readahead) && !atomic_read(&nr_rotate_swap); + return data_race(si->flags & SWP_SOLIDSTATE) && READ_ONCE(enable_vma_readahead); } /* @@ -341,7 +341,7 @@ struct folio *swap_cache_get_folio(swp_entry_t entry, folio = filemap_get_folio(swap_address_space(entry), swp_offset(entry)); if (!IS_ERR(folio)) { - bool vma_ra = swap_use_vma_readahead(); + bool vma_ra = swap_use_vma_readahead(swp_swap_info(entry)); bool readahead; /* @@ -920,16 +920,18 @@ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, struct vm_fault *vmf, bool *swapcached) { + struct swap_info_struct *si; struct mempolicy *mpol; struct page *page; pgoff_t ilx; bool cached; + si = swp_swap_info(entry); mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); - if (swap_use_no_readahead(swp_swap_info(entry), entry)) { + if (swap_use_no_readahead(si, entry)) { page = swapin_no_readahead(entry, gfp_mask, mpol, ilx, vmf->vma->vm_mm); cached = false; - } else if (swap_use_vma_readahead()) { + } else if (swap_use_vma_readahead(si)) { page = swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf); cached = true; } else { From patchwork Sun Nov 19 19:47:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166869 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1809751vqn; Sun, 19 Nov 2023 11:49:05 -0800 (PST) X-Google-Smtp-Source: AGHT+IE7eDafCHrlFxVA6n79VMmALu7HCTaOtslgk/PRNGHvyzdBYUIgp1j1kRObRbecAwwpOv/a X-Received: by 2002:a17:902:e546:b0:1cc:6ec2:d24e with SMTP id n6-20020a170902e54600b001cc6ec2d24emr7828931plf.53.1700423344793; Sun, 19 Nov 2023 11:49:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423344; cv=none; d=google.com; s=arc-20160816; b=tCMgyo1jMtIBFkl+kn0TBb/ho4F5Pom6wiBxRqgDMXlRyFIabp6h3dpmd4qpTZEaeJ egbgAH9Zqv92pezK+03SfykG/O5cHWnfFvnRZk71p5BYhLHPSoX3dYz6ZDmanyr20NLm r4ctgk74Vp4Ay9v1UAdMJuPCpVYQN2VHee+qrewEB3wSjwWVdzUpRko+z3ORJFJL+uDf zdcKGWFsqn2dvOpTz7J8TTX1VuaJxgi97DGGyPBqr8Zz6uEdqq+qqdWrZKksIjkm8AM3 4mEQ2dzdIGx50abqtsqU8WN4IlMANo31CQvmzoXlyJ1cg2X+zPjK3MHKw3JIDxY9h+4y 8KmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=7ikH9nPhVAiKzaSXcyZub9ryWf7e3smOr0BN4n2CRJU=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=Zy50QMbQ47ZRcT8SUuh5zb0Sct1AwBHI3TSi0ONqoSwHMaQ/eUiT1SJVYdoCV8WX6A NAvtK+BUTMHr+1cttK8IqoFzV3Bxdi4dYWgd6H7cns8r/nCuGIHGFgm8fBjAeut0vc4V uEArekxUnl15z+gd0dt8WsBMohORbZlxOkU9LIKowtkvswQspZuDj7gap+6WDvhRgnQ2 ZhxqoTn3pi4dBnoh0T10b6NNTCd+HHXUK0k6RSJ51W535f7JArDP7NMx5SbWToiTmRFi OoMBiaZfTQoF6EGMSzBy1X4/H0UkgF1FskZodIK0nhlWu+sAXW0laSoE0wtx+dgxeKus UGTA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b="Qs3l1A/9"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id p3-20020a17090ab90300b0028511652328si3230128pjr.137.2023.11.19.11.49.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:49:04 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b="Qs3l1A/9"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id F3E0980B01C0; Sun, 19 Nov 2023 11:49:03 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231574AbjKSTs5 (ORCPT + 29 others); Sun, 19 Nov 2023 14:48:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36464 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231614AbjKSTsq (ORCPT ); Sun, 19 Nov 2023 14:48:46 -0500 Received: from mail-pg1-x536.google.com (mail-pg1-x536.google.com [IPv6:2607:f8b0:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C8BEBD4F for ; Sun, 19 Nov 2023 11:48:31 -0800 (PST) Received: by mail-pg1-x536.google.com with SMTP id 41be03b00d2f7-517ab9a4a13so2840780a12.1 for ; Sun, 19 Nov 2023 11:48:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423311; x=1701028111; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=7ikH9nPhVAiKzaSXcyZub9ryWf7e3smOr0BN4n2CRJU=; b=Qs3l1A/9czxQrrdc1WwtlfYX4eberyGOUowS2hKfGCiHaDX6SpF+9XAVYMsAvTs/eU Pd3l5wy9As8VQTpSIPuZdkKWG9+yqXZTpceiKeVQ9sjh55U5v3rN2Tbra7DdMreB/qTR IiwAa9BgqJxCNY181WuTdRe/kguYEsgBO/x8rorQ2z/mPQxQPqn4eAn2sELpMyhZaZ6k kdR+9ZPCcnsDL0gGRCxcTQ6s0EKe/+KZwqhqPaIK3fRzphTCTBpV5yRtqNsIbToe5/C9 KBT+IyMs7LZBA+KyWtMXLMwj/pWltTgRzM+cOP4mmVmPuCOkFB95skafO/E+GPoZ2Q1v IH1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423311; x=1701028111; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=7ikH9nPhVAiKzaSXcyZub9ryWf7e3smOr0BN4n2CRJU=; b=IyhZVIjjmFIGJk+0rKEOo4vSwX5WxhwYS7TNtiIavRfNkAg585rcVl77FN6/Bc2VHt JaTe7LXM6QtA1flosLnMBd6OuYEp8U/ZJnfPEGQB3ZTslKz2ngX/iotB/dZfEXLGz8EU 71/wfpH187IHm3fVcO2cF/G/fVBGHnm6EHOZ8IWeoVUuUfVBeQ7LfZAo8kqJac2nDE+g 8PzVJ3uHTrRHhbmyyJmsouuwMUthJULYGHOoUA2C0zGNm0yi3YMyaQ40cxC3xOI/kran i8UBxjnatQPWJv6/izHSh0l8lVpY50AsB5282NSEyNUnsq0yUYFX9iO15IxKGJatVaA2 D8Ng== X-Gm-Message-State: AOJu0YwAdpKS9Lpn1DLn7ROlCJt4vQNnMG34rc5MSMGULL6kpQ9wQC8T bDsT+wMx37yLHcfZrrYGZm8= X-Received: by 2002:a05:6a20:552a:b0:188:1125:88bd with SMTP id ko42-20020a056a20552a00b00188112588bdmr6787703pzb.43.1700423311039; Sun, 19 Nov 2023 11:48:31 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.28 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:30 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 09/24] mm/swap: inline __swap_count Date: Mon, 20 Nov 2023 03:47:25 +0800 Message-ID: <20231119194740.94101-10-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:49:04 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023109082264354 X-GMAIL-MSGID: 1783023109082264354 From: Kairui Song There is only one caller in swap subsystem now, where it can be inline smoothly, avoid the memory access and function call overheads. Signed-off-by: Kairui Song --- include/linux/swap.h | 6 ------ mm/swap_state.c | 6 +++--- mm/swapfile.c | 8 -------- 3 files changed, 3 insertions(+), 17 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 2401990d954d..64a37819a9b3 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -485,7 +485,6 @@ int swap_type_of(dev_t device, sector_t offset); int find_first_swap(dev_t *device); extern unsigned int count_swap_pages(int, int); extern sector_t swapdev_block(int, pgoff_t); -extern int __swap_count(swp_entry_t entry); extern int swap_swapcount(struct swap_info_struct *si, swp_entry_t entry); extern int swp_swapcount(swp_entry_t entry); extern struct swap_info_struct *page_swap_info(struct page *); @@ -559,11 +558,6 @@ static inline void put_swap_folio(struct folio *folio, swp_entry_t swp) { } -static inline int __swap_count(swp_entry_t entry) -{ - return 0; -} - static inline int swap_swapcount(struct swap_info_struct *si, swp_entry_t entry) { return 0; diff --git a/mm/swap_state.c b/mm/swap_state.c index fb78f7f18ed7..d87c20f9f7ec 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -316,9 +316,9 @@ void free_pages_and_swap_cache(struct encoded_page **pages, int nr) release_pages(pages, nr); } -static inline bool swap_use_no_readahead(struct swap_info_struct *si, swp_entry_t entry) +static inline bool swap_use_no_readahead(struct swap_info_struct *si, pgoff_t offset) { - return data_race(si->flags & SWP_SYNCHRONOUS_IO) && __swap_count(entry) == 1; + return data_race(si->flags & SWP_SYNCHRONOUS_IO) && swap_count(si->swap_map[offset]) == 1; } static inline bool swap_use_vma_readahead(struct swap_info_struct *si) @@ -928,7 +928,7 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, si = swp_swap_info(entry); mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); - if (swap_use_no_readahead(si, entry)) { + if (swap_use_no_readahead(si, swp_offset(entry))) { page = swapin_no_readahead(entry, gfp_mask, mpol, ilx, vmf->vma->vm_mm); cached = false; } else if (swap_use_vma_readahead(si)) { diff --git a/mm/swapfile.c b/mm/swapfile.c index a8ae472ed2b6..e15a6c464a38 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1431,14 +1431,6 @@ void swapcache_free_entries(swp_entry_t *entries, int n) spin_unlock(&p->lock); } -int __swap_count(swp_entry_t entry) -{ - struct swap_info_struct *si = swp_swap_info(entry); - pgoff_t offset = swp_offset(entry); - - return swap_count(si->swap_map[offset]); -} - /* * How many references to @entry are currently swapped out? * This does not give an exact answer when swap count is continued, From patchwork Sun Nov 19 19:47:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166871 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1810087vqn; Sun, 19 Nov 2023 11:50:09 -0800 (PST) X-Google-Smtp-Source: AGHT+IF8n0giWbOq9DFF2ozbHMFayZK/joyXhTBBJ44tAtN6liuMRId/X4hb2bM04UqKq5JpJyQj X-Received: by 2002:a17:902:c941:b0:1cc:5ce4:f64b with SMTP id i1-20020a170902c94100b001cc5ce4f64bmr7810902pla.8.1700423409519; Sun, 19 Nov 2023 11:50:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423409; cv=none; d=google.com; s=arc-20160816; b=ZbZmuyJKujsC2M422jgKDQVwGxaqXCh4DqWughzx84W6OabShMopPBw7WfNlg3d+4+ +7pvf2Qwfjl3FlkXaFaUxC7ojtIYEdry1eEPBhw/kGeEFyQrQw37Ymun5Vspfr7z1XlU WrxgsCNA27T9GASdR51p3bmR7+18fSYnLLUf25DB6yeQK8gcbEl5Ud25N8nQYLIYkMmg 0sw2GdBstdG59IG22SjxMUTWeSJ2mYUdttgAzfAh7r0aO+JOtyv/Bz8sdG8TFRG0Znlk rLXDcLIRfxr+pF1ViE3HbSt1P04xngRjTT6Z1+JqLzlC3U21F72JvVPoaRIKGwsZ1G2c /WDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=C/w+GrPz4ktaqTeTQ4DmVKbN9IqMk3W+D3u9IVSxuMQ=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=get6FQJNCwvgoQCQqiCUa+kafTTjLEekUfWcdM3B9YLb8bjBWcfKwG1Kkgyey+w2Kt RJqGYFFMw4ZViJnHkoeaQDzCT99udLjgi0v986nHBb/Trd2T1tIDM+5+yRFgTaSqZBzm kqOb0/1E+5KsYQ22Le+2Rgt7uNQZmhj8DOwT9T6g9aq/tqs0vCVKE2cA7Sp+xc7/GJG2 cIoY/4svm1hB+R/yj9jgqodwsixfQoDV4cadhRDi0BjbVx15A6uJE8PInYg7CNXJlVku eXAdBzZUJzDuFicYCTd+rdBR4zTARTkrgSySInZCV5SfcvqQtcWFUAK/1hf9VxrqoR2d C7Rg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=hWeuClbw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id ju5-20020a170903428500b001cc76bea519si6415071plb.87.2023.11.19.11.50.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:50:09 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=hWeuClbw; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 7D2198057996; Sun, 19 Nov 2023 11:49:56 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231814AbjKSTtL (ORCPT + 29 others); Sun, 19 Nov 2023 14:49:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36604 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231670AbjKSTsx (ORCPT ); Sun, 19 Nov 2023 14:48:53 -0500 Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EC189D7A for ; Sun, 19 Nov 2023 11:48:34 -0800 (PST) Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-6cb7951d713so352280b3a.1 for ; Sun, 19 Nov 2023 11:48:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423314; x=1701028114; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=C/w+GrPz4ktaqTeTQ4DmVKbN9IqMk3W+D3u9IVSxuMQ=; b=hWeuClbwt9dh9tON8JEyhbawHTrAbClV9oMYSRZp+VU1ZqxrXjtOSlT5yTqQ2SwFr1 bFG9WkEqFygF8GH4CsSzOhyaq5QiUjeUN9w6BKZs8ne0PaC8LxqNd1d6FLPJa5MnUEPX SucgF4XXsth99wKI0POSRLrdtijBlSgrqPvjvBlPgg7ZWy/L+ZDWDYFxYbYtZ0WX3r9o mDS4IosmsloEDJu7no+e+KllNv6tR6c9p/+zcABzir6eL5XcH5BoKz4ngTYdmt8wXvQq rIiWHerw3FOjOrdZq9A/sk1/OpwVCgo3g3RsRtEZxN+NQ7eXyGUT3mx45HF+FWKL0x5o abfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423314; x=1701028114; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=C/w+GrPz4ktaqTeTQ4DmVKbN9IqMk3W+D3u9IVSxuMQ=; b=U0JYih8qxa8GHrkBp8HIf1dn4UH+tUajW6JwYTUtXBqKtzS85GFcio0kxGrlnGgoJC YpLE5pRL2wzPczaC+TZfAtiUV0GrICS7uyO4nirBPaPqBwTmtqIrRWDUvvUGBHU4+vi5 P3Fo+jcX0q3uGaOyben5T00KSEITXYApPuPpb9m0P4Xm4LlinNUy4xAb4GyUNeu4Fbcc fb2l48Da/H9zj8jlRS5wh4v8b0DDuPAD68Wo57h1t276aOXA1etAz0G2E7/uNVUvRUtm QX8TJKm8bcfZKrV1lBO0YbEi4Knhsd9xu4k/0Aerlk0fWrdpmdP3MI/yXQk3kZjgspeT 4qTg== X-Gm-Message-State: AOJu0YwzhEngVfgAyDJAbUxtRXijO2ErFEkjRRQTk1oRce7TMSSahzVq kCviY83GJsS11m0f3WsxQow= X-Received: by 2002:a05:6a00:2e99:b0:6c9:88ee:1d87 with SMTP id fd25-20020a056a002e9900b006c988ee1d87mr19403560pfb.17.1700423314274; Sun, 19 Nov 2023 11:48:34 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.31 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:33 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 10/24] mm/swap: remove nr_rotate_swap and related code Date: Mon, 20 Nov 2023 03:47:26 +0800 Message-ID: <20231119194740.94101-11-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:49:56 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023177175090080 X-GMAIL-MSGID: 1783023177175090080 From: Kairui Song No longer needed after we switched to per entry swap readhead policy. Signed-off-by: Kairui Song --- include/linux/swap.h | 1 - mm/swapfile.c | 11 ----------- 2 files changed, 12 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 64a37819a9b3..cc83fb884757 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -454,7 +454,6 @@ extern void free_pages_and_swap_cache(struct encoded_page **, int); /* linux/mm/swapfile.c */ extern atomic_long_t nr_swap_pages; extern long total_swap_pages; -extern atomic_t nr_rotate_swap; extern bool has_usable_swap(void); /* Swap 50% full? Release swapcache more aggressively.. */ diff --git a/mm/swapfile.c b/mm/swapfile.c index e15a6c464a38..01c3f53b6521 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -104,8 +104,6 @@ static DECLARE_WAIT_QUEUE_HEAD(proc_poll_wait); /* Activity counter to indicate that a swapon or swapoff has occurred */ static atomic_t proc_poll_event = ATOMIC_INIT(0); -atomic_t nr_rotate_swap = ATOMIC_INIT(0); - static struct swap_info_struct *swap_type_to_swap_info(int type) { if (type >= MAX_SWAPFILES) @@ -2486,9 +2484,6 @@ SYSCALL_DEFINE1(swapoff, const char __user *, specialfile) if (p->flags & SWP_CONTINUED) free_swap_count_continuations(p); - if (!p->bdev || !bdev_nonrot(p->bdev)) - atomic_dec(&nr_rotate_swap); - mutex_lock(&swapon_mutex); spin_lock(&swap_lock); spin_lock(&p->lock); @@ -2990,7 +2985,6 @@ SYSCALL_DEFINE2(swapon, const char __user *, specialfile, int, swap_flags) struct swap_cluster_info *cluster_info = NULL; struct page *page = NULL; struct inode *inode = NULL; - bool inced_nr_rotate_swap = false; if (swap_flags & ~SWAP_FLAGS_VALID) return -EINVAL; @@ -3112,9 +3106,6 @@ SYSCALL_DEFINE2(swapon, const char __user *, specialfile, int, swap_flags) cluster = per_cpu_ptr(p->percpu_cluster, cpu); cluster_set_null(&cluster->index); } - } else { - atomic_inc(&nr_rotate_swap); - inced_nr_rotate_swap = true; } error = swap_cgroup_swapon(p->type, maxpages); @@ -3218,8 +3209,6 @@ SYSCALL_DEFINE2(swapon, const char __user *, specialfile, int, swap_flags) spin_unlock(&swap_lock); vfree(swap_map); kvfree(cluster_info); - if (inced_nr_rotate_swap) - atomic_dec(&nr_rotate_swap); if (swap_file) filp_close(swap_file, NULL); out: From patchwork Sun Nov 19 19:47:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166877 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1810314vqn; Sun, 19 Nov 2023 11:50:51 -0800 (PST) X-Google-Smtp-Source: AGHT+IGWA66fU8lg+qd6NXR6rSktp5GNmdGwzNIkAOHo2/JQmw/dHs18jJXpRQiPoQXdJkfZd89b X-Received: by 2002:a17:90b:1c83:b0:280:1689:4aac with SMTP id oo3-20020a17090b1c8300b0028016894aacmr3502471pjb.2.1700423451592; Sun, 19 Nov 2023 11:50:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423451; cv=none; d=google.com; s=arc-20160816; b=o2XY+m2LG9gH4aszXMcq1Ia2GTtjxBwDt8Vx1AaUHi/vukZTCN8bmKox4ANmjl4+Qq pEwpO6w6mte0FoilKPJWP0X5xZEM9f0IGLZR7Cbk0Dm8OVBUD1jLva2mhQe/w/8gsaYi aboIO91RdtWmA2pxsVPYGuWtelaYdnurgvoa5k3e65y+hL2UoOil9pszYXN4C0Kcinkn A2zWdU+hWW5/WYePP2R4az05dqj1G2VjlZWGMwXI0BI9NsoCZdXoLndFBFPF6EsVAuHm 0qTfaZR7oJqAb0Ii/yT02gRxiypHYqXPxrxQwkrscZPjLPBN6PuBxZU7lXf12oUNkwPj l8Sw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Vy77DasHH4iZw3fxXV06AQqlBWBp94UgYk7CDc75jBU=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=cU+EhI9ZjylceCSrSHfRvInWubq05E1ahWlRdoJTISOcytj6nOMwRSSnM+ii0MTdA0 yGZcApK6cgGfjLVQAr1UH7sXqrHCOCW4SXNlhhR4KTuT2Xk/HHtX8DJEvas+U7OqGOtG 5hTOnB23pNpj5/03u/r8n34Qy1NoPPfo2E+RIFePaFgBNds6UA5o0f/v6hshfEJB6Vdm 4yfySgG/vZMPfKqspX3Wv+mp/2u3vzXTMBms1T06o/hN/CknN9jMVDBkcXyZ2spE5S2A MhdZ5SlsZfB1Rv0ZllU0MCm5WFGUNDrSQdemgmkP2W8+6MVoHdoZUF3FtNv7+9A6ECdT fdew== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b="jmI9Qr/Y"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from fry.vger.email (fry.vger.email. [23.128.96.38]) by mx.google.com with ESMTPS id h9-20020a17090acf0900b002849048f576si4109248pju.148.2023.11.19.11.50.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:50:51 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b="jmI9Qr/Y"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id B950280785DB; Sun, 19 Nov 2023 11:50:26 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231750AbjKSTtT (ORCPT + 29 others); Sun, 19 Nov 2023 14:49:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56170 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231728AbjKSTs6 (ORCPT ); Sun, 19 Nov 2023 14:48:58 -0500 Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2EBCBD7F for ; Sun, 19 Nov 2023 11:48:38 -0800 (PST) Received: by mail-io1-xd32.google.com with SMTP id ca18e2360f4ac-7a93b7fedc8so175626939f.1 for ; Sun, 19 Nov 2023 11:48:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423317; x=1701028117; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=Vy77DasHH4iZw3fxXV06AQqlBWBp94UgYk7CDc75jBU=; b=jmI9Qr/YCZXj/KwbpX3E7Cms6SpwqzungcU2z0+6qyeldu6v/wBxEalqyKFDN5PYIk FFmB3z+erBwNko1TXY+0rHl277kJYbHPxn0GlsnkhBkxj93F5i60QZJNTy47Fb7kU+zR 0UyLC9wlR8pHX8o7kfEpGC7KKrwSLneu4PMriwcWZSSOd30Ib5MXo7kaCbyKDEaOsP3u x8jIf2arU/JqbxIYsAzCEW6c1paImgaHXM/itra9VtgHq4rt2Zcs9KLFqbiDdqafT/1H o+oi6hlqhGxPH7tCXBkRpu+HN5q1se2JOa/WCd9ZKs6XWa4xH54ujElKbKZ6nqG8TIDw DUKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423317; x=1701028117; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Vy77DasHH4iZw3fxXV06AQqlBWBp94UgYk7CDc75jBU=; b=IqBH1MemFrg7sB8fpU0GgP1DFuUjwu69KfuXM0NEHUNGVCj2qWX/PtJBACYfH3bEB8 VBD2Oeuhf+19kMWLONrUKDajP1AW0VCzzVgFgpBJO8uBaIchKIdL6oxttoOty2nMqdVm 6Pop5HAQABvbq2jm9j3e38u1VvSFX/wi7ofM762EI1o3CgE/KJSSYV4jckpORSMo4955 oGcuTbsGPfzKutEpBStS2UdBum6pb9E0rcsVZc1gGRCk4wrcBPrs8xOsOYwBA58JZ+Qc L2mN9UprzOplDSEMgvx8T9bYTlvrHu+EI5mpNXCuHd+aZ9+LYLlbkokNDOH9inRht8rv /91w== X-Gm-Message-State: AOJu0YzOc544/ZnCCQrsjI2ErhGhKT7+0pi+9tkLs6wbFfE8o/ern/5o n/ilic4q9GOZaGQhTG0zfto= X-Received: by 2002:a92:cbc2:0:b0:35a:f493:5667 with SMTP id s2-20020a92cbc2000000b0035af4935667mr5462158ilq.20.1700423317470; Sun, 19 Nov 2023 11:48:37 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.34 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:36 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 11/24] mm/swap: also handle swapcache lookup in swapin_readahead Date: Mon, 20 Nov 2023 03:47:27 +0800 Message-ID: <20231119194740.94101-12-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:50:26 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023220999876282 X-GMAIL-MSGID: 1783023220999876282 From: Kairui Song No feature change, just prepare for later commits. Signed-off-by: Kairui Song --- mm/memory.c | 61 +++++++++++++++++++++++-------------------------- mm/swap.h | 10 ++++++-- mm/swap_state.c | 26 +++++++++++++-------- mm/swapfile.c | 30 +++++++++++------------- 4 files changed, 66 insertions(+), 61 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index f4237a2e3b93..22af9f3e8c75 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3786,13 +3786,13 @@ static vm_fault_t handle_pte_marker(struct vm_fault *vmf) vm_fault_t do_swap_page(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; - struct folio *swapcache, *folio = NULL; + struct folio *swapcache = NULL, *folio = NULL; + enum swap_cache_result cache_result; struct page *page; struct swap_info_struct *si = NULL; rmap_t rmap_flags = RMAP_NONE; bool exclusive = false; swp_entry_t entry; - bool swapcached; pte_t pte; vm_fault_t ret = 0; @@ -3850,42 +3850,37 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) if (unlikely(!si)) goto out; - folio = swap_cache_get_folio(entry, vma, vmf->address); - if (folio) - page = folio_file_page(folio, swp_offset(entry)); - swapcache = folio; - - if (!folio) { - page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, - vmf, &swapcached); - if (page) { - folio = page_folio(page); - if (swapcached) - swapcache = folio; - } else { + page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, + vmf, &cache_result); + if (page) { + folio = page_folio(page); + if (cache_result != SWAP_CACHE_HIT) { + /* Had to read the page from swap area: Major fault */ + ret = VM_FAULT_MAJOR; + count_vm_event(PGMAJFAULT); + count_memcg_event_mm(vma->vm_mm, PGMAJFAULT); + } + if (cache_result != SWAP_CACHE_BYPASS) + swapcache = folio; + if (PageHWPoison(page)) { /* - * Back out if somebody else faulted in this pte - * while we released the pte lock. + * hwpoisoned dirty swapcache pages are kept for killing + * owner processes (which may be unknown at hwpoison time) */ - vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd, - vmf->address, &vmf->ptl); - if (likely(vmf->pte && - pte_same(ptep_get(vmf->pte), vmf->orig_pte))) - ret = VM_FAULT_OOM; - goto unlock; + ret = VM_FAULT_HWPOISON; + goto out_release; } - - /* Had to read the page from swap area: Major fault */ - ret = VM_FAULT_MAJOR; - count_vm_event(PGMAJFAULT); - count_memcg_event_mm(vma->vm_mm, PGMAJFAULT); - } else if (PageHWPoison(page)) { + } else { /* - * hwpoisoned dirty swapcache pages are kept for killing - * owner processes (which may be unknown at hwpoison time) + * Back out if somebody else faulted in this pte + * while we released the pte lock. */ - ret = VM_FAULT_HWPOISON; - goto out_release; + vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd, + vmf->address, &vmf->ptl); + if (likely(vmf->pte && + pte_same(ptep_get(vmf->pte), vmf->orig_pte))) + ret = VM_FAULT_OOM; + goto unlock; } ret |= folio_lock_or_retry(folio, vmf); diff --git a/mm/swap.h b/mm/swap.h index a9a654af791e..ac9136eee690 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -30,6 +30,12 @@ extern struct address_space *swapper_spaces[]; (&swapper_spaces[swp_type(entry)][swp_offset(entry) \ >> SWAP_ADDRESS_SPACE_SHIFT]) +enum swap_cache_result { + SWAP_CACHE_HIT, + SWAP_CACHE_MISS, + SWAP_CACHE_BYPASS, +}; + void show_swap_cache_info(void); bool add_to_swap(struct folio *folio); void *get_shadow_from_swap_cache(swp_entry_t entry); @@ -55,7 +61,7 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t flag, struct mempolicy *mpol, pgoff_t ilx); struct page *swapin_readahead(swp_entry_t entry, gfp_t flag, - struct vm_fault *vmf, bool *swapcached); + struct vm_fault *vmf, enum swap_cache_result *result); static inline unsigned int folio_swap_flags(struct folio *folio) { @@ -92,7 +98,7 @@ static inline struct page *swap_cluster_readahead(swp_entry_t entry, } static inline struct page *swapin_readahead(swp_entry_t swp, gfp_t gfp_mask, - struct vm_fault *vmf, bool *swapcached) + struct vm_fault *vmf, enum swap_cache_result *result) { return NULL; } diff --git a/mm/swap_state.c b/mm/swap_state.c index d87c20f9f7ec..e96d63bf8a22 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -908,8 +908,7 @@ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, * @entry: swap entry of this memory * @gfp_mask: memory allocation flags * @vmf: fault information - * @swapcached: pointer to a bool used as indicator if the - * page is swapped in through swapcache. + * @result: a return value to indicate swap cache usage. * * Returns the struct page for entry and addr, after queueing swapin. * @@ -918,30 +917,39 @@ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, * or vma-based(ie, virtual address based on faulty address) readahead. */ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, - struct vm_fault *vmf, bool *swapcached) + struct vm_fault *vmf, enum swap_cache_result *result) { + enum swap_cache_result cache_result; struct swap_info_struct *si; struct mempolicy *mpol; + struct folio *folio; struct page *page; pgoff_t ilx; - bool cached; + + folio = swap_cache_get_folio(entry, vmf->vma, vmf->address); + if (folio) { + page = folio_file_page(folio, swp_offset(entry)); + cache_result = SWAP_CACHE_HIT; + goto done; + } si = swp_swap_info(entry); mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); if (swap_use_no_readahead(si, swp_offset(entry))) { page = swapin_no_readahead(entry, gfp_mask, mpol, ilx, vmf->vma->vm_mm); - cached = false; + cache_result = SWAP_CACHE_BYPASS; } else if (swap_use_vma_readahead(si)) { page = swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf); - cached = true; + cache_result = SWAP_CACHE_MISS; } else { page = swap_cluster_readahead(entry, gfp_mask, mpol, ilx); - cached = true; + cache_result = SWAP_CACHE_MISS; } mpol_cond_put(mpol); - if (swapcached) - *swapcached = cached; +done: + if (result) + *result = cache_result; return page; } diff --git a/mm/swapfile.c b/mm/swapfile.c index 01c3f53b6521..b6d57fff5e21 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1822,13 +1822,21 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, si = swap_info[type]; do { - struct folio *folio; + struct page *page; unsigned long offset; unsigned char swp_count; + struct folio *folio = NULL; swp_entry_t entry; int ret; pte_t ptent; + struct vm_fault vmf = { + .vma = vma, + .address = addr, + .real_address = addr, + .pmd = pmd, + }; + if (!pte++) { pte = pte_offset_map(pmd, addr); if (!pte) @@ -1847,22 +1855,10 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, offset = swp_offset(entry); pte_unmap(pte); pte = NULL; - - folio = swap_cache_get_folio(entry, vma, addr); - if (!folio) { - struct page *page; - struct vm_fault vmf = { - .vma = vma, - .address = addr, - .real_address = addr, - .pmd = pmd, - }; - - page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, - &vmf, NULL); - if (page) - folio = page_folio(page); - } + page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, + &vmf, NULL); + if (page) + folio = page_folio(page); if (!folio) { /* * The entry could have been freed, and will not From patchwork Sun Nov 19 19:47:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166882 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1810544vqn; Sun, 19 Nov 2023 11:51:46 -0800 (PST) X-Google-Smtp-Source: AGHT+IGUGfWlPqWTWXXuzEDCTdawiupNIdmasmfJh77gSAM6nLgq3lD/s11QgMHmvqoUjb/oEGEB X-Received: by 2002:a05:6a00:23d5:b0:6c4:d1cd:c5da with SMTP id g21-20020a056a0023d500b006c4d1cdc5damr6805170pfc.18.1700423506183; Sun, 19 Nov 2023 11:51:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423506; cv=none; d=google.com; s=arc-20160816; b=Xnbp9l0olnAOJOObgeuYchqmXqk2mUiY5z3OA4EfX8T3SFbRzSxvIG6ajrdjGWXFs3 /sBLJRUyabTVH6n+kqcfi4xT9pbMrgX1nGnnZoBh+1WJvRRELxuih/g3aJTKuv7MRFj7 Ygft5oKhmCRib3pfVwFiaNJ5IboUqZ1t7P9PBWwW9cWgONMWv/jYM9X0dZo9JlG4E132 Lnsg64mdq2RiPh6EKSTGRh7HgeZuhnnT9Pue9LAuRbjnVOUAHnK7G8PYEmczNC0btW0j BF4wYUf/1EkWrVxfE7KSyd5cDsG6j4cIoaHrdZ3lrv7OTF6SI0LElPexSD20SRj65A+s olgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ppA1yq5tSXPW7Cplga4R0sTrIzgEEoc6Te1dihtlR90=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=vKq1MkgPx0Gc21CvSnuYaeIuACpp3exYUpPlHYvXFrJWg905UZ42AhraGOQlJ6cDlR +3KHtHyKmnvV6gZNFCieG0t0UtKPjMkhHNrSQOOYxaVlev0io+OXA/6iyUR6ZtdksPnI 0Jeeiqfo/S0nft8/KQqbmmKPiMnm0VWdKf60iGg3VGaHuZP/w2O91XxzzuRekgG33Vkl 0O1zMVJOfPHm6ck9Wqj74E9Xt5AcGHJ6OnKMuZzlXQW0xKvz9QdQkwe4PzW2GGh7Ix0N pkDZ8HFR6b/jgt5C0rTrovcEj8IToxzWlbNB5wSK+/Io542BMxi4zYDu/qJVgA2AVaax 4Dpw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=WSg1wBRp; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from groat.vger.email (groat.vger.email. [23.128.96.35]) by mx.google.com with ESMTPS id s18-20020a056a0008d200b0068e3f55075esi6727568pfu.134.2023.11.19.11.51.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:51:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=WSg1wBRp; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 55EA68080E0D; Sun, 19 Nov 2023 11:51:37 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231593AbjKSTtV (ORCPT + 29 others); Sun, 19 Nov 2023 14:49:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34266 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231644AbjKSTtB (ORCPT ); Sun, 19 Nov 2023 14:49:01 -0500 Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 32D16D6E for ; Sun, 19 Nov 2023 11:48:41 -0800 (PST) Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-6c4cf0aea06so3616039b3a.0 for ; Sun, 19 Nov 2023 11:48:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423320; x=1701028120; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=ppA1yq5tSXPW7Cplga4R0sTrIzgEEoc6Te1dihtlR90=; b=WSg1wBRpJmuTftOP9HXoBZo5KW4YFAwao7tV1HOTnZUWYybr4Vw5Vtm8iGjBbLHDt2 rUKfm49N5LoVnKrQIy5c0pHG3t6wT0gAj7cuP6A6ADVwvDxGXFrj31z5JFGjVIg5U5yo lB6veVIkWScSgwLVHu02G+rK23A3ex5VL95WJ/x9lhj0CJPwp5p6/R6bmTNYvAPsdOki lMK/ho/QlpYtFnndd6ATcsy2Ex/Hprfx77X9Y0zbWKFrWYKJKkSsElGJdJq+yfn+47zO TmGadhtt3i17FSs4vnDZilPH4Dc8nQkdCGU/sJZt3ukcEcnC7rkUckv/PQpbFYYhVcxz OTmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423320; x=1701028120; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ppA1yq5tSXPW7Cplga4R0sTrIzgEEoc6Te1dihtlR90=; b=USGLHsPSwhnMNI+myGSlSaTSIYB6Vr2iYkUfedA0UciFpWaoX8AbbbDLeyuuVlu6rp lmdRy6NfKpmDsZHGMc70MBb+UEJLmzRnS+iOjBkGQQavmnY9BxXPON06YC8Q7fdOx037 N2ltz6APhtLpqBk9O8tdEFxjui08584qbIY5RoJY6EmVA7zY0T2OGBa9dBO7wT1G1G6c W2B/s1w35CTERmUhdEEqpq5WqNVv4svI6MtedxzSSglXbM5Nz/UJU+9leOBlFc0PFhaC 0GHGIKDQ6++RhAYMBvU58v+T8bvmAibgixhJu8Mx9v51hy7DQM6Zfoa9ADoqNVyNl1dk gfQw== X-Gm-Message-State: AOJu0Yzj+pwdJRlgqOTS36CH6eTqT4UilA050mB6wrb2aAaXciXwXCDZ uijzgzsFF1dVn1kZcJ8XQQ0= X-Received: by 2002:a05:6a00:1ca9:b0:6cb:a434:b58f with SMTP id y41-20020a056a001ca900b006cba434b58fmr619405pfw.33.1700423320649; Sun, 19 Nov 2023 11:48:40 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.37 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:40 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 12/24] mm/swap: simplify arguments for swap_cache_get_folio Date: Mon, 20 Nov 2023 03:47:28 +0800 Message-ID: <20231119194740.94101-13-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:51:37 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023278281937570 X-GMAIL-MSGID: 1783023278281937570 From: Kairui Song There are only two caller now, simplify the arguments. Signed-off-by: Kairui Song --- mm/shmem.c | 2 +- mm/swap.h | 2 +- mm/swap_state.c | 15 +++++++-------- 3 files changed, 9 insertions(+), 10 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 0d1ce70bce38..72239061c655 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1875,7 +1875,7 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, } /* Look it up and read it in.. */ - folio = swap_cache_get_folio(swap, NULL, 0); + folio = swap_cache_get_folio(swap, NULL); if (!folio) { /* Or update major stats only when swapin succeeds?? */ if (fault_type) { diff --git a/mm/swap.h b/mm/swap.h index ac9136eee690..e43e965f123f 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -47,7 +47,7 @@ void delete_from_swap_cache(struct folio *folio); void clear_shadow_from_swap_cache(int type, unsigned long begin, unsigned long end); struct folio *swap_cache_get_folio(swp_entry_t entry, - struct vm_area_struct *vma, unsigned long addr); + struct vm_fault *vmf); struct folio *filemap_get_incore_folio(struct address_space *mapping, pgoff_t index); diff --git a/mm/swap_state.c b/mm/swap_state.c index e96d63bf8a22..91461e26a8cc 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -334,8 +334,7 @@ static inline bool swap_use_vma_readahead(struct swap_info_struct *si) * * Caller must lock the swap device or hold a reference to keep it valid. */ -struct folio *swap_cache_get_folio(swp_entry_t entry, - struct vm_area_struct *vma, unsigned long addr) +struct folio *swap_cache_get_folio(swp_entry_t entry, struct vm_fault *vmf) { struct folio *folio; @@ -352,22 +351,22 @@ struct folio *swap_cache_get_folio(swp_entry_t entry, return folio; readahead = folio_test_clear_readahead(folio); - if (vma && vma_ra) { + if (vmf && vma_ra) { unsigned long ra_val; int win, hits; - ra_val = GET_SWAP_RA_VAL(vma); + ra_val = GET_SWAP_RA_VAL(vmf->vma); win = SWAP_RA_WIN(ra_val); hits = SWAP_RA_HITS(ra_val); if (readahead) hits = min_t(int, hits + 1, SWAP_RA_HITS_MAX); - atomic_long_set(&vma->swap_readahead_info, - SWAP_RA_VAL(addr, win, hits)); + atomic_long_set(&vmf->vma->swap_readahead_info, + SWAP_RA_VAL(vmf->address, win, hits)); } if (readahead) { count_vm_event(SWAP_RA_HIT); - if (!vma || !vma_ra) + if (!vmf || !vma_ra) atomic_inc(&swapin_readahead_hits); } } else { @@ -926,7 +925,7 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, struct page *page; pgoff_t ilx; - folio = swap_cache_get_folio(entry, vmf->vma, vmf->address); + folio = swap_cache_get_folio(entry, vmf); if (folio) { page = folio_file_page(folio, swp_offset(entry)); cache_result = SWAP_CACHE_HIT; From patchwork Sun Nov 19 19:47:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166872 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1810161vqn; Sun, 19 Nov 2023 11:50:24 -0800 (PST) X-Google-Smtp-Source: AGHT+IGdWBAkEWZcsrhqu/WrwO+v+ob8MseBDj+h/AX5KtA/6QEG3zkBw0AOTKH5pQ64draiucm1 X-Received: by 2002:a05:6808:1814:b0:3ae:5c48:6f41 with SMTP id bh20-20020a056808181400b003ae5c486f41mr8266060oib.28.1700423424623; Sun, 19 Nov 2023 11:50:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423424; cv=none; d=google.com; s=arc-20160816; b=J0/Fm+FsaYNDPDk2RZwflZr1XU4WlPTZpt/+5V31mduDxaIIHwc4hSqhZQLx5al2D4 U/47R49njgavfVV5/WY5MWhZeTZeMD0Myygm+g15nXvxWpZsytl5UVCp0SJLB9uPbUk7 M8Tpja5E64kw1MtEkXQvfF3tgJy+eBFCcoUNMzQ/kMWWoRR2KLv87+Jax9cmNy5Ivv9S 7hNUEXsj3jkCsBPGF4z1F132NbxllVVKW3BskWoFu0bVl2joEkiHNull+tmlKfX8It0j H3xKzKDvJJmyK3W/bJDy0CSVHp0Ske9mv7ZpT2DFveWdVlA/uUQehL4lgVlU5jAwFxY1 IXkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=5hBTip8l+rVWgXZvgeAFfyX4NMH7GqXDBsHUYVWi3uA=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=mQ8n8x6ipqYYoXk4g2yyaHjPphbAPpFLbENqr0Bwiyq2abz8zyJjJ0dLbqbepCZlvO P0fTLUoivG3658R3YqRHgyJc9xwP4bxNSou5mGmUpic44z9ObD7FFFDP2ArXozTaFCk0 P3aVFiCcbRCYiwVXyJELMfF8LueQ4sw0OC5mK+BEHrueLEaBVyYshCwhxQL0lppj3QCQ oEFNIy0AWpmtK7o1jHdnMq3cCibjG9xqWeNnaAJt0txF+I8HVEBCSbgSToubpjvBXQwE 5ayK6NAROorbZV6Uq2PCenZteabZ1bOd/9HEdTLdgbkVD6vdp4CnNTQ/kw/Zi9XQJSys iUXA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=UtyXFX0e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id u9-20020a632349000000b005c1589db3e6si6695130pgm.647.2023.11.19.11.50.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:50:24 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=UtyXFX0e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 21349805740A; Sun, 19 Nov 2023 11:50:10 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231667AbjKSTt2 (ORCPT + 29 others); Sun, 19 Nov 2023 14:49:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231661AbjKSTtJ (ORCPT ); Sun, 19 Nov 2023 14:49:09 -0500 Received: from mail-oi1-x231.google.com (mail-oi1-x231.google.com [IPv6:2607:f8b0:4864:20::231]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B842CD49 for ; Sun, 19 Nov 2023 11:48:44 -0800 (PST) Received: by mail-oi1-x231.google.com with SMTP id 5614622812f47-3b2ec5ee2e4so2610802b6e.3 for ; Sun, 19 Nov 2023 11:48:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423324; x=1701028124; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=5hBTip8l+rVWgXZvgeAFfyX4NMH7GqXDBsHUYVWi3uA=; b=UtyXFX0e8/Ek/UERAhjAlWMzQLzTdg1YsOffvyqWYEG5x7xqYEA9coS/BQHM1Cx7ZG tI/6PmGbi7sUpz5Tq/0QuBh3z0CILn4eD0vrPpcNx2tgfkp1qPipIaWVSPUzQThbLdwN jlhbGQlKII2fg4nKvDsIkcqqfXIOQhJJFHgqSqIaJYIMQVNe6xzCdwWD79+/3KnFxkIN TcDMpUvQHTpDKP4f4GyW630CuWVXw9BO1jMVQajqB0yb+YsXe7WCcRiA0qK1de1MJyg+ jkyaB/0+PCZ0ulxvictw8dWMjgSQP2EgMZVMsJwgvOz/9nsdzaal7u+BNGcF6iepH6LW GH3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423324; x=1701028124; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=5hBTip8l+rVWgXZvgeAFfyX4NMH7GqXDBsHUYVWi3uA=; b=IocGKSInJPDQxeJAX74SOzPcR/aQYuKSLAwpo10JZhbIWYBsws+Ga83ReVt0EGQtiu +LMju/kXgzw0xd/EVtP+/ExMt5HugZddezpE1JCTS4W5HbGjDC53m+Zna75FrUBhCM3+ Z7mBJU203kScmNGpY8x2ZDEQKiK/LN8Z+0TS6oAmmhNrRN/Ld96yCXEoPQmfC8hcxCuT 5b4C2bguz/LRuWkYXSUoU+RQZbPKTtiqHUCy38qWGjpvT25FSdot6GMZpSjeLzXYAmz2 q2KE9Q/iZZED7lspvd3TFSwdZR6tdD0fSxPNT2ITQcAmUVSk0/ryCF3mS2JXoJXULVcY JbDA== X-Gm-Message-State: AOJu0Yzm8pgyUo1gRv9vKc/yrbniJtnpP7BTnHktKfsqhIY4c2ZhVQeE bTU+SJidFmF4h7wdXQui5aw= X-Received: by 2002:a05:6808:3a10:b0:3ae:bae2:fa76 with SMTP id gr16-20020a0568083a1000b003aebae2fa76mr10473632oib.36.1700423323924; Sun, 19 Nov 2023 11:48:43 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.40 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:43 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 13/24] swap: simplify swap_cache_get_folio Date: Mon, 20 Nov 2023 03:47:29 +0800 Message-ID: <20231119194740.94101-14-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:50:10 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023193226142834 X-GMAIL-MSGID: 1783023193226142834 From: Kairui Song Rearrange the if statement, reduce the code indent, no feature change. Signed-off-by: Kairui Song --- mm/swap_state.c | 58 ++++++++++++++++++++++++------------------------- 1 file changed, 28 insertions(+), 30 deletions(-) diff --git a/mm/swap_state.c b/mm/swap_state.c index 91461e26a8cc..3b5a34f47192 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -336,41 +336,39 @@ static inline bool swap_use_vma_readahead(struct swap_info_struct *si) */ struct folio *swap_cache_get_folio(swp_entry_t entry, struct vm_fault *vmf) { + bool vma_ra, readahead; struct folio *folio; folio = filemap_get_folio(swap_address_space(entry), swp_offset(entry)); - if (!IS_ERR(folio)) { - bool vma_ra = swap_use_vma_readahead(swp_swap_info(entry)); - bool readahead; + if (IS_ERR(folio)) + return NULL; - /* - * At the moment, we don't support PG_readahead for anon THP - * so let's bail out rather than confusing the readahead stat. - */ - if (unlikely(folio_test_large(folio))) - return folio; - - readahead = folio_test_clear_readahead(folio); - if (vmf && vma_ra) { - unsigned long ra_val; - int win, hits; - - ra_val = GET_SWAP_RA_VAL(vmf->vma); - win = SWAP_RA_WIN(ra_val); - hits = SWAP_RA_HITS(ra_val); - if (readahead) - hits = min_t(int, hits + 1, SWAP_RA_HITS_MAX); - atomic_long_set(&vmf->vma->swap_readahead_info, - SWAP_RA_VAL(vmf->address, win, hits)); - } + /* + * At the moment, we don't support PG_readahead for anon THP + * so let's bail out rather than confusing the readahead stat. + */ + if (unlikely(folio_test_large(folio))) + return folio; - if (readahead) { - count_vm_event(SWAP_RA_HIT); - if (!vmf || !vma_ra) - atomic_inc(&swapin_readahead_hits); - } - } else { - folio = NULL; + vma_ra = swap_use_vma_readahead(swp_swap_info(entry)); + readahead = folio_test_clear_readahead(folio); + if (vmf && vma_ra) { + unsigned long ra_val; + int win, hits; + + ra_val = GET_SWAP_RA_VAL(vmf->vma); + win = SWAP_RA_WIN(ra_val); + hits = SWAP_RA_HITS(ra_val); + if (readahead) + hits = min_t(int, hits + 1, SWAP_RA_HITS_MAX); + atomic_long_set(&vmf->vma->swap_readahead_info, + SWAP_RA_VAL(vmf->address, win, hits)); + } + + if (readahead) { + count_vm_event(SWAP_RA_HIT); + if (!vmf || !vma_ra) + atomic_inc(&swapin_readahead_hits); } return folio; From patchwork Sun Nov 19 19:47:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166884 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1810597vqn; Sun, 19 Nov 2023 11:51:54 -0800 (PST) X-Google-Smtp-Source: AGHT+IHBfhyriyZPgVJ6FE6A9rYAaBGA+/zV01BoIGDdGS5wByWWK3fSc9TDdlhDUfkCGI0Kt6mK X-Received: by 2002:a05:6808:2f12:b0:3b6:d665:4938 with SMTP id gu18-20020a0568082f1200b003b6d6654938mr6822384oib.43.1700423514525; Sun, 19 Nov 2023 11:51:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423514; cv=none; d=google.com; s=arc-20160816; b=gAK3W1UwHogNgcyx/wXcULOSUMrZ1ijPUNGHFbYAvEBhglJ5rNe8EalcN2kZEOsiy8 qXRXv1ZzKUtGl/frSSnA5nYwHU9TlPMIa6z3YuibwwIOuaEFgaPlE6Ou0UYKR+1xGSpP C9zqFDUuzTCpdwpMG0Dp+rGJZ9HqZ7kEroydzHXLTWLc1RuCCGl7XXU+MlZY/xSb+Ghj zcSYslId6JfKrRNqdTKAb1L4upNKmG4kvqmlu6/jmkbhDW1zat6qlryJns6Q6kNQRs3h 02aKxuxl0BYesFBVXxIBhb+oa5JipvPrfe+muOAxCicBlOItBY+o35wFH5deHWwMvuoC e15A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=VCznjXM2MgFt7+7dcBajlWUiOZC/KHYcLFwrnZNwV5s=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=ZuYsyT7OTmexWtDEVEEoVl+qS245V+4OD9pyHgbsNz3WiG2BPhDTrRl9vUX1F2bQjB Y4CoCZy3UpFTRaVLljyc1+hl7KWNx+MdXB80kJ/kZ1SAdTTywAX1hYpUw/EsQESNzkMw Fj2ztB6mh7FceM8xiXFl5Mq7JN+idg7IVLcHUejMnZ6lqz/sRFokOvMpWG7wGLYqBX5w ow9SvOjTk3dP1LHT9k3/8zGm/GsKJXn3RiAc/FRXSbM22WT23IfG039IvVKktTle1Ln1 13R8/3i6CnlBKVE+tLACyE/jhAeaF9Jdknp2CZuPof7FLzHfFEjIvwx9EPwkGbHkbASy NmsA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=OL7qQYlb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from groat.vger.email (groat.vger.email. [23.128.96.35]) by mx.google.com with ESMTPS id a21-20020a656415000000b005b8f38f9975si6393712pgv.751.2023.11.19.11.51.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:51:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=OL7qQYlb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 5C9768087E11; Sun, 19 Nov 2023 11:51:46 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231865AbjKSTtf (ORCPT + 29 others); Sun, 19 Nov 2023 14:49:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59094 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231819AbjKSTtL (ORCPT ); Sun, 19 Nov 2023 14:49:11 -0500 Received: from mail-pf1-x42e.google.com (mail-pf1-x42e.google.com [IPv6:2607:f8b0:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02CCD10F9 for ; Sun, 19 Nov 2023 11:48:47 -0800 (PST) Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-6c10f098a27so2928347b3a.2 for ; Sun, 19 Nov 2023 11:48:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423327; x=1701028127; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=VCznjXM2MgFt7+7dcBajlWUiOZC/KHYcLFwrnZNwV5s=; b=OL7qQYlbB8hZlxpYO7qV2qBGwlsMJgdoEyrmUmaQqxtNVB9rDAhwZdFWAhlve4OSi9 /t7H1ppiyiZZMaIS8lSi8TZwTSJmtNfeFvwPwN4xtFw+Lb6Pw6MAP8VZGobsgNW2HqOe qXkiXd0BFvibp25yOSFpiVT46xVrwUvPv6Qpl5iVMOeQDhJ8khrvCnYI6DI5qWalOehl X8H/Ys4Dhip8B5X9NfpvXcw+v/NJXh5Nknrpxfhgrmqfen8w0TBjrbc9TKyOl2MqTNGF lIxALhuVLsXL0xlnv7wO+q0cXsd2Dh3xLmtCewkTyGRY5E8MFX2vpKanUPsJ7gTEWbIs UYhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423327; x=1701028127; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=VCznjXM2MgFt7+7dcBajlWUiOZC/KHYcLFwrnZNwV5s=; b=sG3CPKfP3eyDZGu/L8xdIP8hq7wFPwZ2gmbT4pkStfDstUN3WuOdx0iS/65yz/P8v3 TDJYLvrvFVD8Kjup1RMwVH+BN1UcacIsQeZFmZc59mvusQsSoa4dTGr4o2mTPohZYRD5 FCFIxUCGghR0oDECMS2eldHmN6o71Jzv1kZB7yPRY1cmpjyY8gFgADakXLY/pl5g7/gB 9Xjzg3RjCC+z1j74theCD1SNeF0VDGqqND44ft9QG02orMzrm7UpQAHJp4ozsX7/pjqS 8wuFOcBd/2HVRNskt6N4DypObx7AfR00x5/AWkW9QMNY6ZcOAkuyh6kI1d4veQ3styRb cPGQ== X-Gm-Message-State: AOJu0YxJ7CSMlf9tUryB+FhQrhJouHhghqsYE0DFRWfxAgeLa3gefD2q 3nzG1eScsT6x5ymjL3SnZVw= X-Received: by 2002:a05:6a20:6a1a:b0:189:df1b:6616 with SMTP id p26-20020a056a206a1a00b00189df1b6616mr3107739pzk.15.1700423327040; Sun, 19 Nov 2023 11:48:47 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.44 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:46 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 14/24] mm/swap: do shadow lookup as well when doing swap cache lookup Date: Mon, 20 Nov 2023 03:47:30 +0800 Message-ID: <20231119194740.94101-15-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:51:46 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023287093220461 X-GMAIL-MSGID: 1783023287093220461 From: Kairui Song Make swap_cache_get_folio capable of returning the shadow value when the xarray contains a shadow instead of a valid folio. Just extend the arguments to be used later. Signed-off-by: Kairui Song --- mm/shmem.c | 2 +- mm/swap.h | 2 +- mm/swap_state.c | 11 +++++++---- 3 files changed, 9 insertions(+), 6 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 72239061c655..f9ce4067c742 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1875,7 +1875,7 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, } /* Look it up and read it in.. */ - folio = swap_cache_get_folio(swap, NULL); + folio = swap_cache_get_folio(swap, NULL, NULL); if (!folio) { /* Or update major stats only when swapin succeeds?? */ if (fault_type) { diff --git a/mm/swap.h b/mm/swap.h index e43e965f123f..da9deb5ba37d 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -47,7 +47,7 @@ void delete_from_swap_cache(struct folio *folio); void clear_shadow_from_swap_cache(int type, unsigned long begin, unsigned long end); struct folio *swap_cache_get_folio(swp_entry_t entry, - struct vm_fault *vmf); + struct vm_fault *vmf, void **shadowp); struct folio *filemap_get_incore_folio(struct address_space *mapping, pgoff_t index); diff --git a/mm/swap_state.c b/mm/swap_state.c index 3b5a34f47192..e057c79fb06f 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -334,14 +334,17 @@ static inline bool swap_use_vma_readahead(struct swap_info_struct *si) * * Caller must lock the swap device or hold a reference to keep it valid. */ -struct folio *swap_cache_get_folio(swp_entry_t entry, struct vm_fault *vmf) +struct folio *swap_cache_get_folio(swp_entry_t entry, struct vm_fault *vmf, void **shadowp) { bool vma_ra, readahead; struct folio *folio; - folio = filemap_get_folio(swap_address_space(entry), swp_offset(entry)); - if (IS_ERR(folio)) + folio = filemap_get_entry(swap_address_space(entry), swp_offset(entry)); + if (xa_is_value(folio)) { + if (shadowp) + *shadowp = folio; return NULL; + } /* * At the moment, we don't support PG_readahead for anon THP @@ -923,7 +926,7 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, struct page *page; pgoff_t ilx; - folio = swap_cache_get_folio(entry, vmf); + folio = swap_cache_get_folio(entry, vmf, NULL); if (folio) { page = folio_file_page(folio, swp_offset(entry)); cache_result = SWAP_CACHE_HIT; From patchwork Sun Nov 19 19:47:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166870 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1810070vqn; Sun, 19 Nov 2023 11:50:06 -0800 (PST) X-Google-Smtp-Source: AGHT+IFJqZkGPaU5dY+WCXu39aymL3oFIb3KrRYmHuhAJ/9gfGcDUTijgU14DB7aHrTwCq5ktp+f X-Received: by 2002:a05:6808:23c5:b0:3af:e556:4602 with SMTP id bq5-20020a05680823c500b003afe5564602mr7648697oib.11.1700423405931; Sun, 19 Nov 2023 11:50:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423405; cv=none; d=google.com; s=arc-20160816; b=KRZqSz7a5WfUmDDVH9pfYuCTDq0PSZQ2q5e0Q0mu1yPPHpwXMtjdQMdI6rxBl21I7n G851hbc9ZqwtPNCwl/D09ikkRhimj6h4J9iuVzyTajGc6PZgZKFjefdDTFP8PPS43MNa 9bKgZXXd78w9/e4qPNSWrSpoKrFCco1pTqMD5P6X0M/snXJvbDqN+N7EpBv8BOM14M1K vf7FRf44WPfpORAnN+A58f38PLwinR7myb6WBT6RNMUO457cMFMeVyYD1IxGj6EyzeSd XdYlWouLEPiYYe6FPL7X7tcuere5x1Ck3R6B74q4V4Qk79K2V960rjSMX+kx2ts0yieD qaSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=aDkLnQICnVgpq4/orPONBtwFkKKcmzZtjb3XACAs/tw=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=kq1Y+XOfOY9/h1+bP4YUXdiLF5yTrBYoA7Y20dMqCbFilkzJaYAk1NRHr9tC1rPlYn h+zHQqoFkOrHYE3kjsiyyywMWuffBYx+E5sOidYoYJ2I9FR/m6zXFNOLXgXHF4hoSe62 TasKgVEe7xWi/70SdP7cCEpTl1UFyeLEeSSkWI7PZvQm6/TUSJaAfT/n1SquJoyyZtXI GkgK4vJcKwOlXbr3RVntT6b6JbdRAeVncW/wdDjwsTqOcUetbOqy3l1YIxxWPvi63cqC JHe79IzaW2FoODomJE/+tD3OcJi5JToe1MSijVs3ERqfC9WU2iPytuYParb/mfcbVdUX kmqA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=RXAJrU5+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id j16-20020a634a50000000b005a0737404a7si6651803pgl.258.2023.11.19.11.50.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:50:05 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=RXAJrU5+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id E3C9D80AD8E3; Sun, 19 Nov 2023 11:50:04 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231607AbjKSTtk (ORCPT + 29 others); Sun, 19 Nov 2023 14:49:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60996 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231716AbjKSTtO (ORCPT ); Sun, 19 Nov 2023 14:49:14 -0500 Received: from mail-oi1-x22a.google.com (mail-oi1-x22a.google.com [IPv6:2607:f8b0:4864:20::22a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 19686198C for ; Sun, 19 Nov 2023 11:48:51 -0800 (PST) Received: by mail-oi1-x22a.google.com with SMTP id 5614622812f47-3b6ce6fac81so2425834b6e.1 for ; Sun, 19 Nov 2023 11:48:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423330; x=1701028130; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=aDkLnQICnVgpq4/orPONBtwFkKKcmzZtjb3XACAs/tw=; b=RXAJrU5+u3NIqZYjCeSgFrmFpHZ/y80OxLUHyDQIEANThnEXlB8Gkpn81diIPTlg0+ LNxnLDXmDvmkERe656iIzGP7k3hNqIhB6SSi/J9+OKY6c+KWH8oRWIzOBBMJO6wcwlGi DRcpDg4ti1nvZUwie9yI3HvBJjdwO6VAIZXFGdNan0Gvmhucnm83P8QjnnhNY+YoEcyM Lm9MWaXUevxHsmyQ2IFJUYneVbg48NBUWcJC7amwYWZ7FJPkan8bntgg3v56+ClzdKG0 +zYvV0F+r2Ow54NGWaxxFozYP+FGEXgysJ6pe93gP/MKf64MMSRXmz+L5Vh/W3FpuXe/ Fk2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423330; x=1701028130; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=aDkLnQICnVgpq4/orPONBtwFkKKcmzZtjb3XACAs/tw=; b=Za39uM4eYlCQfBlWgYJXm5pe9euZPGAWvfjMgc/jRVMV+VX7+m2zGCb0YAQxsZOUmg s9EXHErISWWJrZ6fSK7D3RvSnK5pXVEMp5UoQ0ziF+GjMLWb0n9KhnXX73wrf4P89qCv zpQ/lJNch9zGDqRX7xtPwR4cXdFdb+alZ71APkYD6+8tStXiT1KDRzR6toUjWvUr4U1N +xh5f7KrkfAbV8e4GvbhJexiriJDwLRlxFABc83htJZ1P176sW6tRirkn5rWhVe7CiWP brQXhKfpgtCmQOUFe2wATPrX6eg/19ytwhJ9b5yNWwksaXxqUURn/SgIuXeCLz854PY9 iBVA== X-Gm-Message-State: AOJu0YxtSdU9Kc5gCbMaSsGWeFkMDgGriAUEhTUlb6OOKfhB8eqyMOej i6GdzL0P6gdRYQHSTTpmXd4= X-Received: by 2002:a05:6808:1452:b0:3af:983a:8129 with SMTP id x18-20020a056808145200b003af983a8129mr7743023oiv.53.1700423330257; Sun, 19 Nov 2023 11:48:50 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.47 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:49 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 15/24] mm/swap: avoid an duplicated swap cache lookup for SYNCHRONOUS_IO device Date: Mon, 20 Nov 2023 03:47:31 +0800 Message-ID: <20231119194740.94101-16-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:50:04 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023173072557321 X-GMAIL-MSGID: 1783023173072557321 From: Kairui Song When a xa_value is returned by the cache lookup, keep it to be used later for workingset refault check instead of doing the looking up again in swapin_no_readahead. This does have a side effect of making swapoff also triggers workingset check, but should be fine since swapoff does effect the workload in many ways already. Signed-off-by: Kairui Song --- mm/swap_state.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/mm/swap_state.c b/mm/swap_state.c index e057c79fb06f..51de2a0412df 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -872,7 +872,6 @@ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, { struct folio *folio; struct page *page; - void *shadow = NULL; page = alloc_pages_mpol(gfp_mask, 0, mpol, ilx, numa_node_id()); folio = (struct folio *)page; @@ -888,10 +887,6 @@ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, mem_cgroup_swapin_uncharge_swap(entry); - shadow = get_shadow_from_swap_cache(entry); - if (shadow) - workingset_refault(folio, shadow); - folio_add_lru(folio); /* To provide entry to swap_readpage() */ @@ -922,11 +917,12 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, enum swap_cache_result cache_result; struct swap_info_struct *si; struct mempolicy *mpol; + void *shadow = NULL; struct folio *folio; struct page *page; pgoff_t ilx; - folio = swap_cache_get_folio(entry, vmf, NULL); + folio = swap_cache_get_folio(entry, vmf, &shadow); if (folio) { page = folio_file_page(folio, swp_offset(entry)); cache_result = SWAP_CACHE_HIT; @@ -938,6 +934,8 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, if (swap_use_no_readahead(si, swp_offset(entry))) { page = swapin_no_readahead(entry, gfp_mask, mpol, ilx, vmf->vma->vm_mm); cache_result = SWAP_CACHE_BYPASS; + if (shadow) + workingset_refault(page_folio(page), shadow); } else if (swap_use_vma_readahead(si)) { page = swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf); cache_result = SWAP_CACHE_MISS; From patchwork Sun Nov 19 19:47:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166878 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1810433vqn; Sun, 19 Nov 2023 11:51:18 -0800 (PST) X-Google-Smtp-Source: AGHT+IEohPYgrej6DQkZHzzA2pxwDE6pD+0QPiV7qoqO7dgCyWyjYpeUP9szBvS4UAXZZeeAlFb8 X-Received: by 2002:a05:6a20:3944:b0:188:20e0:2bca with SMTP id r4-20020a056a20394400b0018820e02bcamr4113378pzg.13.1700423478232; Sun, 19 Nov 2023 11:51:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423478; cv=none; d=google.com; s=arc-20160816; b=JDfeJNYcR9ZQRLDXG79LWc51hcwquwb6qLsy4wKK/UY/wA5YcfZklrIgbZR2kIO3Oh i8ZlWMscIThqyhYXMmgzFbaAZ0Wp3BZPgx5xfeLiRiwD8pPGZN6KGUl4Mm09VdarU6Ot l3wxXOQ7GBnBkGnzAZLdHjhZxJYFY6O4s+JsEPxkz6itksJMc0kQJ9Kl95W2FK/EdEFl usUpxgrSn+7LBcEpuFcmRU6y94r+uf91xn+NtKOCe33ujhsL4eQPyHTSxEUkZQ2H6TWn Nlhnsx50rja3AQC8xzo+mTABgUaZZIPraVoPEOHk/yOQ8V3E5i5F5I+PgnQthbDF6Rm4 cIpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=9XjpTi/p1jP45iq6KrQ/NXIY9Wo9hQ4YnGSNNoLyAOA=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=uai151YMDU+jmDAysL6fvRL5wiAbep3o8OmtT6o4F1bvti1WsX4XME5/Rz3l17No9F agI1d18J1No/vB9FUl0cPUUwAiWiURzp/pTq+KqpINKJ5lsa5+DDqIbldvCXUbdBdv2A DQ6qq3Kb6VWa3msDeB/+B+zMuhe+IjVPn7YNoqBDk9hdae5TwA05Eyj3K4SruDFI1FIk 5VZ39t6GJE+U/PNtglY/aTVa3jX9j0kZRlsWOldTDWvVosJF28TH4GU2/TdMIx4Do22c +VUXmEWKWYDyOVeUZM1LFgmGp719HTJ3SJQfrSxe7MVkHeC3oIt/IxO+Q1PiJW84mUy+ 0HmA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=OV7vKBtM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from groat.vger.email (groat.vger.email. [23.128.96.35]) by mx.google.com with ESMTPS id h9-20020a170902f54900b001cc54b5b4dasi6819418plf.388.2023.11.19.11.51.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:51:18 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=OV7vKBtM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id A33C58087DE5; Sun, 19 Nov 2023 11:51:04 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231753AbjKSTtt (ORCPT + 29 others); Sun, 19 Nov 2023 14:49:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34474 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231759AbjKSTtT (ORCPT ); Sun, 19 Nov 2023 14:49:19 -0500 Received: from mail-il1-x12b.google.com (mail-il1-x12b.google.com [IPv6:2607:f8b0:4864:20::12b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4722AD56 for ; Sun, 19 Nov 2023 11:48:54 -0800 (PST) Received: by mail-il1-x12b.google.com with SMTP id e9e14a558f8ab-3574297c79eso14110605ab.1 for ; Sun, 19 Nov 2023 11:48:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423333; x=1701028133; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=9XjpTi/p1jP45iq6KrQ/NXIY9Wo9hQ4YnGSNNoLyAOA=; b=OV7vKBtM8QcyzCdAPX2k3P2b3HevaAp9wu9iHRuPa4/0gMJrOxaOgeg65Tr1JJe1az 6BwJEQF+1Af6fYV3TsevmGa7AH1ExN4MdYI31HH5+ab0NH+IkFs1c7pLpU9XZ9pX3oH3 rpkZ8mxpfjHMDLoBM6FMz2uBdiOjmJyWRkYw1aoCarjmAIVDTvh6z4+0ju77seNKv5YG vV5faKNmy6+it8DLlGQLOVFDPo366HXhDSM9+z1FqOcmluCS116AAO8YpiJ72+Co/pVX Yib44qDni3Y9lU6gKm0XMhBsjCQex8HvlnBJvejLCKAD3g5NdTgJVPGG6f8lSRm0fuXX oxjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423333; x=1701028133; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=9XjpTi/p1jP45iq6KrQ/NXIY9Wo9hQ4YnGSNNoLyAOA=; b=w4wKDOuyZVN8Aj4Yv6ZQNVsiuosFV+IQgxpljsh6psSPUU6TzRBYhKDU07+Z9u6+0t URQzsnwB0Mg4hGwbBICrZXAjgAwJDl7S5owrSqrkdKvWgCfEE5htUQn/0BLwKCmc7s/+ A/EmIYJQhkTiCNeX6U/lrckFcwo3vsPS3a23szzhe0gGoAH7rpxhbzpIwNdKeoNSIxQA l0pvzAzHanw2YxfN3DKWkP/v62mAuDu5AhklBD+Md1mJHBX5efnu5QPKi1+HFf5a7TiZ hJo2H4jFJJxIVcuRtvrmgsH6V6xtPEX9q3VsfYUWlfMGK9KkF2wt0+i4NOK7ldklE6fw IHIg== X-Gm-Message-State: AOJu0YyAZ23s7N/L4ATLeQE0e4DHZ1iN8Lt5mshtLWZIsCt7wRtSNefy +QCdFF2SO1bHJ17c5oPD/ng= X-Received: by 2002:a92:7a06:0:b0:359:5389:c0dc with SMTP id v6-20020a927a06000000b003595389c0dcmr6674915ilc.7.1700423333536; Sun, 19 Nov 2023 11:48:53 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.50 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:52 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 16/24] mm/swap: reduce scope of get_swap_device in swapin path Date: Mon, 20 Nov 2023 03:47:32 +0800 Message-ID: <20231119194740.94101-17-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:51:04 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023248923930433 X-GMAIL-MSGID: 1783023248923930433 From: Kairui Song Move get_swap_device into swapin_readahead, simplify the code and prepare for follow up commits. For the later part in do_swap_page, using swp_swap_info directly is fine since in that context, the swap device is pinned by swapcache reference. Signed-off-by: Kairui Song --- mm/memory.c | 16 ++++------------ mm/swap_state.c | 8 ++++++-- mm/swapfile.c | 4 +++- 3 files changed, 13 insertions(+), 15 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 22af9f3e8c75..e399b37ef395 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3789,7 +3789,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) struct folio *swapcache = NULL, *folio = NULL; enum swap_cache_result cache_result; struct page *page; - struct swap_info_struct *si = NULL; rmap_t rmap_flags = RMAP_NONE; bool exclusive = false; swp_entry_t entry; @@ -3845,14 +3844,11 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) goto out; } - /* Prevent swapoff from happening to us. */ - si = get_swap_device(entry); - if (unlikely(!si)) - goto out; - page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, vmf, &cache_result); - if (page) { + if (PTR_ERR(page) == -EBUSY) { + goto out; + } else if (page) { folio = page_folio(page); if (cache_result != SWAP_CACHE_HIT) { /* Had to read the page from swap area: Major fault */ @@ -3964,7 +3960,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) */ exclusive = true; } else if (exclusive && folio_test_writeback(folio) && - data_race(si->flags & SWP_STABLE_WRITES)) { + (swp_swap_info(entry)->flags & SWP_STABLE_WRITES)) { /* * This is tricky: not all swap backends support * concurrent page modifications while under writeback. @@ -4068,8 +4064,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) if (vmf->pte) pte_unmap_unlock(vmf->pte, vmf->ptl); out: - if (si) - put_swap_device(si); return ret; out_nomap: if (vmf->pte) @@ -4082,8 +4076,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) folio_unlock(swapcache); folio_put(swapcache); } - if (si) - put_swap_device(si); return ret; } diff --git a/mm/swap_state.c b/mm/swap_state.c index 51de2a0412df..ff8a166603d0 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -922,6 +922,11 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, struct page *page; pgoff_t ilx; + /* Prevent swapoff from happening to us */ + si = get_swap_device(entry); + if (unlikely(!si)) + return ERR_PTR(-EBUSY); + folio = swap_cache_get_folio(entry, vmf, &shadow); if (folio) { page = folio_file_page(folio, swp_offset(entry)); @@ -929,7 +934,6 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, goto done; } - si = swp_swap_info(entry); mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); if (swap_use_no_readahead(si, swp_offset(entry))) { page = swapin_no_readahead(entry, gfp_mask, mpol, ilx, vmf->vma->vm_mm); @@ -944,8 +948,8 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, cache_result = SWAP_CACHE_MISS; } mpol_cond_put(mpol); - done: + put_swap_device(si); if (result) *result = cache_result; diff --git a/mm/swapfile.c b/mm/swapfile.c index b6d57fff5e21..925ad92486a4 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1857,7 +1857,9 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, pte = NULL; page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, &vmf, NULL); - if (page) + if (IS_ERR(page)) + return PTR_ERR(page); + else if (page) folio = page_folio(page); if (!folio) { /* From patchwork Sun Nov 19 19:47:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166883 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1810571vqn; Sun, 19 Nov 2023 11:51:51 -0800 (PST) X-Google-Smtp-Source: AGHT+IECffCc1ERmvLA/1oY2HuoBzdQp9F5IHd8fIOChoUq0lFoVECq4fBhlNhJtjnE1fvOjTvI1 X-Received: by 2002:a05:6a20:3942:b0:181:15:5755 with SMTP id r2-20020a056a20394200b0018100155755mr8696893pzg.56.1700423510991; Sun, 19 Nov 2023 11:51:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423510; cv=none; d=google.com; s=arc-20160816; b=OGm+ovtyVSZOOvQjDt+ULWmdrMxAPEm1s9bOhviF1c2pGga6VpB0ioXB7Z6huplZaW ovoTdmbmjKTjhux9cDkvPZmdWAkKjFOxj9OgrW3EfnooDg5KZ68nIoKfpTriG0esBck7 UGSXlfA27k4rSujxMcyc1wJJbc7bbR1zpwlBuSLFvNR2apRYyS7gqskB1/ixzhNVQu7Y /hsWBn7/PkUA3oCCjvCv2QgFwIR7CsD6hqd+1ymcDa0o+1FFw0HcBAR4acADvLGDB6Wl PmE7idqNB3DxVOt8w5HDldQwARPq1kpza/UTEZaLHdCAqWrCMV6u22k6wrAmL/pUn018 WcbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=FCPW0dI33gWre1bmPp4wG3A/kpp2Dtpn+fZQd/yP8/o=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=SUSbeQSh0YUEYsWAEzP1S9UjPEt7bGp2QzBO8gTTiQ+7b1G/LU5UavkRe7hHG+rrbQ qCJl1oi+eSWBaYSNMDHvC0BsiI18gMqqsiW3V9g3FLEbiz1O6Z3YJo9KaXpXPvY+8xWl IjHs1M4EV+cpHbwkCTKICjfCmqWn3aHgBgAUfSSTHyPdnjLd7tnBUjvtPhdi/n7v7AJx ae5pcmMAjpaQKiZF9NnOQ7vfjDpoWWK999ybMKT8aJjGAYe65jg+RUzkF87HmyMFmZPz 402wQFRFvcVSIl0MPgvniVcdhxRftj5hbjKDWMqg/o8IjAHK48N6FAZ44gNSkLo/m29f GDag== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=kb+kvJTu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id x6-20020a636306000000b00564929df8besi6753478pgb.568.2023.11.19.11.51.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:51:50 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=kb+kvJTu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 4E14F8089854; Sun, 19 Nov 2023 11:50:53 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231785AbjKSTty (ORCPT + 29 others); Sun, 19 Nov 2023 14:49:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34496 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231794AbjKSTt2 (ORCPT ); Sun, 19 Nov 2023 14:49:28 -0500 Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8600D19A2 for ; Sun, 19 Nov 2023 11:48:57 -0800 (PST) Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-6c115026985so3870297b3a.1 for ; Sun, 19 Nov 2023 11:48:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423337; x=1701028137; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=FCPW0dI33gWre1bmPp4wG3A/kpp2Dtpn+fZQd/yP8/o=; b=kb+kvJTuqgPIZdsg4KJJjy4Vh03F0rOOeZzSaU+peAxitPCQ0MWo9ABfEVs7rZwyfS z31d/fxAwEsdIqbjZigitUmP9kWKcjX+K7kvGXrk+fb5YgU8iRigF+GxYXaKWB9SfiOQ o1TE0qSQmYaTMuuPz62JyWBVG5Lt7KoaU0BiaG0w3fXdqFmbMohiTWl3FQD7v91bOdev F3ujCKuGqWI5jqzdWoXJY6fHyQJlfyOqEP4nTHU21LO6fAwJZxQuzqMnsXdpd3HNCroY W7dC4sMoYYvi+BDePxgluzFC8G5PJ8JxxGPsDNhg14Snar4a8rukXcTwGxz9wwFLh9aM USlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423337; x=1701028137; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=FCPW0dI33gWre1bmPp4wG3A/kpp2Dtpn+fZQd/yP8/o=; b=ghxlr0cmFUU6IM9BG3A5q1d1ZbEsQ8uQuAWia1HdXcXoZc5CO4csurkVoQc0KuMNJV +I7ok0PGWjwDJCxQilEj2atfxeOrowANrzkvGS7rEnkcfKOnA8aN4zDTcdlB4kG3xTlj o1Qd44hoRUtsnjwAIDxnecYTAWTt8+fYajUXG7u48TM/LOEs7QvghKwPD4JF0f4yBt/L VS4V5hvvkZWBsj0S/6lw0szRhnqJZBDtds5paY2CcJDyA5TL5p26NMq6Bb7AO6zxnXEG oE2UBF0MfSyTtiHK4K+4MhGE4sX1/m2wfEVyjU2EUOnk7Oz1QdHeLYxYBX1gAYWDORLN SPbg== X-Gm-Message-State: AOJu0YyCwTVLjsMQOax9N3n0KiqcN4EagWRPhLyxNv+I3WTA4bWHK8Pf SL76K+Zj9t6UHxzVfsO9T7s= X-Received: by 2002:a05:6a20:244e:b0:187:f23d:f9f2 with SMTP id t14-20020a056a20244e00b00187f23df9f2mr6715469pzc.58.1700423336700; Sun, 19 Nov 2023 11:48:56 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.53 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:56 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 17/24] mm/swap: fix false error when swapoff race with swapin Date: Mon, 20 Nov 2023 03:47:33 +0800 Message-ID: <20231119194740.94101-18-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:50:53 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023283327749498 X-GMAIL-MSGID: 1783023283327749498 From: Kairui Song When swapoff race with swapin, get_swap_device may fail and cause swapin_readahead to return -EBUSY. In such case check if the page is already swapped in by swapoff path. Signed-off-by: Kairui Song --- mm/memory.c | 29 +++++++++++++++-------------- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index e399b37ef395..620fa87557fd 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3846,9 +3846,21 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, vmf, &cache_result); - if (PTR_ERR(page) == -EBUSY) { - goto out; - } else if (page) { + if (IS_ERR_OR_NULL(page)) { + /* + * Back out if somebody else faulted in this pte + * while we released the pte lock. + */ + vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd, + vmf->address, &vmf->ptl); + if (likely(vmf->pte && pte_same(ptep_get(vmf->pte), vmf->orig_pte))) { + if (!page) + ret = VM_FAULT_OOM; + else + ret = VM_FAULT_RETRY; + } + goto unlock; + } else { folio = page_folio(page); if (cache_result != SWAP_CACHE_HIT) { /* Had to read the page from swap area: Major fault */ @@ -3866,17 +3878,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) ret = VM_FAULT_HWPOISON; goto out_release; } - } else { - /* - * Back out if somebody else faulted in this pte - * while we released the pte lock. - */ - vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd, - vmf->address, &vmf->ptl); - if (likely(vmf->pte && - pte_same(ptep_get(vmf->pte), vmf->orig_pte))) - ret = VM_FAULT_OOM; - goto unlock; } ret |= folio_lock_or_retry(folio, vmf); From patchwork Sun Nov 19 19:47:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166879 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1810444vqn; Sun, 19 Nov 2023 11:51:23 -0800 (PST) X-Google-Smtp-Source: AGHT+IHnaIdYzeoMa8Zi1RJlH1sGxv4UnfhRUTae+6nlfpVRbuw7HLhJcWGkFzEVtdZVgt2dOlC6 X-Received: by 2002:a05:6808:2e9a:b0:3af:744e:92ff with SMTP id gt26-20020a0568082e9a00b003af744e92ffmr8515428oib.34.1700423483787; Sun, 19 Nov 2023 11:51:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423483; cv=none; d=google.com; s=arc-20160816; b=EcthNOhdz3K2J1h57qPsOuD+ao5RkPSVMknZ6hM1gzTOkn3Mn7svmdi+PZvPxfm/L0 oTFNfG2R0hi2ylTQzFXEvTyiE0oLUnhxOvEuTzyi8/cq+YgbQBngMV6hfgQkzDksbVTl w5fVGZGNLPDDLU/+3t6p3H1Fc2c3ee17VQcxDS2gGfo/uXcc/2UdfS4QNLbfQSSQAJKk AQaVjPzTDxw8BPgvRXV4OcciRCpQ1+OHBKQJkX5m+tAMVLKF6k+a0Unsi0kbvnQDsGjm MukbAAsJC5WhAWkGsELF4wHxiWhPWShR2dWpL5j3vwpx57jBF0M4eyiYB/9QqIPWCr8I hxmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ogKmBoCop4FjzMsXxzq+m99gJioFg3IUepp9Cnvtmi4=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=0EVT6EJaSbzouu7IPxoFT0s+IJ0i7YM0gBBV63TMUlq4JKb0yb635u5Hkc9jyGTn1b 1JJdThpB5OTh5F+15Xn8kkoZkuaOQkFrb1jWhBGk97mTq5BrBCIX63OSZmH/tcl34jeL AW9yvOfzwUS0cr5Qd8DozGrITLKcR2AP1xwNF6Vq+E7ehwJ3rUQV2iQzpE7mSMfLrMWr 2lTXvxUquhGycWnAehx+NqPff1owKu6iCl4Pq/eAJg09maP1DkK5O5FbWKuOq47Hh1Xg r/A7NNiuPLuvkpCrOwMfEkD3J1twkcMOMH941vceAAwhK5mK9N89goWdXP23hViUuuj7 Jirw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=NT0k2DET; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from groat.vger.email (groat.vger.email. [23.128.96.35]) by mx.google.com with ESMTPS id ca16-20020a056a02069000b005c1b2fdfe6dsi7156744pgb.573.2023.11.19.11.51.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:51:23 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=NT0k2DET; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 4AA10807831A; Sun, 19 Nov 2023 11:51:12 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231873AbjKSTuC (ORCPT + 29 others); Sun, 19 Nov 2023 14:50:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34562 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231840AbjKSTtg (ORCPT ); Sun, 19 Nov 2023 14:49:36 -0500 Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 91A90D78 for ; Sun, 19 Nov 2023 11:49:00 -0800 (PST) Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-6c396ef9a3dso3070117b3a.1 for ; Sun, 19 Nov 2023 11:49:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423340; x=1701028140; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=ogKmBoCop4FjzMsXxzq+m99gJioFg3IUepp9Cnvtmi4=; b=NT0k2DETo/JYhGoZQ4D+JumGWDBfOY/56XsePUpsVAtRJmYdQk5TTXqqAJ1s9666nv sinsGxqPSBJrDP2+9F+KHJa1JXBrgXoqEe9VBaSDupZgufHG9KOIqIMISmIp3uQLO0RC AZVh+0Y0gkXcn/oaDGpZU4DRnbptL88CNGcb+Jmku6wCPRJgX4O9a4HRLuo52rR6vQwg Ez+BTVheKCadcnoUPjfqacuJ9BDnhknie24MJp6TPHjAmplsNhHazcPd/B9Z0UsSG5yT z2Tu2pBqFZjZSLjeDEr50AxaIfSVjoUHaC42f/Y9OALZJ1EiktzwAlAyH7ktbeS6N7Ju dQ5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423340; x=1701028140; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ogKmBoCop4FjzMsXxzq+m99gJioFg3IUepp9Cnvtmi4=; b=d2A6+cxVQoy7DZF/jLhRZ0CHramW+Xe3wMpRDQMCWKmpPmut80IEf9E4Poti5nC9/R svmdQPpCjpg6ZOvisJnklNXyFD254STt5XQ7BggF47VZjTJstda2NbBI2OyRv/DIpFSO jOHnbhq5G0RVtGuB8IXmiw+1DNrbueZQKkVaTKQEnkh6+Q/57X7DtGb80t3+q5Bvjl88 71dCdLZP83Hhk2XfM87LrVjd/A3ulYB2zOIxFitnLLP0B7YjGSF3orO8cDX3p/r7vTOA gICsEpGdN3mt13Yvs8OM/CWFTfl42HtWk3TnktcEzZBUER1zk7Py61CvsrXqg6rGl3kY IBMA== X-Gm-Message-State: AOJu0YzJc0c6hdbX0b6qOoeuwF9u5xpOrPQy2l0cccFq+MDNzJ3CtqCx gyeKeXxMzztncaPsgHeaISo= X-Received: by 2002:a05:6a00:301d:b0:693:3c11:4293 with SMTP id ay29-20020a056a00301d00b006933c114293mr4840721pfb.14.1700423339887; Sun, 19 Nov 2023 11:48:59 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.48.56 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:48:59 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 18/24] mm/swap: introduce a helper non fault swapin Date: Mon, 20 Nov 2023 03:47:34 +0800 Message-ID: <20231119194740.94101-19-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:51:12 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023254891107756 X-GMAIL-MSGID: 1783023254891107756 From: Kairui Song There are two places where swapin is not direct caused by page fault: shmem swapin is invoked through shmem mapping, swapoff cause swapin by walking the page table. They used to construct a pseudo vmfault struct for swapin function. Shmem has dropped the pseudo vmfault recently in commit ddc1a5cbc05d ("mempolicy: alloc_pages_mpol() for NUMA policy without vma"). Swapoff path is still using a pseudo vmfault. Introduce a helper for them both, this help save stack usage for swapoff path, and help apply a unified swapin cache and readahead policy check. Also prepare for follow up commits. Signed-off-by: Kairui Song --- mm/shmem.c | 51 ++++++++++++++++--------------------------------- mm/swap.h | 11 +++++++++++ mm/swap_state.c | 38 ++++++++++++++++++++++++++++++++++++ mm/swapfile.c | 23 +++++++++++----------- 4 files changed, 76 insertions(+), 47 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index f9ce4067c742..81d129aa66d1 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1565,22 +1565,6 @@ static inline struct mempolicy *shmem_get_sbmpol(struct shmem_sb_info *sbinfo) static struct mempolicy *shmem_get_pgoff_policy(struct shmem_inode_info *info, pgoff_t index, unsigned int order, pgoff_t *ilx); -static struct folio *shmem_swapin_cluster(swp_entry_t swap, gfp_t gfp, - struct shmem_inode_info *info, pgoff_t index) -{ - struct mempolicy *mpol; - pgoff_t ilx; - struct page *page; - - mpol = shmem_get_pgoff_policy(info, index, 0, &ilx); - page = swap_cluster_readahead(swap, gfp, mpol, ilx); - mpol_cond_put(mpol); - - if (!page) - return NULL; - return page_folio(page); -} - /* * Make sure huge_gfp is always more limited than limit_gfp. * Some of the flags set permissions, while others set limitations. @@ -1854,9 +1838,12 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, { struct address_space *mapping = inode->i_mapping; struct shmem_inode_info *info = SHMEM_I(inode); - struct swap_info_struct *si; + enum swap_cache_result result; struct folio *folio = NULL; + struct mempolicy *mpol; + struct page *page; swp_entry_t swap; + pgoff_t ilx; int error; VM_BUG_ON(!*foliop || !xa_is_value(*foliop)); @@ -1866,34 +1853,30 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, if (is_poisoned_swp_entry(swap)) return -EIO; - si = get_swap_device(swap); - if (!si) { + mpol = shmem_get_pgoff_policy(info, index, 0, &ilx); + page = swapin_page_non_fault(swap, gfp, mpol, ilx, fault_mm, &result); + mpol_cond_put(mpol); + + if (PTR_ERR(page) == -EBUSY) { if (!shmem_confirm_swap(mapping, index, swap)) return -EEXIST; else return -EINVAL; - } - - /* Look it up and read it in.. */ - folio = swap_cache_get_folio(swap, NULL, NULL); - if (!folio) { - /* Or update major stats only when swapin succeeds?? */ - if (fault_type) { + } else if (!page) { + error = -ENOMEM; + goto failed; + } else { + folio = page_folio(page); + if (fault_type && result != SWAP_CACHE_HIT) { *fault_type |= VM_FAULT_MAJOR; count_vm_event(PGMAJFAULT); count_memcg_event_mm(fault_mm, PGMAJFAULT); } - /* Here we actually start the io */ - folio = shmem_swapin_cluster(swap, gfp, info, index); - if (!folio) { - error = -ENOMEM; - goto failed; - } } /* We have to do this with folio locked to prevent races */ folio_lock(folio); - if (!folio_test_swapcache(folio) || + if ((result != SWAP_CACHE_BYPASS && !folio_test_swapcache(folio)) || folio->swap.val != swap.val || !shmem_confirm_swap(mapping, index, swap)) { error = -EEXIST; @@ -1930,7 +1913,6 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, delete_from_swap_cache(folio); folio_mark_dirty(folio); swap_free(swap); - put_swap_device(si); *foliop = folio; return 0; @@ -1944,7 +1926,6 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, folio_unlock(folio); folio_put(folio); } - put_swap_device(si); return error; } diff --git a/mm/swap.h b/mm/swap.h index da9deb5ba37d..b073c29c9790 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -62,6 +62,10 @@ struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t flag, struct mempolicy *mpol, pgoff_t ilx); struct page *swapin_readahead(swp_entry_t entry, gfp_t flag, struct vm_fault *vmf, enum swap_cache_result *result); +struct page *swapin_page_non_fault(swp_entry_t entry, gfp_t gfp_mask, + struct mempolicy *mpol, pgoff_t ilx, + struct mm_struct *mm, + enum swap_cache_result *result); static inline unsigned int folio_swap_flags(struct folio *folio) { @@ -103,6 +107,13 @@ static inline struct page *swapin_readahead(swp_entry_t swp, gfp_t gfp_mask, return NULL; } +static inline struct page *swapin_page_non_fault(swp_entry_t entry, gfp_t gfp_mask, + struct mempolicy *mpol, pgoff_t ilx, struct mm_struct *mm, + enum swap_cache_result *result) +{ + return NULL; +} + static inline int swap_writepage(struct page *p, struct writeback_control *wbc) { return 0; diff --git a/mm/swap_state.c b/mm/swap_state.c index ff8a166603d0..eef66757c615 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -956,6 +956,44 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, return page; } +struct page *swapin_page_non_fault(swp_entry_t entry, gfp_t gfp_mask, + struct mempolicy *mpol, pgoff_t ilx, + struct mm_struct *mm, enum swap_cache_result *result) +{ + enum swap_cache_result cache_result; + struct swap_info_struct *si; + void *shadow = NULL; + struct folio *folio; + struct page *page; + + /* Prevent swapoff from happening to us */ + si = get_swap_device(entry); + if (unlikely(!si)) + return ERR_PTR(-EBUSY); + + folio = swap_cache_get_folio(entry, NULL, &shadow); + if (folio) { + page = folio_file_page(folio, swp_offset(entry)); + cache_result = SWAP_CACHE_HIT; + goto done; + } + + if (swap_use_no_readahead(si, swp_offset(entry))) { + page = swapin_no_readahead(entry, gfp_mask, mpol, ilx, mm); + if (shadow) + workingset_refault(page_folio(page), shadow); + cache_result = SWAP_CACHE_BYPASS; + } else { + page = swap_cluster_readahead(entry, gfp_mask, mpol, ilx); + cache_result = SWAP_CACHE_MISS; + } +done: + put_swap_device(si); + if (result) + *result = cache_result; + return page; +} + #ifdef CONFIG_SYSFS static ssize_t vma_ra_enabled_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) diff --git a/mm/swapfile.c b/mm/swapfile.c index 925ad92486a4..f8c5096fe0f0 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1822,20 +1822,15 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, si = swap_info[type]; do { + int ret; + pte_t ptent; + pgoff_t ilx; + swp_entry_t entry; struct page *page; unsigned long offset; + struct mempolicy *mpol; unsigned char swp_count; struct folio *folio = NULL; - swp_entry_t entry; - int ret; - pte_t ptent; - - struct vm_fault vmf = { - .vma = vma, - .address = addr, - .real_address = addr, - .pmd = pmd, - }; if (!pte++) { pte = pte_offset_map(pmd, addr); @@ -1855,8 +1850,12 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, offset = swp_offset(entry); pte_unmap(pte); pte = NULL; - page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, - &vmf, NULL); + + mpol = get_vma_policy(vma, addr, 0, &ilx); + page = swapin_page_non_fault(entry, GFP_HIGHUSER_MOVABLE, + mpol, ilx, vma->vm_mm, NULL); + mpol_cond_put(mpol); + if (IS_ERR(page)) return PTR_ERR(page); else if (page) From patchwork Sun Nov 19 19:47:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166874 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1810247vqn; Sun, 19 Nov 2023 11:50:40 -0800 (PST) X-Google-Smtp-Source: AGHT+IF2urK0MFxuqY6kP50ily29RYtB2LdTx0KXuSgJc77T2sy5IWIzrqHbKa/tr5AWegZvqdG0 X-Received: by 2002:a17:902:ec8f:b0:1cc:54fb:60f9 with SMTP id x15-20020a170902ec8f00b001cc54fb60f9mr7054269plg.37.1700423440210; Sun, 19 Nov 2023 11:50:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423440; cv=none; d=google.com; s=arc-20160816; b=NxJF5sesMVsSvIK9O6FIRl5B7uKkGIrKZcIEFooXRjOTgs4EWaBh4mVTT5bBnI/VR2 z4VRQv50QRDW9M/KjUZuNhnUUinS98MpKf9DAxjgNsrLm5EfqBn9vNcGIs+efrOWotxL Y49v3bcA28Pi1tqvgrfOSOhwQ/Lb4LWfKSk8da5/zHIrDeHNam3jK+wlvSNBOS4Zecef 32rslR8ZqQKY6cvy5MaW2meWFOCWLeo7hJbGaapSV9A3rusR9w889g/eIGV40+mCosao SRxghFbMxA1ud1S31G7zzt6KL9DDE8MBFlH2/oLulHD63dGG+YCbPvYp5pGR//p9ceHR Tmpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=CM+OPz8sAX9/+pckE36Q2lCrK7XvSIfXWi261rZELtQ=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=mBppBNjSJR6mEU8r8kjPlSGF3LZaRY+nSl3Q/x/CYA6iHzeB8rWCUBGah6fZu4B17p x6HFJmxLYRAooBGWSbkq5C1KE0uYTpy+fhwT/Ok8eKTQwCNwTq4EF69wcb3dQ/DO4V8p YcKStkF+xRTJc0gb2A3YR2ZIo16Q+srG6K4BAJ3Hgah+FM77xWkklO5Q06oTqv4ga4Xj +NJIxYEGqwMWvDdh5AkEFrFO3SGhZDQNZHdefZxHgmAtbiUx/2RFnFFho0K08NjxM8PU PPH3O1rdlyOXQFlN/eu1W0vmyssL2u8Pb3WbKxsp3Y+u0nGI+2eI6lJOAZ0ikwf9i981 1Xig== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=Afvt9npR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id j8-20020a170902690800b001c9e890f42bsi6535131plk.78.2023.11.19.11.50.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:50:40 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=Afvt9npR; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id D98FD80C1139; Sun, 19 Nov 2023 11:50:38 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231795AbjKSTuU (ORCPT + 29 others); Sun, 19 Nov 2023 14:50:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59026 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231665AbjKSTtv (ORCPT ); Sun, 19 Nov 2023 14:49:51 -0500 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E7DA91BD9 for ; Sun, 19 Nov 2023 11:49:03 -0800 (PST) Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-6b1d1099a84so3606091b3a.1 for ; Sun, 19 Nov 2023 11:49:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423343; x=1701028143; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=CM+OPz8sAX9/+pckE36Q2lCrK7XvSIfXWi261rZELtQ=; b=Afvt9npRwp95Im/FJLQ4+yLe4XtjOwRQlS5Bcenqfiu6s4zOVDNDC9TnLd2LDWhRkG Lf2Rk1xnJAxJCu05N3e5x7+kA52CyzE4QJvMnOej7jgkD7/TrKSYRxbeOZEiENa+1ffx vaITqRi5dxBNDB6InBBqOjlYYUHzoHuLXQxS3uUm9Mm/TnDtbYFej+7QIU3OEdCuKRKS UQG2QXnrODblRWZXfWxIhATw7KrBUVCu+krtYU7WYC5sGIqFPI8CE/QBuZK7Otg+aMeK w5ww+J45bGxSYsmDtMx2bO7Ot7hUiqUc+FTLPFYeQUCZv/sjEXwEGjoAB6Y3u1k5xelF gCQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423343; x=1701028143; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=CM+OPz8sAX9/+pckE36Q2lCrK7XvSIfXWi261rZELtQ=; b=hcr1MX/sniu/lkyTjcNc2ScEKn2CY5CQTz1Ha4oLyM/fySwiB1asq7RWacA4ri1yGq yYWiMt2khDWNXvL7kHkj4a4XQhxLEHFfG3UPhnyEPOw+V5WWmrdv/+TfOI1ABPkIHtjA G4TRwR/O1l6oV1wkAjPvhA9s2gIi539cbYqPDmacaUcRNp3yzLHRSidzj0rl7W6IE5Cl Ik1IsK3XR7L1IJSuNqBsfKqhA9TCX70A05aYe12ZvzQ19bW0gLbWB19K6x4RzwH1RURD dMfR6Gk3wi2LFpJClvYimhmK/8VCpvt/CE/JuCl//8SJgZdmMl8C2wkfK/BGA9CLojbl mhtw== X-Gm-Message-State: AOJu0YxMWPGOpR9DEg3WP4XUy3GEhnANX0smxKd1gSjuHkt7Mdc9XQ/G nu7Rxnjvxm3tbMyUpkjapR4= X-Received: by 2002:a05:6a20:748e:b0:186:58d6:ca65 with SMTP id p14-20020a056a20748e00b0018658d6ca65mr7241893pzd.32.1700423343119; Sun, 19 Nov 2023 11:49:03 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.49.00 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:49:02 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 19/24] shmem, swap: refactor error check on OOM or race Date: Mon, 20 Nov 2023 03:47:35 +0800 Message-ID: <20231119194740.94101-20-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:50:38 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023209009242930 X-GMAIL-MSGID: 1783023209009242930 From: Kairui Song It should always check if a swap entry is already swaped in on error, fix potential false error issue. Signed-off-by: Kairui Song --- mm/shmem.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 81d129aa66d1..6154b5b68b8f 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1857,13 +1857,11 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, page = swapin_page_non_fault(swap, gfp, mpol, ilx, fault_mm, &result); mpol_cond_put(mpol); - if (PTR_ERR(page) == -EBUSY) { - if (!shmem_confirm_swap(mapping, index, swap)) - return -EEXIST; + if (IS_ERR_OR_NULL(page)) { + if (!page) + error = -ENOMEM; else - return -EINVAL; - } else if (!page) { - error = -ENOMEM; + error = -EINVAL; goto failed; } else { folio = page_folio(page); From patchwork Sun Nov 19 19:47:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166881 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1810509vqn; Sun, 19 Nov 2023 11:51:36 -0800 (PST) X-Google-Smtp-Source: AGHT+IFruHut2RnfjLrrL/+OYbNnH7zxzwJxZxp8TnfxSg7/AL8J/xF+n91Bl3t9z+p267JR8HVx X-Received: by 2002:a05:6808:1704:b0:3b6:d617:a6f7 with SMTP id bc4-20020a056808170400b003b6d617a6f7mr8354927oib.3.1700423496795; Sun, 19 Nov 2023 11:51:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423496; cv=none; d=google.com; s=arc-20160816; b=KIeaVMqzWvyDdjpWOfTAFQIZ9qqBttGOHBH0MZLhzZG7SZcPUIQMVla4VwLNIPtrZ0 vvYL/r9BmbmpZ5s+m3kbMsqZbxoXD6RDIfaAruHZD4y17E+j4/DnXA2JW8C+QkiwiX3Z e2cDcEUoDoF5M0KgnVHBQRK7NIJPjcxzKIKmsFVcA1I2eow2hr6QWopCQnaXFIpZM+HU EmHaDX50HbCYA7uM6DOPH9GQlxR0cOIcde/SrKSEf1amtqWkwx//JNHV8LPgpxPZBzmN mFcco88r8vuUYnW7ysCk7m2AMYs3uAEzENTGGxnqZPqYdPyT/A5wBOnq1UXB0ucvS/5z 50IA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Ze1Psd2Ie5OovBPk7SmUTsWgEFqDeyPvhXbgqpZhXY8=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=EABVnFcK8Il3ES5HfwSqvJlGKn50V9laWBclB8kpsVTITL8I8AYSShxeH0P/ysXRsr pQbl7LP53vVONqa+O3ICFth5qq+3Burt9bcYROcSCf9aJTdXE8NHzfZzlq5SH7GWUh1F YeqIzklTWCONfnlJlZaz5HiEfXmGz7HgNHM1QXs126qFYN+ZgYuLNTpdDDgsId5HXK/E rR7DmvPRDk63vhztzOOLBPIqaAH4UO1qjwjvIlU2CeKmlUNPGHjhpu+hrP4QwGUn8zW/ ppJsGf9Zgo4LHMJOj/hQouaODe5r7Lxxe4VvmCHEc+iZmEt+HYhZgn9FAVksOcLkERIB IXBQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=N1NnsxV1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from fry.vger.email (fry.vger.email. [23.128.96.38]) by mx.google.com with ESMTPS id c7-20020a631c47000000b005ad8009e304si6699104pgm.784.2023.11.19.11.51.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:51:36 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=N1NnsxV1; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id 39E34809583D; Sun, 19 Nov 2023 11:51:24 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231617AbjKSTuI (ORCPT + 29 others); Sun, 19 Nov 2023 14:50:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231633AbjKSTtr (ORCPT ); Sun, 19 Nov 2023 14:49:47 -0500 Received: from mail-oi1-x22a.google.com (mail-oi1-x22a.google.com [IPv6:2607:f8b0:4864:20::22a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3A3EDD7A for ; Sun, 19 Nov 2023 11:49:07 -0800 (PST) Received: by mail-oi1-x22a.google.com with SMTP id 5614622812f47-3b6d88dbaa3so2548183b6e.1 for ; Sun, 19 Nov 2023 11:49:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423346; x=1701028146; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=Ze1Psd2Ie5OovBPk7SmUTsWgEFqDeyPvhXbgqpZhXY8=; b=N1NnsxV11ewog1K925/BlU5b9EhM0d5Pcde3SG+GzZSzkIjxMU7Mw8+uWuK+BkbY84 TyBrUGD4XlOXC2ZI6u14MuK/fTa33hbbdjdSLr9Ua+hsipTZgnm0Pb4JHrlW1dhrFv1t cp5cLbIvJto4kuOJ0CDqftFzZfT6vJEOxRq+YMd6So8r8qnK9RSf31fk6LV5USdIp7OY ohyenVI7/+mufGpLAUORUEQF8/sWaX6CZIqrRTTLPXdccE+l8KtO45RuoOPlGvq7fX7D wD7CB628QZWTED2q2ryqxvbVKdm0Qto+MEthPwassMAedVrSec3IOYMF9Do+KcJfzWfu VJCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423346; x=1701028146; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Ze1Psd2Ie5OovBPk7SmUTsWgEFqDeyPvhXbgqpZhXY8=; b=AQ5aDUtZXCbzp+RRnLZdJxiS6PDBSFXcBlFTcwZPqAW5Tk7s/NFkxswD8f9YcQHEvw lONTDkHayQLYyF1IW4cFeCSiPWZ8CQusK4hTRLcgVFVZtnikE3BC+t1FKrzsxZUrBk0K PqerYId5RR/j83evVbmSJYnLSftA/PPYjEZ/VuQ8jK6LFXk/QytbFjmVQBI9Q8yM7tN/ 1yswGNfjADjUdPtj+WX9Mjfj3/x+urZJnZa3N3hheipMfCKvkNQkVFMbXAYpzduzUvPo VGEMUM641U+cE5I0fZ85NxYvBX9Mt1nbDjLZl0Dm0qfXbmBuH5nZGFFP8ZyIPFRHvPa+ jSbg== X-Gm-Message-State: AOJu0YzfX7EIrjn4sG/aqytEO53YzLyGm0rp9Js73RdLRUH/mRyF/zhV sVMYRoCfStjyysmADIkKWN8= X-Received: by 2002:a05:6808:1451:b0:3b2:dcff:9e54 with SMTP id x17-20020a056808145100b003b2dcff9e54mr8319167oiv.24.1700423346359; Sun, 19 Nov 2023 11:49:06 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.49.03 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:49:05 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 20/24] swap: simplify and make swap_find_cache static Date: Mon, 20 Nov 2023 03:47:36 +0800 Message-ID: <20231119194740.94101-21-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:51:24 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023268743991233 X-GMAIL-MSGID: 1783023268743991233 From: Kairui Song There are only two callers within the same file now, make it static and drop the redundant if check on the shadow variable. Signed-off-by: Kairui Song --- mm/swap.h | 2 -- mm/swap_state.c | 5 ++--- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/mm/swap.h b/mm/swap.h index b073c29c9790..4402970547e7 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -46,8 +46,6 @@ void __delete_from_swap_cache(struct folio *folio, void delete_from_swap_cache(struct folio *folio); void clear_shadow_from_swap_cache(int type, unsigned long begin, unsigned long end); -struct folio *swap_cache_get_folio(swp_entry_t entry, - struct vm_fault *vmf, void **shadowp); struct folio *filemap_get_incore_folio(struct address_space *mapping, pgoff_t index); diff --git a/mm/swap_state.c b/mm/swap_state.c index eef66757c615..6f39aa8394f1 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -334,15 +334,14 @@ static inline bool swap_use_vma_readahead(struct swap_info_struct *si) * * Caller must lock the swap device or hold a reference to keep it valid. */ -struct folio *swap_cache_get_folio(swp_entry_t entry, struct vm_fault *vmf, void **shadowp) +static struct folio *swap_cache_get_folio(swp_entry_t entry, struct vm_fault *vmf, void **shadowp) { bool vma_ra, readahead; struct folio *folio; folio = filemap_get_entry(swap_address_space(entry), swp_offset(entry)); if (xa_is_value(folio)) { - if (shadowp) - *shadowp = folio; + *shadowp = folio; return NULL; } From patchwork Sun Nov 19 19:47:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166873 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1810195vqn; Sun, 19 Nov 2023 11:50:31 -0800 (PST) X-Google-Smtp-Source: AGHT+IGOomIzgKgzoBw+iKw77J2FeYrHzvSg3gmNpp7pnIN3uHGRYXQi9TP7ZyCVOkZJhQbnU2P0 X-Received: by 2002:a17:903:1108:b0:1cc:3bd3:73d8 with SMTP id n8-20020a170903110800b001cc3bd373d8mr6208541plh.59.1700423431228; Sun, 19 Nov 2023 11:50:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423431; cv=none; d=google.com; s=arc-20160816; b=O3on4ErSZnRSPTNZENnPTg+R1U5V9vFPo8gwA1TF7rmsrY/hn2HxMC9X/llY/UyxDu QN34q4DRvZ8CT+yodDS3GCl5t2ZIYO13XBnwoZzxKA5SpJaj/sgPGJbbILZ8wujqpDth gfP5BEahp6OKMDFk9ZRzXuWKr3jLJRPVzUsbAHyTgolPor4yqi2fG4kUT2RrP3KPZTVy P3Q6NUETJ6QIxQdtyN0BEOFs2cQybHE1heIfPIJO5BJfq2nLSjVNOpv9gZfUFB/QOt3Z nf+S9nirFrCxXa0Hj8lTN7wkKd53jnoaFpgHZLaZ7+MzmXUJeHpFteKphpsRSTNGY9Yh X/LQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=w2h0VsFq1SfFmbRfFbDLLmtclse5HYqoYk4CcDA6qDI=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=mVVNRmDOX+T1iRwgq+iM/f+rB0WCeyJDK+24e0peDnal/mjsFtjVUvgMgK4sFnflj4 tjleKjkIp/zxJRRgGpLRc7FW/kjiOVRVJgkgD4fTKY4Ln1FxahPIX0jrqAQLNk9iMlxX xz7TJEGPkbjz97E4yteFVNlEISqZzVlSw67Dr0NvJmN8Eg4T4sf9w3+0AN0ZAQS8KthZ x+ehA3siSXv1bZw7cGti6rVn6NBB6ZWGT+TZgEj7Gx/AFyqrZwoCmSJ9TjOScY91cFcd 4W7i0OfCiHgy55/NUv/Q0qlkdA4oWBhJZ/b0ZcjsnJBknh7Zxa4oQdIgn1e2CQFWLLxs OYnw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=kPVLQgwQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id i17-20020a170902c95100b001c3411c9b83si6852690pla.454.2023.11.19.11.50.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:50:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=kPVLQgwQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id A476E803E796; Sun, 19 Nov 2023 11:50:29 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231739AbjKSTuM (ORCPT + 29 others); Sun, 19 Nov 2023 14:50:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58926 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231748AbjKSTtt (ORCPT ); Sun, 19 Nov 2023 14:49:49 -0500 Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1671B1704 for ; Sun, 19 Nov 2023 11:49:10 -0800 (PST) Received: by mail-pf1-x42f.google.com with SMTP id d2e1a72fcca58-6c115026985so3870434b3a.1 for ; Sun, 19 Nov 2023 11:49:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423349; x=1701028149; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=w2h0VsFq1SfFmbRfFbDLLmtclse5HYqoYk4CcDA6qDI=; b=kPVLQgwQevJOFiEY0KJ3SPO+qeDa6dHv8o5dvT8mV5RwVnvACWUcAZ09RNYS17O0uJ F5rUlZSZYZJSP3McplnFbayZBp3zJi4oniSKsO9H8xecRURD72zLnK9uzZs31NqN+FrE LF3atFZV3hgZHgdhmuSJ/RuCscN2YzlkQSPcYOHSmqnb3XOY4MxFL9qBy51K9+1mQPoZ ck8UYcQqX568dEFMmZBWVs8elpC8wi4GDirZVqtBxZVK4E8o5R2ZNdLTL325oViQy4W9 F/gRx4taMjSmqYErjPpedjaO3X5zf6tTBo8pad0BnIk+u5a2JlxqT1YPZpkEs9kWg3oO DkqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423349; x=1701028149; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=w2h0VsFq1SfFmbRfFbDLLmtclse5HYqoYk4CcDA6qDI=; b=A7tSNiJiiEpGtL4wf3bUp04p93syIY+rRQWQ85OXb2xmPneZKkyvnfHYM5FJDbkquE CI8nWL3aQUeKRJ0mFVlGEhFZmjh2w9tyeP4s9CouR1MbVbNKSGw/qL40EDAjTsRCZ5SM UmmsBrck9jpj8k4NwP9LjEnfpA7bhCjb52Ub40P+EGJVcksgPS5qnKzc2oXMmrwyeEc/ OXZC/tQZ8TMXHJ9tf46ywF/bIrF9WdsdSE8QO5Ym0wJm7ik1FAn8t7m7KJsTjWRbTIqb z8nohzw1pKOkZO/Bgvz3Msl6mcL9elP6jIQtQ6i2dKhbrpv1aEXA5zSjnQUxhiMqaWLF lggg== X-Gm-Message-State: AOJu0YwroDZ4VCarqa6mw9jY2e/KLe3DLC7JbleeCYdZueQkAEC3fOPZ aITERZ1MYRnVW15rOoI5zq0= X-Received: by 2002:a05:6a00:1988:b0:6cb:a1fe:5217 with SMTP id d8-20020a056a00198800b006cba1fe5217mr787020pfl.16.1700423349552; Sun, 19 Nov 2023 11:49:09 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.49.06 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:49:08 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 21/24] swap: make swapin_readahead result checking argument mandatory Date: Mon, 20 Nov 2023 03:47:37 +0800 Message-ID: <20231119194740.94101-22-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:50:29 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023199821762830 X-GMAIL-MSGID: 1783023199821762830 From: Kairui Song This is only one caller now in page fault path, make the result return argument mandatory. Signed-off-by: Kairui Song --- mm/swap_state.c | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/mm/swap_state.c b/mm/swap_state.c index 6f39aa8394f1..0433a2586c6d 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -913,7 +913,6 @@ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, struct vm_fault *vmf, enum swap_cache_result *result) { - enum swap_cache_result cache_result; struct swap_info_struct *si; struct mempolicy *mpol; void *shadow = NULL; @@ -928,29 +927,27 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, folio = swap_cache_get_folio(entry, vmf, &shadow); if (folio) { + *result = SWAP_CACHE_HIT; page = folio_file_page(folio, swp_offset(entry)); - cache_result = SWAP_CACHE_HIT; goto done; } mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); if (swap_use_no_readahead(si, swp_offset(entry))) { + *result = SWAP_CACHE_BYPASS; page = swapin_no_readahead(entry, gfp_mask, mpol, ilx, vmf->vma->vm_mm); - cache_result = SWAP_CACHE_BYPASS; if (shadow) workingset_refault(page_folio(page), shadow); - } else if (swap_use_vma_readahead(si)) { - page = swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf); - cache_result = SWAP_CACHE_MISS; } else { - page = swap_cluster_readahead(entry, gfp_mask, mpol, ilx); - cache_result = SWAP_CACHE_MISS; + *result = SWAP_CACHE_MISS; + if (swap_use_vma_readahead(si)) + page = swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf); + else + page = swap_cluster_readahead(entry, gfp_mask, mpol, ilx); } mpol_cond_put(mpol); done: put_swap_device(si); - if (result) - *result = cache_result; return page; } From patchwork Sun Nov 19 19:47:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166875 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1810276vqn; Sun, 19 Nov 2023 11:50:46 -0800 (PST) X-Google-Smtp-Source: AGHT+IGo95CYm+MxlfZM5AnHupl9wzlZ1fTmrlPlCC4aQrJxJepGE/tCVP/Vm/heTIyJzCRu2HxF X-Received: by 2002:a05:6808:211c:b0:3b5:656b:f962 with SMTP id r28-20020a056808211c00b003b5656bf962mr8654230oiw.5.1700423446174; Sun, 19 Nov 2023 11:50:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423446; cv=none; d=google.com; s=arc-20160816; b=e8Qsb5irFICjGFiISDJm2HKx8KAjoXd65O+2SoGJxEfREMZhDuBsEympLN6F7jF5mv a3r+tBSSUBH85UfkZzaL/MsbkyTRIH9GjCWI87nWVguOAqMTodTTWQ3QAh7P7W7w95Zb /8Jt5a23kVCIknruAYqrbohOrfRfakOG+P8JIaaJEcGaAyKC1iQVzTVsVWL/ndgLh3PE kkJt9i+3E6+Zfzgmh7HHKhvFceuLt+kNjX7JPC0rPh8P4ArkAmbNjozQRlM+RAi68IOv UOujRE0ye2LtcZaeteYvhhbdZU9jXvNb9wRxJPoqj/ErXLI10UjbcL4fR16GcehtZ7a5 BFeQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=YaE/AcfmvaBX4nhUPBOl45o2uIqec3EidHpbbgh18/A=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=XhV8xWANgfh7LFeCRij57W8WcMwarfaJmfVVesoSihQCvZ7PFMVetLieP8DoMhsFdM mnTadpYKaOrQdmA8KRAHWZQfOcXkhnD4tV9NBtXBFm9RnZ7iA/R6vPT/BcilS3/Ykhjy 4kugTATS6VL+KnZBgUZdm+szVsqyrERmskfQH45FydSTzNkDIlmQ96TjeOgyFfbw37pr xGslHFpAiqy7aFtT7LZlKu/tA3j4XjC8iFcBAUGbU7GAwrW+fruoM3Vdts42248VROZs 7PY6Tw30oRwk0hTcVDcQ5V4pqpB0UEMx2nxX+p2U8v9FTPgVc9UGYr43egrFopn2YzL3 MxGA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=GMgAGHAG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id e32-20020a631e20000000b005bde6fae09asi6293707pge.135.2023.11.19.11.50.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:50:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=GMgAGHAG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 8377280C8D4B; Sun, 19 Nov 2023 11:50:45 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231834AbjKSTuY (ORCPT + 29 others); Sun, 19 Nov 2023 14:50:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59094 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231800AbjKSTt4 (ORCPT ); Sun, 19 Nov 2023 14:49:56 -0500 Received: from mail-io1-xd29.google.com (mail-io1-xd29.google.com [IPv6:2607:f8b0:4864:20::d29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BB8EE1FC3 for ; Sun, 19 Nov 2023 11:49:13 -0800 (PST) Received: by mail-io1-xd29.google.com with SMTP id ca18e2360f4ac-7a6774da682so175596139f.3 for ; Sun, 19 Nov 2023 11:49:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423352; x=1701028152; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=YaE/AcfmvaBX4nhUPBOl45o2uIqec3EidHpbbgh18/A=; b=GMgAGHAGjIC3Ddy0feaX8VCRlLMutjNTDBX5rAcWdVC/FdX7vDfmOhxlW98XyoqPxM Mgw8BRbJ+gnw6un1cyxgndjsRWrHGR22bFefYaRmUIIIZBKYCO5HkcUcZN/mS3xmH/u2 TVHnUCPjFhKhfdy4CUfOhSTO7r0KOV7fAjbIbR/gkU+/OU2V1EGcCgnx+/MDwjL2azmD pqlxjlMOoIZnN9mcj9iOsXEWo21ezWM08nYbHt2iUnVCtJd5odLzC2wiRo27iyGVH2mc xH85SpjFZtJqOzFdAuMf1R0mDN7hE/LB0iYvnLWE1hZcwZ0pIRbp7G5/zGDuNCYrz/nr KJHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423352; x=1701028152; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=YaE/AcfmvaBX4nhUPBOl45o2uIqec3EidHpbbgh18/A=; b=MkREllb1Kik/odG9IqVmVxUSa1FOjqfID1SnORrScy44i4Xek/3RiGwFfJS0akUj0L 7bQ8cuPycHr3Dwva8vBLOSRJUftCuAsAiaBfPiN0nLWaavfpPauuPK0W/X+SRwIFFEYy 4cna5UJ7UlqwRig2iKk9Z6JbAS5bTJhFkib7ww0TCFPdBzRIMHv4cuQ+orfJK9Vo/1F8 TltlHM9RIt1+ff8yWX10E06CZny8nVnXL0NDHIBKVgiUqQXOTvJ+LbEUS3PGLRPsKemj YdNMXqCUnlqKOG9hUhRLSGUIsaYKWuY+wObJkWfpA2k+ejaRiMAEQgfr1vkcXElmRg3S T2Wg== X-Gm-Message-State: AOJu0YzweVJ338LxHOPIjB0HxjksYt7n6B5jcp172vhrL42BaCNoHdIR nqIsX7XZpjV9Mz7gShRarPc= X-Received: by 2002:a05:6e02:2607:b0:35a:fae1:1358 with SMTP id by7-20020a056e02260700b0035afae11358mr5301806ilb.12.1700423352719; Sun, 19 Nov 2023 11:49:12 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.49.09 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:49:12 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 22/24] swap: make swap_cluster_readahead static Date: Mon, 20 Nov 2023 03:47:38 +0800 Message-ID: <20231119194740.94101-23-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:50:45 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023215739573052 X-GMAIL-MSGID: 1783023215739573052 From: Kairui Song Now there is no caller outside the same file, make it static. Signed-off-by: Kairui Song --- mm/swap.h | 8 -------- mm/swap_state.c | 4 ++-- 2 files changed, 2 insertions(+), 10 deletions(-) diff --git a/mm/swap.h b/mm/swap.h index 4402970547e7..795a25df87da 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -56,8 +56,6 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct mempolicy *mpol, pgoff_t ilx, bool *new_page_allocated); -struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t flag, - struct mempolicy *mpol, pgoff_t ilx); struct page *swapin_readahead(swp_entry_t entry, gfp_t flag, struct vm_fault *vmf, enum swap_cache_result *result); struct page *swapin_page_non_fault(swp_entry_t entry, gfp_t gfp_mask, @@ -93,12 +91,6 @@ static inline void show_swap_cache_info(void) { } -static inline struct page *swap_cluster_readahead(swp_entry_t entry, - gfp_t gfp_mask, struct mempolicy *mpol, pgoff_t ilx) -{ - return NULL; -} - static inline struct page *swapin_readahead(swp_entry_t swp, gfp_t gfp_mask, struct vm_fault *vmf, enum swap_cache_result *result) { diff --git a/mm/swap_state.c b/mm/swap_state.c index 0433a2586c6d..b377e55cb850 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -627,8 +627,8 @@ static unsigned long swapin_nr_pages(unsigned long offset) * are used for every page of the readahead: neighbouring pages on swap * are fairly likely to have been swapped out from the same node. */ -struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t gfp_mask, - struct mempolicy *mpol, pgoff_t ilx) +static struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t gfp_mask, + struct mempolicy *mpol, pgoff_t ilx) { struct page *page; unsigned long entry_offset = swp_offset(entry); From patchwork Sun Nov 19 19:47:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166876 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1810294vqn; Sun, 19 Nov 2023 11:50:49 -0800 (PST) X-Google-Smtp-Source: AGHT+IE7oIk6ArIPw4qm4mV/GT5yYTqQmkF7nPBH+PR6fNXDBi/y4woPE6qBGCePjNzcC7skReMi X-Received: by 2002:a17:902:a5c9:b0:1cc:c273:603 with SMTP id t9-20020a170902a5c900b001ccc2730603mr3579957plq.42.1700423448784; Sun, 19 Nov 2023 11:50:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423448; cv=none; d=google.com; s=arc-20160816; b=ApbY9k+tjxqmlFu9DFHMRNhe3ky+VTdRXYUll3sClOyYOVuvueKhWXRwuhY5CAbPnp yMhKaycRnbf9KmDROc0u+MwQm6TUdMU3j2QSr0cnyVOImom/AjqWmkxr92ybaLDiir0u dHCGRvEcNTwQGDXozlF7EImXspbJhvUmCHpDiV4kFdkK67uwamMZkH8Q1BESrv/Hbc+x HNT9FK/Eg4vfnW57f7OQyXVddg9qdczDLpkmwT8Zq6XDjR9+9qKif44HMMF7hIE4gjqe np7WuYMjNalriefYMfmh1wDCYryaq656lsejvuZJYdToWHt5S6MuA+mW3hbGEh4qx0e2 MdXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=VWrBT21AdvUolArI+Vamx/Z+QXIBnOosXdhjX6fE5gQ=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=keH8kpXzlWloY4ih1ihXB8VYt3subJrvIlMvCVmYCSSRAP7gErGs+N/GiD9sSiyKqs FGVVLUyXtiM9zMJXJITA89nmJZjbBSPQrclLb/qiP+I/ClMd/TJklGS3kLWUuNLUBoXp w4AVUnk7pqG5j4VrHa6qpPqBjOESpO9/u+9zim+BUBq6GnwiYCYauD2TimNs7UxagEzv zrVfxM7F8+z7q+MEbkDYs/GeQmgpOTuVffEdiv2SPWfcGA/EyPcWy0ZeSuzdIjwpFkJv FjNP6bwMyTjymhykC2liTi5+vuE3IUfKJnyZJKM9LGgr96va6k1nEd4fUnszb4VJt66e a/4g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=llCCsues; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id z7-20020a170903018700b001cf56154e9bsi2366613plg.247.2023.11.19.11.50.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:50:48 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=llCCsues; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id B8E2B80B01C5; Sun, 19 Nov 2023 11:50:47 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231852AbjKSTub (ORCPT + 29 others); Sun, 19 Nov 2023 14:50:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60970 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231874AbjKSTuC (ORCPT ); Sun, 19 Nov 2023 14:50:02 -0500 Received: from mail-pf1-x435.google.com (mail-pf1-x435.google.com [IPv6:2607:f8b0:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9513710C0 for ; Sun, 19 Nov 2023 11:49:16 -0800 (PST) Received: by mail-pf1-x435.google.com with SMTP id d2e1a72fcca58-6b709048f32so3117372b3a.0 for ; Sun, 19 Nov 2023 11:49:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423356; x=1701028156; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=VWrBT21AdvUolArI+Vamx/Z+QXIBnOosXdhjX6fE5gQ=; b=llCCsuesSJyPuyRFTLv8zFEmg274BkyFCpdLGpGF3QZ/aVnQaB+b6VShm6BQkURq0r 0yZIb1FpJ6Vd5DSIo+ipxGVOQ7j5r4vTauzyN0Kkmzt8Ex0ited9OJ9b3Wf5IzmgxJ8Q nmRE88aq5dMmiOLORI01TSSnszaSgN7HKmSX9UQrfB8GvMLuhIckKTVUgwllptiawseb omx+/XZX5dzi0b+dmJjTDxnUqr6TNyKCggrpovEC09puWo54Ce6cjptMVeOded2FrN/k iXlXAKtRy2SqCCpVpIT+TvpHzO6Qa+ARpxdNyo1IuKaZVUzpuRVII0X5eOZm0I9WxCw9 J8zg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423356; x=1701028156; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=VWrBT21AdvUolArI+Vamx/Z+QXIBnOosXdhjX6fE5gQ=; b=j98dgkOL6X7iHe1yf52yqm8iId3Z/rp+zwTfhHlo6/J19UdTH0/pq3Jkhbp73znLV/ KrTgnsCRhN9lZRqjImmp0ezkik7jS/Zv5RDWya6lFkOaPoceGQaQDx4PTPzEgY1QgWT7 2zCkx95YWa/LMngZjpG8CvT7JCgnLPj2CvmYeKwWVyHSuU5smZW4kB3Bvnel0wiDXsxS cU/VXKZ4BKlZNRtfN1VRS0fqBS+ngvVcXaar3CtwU3+ySxmZT01GktH4RIkGi6Sjtx6c v/Kic7uLFxwA7WkSzSBJxbEPNMzH64P0DLOi2j4NJW+f5giKCN4IuPBn4xKe7O59SfD4 GlnA== X-Gm-Message-State: AOJu0Ywbu2at4Pgg60ZhDwbMUJzq7gRqV7LbXVsFXPQLJpiqZplpvTQ2 Q0gLE++NEAOo74EY5ECSwOc= X-Received: by 2002:a05:6a00:3492:b0:6cb:a0dc:3d56 with SMTP id cp18-20020a056a00349200b006cba0dc3d56mr660136pfb.0.1700423355969; Sun, 19 Nov 2023 11:49:15 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.49.13 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:49:15 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 23/24] swap: fix multiple swap leak when after cgroup migrate Date: Mon, 20 Nov 2023 03:47:39 +0800 Message-ID: <20231119194740.94101-24-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:50:47 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023218390152787 X-GMAIL-MSGID: 1783023218390152787 From: Kairui Song When a process which previously swapped some memory was moved to another cgroup, and the cgroup it previous in is dead, then swapped in pages will be leaked into rootcg. Previous commits fixed the bug for no readahead path, this commit fix the same issue for readahead path. This can be easily reproduced by: - Setup a SSD or HDD swap. - Create memory cgroup A, B and C. - Spawn process P1 in cgroup A and make it swap out some pages. - Move process P1 to memory cgroup B. - Destroy cgroup A. - Do a swapoff in cgroup C - Swapped in pages is accounted into cgroup C. This patch will fix it make the swapped in pages accounted in cgroup B. Signed-off-by: Kairui Song --- mm/swap.h | 2 +- mm/swap_state.c | 19 ++++++++++--------- mm/zswap.c | 2 +- 3 files changed, 12 insertions(+), 11 deletions(-) diff --git a/mm/swap.h b/mm/swap.h index 795a25df87da..4374bf11ca41 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -55,7 +55,7 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct swap_iocb **plug); struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct mempolicy *mpol, pgoff_t ilx, - bool *new_page_allocated); + struct mm_struct *mm, bool *new_page_allocated); struct page *swapin_readahead(swp_entry_t entry, gfp_t flag, struct vm_fault *vmf, enum swap_cache_result *result); struct page *swapin_page_non_fault(swp_entry_t entry, gfp_t gfp_mask, diff --git a/mm/swap_state.c b/mm/swap_state.c index b377e55cb850..362a6f674b36 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -416,7 +416,7 @@ struct folio *filemap_get_incore_folio(struct address_space *mapping, struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct mempolicy *mpol, pgoff_t ilx, - bool *new_page_allocated) + struct mm_struct *mm, bool *new_page_allocated) { struct swap_info_struct *si; struct folio *folio; @@ -462,7 +462,7 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, mpol, ilx, numa_node_id()); if (!folio) goto fail_put_swap; - if (mem_cgroup_swapin_charge_folio(folio, NULL, gfp_mask, entry)) + if (mem_cgroup_swapin_charge_folio(folio, mm, gfp_mask, entry)) goto fail_put_folio; /* @@ -540,7 +540,7 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, mpol = get_vma_policy(vma, addr, 0, &ilx); page = __read_swap_cache_async(entry, gfp_mask, mpol, ilx, - &page_allocated); + vma->vm_mm, &page_allocated); mpol_cond_put(mpol); if (page_allocated) @@ -628,7 +628,8 @@ static unsigned long swapin_nr_pages(unsigned long offset) * are fairly likely to have been swapped out from the same node. */ static struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t gfp_mask, - struct mempolicy *mpol, pgoff_t ilx) + struct mempolicy *mpol, pgoff_t ilx, + struct mm_struct *mm) { struct page *page; unsigned long entry_offset = swp_offset(entry); @@ -657,7 +658,7 @@ static struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t gfp_mask, /* Ok, do the async read-ahead now */ page = __read_swap_cache_async( swp_entry(swp_type(entry), offset), - gfp_mask, mpol, ilx, &page_allocated); + gfp_mask, mpol, ilx, mm, &page_allocated); if (!page) continue; if (page_allocated) { @@ -675,7 +676,7 @@ static struct page *swap_cluster_readahead(swp_entry_t entry, gfp_t gfp_mask, skip: /* The page was likely read above, so no need for plugging here */ page = __read_swap_cache_async(entry, gfp_mask, mpol, ilx, - &page_allocated); + mm, &page_allocated); if (unlikely(page_allocated)) swap_readpage(page, false, NULL); return page; @@ -830,7 +831,7 @@ static struct page *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask, pte_unmap(pte); pte = NULL; page = __read_swap_cache_async(entry, gfp_mask, mpol, ilx, - &page_allocated); + vmf->vma->vm_mm, &page_allocated); if (!page) continue; if (page_allocated) { @@ -850,7 +851,7 @@ static struct page *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask, skip: /* The page was likely read above, so no need for plugging here */ page = __read_swap_cache_async(targ_entry, gfp_mask, mpol, targ_ilx, - &page_allocated); + vmf->vma->vm_mm, &page_allocated); if (unlikely(page_allocated)) swap_readpage(page, false, NULL); return page; @@ -980,7 +981,7 @@ struct page *swapin_page_non_fault(swp_entry_t entry, gfp_t gfp_mask, workingset_refault(page_folio(page), shadow); cache_result = SWAP_CACHE_BYPASS; } else { - page = swap_cluster_readahead(entry, gfp_mask, mpol, ilx); + page = swap_cluster_readahead(entry, gfp_mask, mpol, ilx, mm); cache_result = SWAP_CACHE_MISS; } done: diff --git a/mm/zswap.c b/mm/zswap.c index 030cc137138f..e2712ff169b1 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1081,7 +1081,7 @@ static int zswap_writeback_entry(struct zswap_entry *entry, /* try to allocate swap cache page */ mpol = get_task_policy(current); page = __read_swap_cache_async(swpentry, GFP_KERNEL, mpol, - NO_INTERLEAVE_INDEX, &page_was_allocated); + NO_INTERLEAVE_INDEX, NULL, &page_was_allocated); if (!page) { ret = -ENOMEM; goto fail; From patchwork Sun Nov 19 19:47:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 166880 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9910:0:b0:403:3b70:6f57 with SMTP id i16csp1810496vqn; Sun, 19 Nov 2023 11:51:33 -0800 (PST) X-Google-Smtp-Source: AGHT+IECiGYvI/xtPVQYvtNPp5bdbRSLhXW7xizPwXbMzYiaMbEx+26lOhI0aOTn6sgx8b5gzTB1 X-Received: by 2002:a05:6871:5cd:b0:1f4:d347:df08 with SMTP id v13-20020a05687105cd00b001f4d347df08mr6675199oan.17.1700423493780; Sun, 19 Nov 2023 11:51:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1700423493; cv=none; d=google.com; s=arc-20160816; b=CbPadiWvHzbz3kcStyzyxGpzXpGmOfycjO6majTnE7TOp41mzNW349bMWR5JTv/nhh bWXeEoYtmS2IP/LXHOfLEMj/8Ap7qJi69JoozMFyIC0bqhKn7MVs3WT+O2V3vGTqZ98N 21hlD9XgqNNrROMVk2IapGDbWMYfCULJ52KtO1P0Xtkiq0vSJkpLMxE+ypt5BJiHjlXO DtKAwxlKPtDN0erTMNFuLkrerb5lacg88uVg1bs8Fzd5iR9uAVJbqhgGUTZJVldwT+r+ d49v95LUFCnB95Fk4+3oeHjW+PFrPIIiI22F91axqfaEEbRx48wrAoYZm6VpJlvgXZgb DPDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ujyhaBmVKqUCILaG0//cQCFqqP/+VsPiP59qcIbNEbE=; fh=4HE/piJoUCKuBTCCBiej4//zvvzywHdOLL9QM/KYjYM=; b=uG5UQC/WWuwndu9NcTmVVT+GHgzf6I1v2zVXrBpf+VHtx49Qr3df1atRfbpdEraX6v P3SxFoSv15fSII43h/ov+LrKxO2w9rGR0g8O4nmfr+XIAnD06jBEkUZwfNT20u1LEFrp NZ3J6XRaFUi7gv6UcOZk2pWENtvGOsSNF7dm8awyoPk/ljWYFpSllvHLOtIyjW8+NMQ5 fIM6j97bYAew1WzwnMqba21O/ZdvuRskz4YlxYPpuUxnzz7rRE1EykdnR8w6MmkTGzLN wg85Z68fbZ76RpAVNUwDUQz6MqViB5A8XR9O31hvpho4sGqEZZiUf5OQZnMusjMolyRN JPuw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=fX0Qw+da; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from groat.vger.email (groat.vger.email. [23.128.96.35]) by mx.google.com with ESMTPS id h1-20020a170902748100b001cc7c079304si6558190pll.97.2023.11.19.11.51.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Nov 2023 11:51:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=fX0Qw+da; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 9D5A38080D44; Sun, 19 Nov 2023 11:51:23 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231748AbjKSTuh (ORCPT + 29 others); Sun, 19 Nov 2023 14:50:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32838 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231919AbjKSTuF (ORCPT ); Sun, 19 Nov 2023 14:50:05 -0500 Received: from mail-oi1-x230.google.com (mail-oi1-x230.google.com [IPv6:2607:f8b0:4864:20::230]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DB17E171E for ; Sun, 19 Nov 2023 11:49:19 -0800 (PST) Received: by mail-oi1-x230.google.com with SMTP id 5614622812f47-3b2e4107f47so2778864b6e.2 for ; Sun, 19 Nov 2023 11:49:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700423359; x=1701028159; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=ujyhaBmVKqUCILaG0//cQCFqqP/+VsPiP59qcIbNEbE=; b=fX0Qw+daECB1rjTwNoau70a5cC0VHEPhHZPMPF3Y4iA51CSDOJ/i8EPffWP3fHqtDR Ws803bFEgujnXb4Qcmmc5kvfCykRnh8s9YfTLjenNd0xIUC6p3tMyTOIB4NEbu8VvBiM XQk08s+K22SPV6bSTWAry3AhQhVhBtjIPBy5hO0/1z/ixGYD5icy0PoTl73jNlU4bO94 wkwhj5UmkiGuyTlq5UcYnWF745ih4oBi7GRSNfHJ3xDJEbBBbcrttCmlFUx4jTV1mYDU rHDpRgtYBZ634H2Bq3IbzKPAC7yq6uvjIaMNUbl0S3tn1X3O2JEAO4aukOQ+52j++IGC 8xAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700423359; x=1701028159; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ujyhaBmVKqUCILaG0//cQCFqqP/+VsPiP59qcIbNEbE=; b=cxqKx6FtQlbcby+UVIL0PKP9YTvmaAQnrq6zO6ICJ7SST9tOMhFfKl//Mn1H7y0opW Lg6neE4lG80XVbhXOXTPbXmn0RVAvbiM3NAl5uE6IfecXMxoN6DVC6Vq97RibBjuv90k vmR5kGPa8YTkspP2z/jsZevegXmh1K7iqS/8wgqHB8vdR2uXuVH4YdGYJAFRjk0JAx0A LFfC/gkd10oeBMKhVGepO22P5ZfMaE50csm08ETgFq/DcuJJ7m3ydtYTrHyJOA8OX6lY eMWScE2lJyEK1c6OW7gc+iUF7iJ3KvsZIr/Xlt1IA7ZGOfuRKo3LCN0BnlqTINZRv2Sp HciQ== X-Gm-Message-State: AOJu0YzH1pIthriCU/JeyE2uX5r0YOGCCOMVj6yxv065ncJakHz1zZou ebZkGmVjz6gjY8FoU2NQuLpSl7WClKC2+01u X-Received: by 2002:a05:6808:2389:b0:3b5:75d3:14aa with SMTP id bp9-20020a056808238900b003b575d314aamr9092741oib.25.1700423359201; Sun, 19 Nov 2023 11:49:19 -0800 (PST) Received: from KASONG-MB2.tencent.com ([115.171.40.79]) by smtp.gmail.com with ESMTPSA id a6-20020aa78646000000b006cb7feae74fsm1237140pfo.164.2023.11.19.11.49.16 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 19 Nov 2023 11:49:18 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , "Huang, Ying" , David Hildenbrand , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH 24/24] mm/swap: change swapin_readahead to swapin_page_fault Date: Mon, 20 Nov 2023 03:47:40 +0800 Message-ID: <20231119194740.94101-25-ryncsn@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231119194740.94101-1-ryncsn@gmail.com> References: <20231119194740.94101-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Sun, 19 Nov 2023 11:51:23 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783023265488396788 X-GMAIL-MSGID: 1783023265488396788 From: Kairui Song Now swapin_readahead is only called from direct page fault path, so rename it and drop the gfp argument, since there is only one caller always using the same flag for userspace page fault. Signed-off-by: Kairui Song --- mm/memory.c | 4 ++-- mm/swap.h | 6 +++--- mm/swap_state.c | 15 +++++++++------ 3 files changed, 14 insertions(+), 11 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 620fa87557fd..4907a5b1b75b 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3844,8 +3844,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) goto out; } - page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, - vmf, &cache_result); + page = swapin_page_fault(entry, GFP_HIGHUSER_MOVABLE, + vmf, &cache_result); if (IS_ERR_OR_NULL(page)) { /* * Back out if somebody else faulted in this pte diff --git a/mm/swap.h b/mm/swap.h index 4374bf11ca41..2f8f8befff89 100644 --- a/mm/swap.h +++ b/mm/swap.h @@ -56,8 +56,8 @@ struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct mempolicy *mpol, pgoff_t ilx, struct mm_struct *mm, bool *new_page_allocated); -struct page *swapin_readahead(swp_entry_t entry, gfp_t flag, - struct vm_fault *vmf, enum swap_cache_result *result); +struct page *swapin_page_fault(swp_entry_t entry, gfp_t flag, + struct vm_fault *vmf, enum swap_cache_result *result); struct page *swapin_page_non_fault(swp_entry_t entry, gfp_t gfp_mask, struct mempolicy *mpol, pgoff_t ilx, struct mm_struct *mm, @@ -91,7 +91,7 @@ static inline void show_swap_cache_info(void) { } -static inline struct page *swapin_readahead(swp_entry_t swp, gfp_t gfp_mask, +static inline struct page *swapin_page_fault(swp_entry_t swp, gfp_t gfp_mask, struct vm_fault *vmf, enum swap_cache_result *result) { return NULL; diff --git a/mm/swap_state.c b/mm/swap_state.c index 362a6f674b36..2f51d2e64e59 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -899,7 +899,7 @@ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, } /** - * swapin_readahead - swap in pages in hope we need them soon + * swapin_page_fault - swap in a page from page fault context * @entry: swap entry of this memory * @gfp_mask: memory allocation flags * @vmf: fault information @@ -911,8 +911,8 @@ static struct page *swapin_no_readahead(swp_entry_t entry, gfp_t gfp_mask, * it will read ahead blocks by cluster-based(ie, physical disk based) * or vma-based(ie, virtual address based on faulty address) readahead. */ -struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, - struct vm_fault *vmf, enum swap_cache_result *result) +struct page *swapin_page_fault(swp_entry_t entry, gfp_t gfp_mask, + struct vm_fault *vmf, enum swap_cache_result *result) { struct swap_info_struct *si; struct mempolicy *mpol; @@ -936,15 +936,18 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); if (swap_use_no_readahead(si, swp_offset(entry))) { *result = SWAP_CACHE_BYPASS; - page = swapin_no_readahead(entry, gfp_mask, mpol, ilx, vmf->vma->vm_mm); + page = swapin_no_readahead(entry, GFP_HIGHUSER_MOVABLE, + mpol, ilx, vmf->vma->vm_mm); if (shadow) workingset_refault(page_folio(page), shadow); } else { *result = SWAP_CACHE_MISS; if (swap_use_vma_readahead(si)) - page = swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf); + page = swap_vma_readahead(entry, GFP_HIGHUSER_MOVABLE, + mpol, ilx, vmf); else - page = swap_cluster_readahead(entry, gfp_mask, mpol, ilx); + page = swap_cluster_readahead(entry, GFP_HIGHUSER_MOVABLE, + mpol, ilx, vmf->vma->vm_mm); } mpol_cond_put(mpol); done: