From patchwork Sat Nov 4 03:55:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 161528 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:8f47:0:b0:403:3b70:6f57 with SMTP id j7csp1451036vqu; Fri, 3 Nov 2023 20:56:56 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGbXDffCldEAXqJYcoE93279+S5EGAz5KW3FQJdCqWhTW2Wsi3YEhuSbV/8atkRb0C3M/hv X-Received: by 2002:a05:620a:2847:b0:77a:6498:7091 with SMTP id h7-20020a05620a284700b0077a64987091mr6198823qkp.67.1699070215811; Fri, 03 Nov 2023 20:56:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1699070215; cv=none; d=google.com; s=arc-20160816; b=ztkKtJ8ZeLBCSQnVjNQyVg6C4NG1nVm3eISnhlR6Wa7f0SLfTaJY04KeVwKmoUhc2W uWgKNKryUWkNrpMtHvIBMcVJgV3tntz1DKNg9AUexsG4edEHDIpVqcqI6mK8MgWSKgH7 jjBlym42ZSoX73ma+r1TZZFybxG6xbCApdzqD4ay/oEHMNlCkiLrBqDKaZrksZ3eYDZz 6B1TvwpdF6Siddejg9NmDI2Jx+ytqvP4pxvjs8BD8Bp+UDM9Sb8sQjnhsTEUWOCe8w+7 i0x0wpv2LMuOOIFwCzfEUBweAGXVQa3yvd+nHpVI99mODjTTHz6XAaYhN05iyBJuiSbX W1cA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=oZZ54VdEIlacFfuk1r0uZ44GYKJKe8HsZ6PDThbFMXE=; fh=kUj/5waaepfljbNx30rj2BALdV0u/mUCTaFWThlgquQ=; b=cK40bRaP6KcLeSrs/v//+kEwGAo36TUlJ51AoNsfHaTDQZLSqQBp/TCHgvYvCiXhQ4 vf68Hx/jrfObYwwefXwxNkp2t6sGYUet4dr9TbtHnA/Y/KG9jTFxVXkh+A2/mDCbAYgX Y0pKiTZas1U+XsEfdg7O0Aj3P9G4dQwpJ7xvtCoqtuBzgbp8cCeeta800jgkx2ZH5F4M 2HNmEe+3CEnOTOCvZaen6VC0f/5+xb81Jo6u94xMx3XvPtITU7sl38xfQjJGNOS2ZWnD TLqY3Kc0Qf6OVhYwoapnNWRQYmbw6BCrKcOdPuzPOpO0dMGP5vK4qiSJxviZ8NIZtXgq K1cg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id nn8-20020a17090b38c800b002768ab837bfsi3175930pjb.48.2023.11.03.20.56.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Nov 2023 20:56:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 5465680AB1FA; Fri, 3 Nov 2023 20:56:08 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346223AbjKDD4A (ORCPT + 35 others); Fri, 3 Nov 2023 23:56:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59648 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234316AbjKDDzs (ORCPT ); Fri, 3 Nov 2023 23:55:48 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B5141AA; Fri, 3 Nov 2023 20:55:44 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4SMkKh6BgFzvQ5W; Sat, 4 Nov 2023 11:55:36 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:40 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 01/10] mm: swap: introduce pfn_swap_entry_to_folio() Date: Sat, 4 Nov 2023 11:55:13 +0800 Message-ID: <20231104035522.2418660-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Fri, 03 Nov 2023 20:56:08 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781604250671608569 X-GMAIL-MSGID: 1781604250671608569 Introduce a new pfn_swap_entry_to_folio(), it is similar to pfn_swap_entry_to_page(), but return a folio, which allow us to completely replace the struct page variables with struct folio variables. Signed-off-by: Kefeng Wang --- include/linux/swapops.h | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/include/linux/swapops.h b/include/linux/swapops.h index bff1e8d97de0..85cb84e4be95 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -468,6 +468,19 @@ static inline struct page *pfn_swap_entry_to_page(swp_entry_t entry) return p; } +static inline struct folio *pfn_swap_entry_to_folio(swp_entry_t entry) +{ + struct folio *folio = pfn_folio(swp_offset_pfn(entry)); + + /* + * Any use of migration entries may only occur while the + * corresponding folio is locked + */ + BUG_ON(is_migration_entry(entry) && !folio_test_locked(folio)); + + return folio; +} + /* * A pfn swap entry is a special type of swap entry that always has a pfn stored * in the swap offset. They are used to represent unaddressable device memory From patchwork Sat Nov 4 03:55:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 161523 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:8f47:0:b0:403:3b70:6f57 with SMTP id j7csp1450901vqu; Fri, 3 Nov 2023 20:56:20 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGPQzAUsf3h3Ycg/0kqLdoNy0GiwwxBSKolxB+ldcKTr4I0rGpZUzpSCOqT0QLufx3DQ+NK X-Received: by 2002:a05:6870:4945:b0:1d6:2476:be2e with SMTP id fl5-20020a056870494500b001d62476be2emr29844513oab.35.1699070180580; Fri, 03 Nov 2023 20:56:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1699070180; cv=none; d=google.com; s=arc-20160816; b=dMFU3QgMYGdugWQImxc6C04qOh4orJkUVksZVVYLDobiq3s84Ap9WWAkzyNq4WUHOp PWSv5JYYvKOufZkA1GnF0QWnjXhfFjJPtCdzDv5jtW79kiOANsXq+SoKwkZxbQnPbbHY EhystzR3Jk392Eyu3qaEeSo0ONF4JkObRHJkyN4Hf0zkiwR51ZNvNIc5Fgnx391Zz5po 6LRIuoIyecXLfIdV8whqUrvsI/GFf/YL7loVnDe4PPO6NN6G45jP4HTD2PYukTtqrq45 zxDrilu1bVHNhMnhETVPSe237fOYdeQeVQhW5VUo03Bt6mD9N/DcMVa0Zgc6pln5jzPZ 1PIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=UmujaejCASjfQtJbBR1EzDPhPNeOSdCiPLhgFsCit5A=; fh=kUj/5waaepfljbNx30rj2BALdV0u/mUCTaFWThlgquQ=; b=D8Uz+W2RBUOI2n/W87Zi167PKECVjStkec30o3Jv0+T7qvkZKrgw1FVyFOQIAOiaqB kradYHVajWehGT9zxTNJPZSbRf1tn/04ar28PBENRG0byeQ6FowsqaLWr4kbrj/vrbZb kPaU1CdOoUJwWq6Oq3XliLrcQseipsVyihu91VMnqmAy/xHWuOShDs7EkhVPwBRV/zDw UyrxKzbhQT5/rN+b4h6Pku9eB1XAdurFl+DEjwR9C8ZPyp2i9ynWkO5sO9Zs3TCc/zYF v8EJfRawzX0HetCQE0YobS3vOBpG8UluuPyAz1MyWOPbh8PRVzGn8JclRkDD/BZ4xxGi AmiQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from agentk.vger.email (agentk.vger.email. [23.128.96.32]) by mx.google.com with ESMTPS id cn8-20020a056a020a8800b005b9b68add80si3076332pgb.361.2023.11.03.20.56.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Nov 2023 20:56:20 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id CE52881E5E75; Fri, 3 Nov 2023 20:56:16 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378001AbjKDD4I (ORCPT + 35 others); Fri, 3 Nov 2023 23:56:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59642 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234297AbjKDDzs (ORCPT ); Fri, 3 Nov 2023 23:55:48 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B5CDD45; Fri, 3 Nov 2023 20:55:44 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4SMkGD0WzxzrTQX; Sat, 4 Nov 2023 11:52:36 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:41 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 02/10] s390: pgtable: use a folio in ptep_zap_swap_entry() Date: Sat, 4 Nov 2023 11:55:14 +0800 Message-ID: <20231104035522.2418660-3-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Fri, 03 Nov 2023 20:56:16 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781604213728290883 X-GMAIL-MSGID: 1781604213728290883 Use a folio in ptep_zap_swap_entry(), which is preparetion for converting mm counter functions to take a folio. Signed-off-by: Kefeng Wang --- arch/s390/mm/pgtable.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c index 3bd2ab2a9a34..2f946b493fff 100644 --- a/arch/s390/mm/pgtable.c +++ b/arch/s390/mm/pgtable.c @@ -730,9 +730,9 @@ static void ptep_zap_swap_entry(struct mm_struct *mm, swp_entry_t entry) if (!non_swap_entry(entry)) dec_mm_counter(mm, MM_SWAPENTS); else if (is_migration_entry(entry)) { - struct page *page = pfn_swap_entry_to_page(entry); + struct folio *folio = pfn_swap_entry_to_folio(entry); - dec_mm_counter(mm, mm_counter(page)); + dec_mm_counter(mm, mm_counter(&folio->page)); } free_swap_and_cache(entry); } From patchwork Sat Nov 4 03:55:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 161529 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:8f47:0:b0:403:3b70:6f57 with SMTP id j7csp1451039vqu; Fri, 3 Nov 2023 20:56:56 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGN/7FLS+IWkDmMCFpOKU+xlX891mGS5pje/9cJg04H1m7o8eoEeZn0CddFXbZxmp0dKtdS X-Received: by 2002:a05:6a00:24cb:b0:6c3:5f49:6da7 with SMTP id d11-20020a056a0024cb00b006c35f496da7mr5484675pfv.2.1699070216186; Fri, 03 Nov 2023 20:56:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1699070216; cv=none; d=google.com; s=arc-20160816; b=cTEQHIWyIoFL4TP2l2Sq3r5Haz99xJZ+sig0J04fWRQAvCv3z7/FSOADWhYo0qjx5L n1zJYHw116MKAWNUW0reLINCimVUKUAlstqA6RgBMe6646GVQw97X7wIIiAv2TFo9l/I jeXU1IiRou2PLLof8jTTQgfs3kMjhlXeJpSinVhRBJBfyrj5afxBfQ0KhPodVxgy60+l FH4ybQ3ul6Sa0hkg0SIKwysIYEexHGz4UihcrMztculHGGId6KO2Gj+OIor/y91APuic udKBUst6KpoOaMdi2GPCa/zotMO2ByVBqwyiSocr6pT2r9Tc/7K3AKeRr9JPt7C9IQ35 J9Sg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=MPGmfuGIWa6O6otMqJXbXH0YFlYM5vFEP+zlcU8Yh7w=; fh=kUj/5waaepfljbNx30rj2BALdV0u/mUCTaFWThlgquQ=; b=U2CL/4C3jqBmjZ446Ww7moEbRmnr3ka6S7l+h8UTBms4VKWlimt3/9kxdlQauEUAFe uQvma7DlrCi8f/AGpV9yhmFTVo6PiiGTVKCcFoqR1hLQVX1gZsMosBKat93pUnut6Q2c H/dHAFE+W6HZ4lLzzgnQDPJfJclV0OlsBZl8iwpVTEhKYc+NHunbAerW6wCa9bpfBsxK 3t8r/dAPtT0aexTsZNk+9gUerxWAHLqAEffyyU0lcDLyTffXeRLlSB+VoF9zBQDb9nSi Yx5flLSLdAHScps/Of8u+nrakJsTja44DbeUZdYYrV5KUfZVs/uPWxfc9o1ykfv/gNPQ O9ZQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id l71-20020a63914a000000b005b982b93780si2935673pge.251.2023.11.03.20.56.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Nov 2023 20:56:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 46EAE804E6BA; Fri, 3 Nov 2023 20:56:13 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1377940AbjKDD4E (ORCPT + 35 others); Fri, 3 Nov 2023 23:56:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59638 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234308AbjKDDzs (ORCPT ); Fri, 3 Nov 2023 23:55:48 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B3CD1A8; Fri, 3 Nov 2023 20:55:44 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.57]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4SMkGD3sw9zrTX9; Sat, 4 Nov 2023 11:52:36 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:41 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 03/10] mm: huge_memory: use a folio in __split_huge_pmd_locked() Date: Sat, 4 Nov 2023 11:55:15 +0800 Message-ID: <20231104035522.2418660-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Fri, 03 Nov 2023 20:56:13 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781604250852067750 X-GMAIL-MSGID: 1781604250852067750 Use a folio in __split_huge_pmd_locked() which replaces five compound_head() call with two page_folio() calls. Signed-off-by: Kefeng Wang --- mm/huge_memory.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f31f02472396..34dd01970927 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2117,6 +2117,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, count_vm_event(THP_SPLIT_PMD); if (!vma_is_anonymous(vma)) { + struct folio *folio; old_pmd = pmdp_huge_clear_flush(vma, haddr, pmd); /* * We are going to unmap this huge page. So @@ -2130,17 +2131,17 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, swp_entry_t entry; entry = pmd_to_swp_entry(old_pmd); - page = pfn_swap_entry_to_page(entry); + folio = pfn_swap_entry_to_folio(entry); } else { - page = pmd_page(old_pmd); - if (!PageDirty(page) && pmd_dirty(old_pmd)) - set_page_dirty(page); - if (!PageReferenced(page) && pmd_young(old_pmd)) - SetPageReferenced(page); - page_remove_rmap(page, vma, true); - put_page(page); + folio = page_folio(pmd_page(old_pmd)); + if (!folio_test_dirty(folio) && pmd_dirty(old_pmd)) + folio_set_dirty(folio); + if (!folio_test_referenced(folio) && pmd_young(old_pmd)) + folio_set_referenced(folio); + page_remove_rmap(&folio->page, vma, true); + folio_put(folio); } - add_mm_counter(mm, mm_counter_file(page), -HPAGE_PMD_NR); + add_mm_counter(mm, mm_counter_file(&folio->page), -HPAGE_PMD_NR); return; } From patchwork Sat Nov 4 03:55:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 161522 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:8f47:0:b0:403:3b70:6f57 with SMTP id j7csp1450864vqu; Fri, 3 Nov 2023 20:56:13 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHyPfoelmwZBbjVVFec1WGU/ITlprJosRGdTPyhSWGO0+z2xIAFEIxY7SQj3NePtAUJwSUW X-Received: by 2002:ac8:57c2:0:b0:41e:3eef:736d with SMTP id w2-20020ac857c2000000b0041e3eef736dmr26104458qta.5.1699070173169; Fri, 03 Nov 2023 20:56:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1699070173; cv=none; d=google.com; s=arc-20160816; b=qzPidc/s4eHiF5YyQOdmGW+dFoQUXXxtUwGz2lho4vx07p9qwsKOhl5AF5JeJKKW7v CIrWhw4QpmLrHw/siSmEW1YT+3fyCQ+YcV3SRN2H44FHywb2TvaxoFL2HfbyS25C/OvL FjobJQnrwlOB/Kch2iVnheWLCADvpsbxdN0LoCeCb4gQZ5JWgxWNXCrPD+aCP2LxtKxo UuQ2vL7qzEJUWxe38S8XiZPprdaGQRuq3f3vjRhh7J09MFfIAcwsggDYgtdOy/gNonDD dg8go/Z65ByG2DxpdcwYe6N9t0FQxeKMP92s6rfw2twFQ6w6YjODCFz0Y9nFPLsdD7x1 LuHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=tP9NWM/ekJcLRFo0kqQKalmLDZuh2TT+P+my24sgZwk=; fh=kUj/5waaepfljbNx30rj2BALdV0u/mUCTaFWThlgquQ=; b=QqV5cpdX49txB2j8ylSm4IDcKTK5ijqJi+KyToEBboA45DP0PzpYsY1SRi0FRvHT5K 1x0fHdPCQVNo0/nMH1fcWdzq16AKqB/5g7e5TsTC5Fi1Pv81JpZdbSFM0Q4VoZMnQEOe Kw3iDa0A4SqfPuN5uNzmclGMIdJQ4L06niQkn767IzGp+eRB3H1zJh+K2FQtCfjHcDJg 2STjyWpYORvd+u1D70L9NWUm4sKe9qys8IGB5CQZEG8xyMIdzTqrTucUiJoslSSbS316 yMfyQk8W8PBXJ3wc+NruMAoNxNutN5CoZ18ap95eKHcI/XFN5x5kZt7M00VtpM8yk//3 lL/g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from fry.vger.email (fry.vger.email. [23.128.96.38]) by mx.google.com with ESMTPS id cl11-20020a056a02098b00b005b979990800si3275644pgb.102.2023.11.03.20.56.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Nov 2023 20:56:13 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id CF0D381067AD; Fri, 3 Nov 2023 20:56:08 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345010AbjKDDzt (ORCPT + 35 others); Fri, 3 Nov 2023 23:55:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59620 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233666AbjKDDzr (ORCPT ); Fri, 3 Nov 2023 23:55:47 -0400 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5A5EB194; Fri, 3 Nov 2023 20:55:44 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4SMkF31wjJzPnp7; Sat, 4 Nov 2023 11:51:35 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:42 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 04/10] mm: huge_memory: use a folio in zap_huge_pmd() Date: Sat, 4 Nov 2023 11:55:16 +0800 Message-ID: <20231104035522.2418660-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Fri, 03 Nov 2023 20:56:08 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781604205885700890 X-GMAIL-MSGID: 1781604205885700890 Use a folio in zap_huge_pmd(), which is a preparetion for converting mm counter functions to take a folio. Signed-off-by: Kefeng Wang --- mm/huge_memory.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 34dd01970927..78a00fe22c2d 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1716,11 +1716,13 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, zap_deposited_table(tlb->mm, pmd); spin_unlock(ptl); } else { - struct page *page = NULL; + struct folio *folio = NULL; int flush_needed = 1; if (pmd_present(orig_pmd)) { - page = pmd_page(orig_pmd); + struct page *page = pmd_page(orig_pmd); + + folio = page_folio(page); page_remove_rmap(page, vma, true); VM_BUG_ON_PAGE(page_mapcount(page) < 0, page); VM_BUG_ON_PAGE(!PageHead(page), page); @@ -1729,12 +1731,12 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, VM_BUG_ON(!is_pmd_migration_entry(orig_pmd)); entry = pmd_to_swp_entry(orig_pmd); - page = pfn_swap_entry_to_page(entry); + folio = pfn_swap_entry_to_folio(entry); flush_needed = 0; } else WARN_ONCE(1, "Non present huge pmd without pmd migration enabled!"); - if (PageAnon(page)) { + if (folio_test_anon(folio)) { zap_deposited_table(tlb->mm, pmd); add_mm_counter(tlb->mm, MM_ANONPAGES, -HPAGE_PMD_NR); } else { @@ -1745,7 +1747,7 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, spin_unlock(ptl); if (flush_needed) - tlb_remove_page_size(tlb, page, HPAGE_PMD_SIZE); + tlb_remove_page_size(tlb, &folio->page, HPAGE_PMD_SIZE); } return 1; } From patchwork Sat Nov 4 03:55:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 161527 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:8f47:0:b0:403:3b70:6f57 with SMTP id j7csp1451028vqu; Fri, 3 Nov 2023 20:56:54 -0700 (PDT) X-Google-Smtp-Source: AGHT+IG2HzElPfZP42TolpS9NtMDTmKyPotsgDnoex0OZXyYpL/QWJfUtOFnWxnXRBN3xuznwGwD X-Received: by 2002:a9d:65d4:0:b0:6b7:5687:8a9e with SMTP id z20-20020a9d65d4000000b006b756878a9emr24403619oth.15.1699070214631; Fri, 03 Nov 2023 20:56:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1699070214; cv=none; d=google.com; s=arc-20160816; b=04F7yT9IJfmqzZuDzK9iEKIBeJkWW4HUHLlgGdIqfppfJWPmG7c6wb6IrWDw5kDD6U OrkWKNAkX0i1zfJ8c4MoyaVnB9wmM71viYHOatx2pVC2cSIhbjIXMkg0Q2N639yqIZzw LfduY08SJe3TJaqESwWGckKJ37DK/N+UuN/F7meaN5lqjvpNNmNHwYNsh6R6gqBYWJmS B8typhwF9cspknxcAwGtrn86RjN2ZXoFj536uXuF2yYQqZrnPkjQCKnvr09qmXOOl08E ZnsnZEkoelSeT1mz0tdft5/nDQPu4+thSolH8mqsS9nPu0giNQCOZ7mmZHOKO81N9XCT S91Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=YGSXYmu+7g7unSHUX92KPz3ikmxmm44e+0Bb9DZZSLk=; fh=kUj/5waaepfljbNx30rj2BALdV0u/mUCTaFWThlgquQ=; b=UJwjQjl4YqE9qnoTZ1uw0xCK/1MLwFMRcWzEXJ3IXZpPJnHE2RpXL/OJNdhvD+9dxF NTqqe3tXURuTHa7YpvVS78+/w9VledI+YVQsQbGhhPx5zHik4abn+qgv14E39VSCG6q1 2YKyHhHw9fC0W5rNumDJSIrvETWwzRgZaXI4mXkB5qo0jF1m5iO0KyrRDEJBc7FqWv6m T1vKtxE5/90nmzovea3+JL5OGgnAL503+ST6uPhzOyGg9THfkPL9jBTz14AGSIVkZkq9 CpQEilI+PwMedVCe7jaYIJvI/xvL0Kyfk/SYAE0JwN/4ztzu1AKzuptVqgkOUzUydSXW buuA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id t2-20020a17090aba8200b002804438ebcfsi2965339pjr.171.2023.11.03.20.56.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Nov 2023 20:56:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 94C5881F8FEB; Fri, 3 Nov 2023 20:56:51 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378139AbjKDD4O (ORCPT + 35 others); Fri, 3 Nov 2023 23:56:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59662 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234331AbjKDDzs (ORCPT ); Fri, 3 Nov 2023 23:55:48 -0400 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C57E6D50; Fri, 3 Nov 2023 20:55:44 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4SMkGG17Kyz1P7rh; Sat, 4 Nov 2023 11:52:38 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:42 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 05/10] mm: memory: use a folio in copy_nonpresent_pte() Date: Sat, 4 Nov 2023 11:55:17 +0800 Message-ID: <20231104035522.2418660-6-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Fri, 03 Nov 2023 20:56:51 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781604248887285529 X-GMAIL-MSGID: 1781604248887285529 Use a folio in copy_nonpresent_pte() to save one compound_head() call. Signed-off-by: Kefeng Wang --- mm/memory.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 1f18ed4a5497..d9314dee355e 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -779,7 +779,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, unsigned long vm_flags = dst_vma->vm_flags; pte_t orig_pte = ptep_get(src_pte); pte_t pte = orig_pte; - struct page *page; + struct folio *folio; swp_entry_t entry = pte_to_swp_entry(orig_pte); if (likely(!non_swap_entry(entry))) { @@ -801,9 +801,9 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, } rss[MM_SWAPENTS]++; } else if (is_migration_entry(entry)) { - page = pfn_swap_entry_to_page(entry); + folio = pfn_swap_entry_to_folio(entry); - rss[mm_counter(page)]++; + rss[mm_counter(&folio->page)]++; if (!is_readable_migration_entry(entry) && is_cow_mapping(vm_flags)) { @@ -822,7 +822,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, set_pte_at(src_mm, addr, src_pte, pte); } } else if (is_device_private_entry(entry)) { - page = pfn_swap_entry_to_page(entry); + folio = pfn_swap_entry_to_folio(entry); /* * Update rss count even for unaddressable pages, as @@ -833,10 +833,10 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, * for unaddressable pages, at some point. But for now * keep things as they are. */ - get_page(page); - rss[mm_counter(page)]++; + folio_get(folio); + rss[mm_counter(&folio->page)]++; /* Cannot fail as these pages cannot get pinned. */ - BUG_ON(page_try_dup_anon_rmap(page, false, src_vma)); + BUG_ON(page_try_dup_anon_rmap(&folio->page, false, src_vma)); /* * We do not preserve soft-dirty information, because so From patchwork Sat Nov 4 03:55:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 161526 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:8f47:0:b0:403:3b70:6f57 with SMTP id j7csp1451024vqu; Fri, 3 Nov 2023 20:56:53 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGlxMGWIiwz5PS4uy/Xroor20qF4MWIMI3uQytYgGIHhPDKNFvCTjUlX4/NPiYtgMfqn0gI X-Received: by 2002:a05:6a00:93a3:b0:68f:d1a7:1a3a with SMTP id ka35-20020a056a0093a300b0068fd1a71a3amr27101252pfb.8.1699070213408; Fri, 03 Nov 2023 20:56:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1699070213; cv=none; d=google.com; s=arc-20160816; b=WiESJPizQI1bsPshRkOChz6iMg0qBEFrpVRpmDVrBWBM5c/o0GgYt/LTUZNjPle/5y 7XqGcgUEwTeJtwSWkoD297eKhGu+yGjc2nSj1E62x8tviItohdu2lCFGvBRRLlvcCy25 4BQ31TTM2YNUEJwx26XfT3Fcs04Upv+jEM6tDQhp3dM0Dzx3f7HwwOHdvRqD2OjOIZjh DYtgC18rw6rDNxx40/NX4YHnc/QoPrV0I5XZ1iYeqFjM1/rqqEVaGTOSKbO3/DRLGfuU rzA0aZtpXV06O4gEYUoQIhFza53mJ1jNSH7kvDW9AzkVxXzI/5RJeEZYeVJPmCA/R3SJ 8CTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=JmshzJlQTZerKlylIFF6GjirPxf+6rDZ/9S5JVPwXNs=; fh=kUj/5waaepfljbNx30rj2BALdV0u/mUCTaFWThlgquQ=; b=pr0qTamWggfTDNAdX4xv0hz2VB5nGQ4UX7iiEnl3DPVJodbuvl7ZUMPtpWKTpRQsqG hZxIp93sde/ythamGu9g3SRoSTMsZFoy6waXwAqwLrXkwtqaG9KS4ORk6P9XEc7cPGj4 K08+mzp7sX++o8DWiBOBz9Urnh/inUIhzDBv5wCkA+vmCUJdc11fN8YB4XzqLFKDLWvN Ib6ynttZScgdSVjUoKlo+qe5TWaEWBsZmSTFwjkNpZacuN57P+9MP73i4/yJH0TcX+Sk jD0mWIyIddJDY0ajTrcymecuXhqYuLO6pMS0cBr6YZ+z3FGo+fsGwmpLC6OLVUm0y6U9 QEeA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id d4-20020a656b84000000b005657ba564bdsi3071801pgw.826.2023.11.03.20.56.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Nov 2023 20:56:53 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id E562281F8FDE; Fri, 3 Nov 2023 20:56:50 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378368AbjKDD4R (ORCPT + 35 others); Fri, 3 Nov 2023 23:56:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59668 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343547AbjKDDzs (ORCPT ); Fri, 3 Nov 2023 23:55:48 -0400 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6FB15D51; Fri, 3 Nov 2023 20:55:45 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.57]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4SMkDl5J28zMlhj; Sat, 4 Nov 2023 11:51:19 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:43 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 06/10] mm: memory: use a folio in zap_pte_range() Date: Sat, 4 Nov 2023 11:55:18 +0800 Message-ID: <20231104035522.2418660-7-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Fri, 03 Nov 2023 20:56:51 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781604248103834394 X-GMAIL-MSGID: 1781604248103834394 Make should_zap_page() to take a folio and use a folio in zap_pte_range(), which save several compound_head() calls. Signed-off-by: Kefeng Wang --- mm/memory.c | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index d9314dee355e..806568f9605b 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1358,19 +1358,19 @@ static inline bool should_zap_cows(struct zap_details *details) return details->even_cows; } -/* Decides whether we should zap this page with the page pointer specified */ -static inline bool should_zap_page(struct zap_details *details, struct page *page) +/* Decides whether we should zap this folio with the folio pointer specified */ +static inline bool should_zap_page(struct zap_details *details, struct folio *folio) { - /* If we can make a decision without *page.. */ + /* If we can make a decision without *folio.. */ if (should_zap_cows(details)) return true; - /* E.g. the caller passes NULL for the case of a zero page */ - if (!page) + /* E.g. the caller passes NULL for the case of a zero folio */ + if (!folio) return true; - /* Otherwise we should only zap non-anon pages */ - return !PageAnon(page); + /* Otherwise we should only zap non-anon folios */ + return !folio_test_anon(folio); } static inline bool zap_drop_file_uffd_wp(struct zap_details *details) @@ -1423,7 +1423,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, arch_enter_lazy_mmu_mode(); do { pte_t ptent = ptep_get(pte); - struct page *page; + struct folio *folio = NULL; if (pte_none(ptent)) continue; @@ -1433,9 +1433,13 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, if (pte_present(ptent)) { unsigned int delay_rmap; + struct page *page; page = vm_normal_page(vma, addr, ptent); - if (unlikely(!should_zap_page(details, page))) + if (page) + folio = page_folio(page); + + if (unlikely(!should_zap_page(details, folio))) continue; ptent = ptep_get_and_clear_full(mm, addr, pte, tlb->fullmm); @@ -1449,16 +1453,16 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, } delay_rmap = 0; - if (!PageAnon(page)) { + if (!folio_test_anon(folio)) { if (pte_dirty(ptent)) { - set_page_dirty(page); + folio_set_dirty(folio); if (tlb_delay_rmap(tlb)) { delay_rmap = 1; force_flush = 1; } } if (pte_young(ptent) && likely(vma_has_recency(vma))) - mark_page_accessed(page); + folio_mark_accessed(folio); } rss[mm_counter(page)]--; if (!delay_rmap) { @@ -1477,9 +1481,10 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, entry = pte_to_swp_entry(ptent); if (is_device_private_entry(entry) || is_device_exclusive_entry(entry)) { - page = pfn_swap_entry_to_page(entry); - if (unlikely(!should_zap_page(details, page))) + folio = pfn_swap_entry_to_folio(entry); + if (unlikely(!should_zap_page(details, folio))) continue; + /* * Both device private/exclusive mappings should only * work with anonymous page so far, so we don't need to @@ -1487,10 +1492,10 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, * see zap_install_uffd_wp_if_needed(). */ WARN_ON_ONCE(!vma_is_anonymous(vma)); - rss[mm_counter(page)]--; + rss[mm_counter(&folio->page)]--; if (is_device_private_entry(entry)) - page_remove_rmap(page, vma, false); - put_page(page); + page_remove_rmap(&folio->page, vma, false); + folio_put(folio); } else if (!non_swap_entry(entry)) { /* Genuine swap entry, hence a private anon page */ if (!should_zap_cows(details)) @@ -1499,10 +1504,10 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, if (unlikely(!free_swap_and_cache(entry))) print_bad_pte(vma, addr, ptent, NULL); } else if (is_migration_entry(entry)) { - page = pfn_swap_entry_to_page(entry); - if (!should_zap_page(details, page)) + folio = pfn_swap_entry_to_folio(entry); + if (!should_zap_page(details, folio)) continue; - rss[mm_counter(page)]--; + rss[mm_counter(&folio->page)]--; } else if (pte_marker_entry_uffd_wp(entry)) { /* * For anon: always drop the marker; for file: only From patchwork Sat Nov 4 03:55:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 161524 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:8f47:0:b0:403:3b70:6f57 with SMTP id j7csp1450941vqu; Fri, 3 Nov 2023 20:56:31 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHWgGe9mgCKnaQRGWBAMBu+lfP6y+35XKSD5f1oDVgdO1TqAxiPJZKYpPwks8Y8bwxay5VG X-Received: by 2002:a17:903:187:b0:1cc:47c1:c2cb with SMTP id z7-20020a170903018700b001cc47c1c2cbmr6531412plg.2.1699070191378; Fri, 03 Nov 2023 20:56:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1699070191; cv=none; d=google.com; s=arc-20160816; b=ShRG48fGXEh/Zk0DvFIweLFCD/6f8ndD/uZ+IsZtlaQHSindGQvTKkB7lPVw1/jAKQ 1tYPS3lQa0+16RAofH+WB6NUX9ty67VjC9WbxSM+m28Y9TeHJZc8H8dxA531C5JixDs0 Oewkl1P+AI2PWU9DBFj1fIsaF19AmoH4vGNOhVZ0hJRTjLrb0R/RsPmQTPWBC/oTZoHo ADvPllT1D0eczj3gJytPHmagGU8aDgUiVdNmrGc5H5HSqc3eDx+pluRHmCNjBB6iAfAY Egqc3iUFSeWIkzvFTVBs9vDArRRfAvdfhGFR6PD31Va8KuWWZN1AW8Z39n8fA30cPenm q9rA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=DR0NwF5aoEPfBIrGPy27I4jQhGFlovTU4h4vQaXGQfk=; fh=kUj/5waaepfljbNx30rj2BALdV0u/mUCTaFWThlgquQ=; b=d/sYwjr41PmT0N/JwKh4s83PitL1sTifBH4B560POaaRO3d9hzlDf1fAy21WhhL2pD +pRKiKlBKZJSS8ymG7WUV2HYrSG7srqnw1bGvtnifZc/LqV9VokncX3dn7bX6rOvd0L1 TJNDV6pf2F+V7uvedo/Xrq34KATfWwoo+o3xFpaGBUpdBWqe56bekgpxA23qmVzlSTdV zaLa65FlmGCQn8tKc1wAoVicjESC+kM+xS62nG2UneMAuLu0k5EMeEcsvuKmylEJ7M6C 2q28cmObXUE8NgWC9f0gZnNgRlHwfaQ+PDyIXI4RvbR+te8gZhqJ6oGTbnSSqW86E2BC ZmfQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from fry.vger.email (fry.vger.email. [2620:137:e000::3:8]) by mx.google.com with ESMTPS id q22-20020a170902b11600b001cc11902882si2797982plr.641.2023.11.03.20.56.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Nov 2023 20:56:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id D4BA581ED52C; Fri, 3 Nov 2023 20:56:25 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378032AbjKDD4L (ORCPT + 35 others); Fri, 3 Nov 2023 23:56:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59644 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234193AbjKDDzs (ORCPT ); Fri, 3 Nov 2023 23:55:48 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 81CFBD53; Fri, 3 Nov 2023 20:55:45 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.56]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4SMkKl453TzvQCR; Sat, 4 Nov 2023 11:55:39 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:43 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 07/10] mm: memory: use a folio in do_set_pmd() Date: Sat, 4 Nov 2023 11:55:19 +0800 Message-ID: <20231104035522.2418660-8-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Fri, 03 Nov 2023 20:56:26 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781604225094735841 X-GMAIL-MSGID: 1781604225094735841 Use a folio in do_set_pmd(), which is a preparetion for converting mm counter functions to take a folio. Signed-off-by: Kefeng Wang --- mm/memory.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 806568f9605b..ac247850919a 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4318,12 +4318,13 @@ vm_fault_t do_set_pmd(struct vm_fault *vmf, struct page *page) unsigned long haddr = vmf->address & HPAGE_PMD_MASK; pmd_t entry; vm_fault_t ret = VM_FAULT_FALLBACK; + struct folio *folio; if (!transhuge_vma_suitable(vma, haddr)) return ret; - page = compound_head(page); - if (compound_order(page) != HPAGE_PMD_ORDER) + folio = page_folio(page); + if (folio_order(folio) != HPAGE_PMD_ORDER) return ret; /* From patchwork Sat Nov 4 03:55:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 161525 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:8f47:0:b0:403:3b70:6f57 with SMTP id j7csp1450994vqu; Fri, 3 Nov 2023 20:56:43 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHP4f7wX9EXtZdSUu2RP++V1eQatfe9M4vm+jOCu3TOwhnH8TduRwY6FHfjd578n8yRV7Qb X-Received: by 2002:a05:6214:1241:b0:675:5924:ef69 with SMTP id r1-20020a056214124100b006755924ef69mr7227971qvv.32.1699070203609; Fri, 03 Nov 2023 20:56:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1699070203; cv=none; d=google.com; s=arc-20160816; b=um5PE5G+gqt44jgRTP8Jk1CjxodKox08wO4EaZD2EJ2txC0pxAyZJwh7Kvf54ag+L+ LrSXJFhx8ZFzfD159FRHgMAx3Nrxa+vonxRoWqGrgpvVtDrrHJF+6WYNwYV+nQ8aPM09 v6JNm5t4WoNSeJvqnjY3Rv1LX2cYHunAAKQRIxFG/NgvBA6Qp+IxW2QteX056Dapn1XS EkEsxRcyfm/soVB52fFb/UezWaJ9sFIbO6CHOFji2MSlUSyM/tv6VSJxbTb6ffF76XBw 2gL5/rEyFq4valY/97+jSWaEdr1xE+HvsnSQxC+uBIgZTox2Q0xJoCgRQH92hvuR/kSW qjeQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=vvr+Gp+CIzn+T6Sky/V+P109WJdRdmjUuqGFwTBrUlc=; fh=kUj/5waaepfljbNx30rj2BALdV0u/mUCTaFWThlgquQ=; b=xN917w40gEjb7w/d06QF8hRZvTtyxa6dqpgsTjJETTqedelSa8WcwOVWDbDam59Phm oX8Qr4kaxATe9qQjuLBXFHMMs2QvU579nM3wF5bufjrpImYIKMOreKocX3+ed7qethBP eWeQJz0MFHhm03pzyJcU7f87PTJ9/1vJ3EEcfHB8lAAK0qWbTwgbS3+an2hEkSK/kMiY 1O0jszXVcoWxbcxz5BeaeyNuLdBPiLZxlZmgqzO8b08Ge7KMPitwuY1Z7Ykg4xpkmrAF DasENSgRIsCPUzep6hPJ7CopYndFU02a2f64a6hzmAkGXXtNZAftk/xTVP34JRF/2WlF bJkQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from fry.vger.email (fry.vger.email. [23.128.96.38]) by mx.google.com with ESMTPS id d124-20020a633682000000b005b86b35778fsi2826024pga.180.2023.11.03.20.56.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Nov 2023 20:56:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id CCD9E81C3D7F; Fri, 3 Nov 2023 20:56:36 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378370AbjKDD4U (ORCPT + 35 others); Fri, 3 Nov 2023 23:56:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59678 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343779AbjKDDzs (ORCPT ); Fri, 3 Nov 2023 23:55:48 -0400 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 08EAFD54; Fri, 3 Nov 2023 20:55:46 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.55]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4SMkDm4Z5TzMmQq; Sat, 4 Nov 2023 11:51:20 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:43 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 08/10] mm: memory: use a folio in insert_page_into_pte_locked() Date: Sat, 4 Nov 2023 11:55:20 +0800 Message-ID: <20231104035522.2418660-9-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Fri, 03 Nov 2023 20:56:36 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781604237701351759 X-GMAIL-MSGID: 1781604237701351759 Use a folio in insert_page_into_pte_locked(), which is preparetion for converting mm counter functions to take a folio. Signed-off-by: Kefeng Wang --- mm/memory.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/mm/memory.c b/mm/memory.c index ac247850919a..a2cf240b1975 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1850,10 +1850,13 @@ static int validate_page_before_insert(struct page *page) static int insert_page_into_pte_locked(struct vm_area_struct *vma, pte_t *pte, unsigned long addr, struct page *page, pgprot_t prot) { + struct folio *folio; + if (!pte_none(ptep_get(pte))) return -EBUSY; + folio = page_folio(page); /* Ok, finally just insert the thing.. */ - get_page(page); + folio_get(folio); inc_mm_counter(vma->vm_mm, mm_counter_file(page)); page_add_file_rmap(page, vma, false); set_pte_at(vma->vm_mm, addr, pte, mk_pte(page, prot)); From patchwork Sat Nov 4 03:55:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 161530 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:8f47:0:b0:403:3b70:6f57 with SMTP id j7csp1451058vqu; Fri, 3 Nov 2023 20:57:01 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEZD9KbtNanygmR5qXfG9RYuTOv/VEiSXrCQh3MCOq9Azl0sXEq4ezyJDfRVcqsBQ3ztexi X-Received: by 2002:a05:6a20:c1aa:b0:17b:7505:8ab7 with SMTP id bg42-20020a056a20c1aa00b0017b75058ab7mr29917103pzb.46.1699070221299; Fri, 03 Nov 2023 20:57:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1699070221; cv=none; d=google.com; s=arc-20160816; b=BpAo5bSx8IBiIQT0wRB3UVs+ipL8dmRID/MPWqYokCDBOyOavYP4h0HQU0AOnPCqf3 pPcuaGFIB+4s6wgqbQR7OvznhVNylqMBUihSWFXry5qD2mOyOF+D8+eBbDQAyBlJgR1D QbtvQX1fhshXxxMgoFotuYn0YJNN5e5bwKPuOKqhu1zFUFuhwIDX8Ld29/FozsDKRZqY bAne5jBToh6iGTUDN45FDUCcccX8/6WhRwDvGnysVoHpB0qoS+rJOpmxV/jxznjkvgar Fr31NFe2hfc7+5xF4BGHv1bm37rYGS/ICM3tMhm8+SnOrWv5m9JobxtklHDpFhd6Il6G 0gMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=/pjTV/UH3Htl4eHQK4ntBuuWBvSNKg8stTZ8yGGw66M=; fh=kUj/5waaepfljbNx30rj2BALdV0u/mUCTaFWThlgquQ=; b=txk9IeztnYUFI8uAZYzyKT7ZTQBgKEpJgV8h5jZdTumvCO5kfSkK4Lua0MVJ/5tWDa PTWHqbGabT4ccZbtLw1SDvtKIhdSVjWHBBEToTDBYaSeuH/WZf0QVRED5yE1L0VaIxMQ PGc81VDv4xuwfmC6qrMbrGCgIoEE/rd7WvdrXvete9wLIwarEf0detglkQ5HLhC5RiyG jqHzYVKXv+/RLHogB/npFG9pUoSqmriCwVJdAUBU7uPpDss6DP/4xmJNzbe7ZPxDszIc lBjXIAWEwze/YHjm24dk1Qa9pPek2jle8m0LGeXKSM2lMHFtdrthw8ZEhq1NpjCSr0wx FLbA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id b25-20020a656699000000b005859a7d38f9si3008209pgw.703.2023.11.03.20.57.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Nov 2023 20:57:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 99ADE81B896F; Fri, 3 Nov 2023 20:56:29 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234594AbjKDD4Z (ORCPT + 35 others); Fri, 3 Nov 2023 23:56:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59668 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345321AbjKDDzu (ORCPT ); Fri, 3 Nov 2023 23:55:50 -0400 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7408F194; Fri, 3 Nov 2023 20:55:46 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.55]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4SMkDn0qXmzMmSn; Sat, 4 Nov 2023 11:51:21 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:44 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 09/10] mm: convert mm_counter() to take a folio Date: Sat, 4 Nov 2023 11:55:21 +0800 Message-ID: <20231104035522.2418660-10-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Fri, 03 Nov 2023 20:56:29 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781604256004745310 X-GMAIL-MSGID: 1781604256004745310 Since all mm_counter() callers with a folio, let's convert mm_counter() to take a folio. Signed-off-by: Kefeng Wang --- arch/s390/mm/pgtable.c | 2 +- include/linux/mm.h | 6 +++--- mm/memory.c | 10 +++++----- mm/rmap.c | 8 ++++---- mm/userfaultfd.c | 2 +- 5 files changed, 14 insertions(+), 14 deletions(-) diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c index 2f946b493fff..54b184648db6 100644 --- a/arch/s390/mm/pgtable.c +++ b/arch/s390/mm/pgtable.c @@ -732,7 +732,7 @@ static void ptep_zap_swap_entry(struct mm_struct *mm, swp_entry_t entry) else if (is_migration_entry(entry)) { struct folio *folio = pfn_swap_entry_to_folio(entry); - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); } free_swap_and_cache(entry); } diff --git a/include/linux/mm.h b/include/linux/mm.h index 418d26608ece..fea78900bf84 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2591,11 +2591,11 @@ static inline int mm_counter_file(struct page *page) return MM_FILEPAGES; } -static inline int mm_counter(struct page *page) +static inline int mm_counter(struct folio *folio) { - if (PageAnon(page)) + if (folio_test_anon(folio)) return MM_ANONPAGES; - return mm_counter_file(page); + return mm_counter_file(&folio->page); } static inline unsigned long get_mm_rss(struct mm_struct *mm) diff --git a/mm/memory.c b/mm/memory.c index a2cf240b1975..ad30d4ad2223 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -803,7 +803,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, } else if (is_migration_entry(entry)) { folio = pfn_swap_entry_to_folio(entry); - rss[mm_counter(&folio->page)]++; + rss[mm_counter(folio)]++; if (!is_readable_migration_entry(entry) && is_cow_mapping(vm_flags)) { @@ -834,7 +834,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, * keep things as they are. */ folio_get(folio); - rss[mm_counter(&folio->page)]++; + rss[mm_counter(folio)]++; /* Cannot fail as these pages cannot get pinned. */ BUG_ON(page_try_dup_anon_rmap(&folio->page, false, src_vma)); @@ -1464,7 +1464,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, if (pte_young(ptent) && likely(vma_has_recency(vma))) folio_mark_accessed(folio); } - rss[mm_counter(page)]--; + rss[mm_counter(folio)]--; if (!delay_rmap) { page_remove_rmap(page, vma, false); if (unlikely(page_mapcount(page) < 0)) @@ -1492,7 +1492,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, * see zap_install_uffd_wp_if_needed(). */ WARN_ON_ONCE(!vma_is_anonymous(vma)); - rss[mm_counter(&folio->page)]--; + rss[mm_counter(folio)]--; if (is_device_private_entry(entry)) page_remove_rmap(&folio->page, vma, false); folio_put(folio); @@ -1507,7 +1507,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, folio = pfn_swap_entry_to_folio(entry); if (!should_zap_page(details, folio)) continue; - rss[mm_counter(&folio->page)]--; + rss[mm_counter(folio)]--; } else if (pte_marker_entry_uffd_wp(entry)) { /* * For anon: always drop the marker; for file: only diff --git a/mm/rmap.c b/mm/rmap.c index 7a27a2b41802..7a563490ce08 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1678,7 +1678,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, set_huge_pte_at(mm, address, pvmw.pte, pteval, hsz); } else { - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); set_pte_at(mm, address, pvmw.pte, pteval); } @@ -1693,7 +1693,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, * migration) will not expect userfaults on already * copied pages. */ - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); } else if (folio_test_anon(folio)) { swp_entry_t entry = page_swap_entry(subpage); pte_t swp_pte; @@ -2075,7 +2075,7 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, set_huge_pte_at(mm, address, pvmw.pte, pteval, hsz); } else { - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); set_pte_at(mm, address, pvmw.pte, pteval); } @@ -2090,7 +2090,7 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, * migration) will not expect userfaults on already * copied pages. */ - dec_mm_counter(mm, mm_counter(&folio->page)); + dec_mm_counter(mm, mm_counter(folio)); } else { swp_entry_t entry; pte_t swp_pte; diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 96d9eae5c7cc..9a6759fa9b06 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -124,7 +124,7 @@ int mfill_atomic_install_pte(pmd_t *dst_pmd, * Must happen after rmap, as mm_counter() checks mapping (via * PageAnon()), which is set by __page_set_anon_rmap(). */ - inc_mm_counter(dst_mm, mm_counter(page)); + inc_mm_counter(dst_mm, mm_counter(folio)); set_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte); From patchwork Sat Nov 4 03:55:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 161531 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:8f47:0:b0:403:3b70:6f57 with SMTP id j7csp1451070vqu; Fri, 3 Nov 2023 20:57:04 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF1VSx0cj1vOMW3P6YcCFDKbcEkyiQRbciNa9UYAora03AGZtXKdt5+Jj/pCTAOAXDBr5kE X-Received: by 2002:a17:90b:1d86:b0:27d:5568:e867 with SMTP id pf6-20020a17090b1d8600b0027d5568e867mr6975368pjb.9.1699070223906; Fri, 03 Nov 2023 20:57:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1699070223; cv=none; d=google.com; s=arc-20160816; b=p9iHm+DBhrKlCeHVfMxFZ9kHYMAbuNNu9BOgwU5QAmPKrMj29A9nXv/r3EOyghJ4xK 3mGHSOtCXiwPJ1TpVMn6Z7o2Y7dWYmYmENzMotxmFtx6qmwJQ/sLOPUQQ4OgibheCjmJ eTOolx73jAcLb1WFZ8yN+zzyNcR7ocMzeGO0dQtpYjqJScJ/tI2S8+he133RdB5ojvT8 vxPg5H6BaZI3vpeP8vFu0RWewH/815m12f4nAtWKady7qV6MFq8tgje513P5UUJsVMqo h75Jr87RdmBZTbRHvuI9baJzziZEDjMsPL8nxQ2nvu7H1Nzz3boA0QoGrh7fKtMsCDXi om7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=GazSDJbamK21W3qr6H03idb2J/BUC8P02Cqa+Ldgz9o=; fh=kUj/5waaepfljbNx30rj2BALdV0u/mUCTaFWThlgquQ=; b=JU163OA1C7E2rpyIR1e8MLf2kCCaB60lPgpjEsvPuzTodoLBIIwlc8EE20FrzAHTZ9 UL1xWE5gvNgt7PZiuDspeG3m2DuGmJcSIGomElHtB2X/WSZaBHFtk6kiTdovC5ZfC2Cx Iumd+C3av0VTgPtWhqvRKEWM6WCWUOGKcWNJABlZlTyY8RraFSGGo2CessETAl3NQUef GXyQG3ZPu+kuA/9RSNJ9SKv7rhYvKFu+XnOAA8FIq4gx1yR+gF3YoV8H56XOXzE/sGzQ xO+lf1xod2+x92YeM0xyJo0Pr6fIlDLFHE+/I2+vDNmsAVqvGntMtdMfjOusHFCyTDgb 9+Gw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id i21-20020a17090acf9500b0027cf6bb27dfsi3090742pju.39.2023.11.03.20.57.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Nov 2023 20:57:03 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 1948B81F473E; Fri, 3 Nov 2023 20:57:01 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378051AbjKDD4b (ORCPT + 35 others); Fri, 3 Nov 2023 23:56:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59686 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345405AbjKDDzu (ORCPT ); Fri, 3 Nov 2023 23:55:50 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D72B5D47; Fri, 3 Nov 2023 20:55:46 -0700 (PDT) Received: from dggpemm100001.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4SMkGH51DZzrTXs; Sat, 4 Nov 2023 11:52:39 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Sat, 4 Nov 2023 11:55:44 +0800 From: Kefeng Wang To: Andrew Morton CC: , , Matthew Wilcox , David Hildenbrand , , Kefeng Wang Subject: [PATCH v2 10/10] mm: convert mm_counter_file() to take a folio Date: Sat, 4 Nov 2023 11:55:22 +0800 Message-ID: <20231104035522.2418660-11-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> References: <20231104035522.2418660-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm100001.china.huawei.com (7.185.36.93) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Fri, 03 Nov 2023 20:57:01 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781604259109982812 X-GMAIL-MSGID: 1781604259109982812 Since all mm_counter_file() callers with a folio, let's convert mm_counter_file() to take a folio. Signed-off-by: Kefeng Wang --- include/linux/mm.h | 8 ++++---- kernel/events/uprobes.c | 2 +- mm/huge_memory.c | 5 +++-- mm/khugepaged.c | 4 ++-- mm/memory.c | 10 +++++----- mm/rmap.c | 2 +- 6 files changed, 16 insertions(+), 15 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index fea78900bf84..95573065a46b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2583,10 +2583,10 @@ static inline void dec_mm_counter(struct mm_struct *mm, int member) mm_trace_rss_stat(mm, member); } -/* Optimized variant when page is already known not to be PageAnon */ -static inline int mm_counter_file(struct page *page) +/* Optimized variant when folio is already known not to be anon */ +static inline int mm_counter_file(struct folio *folio) { - if (PageSwapBacked(page)) + if (folio_test_swapbacked(folio)) return MM_SHMEMPAGES; return MM_FILEPAGES; } @@ -2595,7 +2595,7 @@ static inline int mm_counter(struct folio *folio) { if (folio_test_anon(folio)) return MM_ANONPAGES; - return mm_counter_file(&folio->page); + return mm_counter_file(folio); } static inline unsigned long get_mm_rss(struct mm_struct *mm) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 435aac1d8c27..ce251e3a4ae6 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -188,7 +188,7 @@ static int __replace_page(struct vm_area_struct *vma, unsigned long addr, dec_mm_counter(mm, MM_ANONPAGES); if (!folio_test_anon(old_folio)) { - dec_mm_counter(mm, mm_counter_file(old_page)); + dec_mm_counter(mm, mm_counter_file(old_folio)); inc_mm_counter(mm, MM_ANONPAGES); } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 78a00fe22c2d..88420d067477 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1742,7 +1742,8 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, } else { if (arch_needs_pgtable_deposit()) zap_deposited_table(tlb->mm, pmd); - add_mm_counter(tlb->mm, mm_counter_file(page), -HPAGE_PMD_NR); + add_mm_counter(tlb->mm, mm_counter_file(folio), + -HPAGE_PMD_NR); } spin_unlock(ptl); @@ -2143,7 +2144,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, page_remove_rmap(&folio->page, vma, true); folio_put(folio); } - add_mm_counter(mm, mm_counter_file(&folio->page), -HPAGE_PMD_NR); + add_mm_counter(mm, mm_counter_file(folio), -HPAGE_PMD_NR); return; } diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 064654717843..39393f4262b2 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1630,7 +1630,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, /* step 3: set proper refcount and mm_counters. */ if (nr_ptes) { folio_ref_sub(folio, nr_ptes); - add_mm_counter(mm, mm_counter_file(&folio->page), -nr_ptes); + add_mm_counter(mm, mm_counter_file(folio), -nr_ptes); } /* step 4: remove empty page table */ @@ -1661,7 +1661,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, if (nr_ptes) { flush_tlb_mm(mm); folio_ref_sub(folio, nr_ptes); - add_mm_counter(mm, mm_counter_file(&folio->page), -nr_ptes); + add_mm_counter(mm, mm_counter_file(folio), -nr_ptes); } if (start_pte) pte_unmap_unlock(start_pte, ptl); diff --git a/mm/memory.c b/mm/memory.c index ad30d4ad2223..3418ace5e0ad 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -960,7 +960,7 @@ copy_present_pte(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, } else if (page) { folio_get(folio); page_dup_file_rmap(page, false); - rss[mm_counter_file(page)]++; + rss[mm_counter_file(folio)]++; } /* @@ -1857,7 +1857,7 @@ static int insert_page_into_pte_locked(struct vm_area_struct *vma, pte_t *pte, folio = page_folio(page); /* Ok, finally just insert the thing.. */ folio_get(folio); - inc_mm_counter(vma->vm_mm, mm_counter_file(page)); + inc_mm_counter(vma->vm_mm, mm_counter_file(folio)); page_add_file_rmap(page, vma, false); set_pte_at(vma->vm_mm, addr, pte, mk_pte(page, prot)); return 0; @@ -3166,7 +3166,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) if (likely(vmf->pte && pte_same(ptep_get(vmf->pte), vmf->orig_pte))) { if (old_folio) { if (!folio_test_anon(old_folio)) { - dec_mm_counter(mm, mm_counter_file(&old_folio->page)); + dec_mm_counter(mm, mm_counter_file(old_folio)); inc_mm_counter(mm, MM_ANONPAGES); } } else { @@ -4359,7 +4359,7 @@ vm_fault_t do_set_pmd(struct vm_fault *vmf, struct page *page) if (write) entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma); - add_mm_counter(vma->vm_mm, mm_counter_file(page), HPAGE_PMD_NR); + add_mm_counter(vma->vm_mm, mm_counter_file(folio), HPAGE_PMD_NR); page_add_file_rmap(page, vma, true); /* @@ -4422,7 +4422,7 @@ void set_pte_range(struct vm_fault *vmf, struct folio *folio, folio_add_new_anon_rmap(folio, vma, addr); folio_add_lru_vma(folio, vma); } else { - add_mm_counter(vma->vm_mm, mm_counter_file(page), nr); + add_mm_counter(vma->vm_mm, mm_counter_file(folio), nr); folio_add_file_rmap_range(folio, page, nr, vma, false); } set_ptes(vma->vm_mm, addr, vmf->pte, entry, nr); diff --git a/mm/rmap.c b/mm/rmap.c index 7a563490ce08..9e3d0eff8b05 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1801,7 +1801,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, * * See Documentation/mm/mmu_notifier.rst */ - dec_mm_counter(mm, mm_counter_file(&folio->page)); + dec_mm_counter(mm, mm_counter_file(folio)); } discard: page_remove_rmap(subpage, vma, folio_test_hugetlb(folio));