From patchwork Wed Jan 25 23:41:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Moola X-Patchwork-Id: 48422 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp543734wrn; Wed, 25 Jan 2023 15:42:45 -0800 (PST) X-Google-Smtp-Source: AMrXdXtqouy9zpm0iE3+JcDQjsxY4+4SM0HzNROKhGZSxA3Q+qXW7L2Got+ImmyD3GUwlaE0Hf62 X-Received: by 2002:a17:907:c29b:b0:870:6715:bc8c with SMTP id tk27-20020a170907c29b00b008706715bc8cmr32007682ejc.55.1674690164932; Wed, 25 Jan 2023 15:42:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674690164; cv=none; d=google.com; s=arc-20160816; b=B/nRH0fHnCj8eepWCNrPw3UK2wX79PLzUFuoPgMM5hstO6ka7w7exdsb8VK00Bt57h GnYfShf8mw1QanqWId8LNPIgKxDpzMEwbd5a82hnunlp0iBljJn07S88LX8WPpBZnGWr 9mMUPbZ43u8InYPRvfdav6XpCt29FsLpv1cDQupAh92f3IsMjrh3BYH4ba4xDLJKafsJ 0hHkwz17hi5df5RqR6uNPHwVQD2sKYsH5iTVdb8moF1r0tc1fd/3QH/UUIwZT300hZpA 1vQOBZYRFaJ6kUed5LxoZkrAy/B8gvu4AKRa3Q465974OfnM7hnCMXlGfadtBxm0E+PF 7voQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=BGNwUbDjmHky71GpbBmE9Nh0iBILc0cb1v5YBQZD5HQ=; b=DmY1JIpEzjxYIrHI4ihPWgQa1oxA30yRDNVqvVgnCzeLQIiJABJimhF+1OO9GHHNLn xAG6Za8PBMSTAdlkO1iFqJfIHzc3kCM1hw5MBXppX+EY30Neoc/LjM2hfN7hZT+zYerP fZuQBYnWQf+TSNI+MHxjZO/JOur5br3s+HBHCB+kHb5n1j1CRWwVPSeo2mWUjPK33DSf hU6gEJxpmfIS9hm0VVwe6slzoEybTdq5+UoDHcbt1Czx7SvH7Ac8kSP5IGJcZRIY9eJ6 +nIqeuPPxELJTR6FJSkt+63jcDr1F0Qwoh8mAtov5wDgS4PPglTb1cpR/bITLc5OEGI4 ETzQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=mIXSbS3z; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id eq18-20020a170907291200b008621404f1d6si6582371ejc.892.2023.01.25.15.42.21; Wed, 25 Jan 2023 15:42:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=mIXSbS3z; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236091AbjAYXlu (ORCPT + 99 others); Wed, 25 Jan 2023 18:41:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44008 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236044AbjAYXlr (ORCPT ); Wed, 25 Jan 2023 18:41:47 -0500 Received: from mail-pj1-x1033.google.com (mail-pj1-x1033.google.com [IPv6:2607:f8b0:4864:20::1033]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 665F65FE1 for ; Wed, 25 Jan 2023 15:41:46 -0800 (PST) Received: by mail-pj1-x1033.google.com with SMTP id b10so32100pjo.1 for ; Wed, 25 Jan 2023 15:41:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BGNwUbDjmHky71GpbBmE9Nh0iBILc0cb1v5YBQZD5HQ=; b=mIXSbS3zx/wNtBmDxZy9pZVvSQJv/mHw4CtjWD65+3W9Nc24sZUEDDrNS8oORXHJiB CGLmGyovSMwnhhTJMVQUd2L8GZWQRYAR8SllfoLNKEh7wrEFC3j3GTPcpO6eGTt7IB6K lBtFoVj4NrI54l4nxgO5tQt5fd01MB+QIlXVuAOr/ab4O+2GR41LajuWCD6FMFuOkAbT foNteELlF+BDx6tehXHJkvTXF29r//7Ulc+DmZlFdhCsp3uvvGWYSsETbBZM2P3pBhTp ISJu5TQzkh/8DkQKSyiad0S01AG/Ia9PJmFtvYNhAgSLLO6dL8TA1pu24hsqMP5gFGpv Dniw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BGNwUbDjmHky71GpbBmE9Nh0iBILc0cb1v5YBQZD5HQ=; b=ZDP425oRvoBXKhJMiWjN/F4E9mvo8YiCeCTCfQ1qwCqVpJBv31xGwAJ//iHO79WgqD UDBuB+ZohynQgLd9D4yqw4dxcvxtGmrs4INExQSqFpMHCUpyBvAzWp4wxtDrdR8eJKVB V23xAGW/zJaqj9oESc3/TiYjUjTBHVBbhNJ26Gore5NC1ZS7rUDCtudIOTf2ybnxwoqa +SBzFcNjIcSWEWo8BSzYJ2BjKOkorotO43Ob7e6WP/hVdjn/QwPRkcSzuSqX/rmd52Ts 8vCxJlQkcdMwko5GLSBNg46srHywuiDRGu+Ak26NdxGNNZZSMw9AT21QE75uifyxJ06r MQRg== X-Gm-Message-State: AFqh2kqEMd9+eQMpxFMW7o+qM6qzeHblPAvI5GKl/L2Fka8OUwvwqOtj lMp83cGLeTJ5Mbc4FPx2uYg= X-Received: by 2002:a17:90b:1e4b:b0:229:ff05:691a with SMTP id pi11-20020a17090b1e4b00b00229ff05691amr23275110pjb.16.1674690105896; Wed, 25 Jan 2023 15:41:45 -0800 (PST) Received: from fedora.hsd1.ca.comcast.net ([2601:644:8002:1c20::4e4b]) by smtp.googlemail.com with ESMTPSA id e8-20020a17090a630800b00219752c8ea5sm2226806pjj.37.2023.01.25.15.41.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Jan 2023 15:41:45 -0800 (PST) From: "Vishal Moola (Oracle)" To: linux-mm@kvack.org Cc: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, "Vishal Moola (Oracle)" Subject: [PATCH mm-unstable v3 1/6] mm: Add folio_estimated_mapcount() Date: Wed, 25 Jan 2023 15:41:29 -0800 Message-Id: <20230125234134.227244-2-vishal.moola@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230125234134.227244-1-vishal.moola@gmail.com> References: <20230125234134.227244-1-vishal.moola@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756039914633691977?= X-GMAIL-MSGID: =?utf-8?q?1756039914633691977?= folio_estimated_mapcount() takes in a folio and returns the precise number of times the first subpage of the folio is mapped. This function aims to provide an estimate for the mapcount of a subpage within a folio. This is necessary for folio conversions where we care about the mapcount of a subpage, but not necessarily the whole folio. This is in contrast to folio_mapcount() which calculates the total number of the times a folio and all its subpages are mapped. Signed-off-by: Vishal Moola (Oracle) --- include/linux/mm.h | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index c9db257f09b3..fdd5b77ac209 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1817,6 +1817,23 @@ static inline size_t folio_size(struct folio *folio) return PAGE_SIZE << folio_order(folio); } +/** + * folio_estimated_mapcount - Estimate a folio's per-page mapcount. + * @folio: The folio. + * + * folio_estimated_mapcount() aims to serve as a function to efficiently + * estimate the number of times each page in a folio is mapped. + * This may not be accurate for large folios. If you want exact mapcounts, + * look at page_mapcount() or folio_total_mapcount(). + * + * Return: The precise mapcount of the first subpage, meant to estimate + * the precise mapcount of any subpage. + */ +static inline int folio_estimated_mapcount(struct folio *folio) +{ + return page_mapcount(folio_page(folio, 0)); +} + #ifndef HAVE_ARCH_MAKE_PAGE_ACCESSIBLE static inline int arch_make_page_accessible(struct page *page) { From patchwork Wed Jan 25 23:41:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Moola X-Patchwork-Id: 48423 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp543802wrn; Wed, 25 Jan 2023 15:42:56 -0800 (PST) X-Google-Smtp-Source: AMrXdXtZNbClO9Cx7b0k3znXKBay57Lq0FChsHTwsB+lsJchpoTNYoakKIRDn+c4JIDsIB3NPQlB X-Received: by 2002:a17:907:8c17:b0:871:38cc:7b3a with SMTP id ta23-20020a1709078c1700b0087138cc7b3amr27815718ejc.75.1674690176264; Wed, 25 Jan 2023 15:42:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674690176; cv=none; d=google.com; s=arc-20160816; b=WP6j8DJ2jPhY6gUAnzODCUHbkxBl/psmMHd06hW6WwhDNmx32bBlU3/+FA5ETbYF6S 2Vt1jl9RIjeRTPFik3O3mWv2HDmHSMF6xusgura/B929+FrlKwb5S1MkJ5cvOo2O9II/ GOcoIWhGBtcSIHb9fTUaTJvFnaXrMf/7d4jKJ0CjNtPluZVbiWctyyDJnK3TRaXpkcu5 u5e22blWyUZ31BfpWSBZQieXhJyoFte0Sq8gXs0VUOH4123En8e4ATcPl5zU99Yenzm+ PIWZkpXV3TOj2JrkFOdKSIsd+XBLAl9v8g0y6p7MrtVaMM4jWB2eNtQVo5riPNiC2nHs Guaw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=1khdYzxT7PMCcz7jBx9VZXJLxnPij2EE71ujrUVxn1A=; b=fkWUaNy+XvIhbclure5zZpS7UMG37Uca3aGiTmPldZwRPUFJAWzJd8CRyMCPHYHyD8 +UdFGM5x0A+CalDxUmVRQ3UtR6R7pDiRPB8guTk+jYLptUrHn7yvOsNkfsf640TITMmQ Y2Ac3oebx67dLTNdMFMjvrJbxRc2br+0nH4Wjwyod0T1dycnHEkmj4XSYbZpt1IYpSz/ 0CdyLD37bGO+R0WteKQH/iPUdEJOKNLvVfxZp3CZ8U581bJRFmCgzFmf03NWMOn45YeD WCW9+cHEsUjSjR6BBLEsM02wH7WPOe4OJlKqIBR5DVLm0VDP9uHJc0zLcu94WTEVyQOZ LEvg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b="UBtFM/nS"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 22-20020a170906021600b007c4fb4ee06fsi8261303ejd.531.2023.01.25.15.42.32; Wed, 25 Jan 2023 15:42:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b="UBtFM/nS"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236180AbjAYXlw (ORCPT + 99 others); Wed, 25 Jan 2023 18:41:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44024 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235859AbjAYXls (ORCPT ); Wed, 25 Jan 2023 18:41:48 -0500 Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A257F2D79 for ; Wed, 25 Jan 2023 15:41:47 -0800 (PST) Received: by mail-pl1-x631.google.com with SMTP id d3so418038plr.10 for ; Wed, 25 Jan 2023 15:41:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1khdYzxT7PMCcz7jBx9VZXJLxnPij2EE71ujrUVxn1A=; b=UBtFM/nSuni9xqZ8qtJkIk8LKEjARrE386v/XFm4q/6iNHXQglwU92LD/bg63vOquU XtqMQys200RkZrhJtyGKmphdie9l9Zb+gcF0lR2o2q/lBs2t6ClE41cQ5XjRo+Edpf9h F/md9YBpcoz6dthRROooSVzXlPU/MVHdUIy7JULq17/ImoUOXg3nmk8rQMNBYRjdCoFE s0Ynvr4xdLyHXZRENkORJ/T5iABEwi93EyCQsVc6yO0tidjP/AUx8OD8A94a2VejPLdP H2+pU1CayBQrjSiZzZHcZ4Az8LmDqjh5OsliDpGe4V/0lp3jnPy/iqNSK2qSaG1aZ3rI sqgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1khdYzxT7PMCcz7jBx9VZXJLxnPij2EE71ujrUVxn1A=; b=wBSKQDEB4j0vu/6xTh2Bm+90u+cR5urfhTnd4jZNAGPh+C7E3KEVSvaLH7Y8vYICI8 wjTo5bTaOzevxkSSOZUG2EWMxPFAoDi7CCgeRWUb5bep7OLgLhLMjYYd/ikoaW5aOfER tg+zYtHRQaakOmc1iDRcvIX843mnp3KiWVwOGgFQo8PZSvlRdlg6Aq0PlF/fuIug79de 7rT6+xUP4XIOZxQuw0ry3YtfhM25Ph4Ut/OnckyqyHEzJerntVfFRvuccNqiBM2eJkz6 uOhhZld2ZYGMDb5iurvsPLHCamkfQ+s99/lIG85IqsEFGIgkMS/BeH2d1B9Q5eWK2cP/ JFVA== X-Gm-Message-State: AO0yUKVE9kYtTEFffA+Qlw1ITzrNH7YHwNrZhQnq9SIHQcyaiHNrejhi Q76qN5lhku9qYoAWKa9XMi4= X-Received: by 2002:a17:90a:14a5:b0:22b:b25a:d0a0 with SMTP id k34-20020a17090a14a500b0022bb25ad0a0mr4742pja.15.1674690107112; Wed, 25 Jan 2023 15:41:47 -0800 (PST) Received: from fedora.hsd1.ca.comcast.net ([2601:644:8002:1c20::4e4b]) by smtp.googlemail.com with ESMTPSA id e8-20020a17090a630800b00219752c8ea5sm2226806pjj.37.2023.01.25.15.41.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Jan 2023 15:41:46 -0800 (PST) From: "Vishal Moola (Oracle)" To: linux-mm@kvack.org Cc: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, "Vishal Moola (Oracle)" Subject: [PATCH mm-unstable v3 2/6] mm/mempolicy: Convert queue_pages_pmd() to queue_folios_pmd() Date: Wed, 25 Jan 2023 15:41:30 -0800 Message-Id: <20230125234134.227244-3-vishal.moola@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230125234134.227244-1-vishal.moola@gmail.com> References: <20230125234134.227244-1-vishal.moola@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756039926067792936?= X-GMAIL-MSGID: =?utf-8?q?1756039926067792936?= The function now operates on a folio instead of the page associated with a pmd. This change is in preparation for the conversion of queue_pages_required() to queue_folio_required() and migrate_page_add() to migrate_folio_add(). Signed-off-by: Vishal Moola (Oracle) --- mm/mempolicy.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index fd99d303e34f..00fffa93adae 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -442,21 +442,21 @@ static inline bool queue_pages_required(struct page *page, } /* - * queue_pages_pmd() has three possible return values: - * 0 - pages are placed on the right node or queued successfully, or + * queue_folios_pmd() has three possible return values: + * 0 - folios are placed on the right node or queued successfully, or * special page is met, i.e. huge zero page. - * 1 - there is unmovable page, and MPOL_MF_MOVE* & MPOL_MF_STRICT were + * 1 - there is unmovable folio, and MPOL_MF_MOVE* & MPOL_MF_STRICT were * specified. * -EIO - is migration entry or only MPOL_MF_STRICT was specified and an - * existing page was already on a node that does not follow the + * existing folio was already on a node that does not follow the * policy. */ -static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr, +static int queue_folios_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr, unsigned long end, struct mm_walk *walk) __releases(ptl) { int ret = 0; - struct page *page; + struct folio *folio; struct queue_pages *qp = walk->private; unsigned long flags; @@ -464,19 +464,19 @@ static int queue_pages_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr, ret = -EIO; goto unlock; } - page = pmd_page(*pmd); - if (is_huge_zero_page(page)) { + folio = pfn_folio(pmd_pfn(*pmd)); + if (is_huge_zero_page(&folio->page)) { walk->action = ACTION_CONTINUE; goto unlock; } - if (!queue_pages_required(page, qp)) + if (!queue_pages_required(&folio->page, qp)) goto unlock; flags = qp->flags; - /* go to thp migration */ + /* go to folio migration */ if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) { if (!vma_migratable(walk->vma) || - migrate_page_add(page, qp->pagelist, flags)) { + migrate_page_add(&folio->page, qp->pagelist, flags)) { ret = 1; goto unlock; } @@ -512,7 +512,7 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, ptl = pmd_trans_huge_lock(pmd, vma); if (ptl) - return queue_pages_pmd(pmd, ptl, addr, end, walk); + return queue_folios_pmd(pmd, ptl, addr, end, walk); if (pmd_trans_unstable(pmd)) return 0; From patchwork Wed Jan 25 23:41:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Moola X-Patchwork-Id: 48424 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp544424wrn; Wed, 25 Jan 2023 15:44:39 -0800 (PST) X-Google-Smtp-Source: AMrXdXtRSX6mvhQ4V1W6r0vFKiycRmJ6RRN7STnKLgtFz+xpOHpx2c31U28FNLwdOHYdsUTs0wU7 X-Received: by 2002:a05:6402:4ce:b0:47f:bc9b:46ec with SMTP id n14-20020a05640204ce00b0047fbc9b46ecmr34245002edw.7.1674690279044; Wed, 25 Jan 2023 15:44:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674690279; cv=none; d=google.com; s=arc-20160816; b=PmvLdheoZxljg2ptskRPRrIw+Wo4qDGuFqr7zzuDrQkDWONZbNVhtJg7R6p3iXq6Em OuxP4CC+bJfT2O0UMuHoLLhfHfRIUK56K0rmKl+qwgOfQGaQqd/23HoY6b607FWnXtxB 2M/DmtQJh07SvM+p01CTCXjBNGAU6jkSKhzI9pWuqhCJwq1/UDmxs0CMy1TNaOJvIETy k5OsfYL6kDtAM1yOdAxXDtg0dpJ1/HbQmFKiBLwNFUgm0iDtCbe0KeNAbvoB3NOjidc0 fAhNDUBbOtSL5bImgm4VNVZx5xCT04lvGeITzy9DPgphV6r4PvH+6LmXczx8gcb+tQ7g hNtQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=8vZic6jN1vVBvksvwJz1xQpCGfyGjrm+SSW90x/iIgk=; b=kYh2EMnr23MskdYpA1cK1yNvMw+xbYPPgz9yT2+4OEivK2BqbUobfUJyUW814YVa4X c933J6hGdEZmhd+aL+6JM1e5sqy68n1Ei9G5WMkPlf8HR1rva+kGQhZFz37fA1AwWN75 2DsDw8/C+v/q+pdEewieTySMkCuAeakPdabDpYyKKM45vJ7ilZB9utykdUZa05hMA9O0 3cbVQf0qz2bMdgMjxD9zmeDqJyWAg1T+aDXpX5lxjtKY4gAdcCZ2tVUHSAZ9Uc6/SiZa B8jdreRI1yL9v3hO9/qlX6BL4ahH1dxwzHM2ZtreASCLlIM/0XVzfIK44fq7xJvPqvHw FwRA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=Kh4RfcbG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 12-20020a17090600cc00b00870b95fb7c4si8385368eji.998.2023.01.25.15.44.09; Wed, 25 Jan 2023 15:44:39 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=Kh4RfcbG; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236232AbjAYXlz (ORCPT + 99 others); Wed, 25 Jan 2023 18:41:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44088 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236055AbjAYXlu (ORCPT ); Wed, 25 Jan 2023 18:41:50 -0500 Received: from mail-pj1-x1029.google.com (mail-pj1-x1029.google.com [IPv6:2607:f8b0:4864:20::1029]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 884865FE1 for ; Wed, 25 Jan 2023 15:41:48 -0800 (PST) Received: by mail-pj1-x1029.google.com with SMTP id j5so8592pjn.5 for ; Wed, 25 Jan 2023 15:41:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8vZic6jN1vVBvksvwJz1xQpCGfyGjrm+SSW90x/iIgk=; b=Kh4RfcbGLDn6wlmfeljol0sY1eBsJweLAeurosHOgEoNqinDlrYKhY8araOCBOa8wF I/o3DWf7ZyhOwYrBnjIJRzBu/1oB/NNffx5MEncS4Ysj8sudz0DvCcQtY9fGPeFgMfF3 gKedCJxlAVgpI44tRJIRNaoo2AD/9FsRM5z2GBhyTI2MAjFKxNDv5qbOn4GX0RlVDWbT wOIc3zu6WO9W0NW0LMVWZ/+cPSyma7Hk0MtPYdhVAYyp8QnbYyoyev+Rcuk4Y3LqRtIm OtWmcd7GqPnf+HwQ2pxmx2LZVvI6PszEEAQlIgBPp3TtL0QXQaoEbcrbEc6fAkM9ag6v nOwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8vZic6jN1vVBvksvwJz1xQpCGfyGjrm+SSW90x/iIgk=; b=vf/G0W95z75W6ySqF4yEpZjvYRS9zbYsU22I3kHdNAzGgdCpRD5sU22isuzy2DGIYZ TkDfGSscapbxVBDXgikPmp+MXBETYf133qBppYcMYMVnlyRIN+ZJf9tA7pUWX2rDUiww y3O+nDD4zG0TA7ZNS1sAa32GkQWTxZpxPD/7m1d7XO3rX5wLXsl94d1R4l8oJmqZ/Qan 7ygsosGiyD9nUjgFp6GwJbdySjOj7yQkOhJDMy8z6MiWqchMZhaz1HJNNUhokFN21Ptd PdKGE10bP3Ql4acG0R2L6aWvzsxk/FyJOcZrWWV2W7QIhoAqPuVM8zTYYv7qxZG2edbD 4INQ== X-Gm-Message-State: AFqh2kosJIlstvg3MoRERcrwraO27TGN1RUss+oOR+3j5LDbnTk64QVE Bk4hMRrIXkqXJRqjfVzqYbA= X-Received: by 2002:a17:90a:7347:b0:226:b52e:f1b8 with SMTP id j7-20020a17090a734700b00226b52ef1b8mr35518920pjs.24.1674690107979; Wed, 25 Jan 2023 15:41:47 -0800 (PST) Received: from fedora.hsd1.ca.comcast.net ([2601:644:8002:1c20::4e4b]) by smtp.googlemail.com with ESMTPSA id e8-20020a17090a630800b00219752c8ea5sm2226806pjj.37.2023.01.25.15.41.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Jan 2023 15:41:47 -0800 (PST) From: "Vishal Moola (Oracle)" To: linux-mm@kvack.org Cc: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, "Vishal Moola (Oracle)" Subject: [PATCH mm-unstable v3 3/6] mm/mempolicy: Convert queue_pages_pte_range() to queue_folios_pte_range() Date: Wed, 25 Jan 2023 15:41:31 -0800 Message-Id: <20230125234134.227244-4-vishal.moola@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230125234134.227244-1-vishal.moola@gmail.com> References: <20230125234134.227244-1-vishal.moola@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756040034200563487?= X-GMAIL-MSGID: =?utf-8?q?1756040034200563487?= This function now operates on folios associated with ptes instead of pages. This change is in preparation for the conversion of queue_pages_required() to queue_folio_required() and migrate_page_add() to migrate_folio_add(). Signed-off-by: Vishal Moola (Oracle) --- mm/mempolicy.c | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 00fffa93adae..ae9d16124f45 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -491,19 +491,19 @@ static int queue_folios_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr, * Scan through pages checking if pages follow certain conditions, * and move them to the pagelist if they do. * - * queue_pages_pte_range() has three possible return values: - * 0 - pages are placed on the right node or queued successfully, or + * queue_folios_pte_range() has three possible return values: + * 0 - folios are placed on the right node or queued successfully, or * special page is met, i.e. zero page. - * 1 - there is unmovable page, and MPOL_MF_MOVE* & MPOL_MF_STRICT were + * 1 - there is unmovable folio, and MPOL_MF_MOVE* & MPOL_MF_STRICT were * specified. - * -EIO - only MPOL_MF_STRICT was specified and an existing page was already + * -EIO - only MPOL_MF_STRICT was specified and an existing folio was already * on a node that does not follow the policy. */ -static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, +static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, struct mm_walk *walk) { struct vm_area_struct *vma = walk->vma; - struct page *page; + struct folio *folio; struct queue_pages *qp = walk->private; unsigned long flags = qp->flags; bool has_unmovable = false; @@ -521,16 +521,16 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, for (; addr != end; pte++, addr += PAGE_SIZE) { if (!pte_present(*pte)) continue; - page = vm_normal_page(vma, addr, *pte); - if (!page || is_zone_device_page(page)) + folio = vm_normal_folio(vma, addr, *pte); + if (!folio || folio_is_zone_device(folio)) continue; /* - * vm_normal_page() filters out zero pages, but there might - * still be PageReserved pages to skip, perhaps in a VDSO. + * vm_normal_folio() filters out zero pages, but there might + * still be reserved folios to skip, perhaps in a VDSO. */ - if (PageReserved(page)) + if (folio_test_reserved(folio)) continue; - if (!queue_pages_required(page, qp)) + if (!queue_pages_required(&folio->page, qp)) continue; if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) { /* MPOL_MF_STRICT must be specified if we get here */ @@ -544,7 +544,7 @@ static int queue_pages_pte_range(pmd_t *pmd, unsigned long addr, * temporary off LRU pages in the range. Still * need migrate other LRU pages. */ - if (migrate_page_add(page, qp->pagelist, flags)) + if (migrate_page_add(&folio->page, qp->pagelist, flags)) has_unmovable = true; } else break; @@ -703,7 +703,7 @@ static int queue_pages_test_walk(unsigned long start, unsigned long end, static const struct mm_walk_ops queue_pages_walk_ops = { .hugetlb_entry = queue_pages_hugetlb, - .pmd_entry = queue_pages_pte_range, + .pmd_entry = queue_folios_pte_range, .test_walk = queue_pages_test_walk, }; From patchwork Wed Jan 25 23:41:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Moola X-Patchwork-Id: 48425 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp556wrn; Wed, 25 Jan 2023 15:59:49 -0800 (PST) X-Google-Smtp-Source: AK7set+KsZWdihGkXApyBqqiQVRPOAzDNQMVuUPfyEe4g9YK+nisqiR8LurC+pzyuIy2PeQz1Ung X-Received: by 2002:a17:90b:224c:b0:22b:fd18:f31d with SMTP id hk12-20020a17090b224c00b0022bfd18f31dmr5896954pjb.45.1674691188986; Wed, 25 Jan 2023 15:59:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674691188; cv=none; d=google.com; s=arc-20160816; b=F1ocooT0adkip3ucxfTsyfEKv2HXPcrYNrCiCY1DsBy5LNR3de94amMqNi04MVB7O5 QX3hcIl/+/aW/Y0qs/eAXIwmIF4R2hu6FL3UwtHJFItqFvBNFp08l6TSNcgPk0247vYE /oWfnzGcDXEAjWZl/ETm1nAt6LuWbEt+SAr1ikwLnuL+F2Oyc6hSLSvSE2MjvKhrDC5z C37XJcmduAwz8ky4DZo5NryYzAph/TJ00ntZn0bSIpWdlj1vlzuBD3gs05kkWdulEpiR zwImElmFDWlGquHjgQQ2iq1wE4oJft1lOI7M85ndygy5lKL5VxuhLYH9N2i7GmEi3X9u nU6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Sz4HU32GWbuxfmIkQjW0u4ISqBHZ5RIiVnsfOJ2Lp54=; b=jyo6ZB7y0qNIyvMn0bkwPwB8K/0RXi/3hSacEiwAMWqOvQywA9rwoOVlXQ0k3lMLi5 B5B8QqzQnPccCJqI2sx9eKevFqK/LY5MZ2LSpgrZ1swgc5zUfljwYtFvx3EpA3PjQDUc 9jBVlGmLCvbomzapMR0bIQT6rzp9yl4FDQ617/ih1d9WYY7sknkN14VgSwkwIAwuZyad e6cwCi9Xyu76pyes5a0ickNroUsl4Y8JWXpzkl7pVM7g5TVoPyjH6/OQ93dFOLFqYj5O uch4E2qNLj912Q5uaCQPXMbQ1zdO1rxm5a1MfvxUCS3Yrm0lXKbugp31k+4MjZWmB9sT r79w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=dmcxWY8U; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id mm5-20020a17090b358500b00226e2321ea3si3327719pjb.157.2023.01.25.15.59.36; Wed, 25 Jan 2023 15:59:48 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=dmcxWY8U; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229688AbjAYXl6 (ORCPT + 99 others); Wed, 25 Jan 2023 18:41:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236108AbjAYXlu (ORCPT ); Wed, 25 Jan 2023 18:41:50 -0500 Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A6FC515C8A for ; Wed, 25 Jan 2023 15:41:49 -0800 (PST) Received: by mail-pj1-x102a.google.com with SMTP id z9-20020a17090a468900b00226b6e7aeeaso241884pjf.1 for ; Wed, 25 Jan 2023 15:41:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Sz4HU32GWbuxfmIkQjW0u4ISqBHZ5RIiVnsfOJ2Lp54=; b=dmcxWY8U8wAh2AlCeZFmiUjni88jRf+FuhGzSc73OspKeJg/jDgHIv8asRJVZPLE7M GeKI9hjadJUH2P0ponOvP3pbBraa8S/sk+8nTl4/oqYo8JIFSVUDaHw97FrPx7bfNMeo IJM+6DNiaCAdp7b5r7OTFO2h/cIicEasMLMQulW80HW0FxGMwht/P0Z1SgDxEeIXj8ay 2XqexkXBfBMFdPiiPa5MRPZOB4u9lcaP/PQFrDUw4y91ocEY92PcwC3UgXttppZ+VloB bHpeYO3CnwoDG+CP4gqrrpWVu95EqQynL873hIUCRONzU4/DRWp2/OjNZ5OfpzIHluv5 h5Vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Sz4HU32GWbuxfmIkQjW0u4ISqBHZ5RIiVnsfOJ2Lp54=; b=Us+zlwAzv8xA7AYGLHLg9hXkDIbGswTbtBjHiJieCEw8HYYcQ8flIjYlUq6odp0Wtu JunraBMxuhLhk236MFfk7c5FvE0B+HES5KpLB8oc6kSM4GP9NOoJ0zYZuhibtZjlAll0 mqIzw8yPdeJu9WUA7t+IUbFx3j3CzShZWXqJj7Jk8azIxX5nYik7gS1z6AJ7RUD12sEn AIxhDgaJLD+m9+P9L8hES59bbI1GjGbaNhpXy3Jyppvkk81f41zrQIvAJ/rF2MRaaD1d X5195XtGxc4aynblrdNeQ8hT/fPAlOj7fjzrB0nRvDFIKoeR/zoWUOiuQs7AO7xF2vp0 H3ow== X-Gm-Message-State: AFqh2ko6/Zg+vK1VgzUXtBMxz3iB6dYGqLqjd+awUy46Zl95iwzDQWao Dcvk7aamQ0vEqLv+i8ASkWM= X-Received: by 2002:a17:90b:4fc2:b0:229:680:1729 with SMTP id qa2-20020a17090b4fc200b0022906801729mr34762484pjb.10.1674690108968; Wed, 25 Jan 2023 15:41:48 -0800 (PST) Received: from fedora.hsd1.ca.comcast.net ([2601:644:8002:1c20::4e4b]) by smtp.googlemail.com with ESMTPSA id e8-20020a17090a630800b00219752c8ea5sm2226806pjj.37.2023.01.25.15.41.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Jan 2023 15:41:48 -0800 (PST) From: "Vishal Moola (Oracle)" To: linux-mm@kvack.org Cc: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, "Vishal Moola (Oracle)" Subject: [PATCH mm-unstable v3 4/6] mm/mempolicy: Convert queue_pages_hugetlb() to queue_folios_hugetlb() Date: Wed, 25 Jan 2023 15:41:32 -0800 Message-Id: <20230125234134.227244-5-vishal.moola@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230125234134.227244-1-vishal.moola@gmail.com> References: <20230125234134.227244-1-vishal.moola@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756040988361404587?= X-GMAIL-MSGID: =?utf-8?q?1756040988361404587?= This change is in preparation for the conversion of queue_pages_required() to queue_folio_required() and migrate_page_add() to migrate_folio_add(). Signed-off-by: Vishal Moola (Oracle) --- mm/mempolicy.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index ae9d16124f45..ea8cac447e04 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -558,7 +558,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr, return addr != end ? -EIO : 0; } -static int queue_pages_hugetlb(pte_t *pte, unsigned long hmask, +static int queue_folios_hugetlb(pte_t *pte, unsigned long hmask, unsigned long addr, unsigned long end, struct mm_walk *walk) { @@ -566,7 +566,7 @@ static int queue_pages_hugetlb(pte_t *pte, unsigned long hmask, #ifdef CONFIG_HUGETLB_PAGE struct queue_pages *qp = walk->private; unsigned long flags = (qp->flags & MPOL_MF_VALID); - struct page *page; + struct folio *folio; spinlock_t *ptl; pte_t entry; @@ -574,13 +574,13 @@ static int queue_pages_hugetlb(pte_t *pte, unsigned long hmask, entry = huge_ptep_get(pte); if (!pte_present(entry)) goto unlock; - page = pte_page(entry); - if (!queue_pages_required(page, qp)) + folio = pfn_folio(pte_pfn(entry)); + if (!queue_pages_required(&folio->page, qp)) goto unlock; if (flags == MPOL_MF_STRICT) { /* - * STRICT alone means only detecting misplaced page and no + * STRICT alone means only detecting misplaced folio and no * need to further check other vma. */ ret = -EIO; @@ -591,20 +591,27 @@ static int queue_pages_hugetlb(pte_t *pte, unsigned long hmask, /* * Must be STRICT with MOVE*, otherwise .test_walk() have * stopped walking current vma. - * Detecting misplaced page but allow migrating pages which + * Detecting misplaced folio but allow migrating folios which * have been queued. */ ret = 1; goto unlock; } - /* With MPOL_MF_MOVE, we migrate only unshared hugepage. */ + /* + * With MPOL_MF_MOVE, we try to migrate only unshared folios. If it + * is shared it is likely not worth migrating. + * + * To check if the folio is shared, ideally we want to make sure + * every page is mapped to the same process. Doing that is very + * expensive, so check the estimated mapcount of the folio instead. + */ if (flags & (MPOL_MF_MOVE_ALL) || - (flags & MPOL_MF_MOVE && page_mapcount(page) == 1)) { - if (isolate_hugetlb(page_folio(page), qp->pagelist) && + (flags & MPOL_MF_MOVE && folio_estimated_mapcount(folio) == 1)) { + if (isolate_hugetlb(folio, qp->pagelist) && (flags & MPOL_MF_STRICT)) /* - * Failed to isolate page but allow migrating pages + * Failed to isolate folio but allow migrating folios * which have been queued. */ ret = 1; @@ -702,7 +709,7 @@ static int queue_pages_test_walk(unsigned long start, unsigned long end, } static const struct mm_walk_ops queue_pages_walk_ops = { - .hugetlb_entry = queue_pages_hugetlb, + .hugetlb_entry = queue_folios_hugetlb, .pmd_entry = queue_folios_pte_range, .test_walk = queue_pages_test_walk, }; From patchwork Wed Jan 25 23:41:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Moola X-Patchwork-Id: 48426 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2923wrn; Wed, 25 Jan 2023 16:04:35 -0800 (PST) X-Google-Smtp-Source: AK7set/aMBwOjqf4NCa+Jltmb0EcByKqzt8kBGdtPAnq7UkoQE6uaR4+FRXUpIZgoBxo4n7J1V+l X-Received: by 2002:a17:90b:3b85:b0:22c:19e4:1ec with SMTP id pc5-20020a17090b3b8500b0022c19e401ecmr1341991pjb.11.1674691474685; Wed, 25 Jan 2023 16:04:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674691474; cv=none; d=google.com; s=arc-20160816; b=gHLiXpA++WMjkGCppVSCEPiun+MS9fOc2y8KrVu9njqghQsSG6anYy+mzwmzlY8dPx DOFYzcSZaEVvM8l2Q6Bu08aXwBizE2I1UNFYL/DyBi6NF3nG7Wu4q2U+nVSQJrZaBgoL YZZvkFWgPr2KlpYgkzcBlPns/jCKs0EFaITosnWsViYT7GOFyV3+e7HKPTajGhChNY7h 77GXrSIEvIPLajxwxaAwgbfT9WO2DTdWhw4gy8R8neZZPFtvGkqcL3iXdqoXtPfpWG4x 2cE36YZxVmF6VFejEJIPBrvre04uYL3yPsuQvqSht5hYys561KK7BI9JxVkyHfy7eEYX 59xQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=nE+3UuHIJTsmxu1JqArGmSE4JP62IBIElITMZZ15/p0=; b=CWwO5Zwe4b2uX+yu0IF3n70XoqgGfe/ewsHZoWQn4cfezQxgEj0JS3CeoqwUbYsEdT /ColNERba1PdLs3oxjwLOrn72NE7PcxWIPPkiTp9M3Lm99UEs6ORdYK6jXeOZmiW/nbV q964MGR/psQoGX2K6WL5NI4UbqTFIB5GTyIoVpLSyFNMW4GUSYwrW1bb3KCkv5kIY2QG HZ4sd92GyLw3g5FrfNcuI1+KXRe9vCHPbu7O54X7ma0UFiY02Gr+Ik28Uuevg7KFULg8 3w9jZaXfBp7QZpGF0QlDe9J1U3uxO7f9pBai6ZWJBC9ShJF4geP3REzlsuWnHg5ZYbPl +EPA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=jQrVZxKe; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id v6-20020a63b646000000b004cd1e132878si7180319pgt.764.2023.01.25.16.04.21; Wed, 25 Jan 2023 16:04:34 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=jQrVZxKe; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236268AbjAYXmD (ORCPT + 99 others); Wed, 25 Jan 2023 18:42:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44234 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236147AbjAYXlw (ORCPT ); Wed, 25 Jan 2023 18:41:52 -0500 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D0522298E2 for ; Wed, 25 Jan 2023 15:41:50 -0800 (PST) Received: by mail-pj1-x1032.google.com with SMTP id z1-20020a17090a66c100b00226f05b9595so270801pjl.0 for ; Wed, 25 Jan 2023 15:41:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nE+3UuHIJTsmxu1JqArGmSE4JP62IBIElITMZZ15/p0=; b=jQrVZxKeueZyKOX/9U6abED0nBNMUqF3gG1HIZMoK5tlZcQs9cJ/ODvbcqhthYHpKQ DWBlgZq8Tb2XiUpPQuMtypfYZ9+e6LBX9aHXQkiGdmyvdgULRvOsFnVYvZyBzK93TgON yT+3SjOAyRVIKqC2tyImtBCNxFkufKllds3KVdssuCQ158DP+AUXt5N9a7sXPIwJu0b7 6NUR4WdIza3XMkpwmk7ebXCPL8ZC27KhnroLf9RZKkf8URh1kFoa0wxqKDWcfTalHBmd 5ICDiFBJCDc915aV0MFz6KVHGYDjIaaj6cqBHLkLCcrCG7fiSVT8xzU8d7ZO0yGQVsMP GiWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nE+3UuHIJTsmxu1JqArGmSE4JP62IBIElITMZZ15/p0=; b=K/9AGnEcVwn2xrHEm7y/QuAQ3UE1XyHv6Opemm4anIvYSOj77kkET4oIOe5w9gZJhY gnqxcmYjFpDeF4CjhHRNi2HP/aZWsHJqf9wjysiGVv0zRiHoIiOrWC6KlPE+5TfhN7Vn GLVq3Pi/z5WMJTEMu1l+BlDZzY5Q1CpTIClBNp1tiMUEwwLNlTFqBA6PPj7VHZFl3+Xk QPuz4XprQgALOxfHMIMa7kG86ls32A3HVdqtMxoBlFChqjwlG3ytoEwsS1AEOI6RpbJs 27iVPFxfVVza7yQIM7q0Dxc+w4/F+hRwgA8PgTS7dmIwVrIhowHsk2cuz/D1mdvUc5r1 DIjw== X-Gm-Message-State: AFqh2krdm6A2/f5B2UylzhsgJb70yfv2F/SnBFsfUzn8xmnNTw5EtAQB 54N46ZEF5TfRcODeI9YFWX4= X-Received: by 2002:a17:90a:7804:b0:229:912:1340 with SMTP id w4-20020a17090a780400b0022909121340mr34515942pjk.39.1674690110189; Wed, 25 Jan 2023 15:41:50 -0800 (PST) Received: from fedora.hsd1.ca.comcast.net ([2601:644:8002:1c20::4e4b]) by smtp.googlemail.com with ESMTPSA id e8-20020a17090a630800b00219752c8ea5sm2226806pjj.37.2023.01.25.15.41.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Jan 2023 15:41:49 -0800 (PST) From: "Vishal Moola (Oracle)" To: linux-mm@kvack.org Cc: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, "Vishal Moola (Oracle)" Subject: [PATCH mm-unstable v3 5/6] mm/mempolicy: Convert queue_pages_required() to queue_folio_required() Date: Wed, 25 Jan 2023 15:41:33 -0800 Message-Id: <20230125234134.227244-6-vishal.moola@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230125234134.227244-1-vishal.moola@gmail.com> References: <20230125234134.227244-1-vishal.moola@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756041287932710726?= X-GMAIL-MSGID: =?utf-8?q?1756041287932710726?= Replace queue_pages_required() with queue_folio_required(). queue_folio_required() does the same as queue_pages_required(), except takes in a folio instead of a page. Signed-off-by: Vishal Moola (Oracle) --- mm/mempolicy.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index ea8cac447e04..da87644430e3 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -427,15 +427,15 @@ struct queue_pages { }; /* - * Check if the page's nid is in qp->nmask. + * Check if the folio's nid is in qp->nmask. * * If MPOL_MF_INVERT is set in qp->flags, check if the nid is * in the invert of qp->nmask. */ -static inline bool queue_pages_required(struct page *page, +static inline bool queue_folio_required(struct folio *folio, struct queue_pages *qp) { - int nid = page_to_nid(page); + int nid = folio_nid(folio); unsigned long flags = qp->flags; return node_isset(nid, *qp->nmask) == !(flags & MPOL_MF_INVERT); @@ -469,7 +469,7 @@ static int queue_folios_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr, walk->action = ACTION_CONTINUE; goto unlock; } - if (!queue_pages_required(&folio->page, qp)) + if (!queue_folio_required(folio, qp)) goto unlock; flags = qp->flags; @@ -530,7 +530,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr, */ if (folio_test_reserved(folio)) continue; - if (!queue_pages_required(&folio->page, qp)) + if (!queue_folio_required(folio, qp)) continue; if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) { /* MPOL_MF_STRICT must be specified if we get here */ @@ -575,7 +575,7 @@ static int queue_folios_hugetlb(pte_t *pte, unsigned long hmask, if (!pte_present(entry)) goto unlock; folio = pfn_folio(pte_pfn(entry)); - if (!queue_pages_required(&folio->page, qp)) + if (!queue_folio_required(folio, qp)) goto unlock; if (flags == MPOL_MF_STRICT) { From patchwork Wed Jan 25 23:41:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Moola X-Patchwork-Id: 48427 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp3084wrn; Wed, 25 Jan 2023 16:04:54 -0800 (PST) X-Google-Smtp-Source: AK7set9Dn/Ez5oQ5MLeEyGHiTxaPjTk2gG9DEklKIjGIa1J+CZZPIOx8n99KLqK4ThW668h8e271 X-Received: by 2002:a05:6a20:296:b0:b8:64a2:2f80 with SMTP id 22-20020a056a20029600b000b864a22f80mr15243pza.23.1674691494536; Wed, 25 Jan 2023 16:04:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674691494; cv=none; d=google.com; s=arc-20160816; b=kG0spTFqFIoVuNGQO9U8gdKOqovu8sjGKifPt8I4npnR00j4SXHWDayOAw+Ja2e9F6 g7WycXMioHngxNU7jukcU6dtCDxPUcI+inB2W8NvoeqzmiOqOCtMG68xvM7zgtfQ+n3a 0QtrfmyOtTQvboxkzvyTMc/vO30Fq7FhiWQVh6wl06rpsVT062WQSldNfOpqRxiOlEeM cPOCtMXYTsMWnV3aG2jXQd5tqjmr1NmHQsI5qMZe9Aqu9axPc1uzBPa9gZeXgtmOvmm2 nXiTFmfbalh+FkSwFmw319d/WjOLtpJgI1cPMGeCLb8tH9XnPsBO6Yr8Za+OIeTHDOXa Jjig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=aN57XfC0N2Z9PnMQ9nUjXYfVmJE2FGzydu2ZVvrzX3Q=; b=AqjaJ236lf9nJafHbLDooQTGGvWV+qMnBiVZdj4q42XWWjYxhYJPleKSAu5Sws/amd kKMR5D8o58DPVFtxrRQuYMXxGg4LCp0kI9IHO8uO6NoPCBUkNFXfjIw/knArL6vsU7bM /T1PLx7CPjKRljcqkKmvwj4uXlFl9c3YHW4rAVAtlsQKRtL8R3JuH521hwlHKwB5K3EU rN+E1sJR7HrgThFvgMLdGOOHY4xQtFmLqfa4QmVlBE3m6XnDNYI25Z277thbRivlUoJw w8uTxpREPGRtA0mp8WUiVu0fowTUDm2WUaWv38hso9iW7Hp8Bi2qjxVAdgT1qjvhoC0R a7bQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=P6CDrkzJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w22-20020a63af16000000b004a069d07e13si6923535pge.734.2023.01.25.16.04.35; Wed, 25 Jan 2023 16:04:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=P6CDrkzJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236251AbjAYXmB (ORCPT + 99 others); Wed, 25 Jan 2023 18:42:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44246 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236182AbjAYXlw (ORCPT ); Wed, 25 Jan 2023 18:41:52 -0500 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 991AA5FDB for ; Wed, 25 Jan 2023 15:41:51 -0800 (PST) Received: by mail-pj1-x102b.google.com with SMTP id nm12-20020a17090b19cc00b0022c2155cc0bso211744pjb.4 for ; Wed, 25 Jan 2023 15:41:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aN57XfC0N2Z9PnMQ9nUjXYfVmJE2FGzydu2ZVvrzX3Q=; b=P6CDrkzJSgET54MDlG18KNSNOlZoW3MU7+hkgLqRJ/5Xqzgl8we5PZOg1hkICvX2CV qJuAyZ0zwDf79dalVbx6THwSx5WpRv7irQY4fWrO65cPK9qxlsBx0aRlT9bMDJUF1IWk Vdc3Flt7m1v4/SJ2NceeyIu3ROeYczykNgzpa6MKvRiWkg3DKWUnET3/mPqUh8G50olU TZeUw09UecGDHUoaAyfy+kXlHDrlCQD7OBWh+QrMS+SXz98gVihHyB7H5Lkz5OvooRno IJSQLLalXZorgQX63BTIU5Rmz6BSXpttPAnERpvzhMa9w+rhHUUo1reXjMpAwfuE0P5c DeAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aN57XfC0N2Z9PnMQ9nUjXYfVmJE2FGzydu2ZVvrzX3Q=; b=TeFtzNW+SQGD9ZrMLThSLubp8Al03gq7LI2U5ou7ElCiebmI6+hUEpMEQ1OyCrXcC/ knplikuJeUV7efobtpev3MzJoWQVU0kycFH5qe2xjSOGLt0KhG1jG2H9fAkpWXwtOnGc qZbo2L+CsrO6vyRdSzfw+rsmTJrluJUXfIjKkij4TPPS0iUiZH4A/Tz6z/OhYk7Z2dYN Ve2wD6GjjtoQCaAvJUHuwiyh55WBC861ySf52N6E7DDhxOWyuFByLkEneQx98gkLnynU DBQiXkN7RCeFNI/F+8IddOSWOvXV/dmjG2tTzC4ijA5sBUn9F2O/l7HhBmp/tmuP5pA3 sy4Q== X-Gm-Message-State: AFqh2kodCdR7RqvGQ/QqhRjkQ0HQtu0RJFSDopTY97s5HSR1WjIaAxcC 4k0Xh408sX4hGtgsJdXVgyQVwQejn+t/9w== X-Received: by 2002:a17:90a:86ca:b0:225:d697:41ea with SMTP id y10-20020a17090a86ca00b00225d69741eamr36754049pjv.23.1674690111263; Wed, 25 Jan 2023 15:41:51 -0800 (PST) Received: from fedora.hsd1.ca.comcast.net ([2601:644:8002:1c20::4e4b]) by smtp.googlemail.com with ESMTPSA id e8-20020a17090a630800b00219752c8ea5sm2226806pjj.37.2023.01.25.15.41.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Jan 2023 15:41:50 -0800 (PST) From: "Vishal Moola (Oracle)" To: linux-mm@kvack.org Cc: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, "Vishal Moola (Oracle)" Subject: [PATCH mm-unstable v3 6/6] mm/mempolicy: Convert migrate_page_add() to migrate_folio_add() Date: Wed, 25 Jan 2023 15:41:34 -0800 Message-Id: <20230125234134.227244-7-vishal.moola@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230125234134.227244-1-vishal.moola@gmail.com> References: <20230125234134.227244-1-vishal.moola@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1756041308907551647?= X-GMAIL-MSGID: =?utf-8?q?1756041308907551647?= Replace migrate_page_add() with migrate_folio_add(). migrate_folio_add() does the same a migrate_page_add() but takes in a folio instead of a page. This removes a couple of calls to compound_head(). Signed-off-by: Vishal Moola (Oracle) --- mm/mempolicy.c | 39 ++++++++++++++++++++------------------- 1 file changed, 20 insertions(+), 19 deletions(-) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index da87644430e3..9bb4600c4294 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -414,7 +414,7 @@ static const struct mempolicy_operations mpol_ops[MPOL_MAX] = { }, }; -static int migrate_page_add(struct page *page, struct list_head *pagelist, +static int migrate_folio_add(struct folio *folio, struct list_head *foliolist, unsigned long flags); struct queue_pages { @@ -476,7 +476,7 @@ static int queue_folios_pmd(pmd_t *pmd, spinlock_t *ptl, unsigned long addr, /* go to folio migration */ if (flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) { if (!vma_migratable(walk->vma) || - migrate_page_add(&folio->page, qp->pagelist, flags)) { + migrate_folio_add(folio, qp->pagelist, flags)) { ret = 1; goto unlock; } @@ -544,7 +544,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr, * temporary off LRU pages in the range. Still * need migrate other LRU pages. */ - if (migrate_page_add(&folio->page, qp->pagelist, flags)) + if (migrate_folio_add(folio, qp->pagelist, flags)) has_unmovable = true; } else break; @@ -1029,27 +1029,28 @@ static long do_get_mempolicy(int *policy, nodemask_t *nmask, } #ifdef CONFIG_MIGRATION -/* - * page migration, thp tail pages can be passed. - */ -static int migrate_page_add(struct page *page, struct list_head *pagelist, +static int migrate_folio_add(struct folio *folio, struct list_head *foliolist, unsigned long flags) { - struct page *head = compound_head(page); /* - * Avoid migrating a page that is shared with others. + * We try to migrate only unshared folios. If it is shared it + * is likely not worth migrating. + * + * To check if the folio is shared, ideally we want to make sure + * every page is mapped to the same process. Doing that is very + * expensive, so check the estimated mapcount of the folio instead. */ - if ((flags & MPOL_MF_MOVE_ALL) || page_mapcount(head) == 1) { - if (!isolate_lru_page(head)) { - list_add_tail(&head->lru, pagelist); - mod_node_page_state(page_pgdat(head), - NR_ISOLATED_ANON + page_is_file_lru(head), - thp_nr_pages(head)); + if ((flags & MPOL_MF_MOVE_ALL) || folio_estimated_mapcount(folio) == 1) { + if (!folio_isolate_lru(folio)) { + list_add_tail(&folio->lru, foliolist); + node_stat_mod_folio(folio, + NR_ISOLATED_ANON + folio_is_file_lru(folio), + folio_nr_pages(folio)); } else if (flags & MPOL_MF_STRICT) { /* - * Non-movable page may reach here. And, there may be - * temporary off LRU pages or non-LRU movable pages. - * Treat them as unmovable pages since they can't be + * Non-movable folio may reach here. And, there may be + * temporary off LRU folios or non-LRU movable folios. + * Treat them as unmovable folios since they can't be * isolated, so they can't be moved at the moment. It * should return -EIO for this case too. */ @@ -1241,7 +1242,7 @@ static struct page *new_page(struct page *page, unsigned long start) } #else -static int migrate_page_add(struct page *page, struct list_head *pagelist, +static int migrate_folio_add(struct folio *folio, struct list_head *foliolist, unsigned long flags) { return -EIO;