From patchwork Sat Aug 19 10:52:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 136257 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b82d:0:b0:3f2:4152:657d with SMTP id z13csp2485147vqi; Sun, 20 Aug 2023 06:42:27 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFW9SZ+pHoFf00ST0mPa1fSXcd+oBgG0kkrYPkX1tBXO/Rctr0n0+XGPrga5chnYZsf377j X-Received: by 2002:a05:6a20:1453:b0:131:5b6f:ccb6 with SMTP id a19-20020a056a20145300b001315b6fccb6mr6649653pzi.19.1692538946812; Sun, 20 Aug 2023 06:42:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692538946; cv=none; d=google.com; s=arc-20160816; b=AUEK40DXmqIJClX9+eb1QC2qCxNhiPkDqkV/8TJYLWYzD7eltlIlJ+5WkWy3JTKFHr 9ulWXBsgcke1Mmz9AU8q1JeHubKDY+DlKvsmFBKJJc0IKb3jp9/BNaMPBQRHlz7D/xOt l+mJh5wVXrB3vHhSz1tyytWwUUS+2HYs1Bl7LZ2Zeu6QWTP1pJ8JaFBTmLZl9Mosx4AE NuMZXauVCk+RaYN/KtdrjpG2MBYSfiYf5CZSrrB5+oUm9e6/lT9B81QB+Kz8by9lqlfy DDOFWwoBviDVcQ/Hcey9l6pe2ifG8SqkRoFvjLKyIL3V1cBZPA7DOAcAlCiw57oezaPm SeYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=Gno/qmUTlW9t9/wqxr/fxCmzyH/MTCfjhdDmjzbuz2c=; fh=9/y/4i4wURkq6VSoO06mJGKzAg0FgvlCtHCXS3nYPok=; b=TCYGtx4QdmBpyRAxgUcN4VnRwNmepgqUgh9k3ZYXC7Amaq1KS8XvhxFu+Es3W/xrxz z6OaSAddAYpXm+inTZNLi3Bnsdblsfbit3Lp7lfIWkLn29oWAsEkOx90nU3gsxz+Wdu2 cPSEToHjibk2viYOGXuoEEC5Hdi3bk9ha/fibL9tH60TvrQEDhgYGZEdPos0HwQju7S8 +6pwBOSw0l8pcq6T4lX3CXsGfPj4q0tXGeD+0dU0zcH9hiazjVQLdgdbhNayt9EqKErE ZtHaZQSa+bQnkPkGKlSR4g+lbVEhHd7o9dYMPE2SYSikMM/DAdZp3sHq57zbZ1iuLDEu AA6Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net. [2620:137:e000::1:18]) by mx.google.com with ESMTPS id v35-20020a634663000000b00563e815e202si3694343pgk.674.2023.08.20.06.42.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Aug 2023 06:42:26 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id C23483642B3; Sat, 19 Aug 2023 12:08:04 -0700 (PDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231673AbjHSK4C (ORCPT + 99 others); Sat, 19 Aug 2023 06:56:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44932 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231588AbjHSKz6 (ORCPT ); Sat, 19 Aug 2023 06:55:58 -0400 Received: from out30-119.freemail.mail.aliyun.com (out30-119.freemail.mail.aliyun.com [115.124.30.119]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DF0703768F for ; Sat, 19 Aug 2023 03:52:53 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R831e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045168;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=8;SR=0;TI=SMTPD_---0Vq58.SR_1692442363; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0Vq58.SR_1692442363) by smtp.aliyun-inc.com; Sat, 19 Aug 2023 18:52:44 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: mgorman@techsingularity.net, shy828301@gmail.com, david@redhat.com, ying.huang@intel.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/4] mm: migrate: move the numamigrate_isolate_page() into do_numa_page() Date: Sat, 19 Aug 2023 18:52:35 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1774755718720934278 X-GMAIL-MSGID: 1774755718720934278 Move the numamigrate_isolate_page() into do_numa_page() to simplify the migrate_misplaced_page(), which now only focuses on page migration, and it also serves as a preparation for supporting batch migration for NUMA balancing. Signed-off-by: Baolin Wang --- include/linux/migrate.h | 6 ++++++ mm/huge_memory.c | 10 ++++++++++ mm/memory.c | 10 ++++++++++ mm/migrate.c | 16 ++++------------ 4 files changed, 30 insertions(+), 12 deletions(-) diff --git a/include/linux/migrate.h b/include/linux/migrate.h index 711dd9412561..7c5189043707 100644 --- a/include/linux/migrate.h +++ b/include/linux/migrate.h @@ -144,12 +144,18 @@ const struct movable_operations *page_movable_ops(struct page *page) #ifdef CONFIG_NUMA_BALANCING int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma, int node); +int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page); #else static inline int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma, int node) { return -EAGAIN; /* can't migrate now */ } + +static inline int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page) +{ + return -EAGAIN; +} #endif /* CONFIG_NUMA_BALANCING */ #ifdef CONFIG_MIGRATION diff --git a/mm/huge_memory.c b/mm/huge_memory.c index cb4432792b88..b7cc6828ce9e 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1496,6 +1496,8 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) int target_nid, last_cpupid = (-1 & LAST_CPUPID_MASK); bool migrated = false, writable = false; int flags = 0; + pg_data_t *pgdat; + int isolated; vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd); if (unlikely(!pmd_same(oldpmd, *vmf->pmd))) { @@ -1540,11 +1542,19 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) spin_unlock(vmf->ptl); writable = false; + pgdat = NODE_DATA(target_nid); + isolated = numamigrate_isolate_page(pgdat, page); + if (!isolated) { + put_page(page); + goto isolate_fail; + } + migrated = migrate_misplaced_page(page, vma, target_nid); if (migrated) { flags |= TNF_MIGRATED; page_nid = target_nid; } else { +isolate_fail: flags |= TNF_MIGRATE_FAIL; vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd); if (unlikely(!pmd_same(oldpmd, *vmf->pmd))) { diff --git a/mm/memory.c b/mm/memory.c index bee9b1e86ef0..01b1980d4fb7 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4779,6 +4779,8 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) int target_nid; pte_t pte, old_pte; int flags = 0; + pg_data_t *pgdat; + int isolated; /* * The "pte" at this point cannot be used safely without @@ -4849,11 +4851,19 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) pte_unmap_unlock(vmf->pte, vmf->ptl); writable = false; + pgdat = NODE_DATA(target_nid); + isolated = numamigrate_isolate_page(pgdat, page); + if (!isolated) { + put_page(page); + goto isolate_fail; + } + /* Migrate to the requested node */ if (migrate_misplaced_page(page, vma, target_nid)) { page_nid = target_nid; flags |= TNF_MIGRATED; } else { +isolate_fail: flags |= TNF_MIGRATE_FAIL; vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd, vmf->address, &vmf->ptl); diff --git a/mm/migrate.c b/mm/migrate.c index 9cc98fb1d6ec..5eeeb2cda21c 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2478,7 +2478,7 @@ static struct folio *alloc_misplaced_dst_folio(struct folio *src, return __folio_alloc_node(gfp, order, nid); } -static int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page) +int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page) { int nr_pages = thp_nr_pages(page); int order = compound_order(page); @@ -2523,16 +2523,12 @@ int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma, int node) { pg_data_t *pgdat = NODE_DATA(node); - int isolated; + int migrated = 1; int nr_remaining; unsigned int nr_succeeded; LIST_HEAD(migratepages); int nr_pages = thp_nr_pages(page); - isolated = numamigrate_isolate_page(pgdat, page); - if (!isolated) - goto out; - list_add(&page->lru, &migratepages); nr_remaining = migrate_pages(&migratepages, alloc_misplaced_dst_folio, NULL, node, MIGRATE_ASYNC, @@ -2544,7 +2540,7 @@ int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma, page_is_file_lru(page), -nr_pages); putback_lru_page(page); } - isolated = 0; + migrated = 0; } if (nr_succeeded) { count_vm_numa_events(NUMA_PAGE_MIGRATE, nr_succeeded); @@ -2553,11 +2549,7 @@ int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma, nr_succeeded); } BUG_ON(!list_empty(&migratepages)); - return isolated; - -out: - put_page(page); - return 0; + return migrated; } #endif /* CONFIG_NUMA_BALANCING */ #endif /* CONFIG_NUMA */ From patchwork Sat Aug 19 10:52:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 136289 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b82d:0:b0:3f2:4152:657d with SMTP id z13csp2591574vqi; Sun, 20 Aug 2023 11:06:04 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE86dVKitSr3XPxUvNHd6O/6BKNcV6Q6s59TQDW0cxcGmHJ4AATnnllOcHYwSVeLuoI1N2t X-Received: by 2002:a05:6a20:7da2:b0:148:4c5:9714 with SMTP id v34-20020a056a207da200b0014804c59714mr3694730pzj.13.1692554764518; Sun, 20 Aug 2023 11:06:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692554764; cv=none; d=google.com; s=arc-20160816; b=UIxW49feT6HvdGJ5w2wNI1bBnHtKuGfpjDfvFqBTikk7yVSzDmw27BlcMc2U40wI3L zL9p1rkEf4mfaP962rW1ihEl0OQjBSPKvByIB+d118NZVkHXU7RDV1A0/XwJKNF3i8/f D3+W/usPfODsyMLnYVIdqhm3BB3jww6U8g5MJjZfikVhrUMerMGPyLeDUFVymS2M7EpC V5Mqvlx+KwjnpYSJl0P/n9CKQ4RtMGtekw7gtyd6jEaHEmJS7m7tTdYhI0RCkIr6UgEw JthJJ2not9DRqeR2juGQL4U3y6NU4hBcYeTytZIpnsl1Aqvy1q3KhgiyLHbl11drA+G+ Ih2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=NVfxIFpzUqEqqP2hfbOTsF5yBYikaGXOv+U0WqJGCFI=; fh=9/y/4i4wURkq6VSoO06mJGKzAg0FgvlCtHCXS3nYPok=; b=xSeJVLhP4hOgv5p9rcJ+wEPQROTU9voRK1wSDYwz6Lq780Ywc4tBXsQz7YLIa7sue0 YvM+rAUeytQzzlOeB9EL/ZzSVXVZwg+/p5NGYYpKnmGQxnbHn7o3ywc/jeICvjd/VBAl +j9iqsxFDkr+RoVxx/OFwR6TYYp9bvymgkrTyV4Jm/sUQX46xePqUUgl11s1LZtQPYvv xGrd1PSZFM9tv89ZDZTiMEnqdCITIgaYsX5Cgrw0cDR4EKxQCRILdc/iDt1BKJJT1MGu /uFjkuBPDvMiitr9d52hWjI+BP/0zgwgGiVO9A6Nd0UoIwiUbj1kS/SnzCMX7UtEns1G bYyQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net. [2620:137:e000::1:18]) by mx.google.com with ESMTPS id ks12-20020a170903084c00b001bbd785e1d2si5378531plb.201.2023.08.20.11.06.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Aug 2023 11:06:04 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id B63F9420DFD; Sat, 19 Aug 2023 12:27:59 -0700 (PDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231372AbjHSKzK (ORCPT + 99 others); Sat, 19 Aug 2023 06:55:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42292 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231315AbjHSKzK (ORCPT ); Sat, 19 Aug 2023 06:55:10 -0400 Received: from out30-100.freemail.mail.aliyun.com (out30-100.freemail.mail.aliyun.com [115.124.30.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4731C38E5B for ; Sat, 19 Aug 2023 03:52:56 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R741e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046059;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=8;SR=0;TI=SMTPD_---0Vq58.Sy_1692442365; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0Vq58.Sy_1692442365) by smtp.aliyun-inc.com; Sat, 19 Aug 2023 18:52:46 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: mgorman@techsingularity.net, shy828301@gmail.com, david@redhat.com, ying.huang@intel.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 4/4] mm: migrate: change to return the number of pages migrated successfully Date: Sat, 19 Aug 2023 18:52:37 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1774772304944589734 X-GMAIL-MSGID: 1774772304944589734 Change the migrate_misplaced_page() to return the number of pages migrated successfully, which is used to calculate how many pages are failed to migrate for batch migration. Signed-off-by: Baolin Wang --- mm/huge_memory.c | 7 ++++--- mm/memory.c | 5 +++-- mm/migrate.c | 5 +---- 3 files changed, 8 insertions(+), 9 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 53a9d63cfb1e..a9c454160984 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1494,11 +1494,12 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) unsigned long haddr = vmf->address & HPAGE_PMD_MASK; int page_nid = NUMA_NO_NODE; int target_nid, last_cpupid = (-1 & LAST_CPUPID_MASK); - bool migrated = false, writable = false; + bool writable = false; int flags = 0; pg_data_t *pgdat; int isolated; LIST_HEAD(migratepages); + int nr_successed = 0; vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd); if (unlikely(!pmd_same(oldpmd, *vmf->pmd))) { @@ -1551,8 +1552,8 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) } list_add(&page->lru, &migratepages); - migrated = migrate_misplaced_page(&migratepages, vma, page_nid, target_nid); - if (migrated) { + nr_successed = migrate_misplaced_page(&migratepages, vma, page_nid, target_nid); + if (nr_successed) { flags |= TNF_MIGRATED; page_nid = target_nid; } else { diff --git a/mm/memory.c b/mm/memory.c index 973403a83797..edfd2d528e7e 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4780,7 +4780,7 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) pte_t pte, old_pte; int flags = 0; pg_data_t *pgdat; - int isolated; + int isolated, nr_succeeded; LIST_HEAD(migratepages); /* @@ -4861,7 +4861,8 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) list_add(&page->lru, &migratepages); /* Migrate to the requested node */ - if (migrate_misplaced_page(&migratepages, vma, page_nid, target_nid)) { + nr_succeeded = migrate_misplaced_page(&migratepages, vma, page_nid, target_nid); + if (nr_succeeded) { page_nid = target_nid; flags |= TNF_MIGRATED; } else { diff --git a/mm/migrate.c b/mm/migrate.c index 93d359471b95..45f92376ba6f 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2523,7 +2523,6 @@ int migrate_misplaced_page(struct list_head *migratepages, struct vm_area_struct int source_nid, int target_nid) { pg_data_t *pgdat = NODE_DATA(target_nid); - int migrated = 1; int nr_remaining; unsigned int nr_succeeded; @@ -2533,8 +2532,6 @@ int migrate_misplaced_page(struct list_head *migratepages, struct vm_area_struct if (nr_remaining) { if (!list_empty(migratepages)) putback_movable_pages(migratepages); - - migrated = 0; } if (nr_succeeded) { count_vm_numa_events(NUMA_PAGE_MIGRATE, nr_succeeded); @@ -2543,7 +2540,7 @@ int migrate_misplaced_page(struct list_head *migratepages, struct vm_area_struct nr_succeeded); } BUG_ON(!list_empty(migratepages)); - return migrated; + return nr_succeeded; } #endif /* CONFIG_NUMA_BALANCING */ #endif /* CONFIG_NUMA */