From patchwork Tue Aug 22 00:53:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 136518 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b82d:0:b0:3f2:4152:657d with SMTP id z13csp3559379vqi; Tue, 22 Aug 2023 04:33:16 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE7GhHRZrtQRtfBvXderAi/c8rKKSy0CRfyZX9MKyO5pdAhDthzdIg+gErn63hAkWlXKUtq X-Received: by 2002:a17:903:24f:b0:1bf:cac:13b4 with SMTP id j15-20020a170903024f00b001bf0cac13b4mr8061689plh.5.1692703996550; Tue, 22 Aug 2023 04:33:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692703996; cv=none; d=google.com; s=arc-20160816; b=WcrbfXQqA21fHuQS3zgcO3hJ/9NIwE3nInv6dBbg2I7oCs9o0j3d466oUr19hAbINJ D0mb62DpucT2LoTIswqlatVvJA7Qtlpr1IROFUZIW4w6AYSosoqAlk4eLuJE8HXxNkC/ sVAs6tT4z5O8eg0x+sjNmdrictXOLUBYACkKEI+TeROEPB8PBXry0Yuv9lAAjet+/Ni7 ko6oFf2Ru70J9qUqyrxE8vnqcHYjCCgRfFwVEgZSyq8EcTjAXsuoJFy4o0cxVqK/Xpup TitmeP1wCqUNi2B4wtYLLzVTtWowHuBSWx8S1SCuDwvOb67539MzZrw8+gG5R4hzNZgU WKHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=DlJp3/3E7Xb8ya6yRKzSEIygAUPOQPvI4nkRhXzfxUw=; fh=9/y/4i4wURkq6VSoO06mJGKzAg0FgvlCtHCXS3nYPok=; b=p+jW9GCDvwRQ46qpJG1uNG+IYk4L+/HbEE4HwDsCS20PwO3ZgBIY6boTteWpTOiAq1 MHs5AAM+1iVtGwJl++xjzI+2zEhTfJszFrEE2TsQbL4vHj/+v/m2ziGVhH3F7Dc+GqeJ b8O8QEY+9lrxNfhU/+dSLxwiZ5cia1/oyy8cAc2hzI5PgMu07nDlJizj+emGqPvKYKOA xprRm6bf7UGAQKexTobM5/2iXrXhDIh+wcw2PziO6/OCJbOSBuYlgNQAOMKnjI56o1XY WxlAaWMdemW0ldec42N9muS+gUgGrLAKwucn9+iiZUMDMr46JkULRW/YMBQslkPs14Di ZFcg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id jy5-20020a17090342c500b001b9c7300823si8533106plb.221.2023.08.22.04.33.03; Tue, 22 Aug 2023 04:33:16 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231979AbjHVAyd (ORCPT + 99 others); Mon, 21 Aug 2023 20:54:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58966 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231954AbjHVAyP (ORCPT ); Mon, 21 Aug 2023 20:54:15 -0400 Received: from out30-99.freemail.mail.aliyun.com (out30-99.freemail.mail.aliyun.com [115.124.30.99]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E50BA184 for ; Mon, 21 Aug 2023 17:54:13 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R121e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045192;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=8;SR=0;TI=SMTPD_---0VqK0DQ6_1692665650; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VqK0DQ6_1692665650) by smtp.aliyun-inc.com; Tue, 22 Aug 2023 08:54:11 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: mgorman@techsingularity.net, shy828301@gmail.com, david@redhat.com, ying.huang@intel.com, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 3/4] mm: migrate: change migrate_misplaced_page() to support multiple pages migration Date: Tue, 22 Aug 2023 08:53:51 +0800 Message-Id: <02c3d36270705f0dfec1ea583e252464cb48d802.1692665449.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1774928786257824170 X-GMAIL-MSGID: 1774928786257824170 Expanding the migrate_misplaced_page() function to allow passing in a list to support multiple pages migration as a preparation to support batch migration for NUMA balancing as well as compound page NUMA balancing in future. Signed-off-by: Baolin Wang --- include/linux/migrate.h | 9 +++++---- mm/huge_memory.c | 5 ++++- mm/memory.c | 4 +++- mm/migrate.c | 26 ++++++++++---------------- 4 files changed, 22 insertions(+), 22 deletions(-) diff --git a/include/linux/migrate.h b/include/linux/migrate.h index ddcd62ec2c12..87edce8e939d 100644 --- a/include/linux/migrate.h +++ b/include/linux/migrate.h @@ -142,12 +142,13 @@ const struct movable_operations *page_movable_ops(struct page *page) } #ifdef CONFIG_NUMA_BALANCING -int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma, - int node); +int migrate_misplaced_page(struct list_head *migratepages, struct vm_area_struct *vma, + int source_nid, int target_nid); bool numamigrate_isolate_page(pg_data_t *pgdat, struct page *page); #else -static inline int migrate_misplaced_page(struct page *page, - struct vm_area_struct *vma, int node) +static inline int migrate_misplaced_page(struct list_head *migratepages, + struct vm_area_struct *vma, + int source_nid, int target_nid) { return -EAGAIN; /* can't migrate now */ } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 07149ead11e4..4401a3493544 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1497,6 +1497,7 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) bool migrated = false, writable = false; int flags = 0; pg_data_t *pgdat; + LIST_HEAD(migratepages); vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd); if (unlikely(!pmd_same(oldpmd, *vmf->pmd))) { @@ -1552,7 +1553,9 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) goto migrate_fail; } - migrated = migrate_misplaced_page(page, vma, target_nid); + list_add(&page->lru, &migratepages); + migrated = migrate_misplaced_page(&migratepages, vma, + page_nid, target_nid); if (migrated) { flags |= TNF_MIGRATED; page_nid = target_nid; diff --git a/mm/memory.c b/mm/memory.c index 4e451b041488..9e417e8dd5d5 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4770,6 +4770,7 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) pte_t pte, old_pte; int flags = 0; pg_data_t *pgdat; + LIST_HEAD(migratepages); /* * The "pte" at this point cannot be used safely without @@ -4851,8 +4852,9 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) goto migrate_fail; } + list_add(&page->lru, &migratepages); /* Migrate to the requested node */ - if (migrate_misplaced_page(page, vma, target_nid)) { + if (migrate_misplaced_page(&migratepages, vma, page_nid, target_nid)) { page_nid = target_nid; flags |= TNF_MIGRATED; } else { diff --git a/mm/migrate.c b/mm/migrate.c index 0b2b69a2a7ab..fae7224b8e64 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2519,36 +2519,30 @@ bool numamigrate_isolate_page(pg_data_t *pgdat, struct page *page) * node. Caller is expected to have an elevated reference count on * the page that will be dropped by this function before returning. */ -int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma, - int node) +int migrate_misplaced_page(struct list_head *migratepages, struct vm_area_struct *vma, + int source_nid, int target_nid) { - pg_data_t *pgdat = NODE_DATA(node); + pg_data_t *pgdat = NODE_DATA(target_nid); int migrated = 1; int nr_remaining; unsigned int nr_succeeded; - LIST_HEAD(migratepages); - int nr_pages = thp_nr_pages(page); - list_add(&page->lru, &migratepages); - nr_remaining = migrate_pages(&migratepages, alloc_misplaced_dst_folio, - NULL, node, MIGRATE_ASYNC, + nr_remaining = migrate_pages(migratepages, alloc_misplaced_dst_folio, + NULL, target_nid, MIGRATE_ASYNC, MR_NUMA_MISPLACED, &nr_succeeded); if (nr_remaining) { - if (!list_empty(&migratepages)) { - list_del(&page->lru); - mod_node_page_state(page_pgdat(page), NR_ISOLATED_ANON + - page_is_file_lru(page), -nr_pages); - putback_lru_page(page); - } + if (!list_empty(migratepages)) + putback_movable_pages(migratepages); + migrated = 0; } if (nr_succeeded) { count_vm_numa_events(NUMA_PAGE_MIGRATE, nr_succeeded); - if (!node_is_toptier(page_to_nid(page)) && node_is_toptier(node)) + if (!node_is_toptier(source_nid) && node_is_toptier(target_nid)) mod_node_page_state(pgdat, PGPROMOTE_SUCCESS, nr_succeeded); } - BUG_ON(!list_empty(&migratepages)); + BUG_ON(!list_empty(migratepages)); return migrated; } #endif /* CONFIG_NUMA_BALANCING */