From patchwork Tue Feb 27 13:52:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 207227 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:a81b:b0:108:e6aa:91d0 with SMTP id bq27csp2708177dyb; Tue, 27 Feb 2024 05:55:35 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCW+lvlND7qOi3l+TSdrw0wo9gNdjy7R/IukZYpiHidD6Ke3R66td0BayFtFXffWuPKCV3alxkglkhwclZ+e7d1j1nQ1UQ== X-Google-Smtp-Source: AGHT+IH3l1f3oD36zzDeFfNc9iXiVSy8+hY4Tx99+c6XPkD2V0MKO/QXlTZ2I2OHberYRKyNt3MF X-Received: by 2002:a17:902:7ec8:b0:1db:cb54:5e77 with SMTP id p8-20020a1709027ec800b001dbcb545e77mr11382369plb.22.1709042135521; Tue, 27 Feb 2024 05:55:35 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1709042135; cv=pass; d=google.com; s=arc-20160816; b=cWU8nz2dmD1FA6vBst8y6zuDy9HimP8idvQm22ZK5AiihYmmLqhNXxZi3Y3C5q+tIS EQMzta4O0gfGWRZ9VajCh9GdJUpxa64yxAq2y9W+RSlF2tTd4LTMZoZaZmqB0PB7IHSs HtMeK/bApF0/LdNvzyio1lmK76fEmRxYkKVC5YU6dmzdYgKRnhrw3MyNKuGVAOtUpRE0 Ahe7Mgoe5yX5Kan7Y2bd2sDVu6Dkti9jBXYZpGwV/NAx57XkDX2tIhTiVvURqYASXZ5Y ylvLKr7T8um5X+oKENL3AO3lMuO2S6hP1kHWZ7ssyyeF6VanyS8luhbFF8Y+lrFDRvfe fHpw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=vclFUdHEbEOLvLac5txTbNN5X/YAQNb/V/s0HCQPExw=; fh=Eg1bbkMBB2FpJxkcjuowRNwwz/msSkRTpuEVSvFESpg=; b=pegyWpUEz/yNUAGfvlwmpc4XT+ASUo03Dtbzf5FSgtmWL7KKhmlh06QSPs8APnZSdd xuloKt2Dn925+bvsd3NSypKhPWtrQAcfN/BE6mSDB75/S6CKorXgQCPx1mr7dTQmz4Bh kEi87cTUQ92yGse0E7Lt+53QAuO1NPHMd2BcF67UygZBHKODdd3TsIt3JHcloju7ZD5O OLld8AignmK3XdpN+ylVsqxNF9Kce81eXtc3Ebgiyp1G+/jCXnLI3dwWoUhYT7qVeJhW 79gLsenQOvtR0ZJuPl1YZiXRL8PdYNgJXaRdCvDgWPjXNfS9EIwkEfioh2MF+RwRZuQs IgPA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.alibaba.com header.s=default header.b=V49jqNvO; arc=pass (i=1 spf=pass spfdomain=linux.alibaba.com dkim=pass dkdomain=linux.alibaba.com dmarc=pass fromdomain=linux.alibaba.com); spf=pass (google.com: domain of linux-kernel+bounces-83355-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-83355-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.alibaba.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id t1-20020a170902e84100b001d95f009fd0si1428505plg.367.2024.02.27.05.55.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Feb 2024 05:55:35 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-83355-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.alibaba.com header.s=default header.b=V49jqNvO; arc=pass (i=1 spf=pass spfdomain=linux.alibaba.com dkim=pass dkdomain=linux.alibaba.com dmarc=pass fromdomain=linux.alibaba.com); spf=pass (google.com: domain of linux-kernel+bounces-83355-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-83355-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.alibaba.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id E857C2939B9 for ; Tue, 27 Feb 2024 13:53:56 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 09B9F1420A6; Tue, 27 Feb 2024 13:52:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="V49jqNvO" Received: from out30-97.freemail.mail.aliyun.com (out30-97.freemail.mail.aliyun.com [115.124.30.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 827FE1420D0 for ; Tue, 27 Feb 2024 13:52:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.97 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709041963; cv=none; b=p9OIJw3xraorhXQmFSkE9/UzWeaNNpBWmgcdmJompjweyI668Tfkg5AHWdoYljzxzeDGBrnnXelQV4FhzwErV0C0uGdFlfgMlg+P6n2+ZhNdpq2VRqZVMth9HZNPVorQwSIbOdvOGxjuskNLZsy+0gfv/IRd197YCboWtmFov3I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709041963; c=relaxed/simple; bh=X8NNHUf6EO6XtKsBok97wPMwUXPF3HgsaY7PdvrfU7M=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Pz2tSSeKDIF9p3cimxTUVKntv5EEPhI/9yAwdr6W5uxXYp07nOo4vm1o5rqYR4WfstKvUFceYzyksRUJG+6AK1GNGZU44tNlNNgP7vnTxutat9LX7H54LGsCM2/sXYg3hfIym/QWCnJT3glbP4IH/y/qjHkIq8V6y/nIBBVoGtw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=V49jqNvO; arc=none smtp.client-ip=115.124.30.97 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1709041956; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=vclFUdHEbEOLvLac5txTbNN5X/YAQNb/V/s0HCQPExw=; b=V49jqNvOGlo/kH+WCaweqCN76eS71atV/3oCuHt9MQcG+WKIDJN6PQigadLFgYQx4g8q3FK0gKYennx9aVKT9WSCVW3NEQ4eVKBDLX5YAIqd4udQzAxX2CKyXNpOOHpXbDxPWGdhk4qP18vkjMzZl28xGge2bkrR4Zx56TBZEX8= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R111e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045192;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=10;SR=0;TI=SMTPD_---0W1N-REC_1709041955; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W1N-REC_1709041955) by smtp.aliyun-inc.com; Tue, 27 Feb 2024 21:52:35 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: muchun.song@linux.dev, osalvador@suse.de, david@redhat.com, linmiaohe@huawei.com, naoya.horiguchi@nec.com, mhocko@kernel.org, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/3] mm: record the migration reason for struct migration_target_control Date: Tue, 27 Feb 2024 21:52:25 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1792060566240522050 X-GMAIL-MSGID: 1792060566240522050 To support different hugetlb allocation strategies during hugetlb migration based on various migration reasons, recording the migration reason for the migration_target_control structure as a preparation. Signed-off-by: Baolin Wang Reviewed-by: Oscar Salvador --- mm/gup.c | 1 + mm/internal.h | 1 + mm/memory-failure.c | 1 + mm/memory_hotplug.c | 1 + mm/mempolicy.c | 1 + mm/migrate.c | 1 + mm/page_alloc.c | 1 + mm/vmscan.c | 3 ++- 8 files changed, 9 insertions(+), 1 deletion(-) diff --git a/mm/gup.c b/mm/gup.c index df83182ec72d..959a1a05b059 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -2132,6 +2132,7 @@ static int migrate_longterm_unpinnable_pages( struct migration_target_control mtc = { .nid = NUMA_NO_NODE, .gfp_mask = GFP_USER | __GFP_NOWARN, + .reason = MR_LONGTERM_PIN, }; if (migrate_pages(movable_page_list, alloc_migration_target, diff --git a/mm/internal.h b/mm/internal.h index 2b7efffbe4d7..47edf69b6ee6 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -959,6 +959,7 @@ struct migration_target_control { int nid; /* preferred node id */ nodemask_t *nmask; gfp_t gfp_mask; + enum migrate_reason reason; }; /* diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 9349948f1abf..780bb2aee0af 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -2666,6 +2666,7 @@ static int soft_offline_in_use_page(struct page *page) struct migration_target_control mtc = { .nid = NUMA_NO_NODE, .gfp_mask = GFP_USER | __GFP_MOVABLE | __GFP_RETRY_MAYFAIL, + .reason = MR_MEMORY_FAILURE, }; if (!huge && folio_test_large(folio)) { diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index a444e2d7dd2b..b79ba36e09e0 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1841,6 +1841,7 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) struct migration_target_control mtc = { .nmask = &nmask, .gfp_mask = GFP_USER | __GFP_MOVABLE | __GFP_RETRY_MAYFAIL, + .reason = MR_MEMORY_HOTPLUG, }; int ret; diff --git a/mm/mempolicy.c b/mm/mempolicy.c index f60b4c99f130..98ceb12e0e17 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -1070,6 +1070,7 @@ static long migrate_to_node(struct mm_struct *mm, int source, int dest, struct migration_target_control mtc = { .nid = dest, .gfp_mask = GFP_HIGHUSER_MOVABLE | __GFP_THISNODE, + .reason = MR_SYSCALL, }; nodes_clear(nmask); diff --git a/mm/migrate.c b/mm/migrate.c index 73a052a382f1..bde63010a3cf 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2060,6 +2060,7 @@ static int do_move_pages_to_node(struct list_head *pagelist, int node) struct migration_target_control mtc = { .nid = node, .gfp_mask = GFP_HIGHUSER_MOVABLE | __GFP_THISNODE, + .reason = MR_SYSCALL, }; err = migrate_pages(pagelist, alloc_migration_target, NULL, diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 96839b210abe..8e6dd3a1028b 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6266,6 +6266,7 @@ int __alloc_contig_migrate_range(struct compact_control *cc, struct migration_target_control mtc = { .nid = zone_to_nid(cc->zone), .gfp_mask = GFP_USER | __GFP_MOVABLE | __GFP_RETRY_MAYFAIL, + .reason = MR_CONTIG_RANGE, }; lru_cache_disable(); diff --git a/mm/vmscan.c b/mm/vmscan.c index 402c290fbf5a..510f438bb9e0 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -978,7 +978,8 @@ static unsigned int demote_folio_list(struct list_head *demote_folios, .gfp_mask = (GFP_HIGHUSER_MOVABLE & ~__GFP_RECLAIM) | __GFP_NOWARN | __GFP_NOMEMALLOC | GFP_NOWAIT, .nid = target_nid, - .nmask = &allowed_mask + .nmask = &allowed_mask, + .reason = MR_DEMOTION, }; if (list_empty(demote_folios)) From patchwork Tue Feb 27 13:52:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 207225 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:a81b:b0:108:e6aa:91d0 with SMTP id bq27csp2707470dyb; Tue, 27 Feb 2024 05:54:12 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUFopk/aeBFbejg9SmgLiRQ5psvpml9tvqBy/acFCY3IKA2bmweydkl5o0fk+OSjeFAROqGnFnEllZsYTm0lgCLc4zDIQ== X-Google-Smtp-Source: AGHT+IEk7oNILIY6Ynnk+J16z1GopCBkypc3p6LY2pSptLFXucEZjPqtH36TFmli/hihKizNGjzu X-Received: by 2002:a17:906:a190:b0:a3d:e2e9:a7f7 with SMTP id s16-20020a170906a19000b00a3de2e9a7f7mr6823913ejy.27.1709042052215; Tue, 27 Feb 2024 05:54:12 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1709042052; cv=pass; d=google.com; s=arc-20160816; b=PdHSKWVCWTSgZfPHObXJy5YlzOwwn5UoJ00xMDc6bmfMYIO65OTpqVj74ZE+L/vFNd 2wsRqxH47f4fow3bf9QGl823INYMkYeXuhgWWDZC4RSn/ORIXR9mly0n+f9AnTXcdnjz ky44gq5v0hX59BtJAXpP3Jz11Boh2EjGG58F8V1Otv0VmrCMKlj1fY14guO2ukRt+mQn /QPynKzurpIYtF5KoCvixQd7ZFs+kQ/KQa/kUrxO+OzDk+DGn/6N1pUQwZdgEYhx0QU6 gN+2AUSb0V0HGfBFkCPPSvStBdLlLkgl6vZ8t387zP3FTScQ72Vx5QtPcfC3yUTh06tN Cf8A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=KTx0R6BPzRbPHHXux5hHxOtyW6q21tS4nAQfi/BVpWw=; fh=Eg1bbkMBB2FpJxkcjuowRNwwz/msSkRTpuEVSvFESpg=; b=gaBfEHAHS3EOckzaVOyuBBLmubDxWPJ4biq9t2UY0++HExnnR+Bf1pbDpwTgKWH1p1 U/FY/pq7vBzxAUzC01/WVd+b682RLXoFik8z13/36vqWSBC8fYR7Uo7lSqoIT7IN/pKy gvi+UyPu37agdrG/QhrUsG+wTi586PLPBVR6qQfyJR1U4jxS3pbCK3V0k+q/92tDvo8n j9HqbcXGrSxo+Wrk6UYzKu0frgtsCo6ht5ejRQ/0UZzP7UNfNUgA5JmNBbbAcp93TUIy PLHJRpUGSLYpXXcnBy3S+578WOqofphK3T7NR0NBGVIQ52MxJH1fXSZe4/5pRGzmcIX1 07Sw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.alibaba.com header.s=default header.b="Kg/SiZf2"; arc=pass (i=1 spf=pass spfdomain=linux.alibaba.com dkim=pass dkdomain=linux.alibaba.com dmarc=pass fromdomain=linux.alibaba.com); spf=pass (google.com: domain of linux-kernel+bounces-83356-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-83356-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.alibaba.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id nc37-20020a1709071c2500b00a43b2380759si548635ejc.662.2024.02.27.05.54.11 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Feb 2024 05:54:12 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-83356-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.alibaba.com header.s=default header.b="Kg/SiZf2"; arc=pass (i=1 spf=pass spfdomain=linux.alibaba.com dkim=pass dkdomain=linux.alibaba.com dmarc=pass fromdomain=linux.alibaba.com); spf=pass (google.com: domain of linux-kernel+bounces-83356-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-83356-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.alibaba.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 9CB2C1F22831 for ; Tue, 27 Feb 2024 13:54:11 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5453C1468E7; Tue, 27 Feb 2024 13:52:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="Kg/SiZf2" Received: from out30-113.freemail.mail.aliyun.com (out30-113.freemail.mail.aliyun.com [115.124.30.113]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A6178145FE2 for ; Tue, 27 Feb 2024 13:52:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.113 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709041967; cv=none; b=BGjcyjTjLTBa8YaIuom4/Nr/0JI86omLO+tpdFqR4nFEFAduYc26Dt3D13BTLzTrJOst5tg0kYOqGG6o+f2QxE8xlcoEQeZGVgt/K5ug/xVxstFJopsQUlHjyNw8tHIk+hhdHQCnjnFAIlrYoGpyik8dCoElOM6RGgba4FWanWk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709041967; c=relaxed/simple; bh=H6GuExzJk+Z6cg9yaI81LGGUCrhHBi+gq30dR6i0HWk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=VfsQ6Ew8iEM+hE2ZDkN7bOI2gnNJB5BzQVOEeo+39Qri8f/riM0NR+8xv0A+uixJdxyElhO3RS/oB9mnkXhaIdbgSVvDOVHkVWX3+R5Qv8+/MF1+yhcMn5EiR8FSI0ga0f/puKLqjYHnVdjIoOlu2MdQG85keskiljF2wyPHBG0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=Kg/SiZf2; arc=none smtp.client-ip=115.124.30.113 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1709041957; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=KTx0R6BPzRbPHHXux5hHxOtyW6q21tS4nAQfi/BVpWw=; b=Kg/SiZf23lS4yDTRuJvpuC9JVjUivT/Yqw5FhawV7FrnW/YhQupRgEOSUa/eKGRnTHK1DroTRfWABW5Pb2KnKJN5lRnF+QEWJ8der1hIcOUqRGIKqghsOdRjEpx8oZUJbwboswoZ2fukomAFWheCUdJ1HemEwn8TXnBdMGR7ceI= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R711e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045170;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=10;SR=0;TI=SMTPD_---0W1MvL.U_1709041956; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W1MvL.U_1709041956) by smtp.aliyun-inc.com; Tue, 27 Feb 2024 21:52:36 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: muchun.song@linux.dev, osalvador@suse.de, david@redhat.com, linmiaohe@huawei.com, naoya.horiguchi@nec.com, mhocko@kernel.org, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/3] mm: hugetlb: make the hugetlb migration strategy consistent Date: Tue, 27 Feb 2024 21:52:26 +0800 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1792060478862418414 X-GMAIL-MSGID: 1792060478862418414 As discussed in previous thread [1], there is an inconsistency when handing hugetlb migration. When handling the migration of freed hugetlb, it prevents fallback to other NUMA nodes in alloc_and_dissolve_hugetlb_folio(). However, when dealing with in-use hugetlb, it allows fallback to other NUMA nodes in alloc_hugetlb_folio_nodemask(), which can break the per-node hugetlb pool and might result in unexpected failures when node bound workloads doesn't get what is asssumed available. To make hugetlb migration strategy more clear, we should list all the scenarios of hugetlb migration and analyze whether allocation fallback is permitted: 1) Memory offline: will call dissolve_free_huge_pages() to free the freed hugetlb, and call do_migrate_range() to migrate the in-use hugetlb. Both can break the per-node hugetlb pool, but as this is an explicit offlining operation, no better choice. So should allow the hugetlb allocation fallback. 2) Memory failure: same as memory offline. Should allow fallback to a different node might be the only option to handle it, otherwise the impact of poisoned memory can be amplified. 3) Longterm pinning: will call migrate_longterm_unpinnable_pages() to migrate in-use and not-longterm-pinnable hugetlb, which can break the per-node pool. But we should fail to longterm pinning if can not allocate on current node to avoid breaking the per-node pool. 4) Syscalls (mbind, migrate_pages, move_pages): these are explicit users operation to move pages to other nodes, so fallback to other nodes should not be prohibited. 5) alloc_contig_range: used by CMA allocation and virtio-mem fake-offline to allocate given range of pages. Now the freed hugetlb migration is not allowed to fallback, to keep consistency, the in-use hugetlb migration should be also not allowed to fallback. 6) alloc_contig_pages: used by kfence, pgtable_debug etc. The strategy should be consistent with that of alloc_contig_range(). Based on the analysis of the various scenarios above, determine whether fallback is permitted according to the migration reason in alloc_migrate_hugetlb_folio(). [1] https://lore.kernel.org/all/6f26ce22d2fcd523418a085f2c588fe0776d46e7.1706794035.git.baolin.wang@linux.alibaba.com/ Signed-off-by: Baolin Wang --- include/linux/hugetlb.h | 4 ++-- mm/hugetlb.c | 39 +++++++++++++++++++++++++++++++++++---- mm/mempolicy.c | 2 +- mm/migrate.c | 2 +- 4 files changed, 39 insertions(+), 8 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 77b30a8c6076..fa122dc509cf 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -747,7 +747,7 @@ int isolate_or_dissolve_huge_page(struct page *page, struct list_head *list); struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, unsigned long addr, int avoid_reserve); struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred_nid, - nodemask_t *nmask, gfp_t gfp_mask); + nodemask_t *nmask, gfp_t gfp_mask, int reason); int hugetlb_add_to_page_cache(struct folio *folio, struct address_space *mapping, pgoff_t idx); void restore_reserve_on_error(struct hstate *h, struct vm_area_struct *vma, @@ -1065,7 +1065,7 @@ static inline struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, static inline struct folio * alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred_nid, - nodemask_t *nmask, gfp_t gfp_mask) + nodemask_t *nmask, gfp_t gfp_mask, int reason) { return NULL; } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 418d66953224..e8eb08bbc688 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -2567,13 +2567,38 @@ static struct folio *alloc_surplus_hugetlb_folio(struct hstate *h, } static struct folio *alloc_migrate_hugetlb_folio(struct hstate *h, gfp_t gfp_mask, - int nid, nodemask_t *nmask) + int nid, nodemask_t *nmask, int reason) { struct folio *folio; + bool allowed_fallback = false; if (hstate_is_gigantic(h)) return NULL; + if (gfp_mask & __GFP_THISNODE) + goto alloc_new; + + /* + * Note: the memory offline, memory failure and migration syscalls will + * be allowed to fallback to other nodes due to lack of a better chioce, + * that might break the per-node hugetlb pool. While other cases will + * set the __GFP_THISNODE to avoid breaking the per-node hugetlb pool. + */ + switch (reason) { + case MR_MEMORY_HOTPLUG: + case MR_MEMORY_FAILURE: + case MR_SYSCALL: + case MR_MEMPOLICY_MBIND: + allowed_fallback = true; + break; + default: + break; + } + + if (!allowed_fallback) + gfp_mask |= __GFP_THISNODE; + +alloc_new: folio = alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, NULL); if (!folio) return NULL; @@ -2621,7 +2646,7 @@ struct folio *alloc_buddy_hugetlb_folio_with_mpol(struct hstate *h, /* folio migration callback function */ struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred_nid, - nodemask_t *nmask, gfp_t gfp_mask) + nodemask_t *nmask, gfp_t gfp_mask, int reason) { spin_lock_irq(&hugetlb_lock); if (available_huge_pages(h)) { @@ -2636,7 +2661,7 @@ struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred_nid, } spin_unlock_irq(&hugetlb_lock); - return alloc_migrate_hugetlb_folio(h, gfp_mask, preferred_nid, nmask); + return alloc_migrate_hugetlb_folio(h, gfp_mask, preferred_nid, nmask, reason); } /* @@ -6653,7 +6678,13 @@ static struct folio *alloc_hugetlb_folio_vma(struct hstate *h, gfp_mask = htlb_alloc_mask(h); node = huge_node(vma, address, gfp_mask, &mpol, &nodemask); - folio = alloc_hugetlb_folio_nodemask(h, node, nodemask, gfp_mask); + /* + * This is used to allocate a temporary hugetlb to hold the copied + * content, which will then be copied again to the final hugetlb + * consuming a reservation. Set the migrate reason to -1 to indicate + * that breaking the per-node hugetlb pool is not allowed in this case. + */ + folio = alloc_hugetlb_folio_nodemask(h, node, nodemask, gfp_mask, -1); mpol_cond_put(mpol); return folio; diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 98ceb12e0e17..436e817eeaeb 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -1228,7 +1228,7 @@ static struct folio *alloc_migration_target_by_mpol(struct folio *src, h = folio_hstate(src); gfp = htlb_alloc_mask(h); nodemask = policy_nodemask(gfp, pol, ilx, &nid); - return alloc_hugetlb_folio_nodemask(h, nid, nodemask, gfp); + return alloc_hugetlb_folio_nodemask(h, nid, nodemask, gfp, MR_MEMPOLICY_MBIND); } if (folio_test_large(src)) diff --git a/mm/migrate.c b/mm/migrate.c index bde63010a3cf..0c2b70800da3 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2022,7 +2022,7 @@ struct folio *alloc_migration_target(struct folio *src, unsigned long private) gfp_mask = htlb_modify_alloc_mask(h, gfp_mask); return alloc_hugetlb_folio_nodemask(h, nid, - mtc->nmask, gfp_mask); + mtc->nmask, gfp_mask, mtc->reason); } if (folio_test_large(src)) { From patchwork Tue Feb 27 13:52:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 207224 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:a81b:b0:108:e6aa:91d0 with SMTP id bq27csp2707341dyb; Tue, 27 Feb 2024 05:53:54 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUBpQgwyYl7GNbJseL4vQCXhX+x6RVEKvZHJWtCRFLqSwsWhsDhWsrKqt9stt7IbNw0xzkLS/DfRAdZMjPUY9NF1h9G1g== X-Google-Smtp-Source: AGHT+IHZgCdKVdZCo5Kc5B9khDyY/hGiV4SYYRoZqzXIWmazrDLU4C+1wY8wQFwy06j5tr9pj0Ci X-Received: by 2002:a17:907:b9c7:b0:a3f:da1c:ae4c with SMTP id xa7-20020a170907b9c700b00a3fda1cae4cmr7208413ejc.54.1709042034010; Tue, 27 Feb 2024 05:53:54 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1709042033; cv=pass; d=google.com; s=arc-20160816; b=t8LNeWIzavwDzBLFpijk94W5ES6VcAi8Mak497UE+2edOmPRbfW0UdfiFssXRUyJIJ lruslPv1Gs8LmyUlB5o/DZUonvkoBUlmxPnNPqebpWOhFtOQA96UbPnF/FXKevHlBN+8 3pOqxVVvnSP6UAW+5awtTw9PzZtDRAMooV2Psmx6KB1MINHbmL+CQcOKB+V7twG3NfpD umb33u3V6s8yyiW67uwgjalCDfwPl3SZwB8KG2NYI0okhGfSW6nsAmTK92e77RY7OojY yD8DDmJUWYBJdwlA336oUMPEgN6jDH+8dAqBjoIQDOArQMBFoIGA9sgBPyQphb4d9YjB wOyA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=jQS6ixAjVk0AC8cKYwTBDAm2HkLKEDsUdifzfOmBcs0=; fh=Eg1bbkMBB2FpJxkcjuowRNwwz/msSkRTpuEVSvFESpg=; b=KgeVwS64H69p1QCLwaSt3bpyZ6tkmjGpBEEojh+fuG0Uwp4KUzw6vUBw9Iwy3uATFN 8TWf3vtBZ6oDE8MeLoP0ZMmR11DIh3KzzJPSR+805SgEEwb7Rp5RzcsTbjReBMhRofd2 hMEkNsJ37jPuHkmImgSpfGlibPqckHBh1aNYJ1H/sDpme/IEHmxwLULQsLg/kvILEPdy TBxtCYwZwC0M/A/FbqpGsBiSXuklMAt5HrUlLrzU3Hs2duKscxT6UIZvAEzOICd+QMBv GmSwbKcbSyjZqkgzmK/xOR6HqCVK6WoqOfaOxciwZekGA69h2d/0g9HRwwuiy5aL5mM8 xlxw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.alibaba.com header.s=default header.b=i3R9bWJt; arc=pass (i=1 spf=pass spfdomain=linux.alibaba.com dkim=pass dkdomain=linux.alibaba.com dmarc=pass fromdomain=linux.alibaba.com); spf=pass (google.com: domain of linux-kernel+bounces-83354-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-83354-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.alibaba.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id e21-20020a1709062c1500b00a43115d5551si745370ejh.678.2024.02.27.05.53.53 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Feb 2024 05:53:53 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-83354-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.alibaba.com header.s=default header.b=i3R9bWJt; arc=pass (i=1 spf=pass spfdomain=linux.alibaba.com dkim=pass dkdomain=linux.alibaba.com dmarc=pass fromdomain=linux.alibaba.com); spf=pass (google.com: domain of linux-kernel+bounces-83354-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-83354-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.alibaba.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 9D7F51F227BF for ; Tue, 27 Feb 2024 13:53:53 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 65D58145FFF; Tue, 27 Feb 2024 13:52:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="i3R9bWJt" Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 862601420A6 for ; Tue, 27 Feb 2024 13:52:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.133 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709041962; cv=none; b=DROcn0oUVdUzm0cIhDYShct/P36vURNviLqtPqTM7GX8Rq48sfssUTZWWgmiqRVMJHi9zJIs8iXUo3xlyGJRUbl7v7sg56/jpOOoUcLCIxyCpjDNoN8et6dpl2W2QEg+9voR0ZHnwbNXr4GE0LxEHno6hAgdcG4VWHugDN+iXH0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709041962; c=relaxed/simple; bh=MzCC0AE/rUvpEAo4tb5cVZ7cGsewHKgZszzxyYlWgms=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Mm6ZPQUvEYWEJ7T9IXqE567dVnRCipA6ACoCMHuEqgvT3NPhRb2zP6XLDUetsTUPKjZTElYOwM3qPcCXAg3ZQNrePvxr1RvMmzXDb69CmMd0zlbh02XTkJbtvNSEVNdVkUNQXbYRWxmjfIchXvVrQwm9CP3DFa+MUxYI38fw2BM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=i3R9bWJt; arc=none smtp.client-ip=115.124.30.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1709041958; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=jQS6ixAjVk0AC8cKYwTBDAm2HkLKEDsUdifzfOmBcs0=; b=i3R9bWJt363GKWx5GJRWr6i7VTUoUN3mS7947TEsH0hIcPK9mpi1+rMe+1g2owwUwln/HU+E6tUx4cawEmaXKdIRCNoLh0rkLpUPx8UyCcJaDbTe3bEqJXJkY075nXPZqJzsj3GBiw8iLlmJXH9q9l2GzFLDDFezrn+NrbmgHMs= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R161e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045192;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=10;SR=0;TI=SMTPD_---0W1MvL.r_1709041957; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W1MvL.r_1709041957) by smtp.aliyun-inc.com; Tue, 27 Feb 2024 21:52:37 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: muchun.song@linux.dev, osalvador@suse.de, david@redhat.com, linmiaohe@huawei.com, naoya.horiguchi@nec.com, mhocko@kernel.org, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/3] docs: hugetlbpage.rst: add hugetlb migration description Date: Tue, 27 Feb 2024 21:52:27 +0800 Message-Id: <574e5bfbaa2b6930aad4a64e1c3da25c4ee5c9ee.1709041586.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1792060459581485126 X-GMAIL-MSGID: 1792060459581485126 Add some description of the hugetlb migration strategy. Signed-off-by: Baolin Wang --- Documentation/admin-guide/mm/hugetlbpage.rst | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/Documentation/admin-guide/mm/hugetlbpage.rst b/Documentation/admin-guide/mm/hugetlbpage.rst index e4d4b4a8dc97..f34a0d798d5b 100644 --- a/Documentation/admin-guide/mm/hugetlbpage.rst +++ b/Documentation/admin-guide/mm/hugetlbpage.rst @@ -376,6 +376,13 @@ Note that the number of overcommit and reserve pages remain global quantities, as we don't know until fault time, when the faulting task's mempolicy is applied, from which node the huge page allocation will be attempted. +The hugetlb may be migrated between the per-node hugepages pool in the following +scenarios: memory offline, memory failure, longterm pinning, syscalls(mbind, +migrate_pages and move_pages), alloc_contig_range() and alloc_contig_pages(). +Now only memory offline, memory failure and syscalls allow fallbacking to allocate +a new hugetlb on a different node if the current node is unable to allocate during +hugetlb migration, that means these 3 cases can break the per-node hugepages pool. + .. _using_huge_pages: Using Huge Pages