From patchwork Wed Feb 21 09:27:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 204062 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:108:e6aa:91d0 with SMTP id mn5csp926717dyc; Wed, 21 Feb 2024 01:30:22 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXX80E0cajsfcalSYGupuKprbfO110CaI1bvjnPWYUG043RhNWZVEqQ6knFTDvMERC9QEg1TQsBO3hvcW7vu0m+Z/Xsfg== X-Google-Smtp-Source: AGHT+IGz1v+ZricBA5l+ubX13OK8k3m2i1HNQoch2ZKEAMo9x98q5UtUOtB/Y+pU/mSDxzAxBg+W X-Received: by 2002:aa7:d441:0:b0:564:50e6:93f8 with SMTP id q1-20020aa7d441000000b0056450e693f8mr6318862edr.15.1708507822392; Wed, 21 Feb 2024 01:30:22 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708507822; cv=pass; d=google.com; s=arc-20160816; b=SHQCcRqPtqcPmbBblrj4tnFponVKvW6qgRKhTPJyDQrhv7lO+lVbSnzRA3VRZvbhqe XDrDs3DVizd/EtAmwyFJ2G0ohzrQajUiUqNBNhGS7mgHf+w3MPs2eE1kdFakEI4mdQY2 ygCIikDjWghte1gjJz8LiFjNkOW871eKKs7oKvpAXxZj/RgM+UiuM6qcZBMk8XvhptAG QGlOPe8JUaSInDGwWmHJlo/ZEGvkRtY5T0Z+lJcad5nM7kS5KCos+mhrlwax+2uOQlox XKnVcw1Mls3kFU+smQY4LjWrsR8LGDHPpqCQv8J8ectroeCxCpm6cnNxNV6lQ+ADI09H r8Pw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=dbpOfTRpA7RL14rl+8ySQliZQuoH0mAg7qi1p2q34yY=; fh=Eg1bbkMBB2FpJxkcjuowRNwwz/msSkRTpuEVSvFESpg=; b=A2123umJHcDfPLfbcj35uyOXpMvRju2ycZZAMm7xdPdX0cGAkRneLKAph89QEDIrLv CXMDF5QF7PAJQJIwZ1aLWVikBgk4DhXE/eL/FHAl/4b0P7bHuQQEZ+wUoDBzUu5LSBx7 WMOmZpvDkbj1lz/O4ljtIrvXVFjg5xmlLd5WKNAFymHsIXGn18AwEUaZU2A7vAQ3TKtB T5IgCMnuWqONObt+BfTqqnOtDBsNRAeVYbB57Z8vrHPEzc6YC7NTqRZ2VMvz9JMVd1uD gvlb4BAhU+anpY+JgMoPE10ViltQr891/k4bmc+HB2Ix+lplSeN/rW0lNDIxYtLyfuls lNEA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.alibaba.com header.s=default header.b=h1985OuN; arc=pass (i=1 spf=pass spfdomain=linux.alibaba.com dkim=pass dkdomain=linux.alibaba.com dmarc=pass fromdomain=linux.alibaba.com); spf=pass (google.com: domain of linux-kernel+bounces-74403-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74403-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.alibaba.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id r22-20020aa7d156000000b005649f9fbfe9si2210146edo.472.2024.02.21.01.30.22 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Feb 2024 01:30:22 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-74403-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.alibaba.com header.s=default header.b=h1985OuN; arc=pass (i=1 spf=pass spfdomain=linux.alibaba.com dkim=pass dkdomain=linux.alibaba.com dmarc=pass fromdomain=linux.alibaba.com); spf=pass (google.com: domain of linux-kernel+bounces-74403-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74403-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.alibaba.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 0781C1F239C1 for ; Wed, 21 Feb 2024 09:30:22 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9FD9B3F9F4; Wed, 21 Feb 2024 09:28:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="h1985OuN" Received: from out30-99.freemail.mail.aliyun.com (out30-99.freemail.mail.aliyun.com [115.124.30.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E87E3DB89 for ; Wed, 21 Feb 2024 09:28:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.99 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708507695; cv=none; b=eWwNFPPWijaZ/GG8XZsw/FRJ34UGS7cAvIljB4XP6ZOUwI0wBRJyJSm84rLpPVsPnGgqWbNFDujtkdgkZB841vqTIhLOm++qAJ7JQiaiPGHdNnd2pnLT0o4DytrrueuVc7n36sukmzgRbfqWQ+LReFcPS3PE1jd9gsnNaO+NLW4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708507695; c=relaxed/simple; bh=3uKwHohhB3aic92O+ME8L8BAtG4fdHDuAqPlhAFi+CU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=l+1tuzxVCWjOavqWkRmibeOzd2ubAMGn1Ot68mU7ACvbgLY2gqGXt9H9V6F3GVY/dAmg70VGcW/gGn8Qa+PGnoUFdkcNyQAEqLd5kYHN4eT5ksJhIZx0GMfsdvs+7OrXwHVnPfVgXtm9PDEd0AsnY5XY5vbM0G+9CeTTzwvJoUk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=h1985OuN; arc=none smtp.client-ip=115.124.30.99 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708507685; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=dbpOfTRpA7RL14rl+8ySQliZQuoH0mAg7qi1p2q34yY=; b=h1985OuNmMzPVT2MEjzNvkwjO8iGeZGj5HIyJ840Ku6pHvlVK89oIPp2MBbUJXnbvOiXozhRYD2uEk1PeZ4lZzagswPlsD8T4V9zS2yTS+RtyoBZ8ijq+C46pbk4hgD3IFkj6GkA3qU91OTwHXh8s1tqSVQWZyPOVbwvIWk/Zb0= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R211e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045176;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=10;SR=0;TI=SMTPD_---0W0zPloF_1708507683; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W0zPloF_1708507683) by smtp.aliyun-inc.com; Wed, 21 Feb 2024 17:28:04 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: muchun.song@linux.dev, osalvador@suse.de, david@redhat.com, linmiaohe@huawei.com, naoya.horiguchi@nec.com, mhocko@kernel.org, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH 1/3] mm: record the migration reason for struct migration_target_control Date: Wed, 21 Feb 2024 17:27:53 +0800 Message-Id: <04e445d16ab9f8d5a4cd4082c0c9b5f5e0bbf54c.1708507022.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791500298098591086 X-GMAIL-MSGID: 1791500298098591086 To support different hugetlb allocation strategies during hugetlb migration based on various migration reasons, recording the migration reason for the migration_target_control structure as a preparation. Signed-off-by: Baolin Wang --- mm/gup.c | 1 + mm/internal.h | 1 + mm/memory-failure.c | 1 + mm/memory_hotplug.c | 1 + mm/mempolicy.c | 1 + mm/migrate.c | 1 + mm/page_alloc.c | 1 + mm/vmscan.c | 3 ++- 8 files changed, 9 insertions(+), 1 deletion(-) diff --git a/mm/gup.c b/mm/gup.c index df83182ec72d..959a1a05b059 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -2132,6 +2132,7 @@ static int migrate_longterm_unpinnable_pages( struct migration_target_control mtc = { .nid = NUMA_NO_NODE, .gfp_mask = GFP_USER | __GFP_NOWARN, + .reason = MR_LONGTERM_PIN, }; if (migrate_pages(movable_page_list, alloc_migration_target, diff --git a/mm/internal.h b/mm/internal.h index 93e229112045..7677ee4d8e12 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -958,6 +958,7 @@ struct migration_target_control { int nid; /* preferred node id */ nodemask_t *nmask; gfp_t gfp_mask; + enum migrate_reason reason; }; /* diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 9349948f1abf..780bb2aee0af 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -2666,6 +2666,7 @@ static int soft_offline_in_use_page(struct page *page) struct migration_target_control mtc = { .nid = NUMA_NO_NODE, .gfp_mask = GFP_USER | __GFP_MOVABLE | __GFP_RETRY_MAYFAIL, + .reason = MR_MEMORY_FAILURE, }; if (!huge && folio_test_large(folio)) { diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index a444e2d7dd2b..b79ba36e09e0 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1841,6 +1841,7 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) struct migration_target_control mtc = { .nmask = &nmask, .gfp_mask = GFP_USER | __GFP_MOVABLE | __GFP_RETRY_MAYFAIL, + .reason = MR_MEMORY_HOTPLUG, }; int ret; diff --git a/mm/mempolicy.c b/mm/mempolicy.c index f60b4c99f130..98ceb12e0e17 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -1070,6 +1070,7 @@ static long migrate_to_node(struct mm_struct *mm, int source, int dest, struct migration_target_control mtc = { .nid = dest, .gfp_mask = GFP_HIGHUSER_MOVABLE | __GFP_THISNODE, + .reason = MR_SYSCALL, }; nodes_clear(nmask); diff --git a/mm/migrate.c b/mm/migrate.c index 73a052a382f1..bde63010a3cf 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2060,6 +2060,7 @@ static int do_move_pages_to_node(struct list_head *pagelist, int node) struct migration_target_control mtc = { .nid = node, .gfp_mask = GFP_HIGHUSER_MOVABLE | __GFP_THISNODE, + .reason = MR_SYSCALL, }; err = migrate_pages(pagelist, alloc_migration_target, NULL, diff --git a/mm/page_alloc.c b/mm/page_alloc.c index b0b92ce997dc..81ba73d77921 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6264,6 +6264,7 @@ int __alloc_contig_migrate_range(struct compact_control *cc, struct migration_target_control mtc = { .nid = zone_to_nid(cc->zone), .gfp_mask = GFP_USER | __GFP_MOVABLE | __GFP_RETRY_MAYFAIL, + .reason = MR_CONTIG_RANGE, }; lru_cache_disable(); diff --git a/mm/vmscan.c b/mm/vmscan.c index 87df3a48bdd7..d111c8e3b40e 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -978,7 +978,8 @@ static unsigned int demote_folio_list(struct list_head *demote_folios, .gfp_mask = (GFP_HIGHUSER_MOVABLE & ~__GFP_RECLAIM) | __GFP_NOWARN | __GFP_NOMEMALLOC | GFP_NOWAIT, .nid = target_nid, - .nmask = &allowed_mask + .nmask = &allowed_mask, + .reason = MR_DEMOTION, }; if (list_empty(demote_folios)) From patchwork Wed Feb 21 09:27:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 204064 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:108:e6aa:91d0 with SMTP id mn5csp926821dyc; Wed, 21 Feb 2024 01:30:35 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWbXDQ3XgPKIGo5gmUWi9ceYnTIYT6O8nhY1mnHOmhudFBS4/P32Gzsy8H5t5mHQcZjXB/OjX/ZKmMKfqk5iD6rppBbvg== X-Google-Smtp-Source: AGHT+IGyDsI/YQV1OEvTvFx1E5EupPkO8GT26o+vzdn8j9Vwozzo48kyUrMUQMm5bqRp7f4YW0KF X-Received: by 2002:a05:6808:16a0:b0:3bd:bef9:84b9 with SMTP id bb32-20020a05680816a000b003bdbef984b9mr21973077oib.33.1708507835525; Wed, 21 Feb 2024 01:30:35 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708507835; cv=pass; d=google.com; s=arc-20160816; b=GsakYExdME2wkB3bqcrzVEbOLwjjkV3loqHxVub80ckeaa2fOft/Ot2kTeWNcVJSTA R0W5z1Bsum4dodZQdyRG20tqRlC/nG3C1r4jx5bDhcfNasSdXbZyWrnbspL1ujithzIk GLzllkg+aRLFIz70GzE5Qm/YG7n8RiRk5zlnkg5VZKuY43PKwdeKZPI6WcMdOYCzLXDd 0VStsl7N/fdBryxRQMSpm5LljlI8zRkz0nZy9efMvtjborrR82A2LqbwD2iBBkq/GxhZ lGCcMTgOyd65GIdtMo1hAVaQGAK23pEFKePLFfzhTNOnunR6nWoamPr2SfovowS8qTC+ kFTA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=ImWD/LFam8uPK216oOawCtCWE3yiiD0csaC4SgFveA8=; fh=Eg1bbkMBB2FpJxkcjuowRNwwz/msSkRTpuEVSvFESpg=; b=ui/9mP6CkSOQs84u1fr9jfjPE/pevQd8kIhziekJiiVP0p5ORl3IvpUkuBou2dQODq RfhjaxHgre10c3xQp32292zu5ljEXYB/8ccwhx4qR2fbpu8VbblfXh4DTGzL2yPiSfBs WwnJ1rOG6c2RgVWPGYgzCyKD51oA8t0zz6leV6GqU6LqUCULZVlGgcmyEdQgR0kcJ+Aj lM3jM/PePatVWSRizKNbiM3qV/jhT0Kpgklm+L7H+4ter+Pa6uvFTCfd1vjZ7L5agN03 VkAOgbygh8w+ld2+D+6jCRrVJHmvy0X3hSEg8CagOlZ7g5iEFSzfc583BmiDxcMn/FWU OoiA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.alibaba.com header.s=default header.b=N8dnYYs4; arc=pass (i=1 spf=pass spfdomain=linux.alibaba.com dkim=pass dkdomain=linux.alibaba.com dmarc=pass fromdomain=linux.alibaba.com); spf=pass (google.com: domain of linux-kernel+bounces-74404-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74404-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.alibaba.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id q22-20020a056a00085600b006e3ac788faasi6677001pfk.364.2024.02.21.01.30.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Feb 2024 01:30:35 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-74404-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.alibaba.com header.s=default header.b=N8dnYYs4; arc=pass (i=1 spf=pass spfdomain=linux.alibaba.com dkim=pass dkdomain=linux.alibaba.com dmarc=pass fromdomain=linux.alibaba.com); spf=pass (google.com: domain of linux-kernel+bounces-74404-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74404-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.alibaba.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 50612283A11 for ; Wed, 21 Feb 2024 09:30:35 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 44C0E3DB89; Wed, 21 Feb 2024 09:28:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="N8dnYYs4" Received: from out30-99.freemail.mail.aliyun.com (out30-99.freemail.mail.aliyun.com [115.124.30.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EEEC43D97A for ; Wed, 21 Feb 2024 09:28:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.99 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708507696; cv=none; b=dwwe7Lnp4EZI+JxJF1WJchKKcpuM+dVELrIryUtyrnkVyYFkfTrv4lfY9rvXRlPVQjzFcjVRd7A8ADv+9ute+ZbXO4GxfdNn9ohRXPu/lS7sP+ktXwqBaFdkyluSp1P8dXrjGUAFAAXBwJ1AHVMu3Ba/53u4v/xTKImF19I0kAY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708507696; c=relaxed/simple; bh=XYc4TdVvWWJquVpA+tfkNKBtu2+qh+Pphd2uWChX58M=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=sXHZj6nWkz8WYXC4PnHl3P8LXLkXhkZnOTyOgaUbFUUsSljfK+IQKwaB4Pogf4Hkpdj8TG77ZYn+3YgBvIveZePZFFrnTJtsuqk1kqaQXJ1024Q7VTY/cdJYBN/i6M5ufIDkbimchaIlMo+wut3W8/H0EYB7hRQsHMViIAomXIk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=N8dnYYs4; arc=none smtp.client-ip=115.124.30.99 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708507686; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=ImWD/LFam8uPK216oOawCtCWE3yiiD0csaC4SgFveA8=; b=N8dnYYs4/RxkmL95EU9cddIr52GZltuM1MHI6/tbV4RPoxPXk9YywsZeJpvCq+yb+h+GRpBqbYGz0eOfRhOoOK4DZoYPDa3SlNqtFT0iQgtTOpcc2tV/1WQy7vIwK1QQ+b4NrsPOQ3rv4egKD2VaQ+Kdtn+92jFwoAlKI90t/iU= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R951e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046056;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=10;SR=0;TI=SMTPD_---0W0zUu6x_1708507684; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W0zUu6x_1708507684) by smtp.aliyun-inc.com; Wed, 21 Feb 2024 17:28:05 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: muchun.song@linux.dev, osalvador@suse.de, david@redhat.com, linmiaohe@huawei.com, naoya.horiguchi@nec.com, mhocko@kernel.org, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH 2/3] mm: hugetlb: make the hugetlb migration strategy consistent Date: Wed, 21 Feb 2024 17:27:54 +0800 Message-Id: <0514e5139b17ecf3cd9e09d86c93e586c56688dc.1708507022.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791500312062604124 X-GMAIL-MSGID: 1791500312062604124 As discussed in previous thread [1], there is an inconsistency when handing hugetlb migration. When handling the migration of freed hugetlb, it prevents fallback to other NUMA nodes in alloc_and_dissolve_hugetlb_folio(). However, when dealing with in-use hugetlb, it allows fallback to other NUMA nodes in alloc_hugetlb_folio_nodemask(), which can break the per-node hugetlb pool and might result in unexpected failures when node bound workloads doesn't get what is asssumed available. To make hugetlb migration strategy more clear, we should list all the scenarios of hugetlb migration and analyze whether allocation fallback is permitted: 1) Memory offline: will call dissolve_free_huge_pages() to free the freed hugetlb, and call do_migrate_range() to migrate the in-use hugetlb. Both can break the per-node hugetlb pool, but as this is an explicit offlining operation, no better choice. So should allow the hugetlb allocation fallback. 2) Memory failure: same as memory offline. Should allow fallback to a different node might be the only option to handle it, otherwise the impact of poisoned memory can be amplified. 3) Longterm pinning: will call migrate_longterm_unpinnable_pages() to migrate in-use and not-longterm-pinnable hugetlb, which can break the per-node pool. But we should fail to longterm pinning if can not allocate on current node to avoid breaking the per-node pool. 4) Syscalls (mbind, migrate_pages, move_pages): these are explicit users operation to move pages to other nodes, so fallback to other nodes should not be prohibited. 5) alloc_contig_range: used by CMA allocation and virtio-mem fake-offline to allocate given range of pages. Now the freed hugetlb migration is not allowed to fallback, to keep consistency, the in-use hugetlb migration should be also not allowed to fallback. 6) alloc_contig_pages: used by kfence, pgtable_debug etc. The strategy should be consistent with that of alloc_contig_range(). Based on the analysis of the various scenarios above, determine whether fallback is permitted according to the migration reason in alloc_hugetlb_folio_nodemask(). [1] https://lore.kernel.org/all/6f26ce22d2fcd523418a085f2c588fe0776d46e7.1706794035.git.baolin.wang@linux.alibaba.com/ Signed-off-by: Baolin Wang --- include/linux/hugetlb.h | 4 ++-- mm/hugetlb.c | 28 ++++++++++++++++++++++++++-- mm/mempolicy.c | 2 +- mm/migrate.c | 2 +- 4 files changed, 30 insertions(+), 6 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 77b30a8c6076..fa122dc509cf 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -747,7 +747,7 @@ int isolate_or_dissolve_huge_page(struct page *page, struct list_head *list); struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, unsigned long addr, int avoid_reserve); struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred_nid, - nodemask_t *nmask, gfp_t gfp_mask); + nodemask_t *nmask, gfp_t gfp_mask, int reason); int hugetlb_add_to_page_cache(struct folio *folio, struct address_space *mapping, pgoff_t idx); void restore_reserve_on_error(struct hstate *h, struct vm_area_struct *vma, @@ -1065,7 +1065,7 @@ static inline struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, static inline struct folio * alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred_nid, - nodemask_t *nmask, gfp_t gfp_mask) + nodemask_t *nmask, gfp_t gfp_mask, int reason) { return NULL; } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 68283e54c899..a55cfc7844bc 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -2621,8 +2621,10 @@ struct folio *alloc_buddy_hugetlb_folio_with_mpol(struct hstate *h, /* folio migration callback function */ struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred_nid, - nodemask_t *nmask, gfp_t gfp_mask) + nodemask_t *nmask, gfp_t gfp_mask, int reason) { + bool allowed_fallback = false; + spin_lock_irq(&hugetlb_lock); if (available_huge_pages(h)) { struct folio *folio; @@ -2636,6 +2638,28 @@ struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred_nid, } spin_unlock_irq(&hugetlb_lock); + if (gfp_mask & __GFP_THISNODE) + goto alloc_new; + + /* + * Note: the memory offline, memory failure and migration syscalls can break + * the per-node hugetlb pool. Other cases can not allocate new hugetlb on + * other nodes. + */ + switch (reason) { + case MR_MEMORY_HOTPLUG: + case MR_MEMORY_FAILURE: + case MR_SYSCALL: + case MR_MEMPOLICY_MBIND: + allowed_fallback = true; + break; + default: + break; + } + + if (!allowed_fallback) + gfp_mask |= __GFP_THISNODE; +alloc_new: return alloc_migrate_hugetlb_folio(h, gfp_mask, preferred_nid, nmask); } @@ -6666,7 +6690,7 @@ static struct folio *alloc_hugetlb_folio_vma(struct hstate *h, gfp_mask = htlb_alloc_mask(h); node = huge_node(vma, address, gfp_mask, &mpol, &nodemask); - folio = alloc_hugetlb_folio_nodemask(h, node, nodemask, gfp_mask); + folio = alloc_hugetlb_folio_nodemask(h, node, nodemask, gfp_mask, -1); mpol_cond_put(mpol); return folio; diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 98ceb12e0e17..436e817eeaeb 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -1228,7 +1228,7 @@ static struct folio *alloc_migration_target_by_mpol(struct folio *src, h = folio_hstate(src); gfp = htlb_alloc_mask(h); nodemask = policy_nodemask(gfp, pol, ilx, &nid); - return alloc_hugetlb_folio_nodemask(h, nid, nodemask, gfp); + return alloc_hugetlb_folio_nodemask(h, nid, nodemask, gfp, MR_MEMPOLICY_MBIND); } if (folio_test_large(src)) diff --git a/mm/migrate.c b/mm/migrate.c index bde63010a3cf..0c2b70800da3 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2022,7 +2022,7 @@ struct folio *alloc_migration_target(struct folio *src, unsigned long private) gfp_mask = htlb_modify_alloc_mask(h, gfp_mask); return alloc_hugetlb_folio_nodemask(h, nid, - mtc->nmask, gfp_mask); + mtc->nmask, gfp_mask, mtc->reason); } if (folio_test_large(src)) { From patchwork Wed Feb 21 09:27:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 204060 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:693c:2685:b0:108:e6aa:91d0 with SMTP id mn5csp926405dyc; Wed, 21 Feb 2024 01:29:27 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXqAwZA/yJ5AOKdzQL/0BPrmAJoVCgj46k4wS1KXV0h6eQzOWuVPvKPfQoisOHEaG53eTnIzvk9CdFWQ4ZoW4o+sdh3tg== X-Google-Smtp-Source: AGHT+IEKaa0VRI41PBaexBqKbx7idUUOJ5SIzBHpy/7a2DfxBFa6HTRy3LAYryQnOpNzSn+jGLRr X-Received: by 2002:a05:6808:1646:b0:3c1:3427:fef3 with SMTP id az6-20020a056808164600b003c13427fef3mr22956818oib.48.1708507767787; Wed, 21 Feb 2024 01:29:27 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708507767; cv=pass; d=google.com; s=arc-20160816; b=KXPK0X2lKhGe3RxmFUx2kBES4BthcvZ+fJDT/eAbO901h+vUxoJ3jDxuQMBsUwQQ4f k6iEjokar5WwROuOTJEf9DynE5x6Qy33fQNT0ATqI1LaGV9gy7T/shWH9/dB085m9HkR FyZWTR0rDsgOTO5SYwiDaLNoJ97TAhWC8M9Vhzodu2YeBzxXG5yftpFny+QL3vW+yshz TCXLp81groVcmVJDQLmAC6PoLLiKz6tXDUmgTf60vo7GUbuKDr0hqTnwutmeDRgpnhNB W9ValxkY6aZ6TSdWnPY0Od6nRFB5y2T+THaD3OkefL4MxwH56wom+YDDY8h9ANr9HxF6 9wvw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=m3nqGI/fMVZjL40459w/MnS9DkbLdhc8HEWPtY+H0PM=; fh=Eg1bbkMBB2FpJxkcjuowRNwwz/msSkRTpuEVSvFESpg=; b=LotWfJLZIePI8kUJwfrhRk80buV6+1C9KBfwflB9JbPzQ8SLDfWIMwt6iuO7oISwz7 FAdALBgfnWTdOon3n8FHGR4ftKpJvWxUxmZxRmWdZz+Knn1H+YpxTx09hcR62hFRrkUl PZZFT5Y+Ffa7BPdZzVvwQ4yO9xbnPK4uHymHhcCzb7T5yhjHo1AeK3h+rL2T6zsrijlL M4vNWWJPZCt+iZUl4I9rsWTPJO2x8afRpP8AlFEO5DYrRE05sr5/G+OBu/OGFP2Itk0e fAnqAd/rDwQEdoFY3G+ft72OBKERG1HJ8EZNjgiigcRluXFF4WeeVTYTx35kVGjZk8Gu CwEg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.alibaba.com header.s=default header.b=tbf261B3; arc=pass (i=1 spf=pass spfdomain=linux.alibaba.com dkim=pass dkdomain=linux.alibaba.com dmarc=pass fromdomain=linux.alibaba.com); spf=pass (google.com: domain of linux-kernel+bounces-74400-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74400-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.alibaba.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id j3-20020a654283000000b005dccf9a43ccsi7775363pgp.202.2024.02.21.01.29.27 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Feb 2024 01:29:27 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-74400-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.alibaba.com header.s=default header.b=tbf261B3; arc=pass (i=1 spf=pass spfdomain=linux.alibaba.com dkim=pass dkdomain=linux.alibaba.com dmarc=pass fromdomain=linux.alibaba.com); spf=pass (google.com: domain of linux-kernel+bounces-74400-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74400-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.alibaba.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 946BE28377A for ; Wed, 21 Feb 2024 09:29:27 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7C1EB3EA89; Wed, 21 Feb 2024 09:28:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="tbf261B3" Received: from out30-100.freemail.mail.aliyun.com (out30-100.freemail.mail.aliyun.com [115.124.30.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 385563D556 for ; Wed, 21 Feb 2024 09:28:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.100 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708507692; cv=none; b=I45oBCvl+XBPXOHK/Vv9n4xd6GwCOC5OB9fv+I4/4fqPTIl11NUZKweSlETKygE+RECljTXSae7p9aO2NRwm3f6yssGivnFHp0AtFJ77Kx8RzaSb5suIe5c9y2XirOF5x9c9nUKAyqhv2fD3OcsJP6zi1tBr/b+bl+Lv3raiqw8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708507692; c=relaxed/simple; bh=x2riKMYmISimOeooDuvtdgn5hEARkXgRbBvuG4j/oVE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=K5njrrUZFu41YuUz+xE7cC411/Y/93OgTuPToE9ZWjvWgq9A+pPuRCo19TTc0+AqzbdMkexDWx5b6JMBbv5pLZnB3/8r2S9dfUUsfvkgQqJcG6EUEMWsjSSa92RTnFYmqTp+tMSREalNU5MsX1Sy3E4WZDkq0eqxfpI8tx2H6ao= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=tbf261B3; arc=none smtp.client-ip=115.124.30.100 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1708507687; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=m3nqGI/fMVZjL40459w/MnS9DkbLdhc8HEWPtY+H0PM=; b=tbf261B3zP1aJ8GoO26ZLScrCldl7dzZMaMeJlr4A0bIl/eqetArzNNTiOsFE+fOf/P1yNqrhbc01ou4qQRZMWJoVnjJ3soxij/rRObHr6LyXu5wk8ViEdZ/Bm1Q1GVHnL3JyQopwO4VLGTofhfHTP+Uxm8md5XHJXiDpx/ZCk0= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R101e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046050;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=10;SR=0;TI=SMTPD_---0W0zShrU_1708507686; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W0zShrU_1708507686) by smtp.aliyun-inc.com; Wed, 21 Feb 2024 17:28:06 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: muchun.song@linux.dev, osalvador@suse.de, david@redhat.com, linmiaohe@huawei.com, naoya.horiguchi@nec.com, mhocko@kernel.org, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH 3/3] docs: hugetlbpage.rst: add hugetlb migration description Date: Wed, 21 Feb 2024 17:27:55 +0800 Message-Id: <75b80937a84bd98211cea0607707bfdee8cb5873.1708507022.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791500241131446664 X-GMAIL-MSGID: 1791500241131446664 Add some description of the hugetlb migration strategy. Signed-off-by: Baolin Wang --- Documentation/admin-guide/mm/hugetlbpage.rst | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/Documentation/admin-guide/mm/hugetlbpage.rst b/Documentation/admin-guide/mm/hugetlbpage.rst index e4d4b4a8dc97..68d7bc2165c9 100644 --- a/Documentation/admin-guide/mm/hugetlbpage.rst +++ b/Documentation/admin-guide/mm/hugetlbpage.rst @@ -376,6 +376,13 @@ Note that the number of overcommit and reserve pages remain global quantities, as we don't know until fault time, when the faulting task's mempolicy is applied, from which node the huge page allocation will be attempted. +The hugetlb may be migrated between the per-node hugepages pool in the following +scenarios: memory offline, memory failure, longterm pinning, syscalls(mbind, +migrate_pages, move_pages), alloc_contig_range() and alloc_contig_pages(). Now +only memory offline, memory failure and syscalls allow fallback to allocate a +new hugetlb on a different node if the current node is unable to allocate during +hugetlb migration, that means these 3 cases can break the per-node hugepages pool. + .. _using_huge_pages: Using Huge Pages