| Message ID | 20230428100846.95535-1-yajun.deng@linux.dev |
|---|---|
| State | New |
| Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org>
Delivered-To: ouuuleilei@gmail.com
Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp832352vqo;
Fri, 28 Apr 2023 03:35:43 -0700 (PDT)
X-Google-Smtp-Source:
ACHHUZ6/zXCcABtxHhPtchZB4pzwkZFIk1uGG998kmm/DlY5MIeaOscbYDc6QdysOhCUHd14uuQ1
X-Received: by 2002:a17:90a:ec0a:b0:247:bdeb:594f with SMTP id
l10-20020a17090aec0a00b00247bdeb594fmr4930437pjy.15.1682678142691;
Fri, 28 Apr 2023 03:35:42 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; t=1682678142; cv=none;
d=google.com; s=arc-20160816;
b=yA38CdtbRn0eD9+wRB/vl6JYbVZLoMch6APBr2eWTM+/CkDuKEqVvXlqU1gae+39pv
8shx++UD5ffbPH50EmJ5egzSKMbBfQrXa6ldKv1SQ7/LWpGjxlkN+3rp5oFeAqTGuVYQ
zFyaHGbT6Io56VRMhw2Q/JhIDbBN430qLYggXmXMh2PIFP5stnbMMgRN6iuZXwb/fPDM
gK29zQxjGtz+NcGiJAy6v5Jzf77Lexj6vEF3Zb6QLKzkYfhoDZi6hQ2g0Y1YHIw9M5p7
xbzGVx9W7+VqQj0uWIZckDFbos08eFt/cKb31hUf86DreZUUcLrrcHdb5T/3XyqV/Q2O
bo8g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com;
s=arc-20160816;
h=list-id:precedence:content-transfer-encoding:mime-version
:message-id:date:subject:cc:to:from:dkim-signature;
bh=PVAO+BlDHMujnNKZN8jYnO3UHWhcSJNRlz/2YLsn0QQ=;
b=OMuyD7JNrGZt9Xpm5yozjVeDL5wbi4PbNcrVu6E4YadzxqfMRWMngNfI9eNZHyRHOp
luUmdESvC07FHYv5PBS/3ph/jXdqkQT2SZbwdsbWHqArfHXX2NTejMRoyN3yxKnKOcZ7
PzCFlgt5WgWwObGo2t7O96T4LRPPAq/GRq/6dUOKL6eCoIXMo9QnKFlPQID/pD2qSUDH
X555lNFj13zD505o1N6oLSMN8bVCqOVPkGv9xWDGHhuxApQupe8pv/COFw5zTpJlRgyl
vJjHwsvLC8xreDESGITerGkYzuMUO0OCKMDpIgZeLZUa7KA2Anbs5RqIH10rjWnycimo
sBzQ==
ARC-Authentication-Results: i=1; mx.google.com;
dkim=pass header.i=@linux.dev header.s=key1 header.b=PtApRLYm;
spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org
designates 2620:137:e000::1:20 as permitted sender)
smtp.mailfrom=linux-kernel-owner@vger.kernel.org;
dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev
Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20])
by mx.google.com with ESMTP id
nn17-20020a17090b38d100b002478bba4da2si1956618pjb.127.2023.04.28.03.35.29;
Fri, 28 Apr 2023 03:35:42 -0700 (PDT)
Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org
designates 2620:137:e000::1:20 as permitted sender)
client-ip=2620:137:e000::1:20;
Authentication-Results: mx.google.com;
dkim=pass header.i=@linux.dev header.s=key1 header.b=PtApRLYm;
spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org
designates 2620:137:e000::1:20 as permitted sender)
smtp.mailfrom=linux-kernel-owner@vger.kernel.org;
dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
id S1345632AbjD1KJJ (ORCPT <rfc822;chrisjones.unixmen@gmail.com>
+ 99 others); Fri, 28 Apr 2023 06:09:09 -0400
Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51050 "EHLO
lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
with ESMTP id S229519AbjD1KJG (ORCPT
<rfc822;linux-kernel@vger.kernel.org>);
Fri, 28 Apr 2023 06:09:06 -0400
Received: from out-63.mta1.migadu.com (out-63.mta1.migadu.com [95.215.58.63])
by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5A540268E
for <linux-kernel@vger.kernel.org>;
Fri, 28 Apr 2023 03:09:04 -0700 (PDT)
X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and
include these headers.
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1;
t=1682676542;
h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
to:to:cc:cc:mime-version:mime-version:
content-transfer-encoding:content-transfer-encoding;
bh=PVAO+BlDHMujnNKZN8jYnO3UHWhcSJNRlz/2YLsn0QQ=;
b=PtApRLYmSe4NU9sJuZXacjEEDoHP1mUvWTAOjzUNY7o8SF1NHkhSVS+YleS4s7lmUzGlQ2
zrmrhV1KalImjhj7cWutu7WsxuZNpJjVsJeRb98zodWiH3EZOen6rrnJff00fV2Zxzib60
OeQR9ZEO9U7Ogw+A7sH+fyNN+747kwE=
From: Yajun Deng <yajun.deng@linux.dev>
To: david@redhat.com, osalvador@suse.de, akpm@linux-foundation.org,
mhocko@suse.com
Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org,
Yajun Deng <yajun.deng@linux.dev>
Subject: [PATCH] mm/memory_hotplug: fix dead loop in offline_pages()
Date: Fri, 28 Apr 2023 18:08:46 +0800
Message-Id: <20230428100846.95535-1-yajun.deng@linux.dev>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-Migadu-Flow: FLOW_OUT
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS,
T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no
version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?=
X-GMAIL-THRID: =?utf-8?q?1764415916025366697?=
X-GMAIL-MSGID: =?utf-8?q?1764415916025366697?=
|
| Series |
mm/memory_hotplug: fix dead loop in offline_pages()
|
|
Commit Message
Yajun Deng
April 28, 2023, 10:08 a.m. UTC
When migration failure in do_migrate_range() and then the
scan_movable_pages() will most likely return the same movable pfn.
In this case, there is no condition to bail out, they will
always run like this:
...
[878020.623959] migrating pfn 1727813 failed ret:1
[878020.623960] page:00000000faa9673c refcount:3 mapcount:0 mapping:00000000144ccd79 index:0x14280025 pfn:0x1727813
[878020.623962] memcg:ffffa0ff82d5a000
[878020.623962] aops:def_blk_aops ino:fd00001
[878020.623964] flags: 0x17ffffc000206a(referenced|dirty|active|workingset|private|node=0|zone=2|lastcpupid=0x1fffff)
[878020.623966] raw: 0017ffffc000206a ffffb0d14f50fbd8 ffffb0d14f50fbd8 ffffa0ff9c155018
[878020.623967] raw: 0000000014280025 ffffa10327d702d8 00000003ffffffff ffffa0ff82d5a000
[878020.623968] page dumped because: migration failure
[878020.626196] migrating pfn 1727813 failed ret:1
[878020.626198] page:00000000faa9673c refcount:3 mapcount:0 mapping:00000000144ccd79 index:0x14280025 pfn:0x1727813
[878020.626200] memcg:ffffa0ff82d5a000
[878020.626200] aops:def_blk_aops ino:fd00001
[878020.626202] flags: 0x17ffffc000206a(referenced|dirty|active|workingset|private|node=0|zone=2|lastcpupid=0x1fffff)
[878020.626204] raw: 0017ffffc000206a ffffb0d14f50fbd8 ffffb0d14f50fbd8 ffffa0ff9c155018
[878020.626205] raw: 0000000014280025 ffffa10327d702d8 00000003ffffffff ffffa0ff82d5a000
[878020.626206] page dumped because: migration failure
...
Bail out when migration failures reach 3 times.
Fixes: bb8965bd82fd ("mm, memory_hotplug: deobfuscate migration part of offlining")
Signed-off-by: Yajun Deng <yajun.deng@linux.dev>
---
mm/memory_hotplug.c | 19 ++++++++++++++-----
1 file changed, 14 insertions(+), 5 deletions(-)
Comments
On Fri 28-04-23 18:08:46, Yajun Deng wrote: > When migration failure in do_migrate_range() and then the > scan_movable_pages() will most likely return the same movable pfn. > In this case, there is no condition to bail out, they will > always run like this: > > ... > [878020.623959] migrating pfn 1727813 failed ret:1 > [878020.623960] page:00000000faa9673c refcount:3 mapcount:0 mapping:00000000144ccd79 index:0x14280025 pfn:0x1727813 > [878020.623962] memcg:ffffa0ff82d5a000 > [878020.623962] aops:def_blk_aops ino:fd00001 > [878020.623964] flags: 0x17ffffc000206a(referenced|dirty|active|workingset|private|node=0|zone=2|lastcpupid=0x1fffff) > [878020.623966] raw: 0017ffffc000206a ffffb0d14f50fbd8 ffffb0d14f50fbd8 ffffa0ff9c155018 > [878020.623967] raw: 0000000014280025 ffffa10327d702d8 00000003ffffffff ffffa0ff82d5a000 > [878020.623968] page dumped because: migration failure > [878020.626196] migrating pfn 1727813 failed ret:1 > [878020.626198] page:00000000faa9673c refcount:3 mapcount:0 mapping:00000000144ccd79 index:0x14280025 pfn:0x1727813 > [878020.626200] memcg:ffffa0ff82d5a000 > [878020.626200] aops:def_blk_aops ino:fd00001 > [878020.626202] flags: 0x17ffffc000206a(referenced|dirty|active|workingset|private|node=0|zone=2|lastcpupid=0x1fffff) > [878020.626204] raw: 0017ffffc000206a ffffb0d14f50fbd8 ffffb0d14f50fbd8 ffffa0ff9c155018 > [878020.626205] raw: 0000000014280025 ffffa10327d702d8 00000003ffffffff ffffa0ff82d5a000 > [878020.626206] page dumped because: migration failure > ... > > Bail out when migration failures reach 3 times. > > Fixes: bb8965bd82fd ("mm, memory_hotplug: deobfuscate migration part of offlining") > Signed-off-by: Yajun Deng <yajun.deng@linux.dev> Any hard coded failure retry limit will lead to premature failures. Have a look at 72b39cfc4d75 ("mm, memory_hotplug: do not fail offlining too early"). A proper way to deal with this is to implement termination from the userspace (e.g. timeout $TIMEOUT /bin/echo 0 > $PATH_TO_MEM/online) Nacked-by: Michal Hocko <mhocko@suse.com> > --- > mm/memory_hotplug.c | 19 ++++++++++++++----- > 1 file changed, 14 insertions(+), 5 deletions(-) > > diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c > index 8e0fa209d533..72dd385b8892 100644 > --- a/mm/memory_hotplug.c > +++ b/mm/memory_hotplug.c > @@ -1800,11 +1800,12 @@ int __ref offline_pages(unsigned long start_pfn, unsigned long nr_pages, > struct zone *zone, struct memory_group *group) > { > const unsigned long end_pfn = start_pfn + nr_pages; > - unsigned long pfn, system_ram_pages = 0; > + unsigned long pfn, tmp_pfn, system_ram_pages = 0; > const int node = zone_to_nid(zone); > unsigned long flags; > struct memory_notify arg; > char *reason; > + int count = 0; > int ret; > > /* > @@ -1887,12 +1888,20 @@ int __ref offline_pages(unsigned long start_pfn, unsigned long nr_pages, > > cond_resched(); > > + tmp_pfn = pfn; > ret = scan_movable_pages(pfn, end_pfn, &pfn); > if (!ret) { > - /* > - * TODO: fatal migration failures should bail > - * out > - */ > + if (pfn == tmp_pfn) > + count++; > + else > + count = 0; > + > + if (unlikely(count == 3)) { > + ret = -EBUSY; > + reason = "migration failure"; > + goto failed_removal_isolated; > + } > + > do_migrate_range(pfn, end_pfn); > } > } while (!ret); > -- > 2.25.1
On 28.04.23 13:07, Michal Hocko wrote: > On Fri 28-04-23 18:08:46, Yajun Deng wrote: >> When migration failure in do_migrate_range() and then the >> scan_movable_pages() will most likely return the same movable pfn. >> In this case, there is no condition to bail out, they will >> always run like this: >> >> ... >> [878020.623959] migrating pfn 1727813 failed ret:1 >> [878020.623960] page:00000000faa9673c refcount:3 mapcount:0 mapping:00000000144ccd79 index:0x14280025 pfn:0x1727813 >> [878020.623962] memcg:ffffa0ff82d5a000 >> [878020.623962] aops:def_blk_aops ino:fd00001 >> [878020.623964] flags: 0x17ffffc000206a(referenced|dirty|active|workingset|private|node=0|zone=2|lastcpupid=0x1fffff) >> [878020.623966] raw: 0017ffffc000206a ffffb0d14f50fbd8 ffffb0d14f50fbd8 ffffa0ff9c155018 >> [878020.623967] raw: 0000000014280025 ffffa10327d702d8 00000003ffffffff ffffa0ff82d5a000 >> [878020.623968] page dumped because: migration failure >> [878020.626196] migrating pfn 1727813 failed ret:1 >> [878020.626198] page:00000000faa9673c refcount:3 mapcount:0 mapping:00000000144ccd79 index:0x14280025 pfn:0x1727813 >> [878020.626200] memcg:ffffa0ff82d5a000 >> [878020.626200] aops:def_blk_aops ino:fd00001 >> [878020.626202] flags: 0x17ffffc000206a(referenced|dirty|active|workingset|private|node=0|zone=2|lastcpupid=0x1fffff) >> [878020.626204] raw: 0017ffffc000206a ffffb0d14f50fbd8 ffffb0d14f50fbd8 ffffa0ff9c155018 >> [878020.626205] raw: 0000000014280025 ffffa10327d702d8 00000003ffffffff ffffa0ff82d5a000 >> [878020.626206] page dumped because: migration failure >> ... >> >> Bail out when migration failures reach 3 times. >> >> Fixes: bb8965bd82fd ("mm, memory_hotplug: deobfuscate migration part of offlining") >> Signed-off-by: Yajun Deng <yajun.deng@linux.dev> > > Any hard coded failure retry limit will lead to premature failures. Have > a look at 72b39cfc4d75 ("mm, memory_hotplug: do not fail offlining too > early"). A proper way to deal with this is to implement termination from > the userspace (e.g. timeout $TIMEOUT /bin/echo 0 > $PATH_TO_MEM/online) > Nacked-by: Michal Hocko <mhocko@suse.com> Agreed; it's a bit more complicated when offlining is triggered by ACPI code :(
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 8e0fa209d533..72dd385b8892 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1800,11 +1800,12 @@ int __ref offline_pages(unsigned long start_pfn, unsigned long nr_pages, struct zone *zone, struct memory_group *group) { const unsigned long end_pfn = start_pfn + nr_pages; - unsigned long pfn, system_ram_pages = 0; + unsigned long pfn, tmp_pfn, system_ram_pages = 0; const int node = zone_to_nid(zone); unsigned long flags; struct memory_notify arg; char *reason; + int count = 0; int ret; /* @@ -1887,12 +1888,20 @@ int __ref offline_pages(unsigned long start_pfn, unsigned long nr_pages, cond_resched(); + tmp_pfn = pfn; ret = scan_movable_pages(pfn, end_pfn, &pfn); if (!ret) { - /* - * TODO: fatal migration failures should bail - * out - */ + if (pfn == tmp_pfn) + count++; + else + count = 0; + + if (unlikely(count == 3)) { + ret = -EBUSY; + reason = "migration failure"; + goto failed_removal_isolated; + } + do_migrate_range(pfn, end_pfn); } } while (!ret);