From patchwork Mon Oct 24 06:20:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 8169 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp280021wru; Sun, 23 Oct 2022 23:21:19 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7nbS5ySNpncLz6gN3C9706SmU4Pr5lZPwES9M5R7IOTyO52829ug6MiJ4+WTXAWhLoqeeW X-Received: by 2002:aa7:d889:0:b0:460:62ef:2695 with SMTP id u9-20020aa7d889000000b0046062ef2695mr19359539edq.273.1666592479662; Sun, 23 Oct 2022 23:21:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666592479; cv=none; d=google.com; s=arc-20160816; b=EpZNYBnEZI4UQiHi7LCKNo5wQ3gFCDpOSgelwaYxbWN+vKbAHVXfbHb8moUvsLk1/t EvSiWscYo3TGXnvffZfACZXmi/+ECTVULycspfhotggMf9+uIlkwS8Lb1w4aNXkvGOFa GLXiUErGEjNzsoGGka8xOP+TmdOcjy+FujKO5IY83OBWQap0MOgj3P1v90zZUgX5oIS2 3IxJkPvHUkmLgiANTTt07PxAuGsoSaKhtDViqadb+qmSE6wHoObMndp6y/1ewXdGf2Y8 +Jua1LRYpX+jnkmXUo+YXcdVgB9ArkUU9cRBO164po/+L8dig5Q0UGhrXjCD9bMn6puw b+DQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=xgj0BswIBkVGdA/syrYALGbly/Y6nfTG0sbkmC7GwMs=; b=tJga9gRJgkomVvPnMC0b//4UBK6XooAsFuWtfXkbxYxaiT9iaKgdyOMN2UJoUnXjvY //G/kFnru0105ABrNuyLp+MGRHToY63cPBBw5HR2eRhvLtXHEtTnhL0DJD3eLfaRjwBO xc/OCb6b6oCzTNu482gU2wbS72rutRkq3Nkjd7pAD7EP4x8iF89ZAyEJDKzEEu4d1g8w Eh9asrtOU6+Gx2iWg3PiYSJT4gTDu1v2/jqjLee8i6TyvigygHwc+2FVVeqaM9YKD+pl e/hY+Xunh/xe1EI5eMudSr2EQVGoo5hbhRYeemPwJhYGBHXkTUAmM2XIUwNXrm5VPG2Q gycg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=KSBITgTa; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l15-20020a056402028f00b00459cbb8074asi22977236edv.439.2022.10.23.23.20.55; Sun, 23 Oct 2022 23:21:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=KSBITgTa; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230221AbiJXGUk (ORCPT + 99 others); Mon, 24 Oct 2022 02:20:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60886 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230119AbiJXGUe (ORCPT ); Mon, 24 Oct 2022 02:20:34 -0400 Received: from out0.migadu.com (out0.migadu.com [IPv6:2001:41d0:2:267::]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0CE4513E87 for ; Sun, 23 Oct 2022 23:20:31 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1666592430; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xgj0BswIBkVGdA/syrYALGbly/Y6nfTG0sbkmC7GwMs=; b=KSBITgTaKK2z4iAcjh5BT0FvnZGgOSiDtiMYUNOvZrAomkOtanxjMBeFn8OJ5OHjnMpC47 ps/bfVLcUfvbTOJL8EfmHea1BTcL7QPHRk6tpxkYkEOnUWXU2XY/L3+4naIFYc2H5yDXxC 8sNLI7WEI219Y6fpDl9rcWLmPf2V4ms= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , Miaohe Lin , David Hildenbrand , Mike Kravetz , Yang Shi , Oscar Salvador , Muchun Song , Jane Chu , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v7 1/4] mm,hwpoison,hugetlb,memory_hotplug: hotremove memory section with hwpoisoned hugepage Date: Mon, 24 Oct 2022 15:20:09 +0900 Message-Id: <20221024062012.1520887-2-naoya.horiguchi@linux.dev> In-Reply-To: <20221024062012.1520887-1-naoya.horiguchi@linux.dev> References: <20221024062012.1520887-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_PASS, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747548875660029082?= X-GMAIL-MSGID: =?utf-8?q?1747548875660029082?= From: Naoya Horiguchi HWPoisoned page is not supposed to be accessed once marked, but currently such accesses can happen during memory hotremove because do_migrate_range() can be called before dissolve_free_huge_pages() is called. Clear HPageMigratable for hwpoisoned hugepages to prevent them from being migrated. This should be done in hugetlb_lock to avoid race against isolate_hugetlb(). get_hwpoison_huge_page() needs to have a flag to show it's called from unpoison to take refcount of hwpoisoned hugepages, so add it. Reported-by: Miaohe Lin Signed-off-by: Naoya Horiguchi Reviewed-by: Oscar Salvador Reviewed-by: Miaohe Lin --- ChangeLog v3 -> v7: - introduce TESTCLEARHPAGEFLAG() to determine the value of migratable_cleared ChangeLog v3 -> v6: - introduce migratable_cleared to remember that HPageMigratable is cleared in error handling. It's needed to cancel when an error event is filtered by hwpoison_filter(). (Thanks to Miaohe) ChangeLog v2 -> v3 - move to the approach of clearing HPageMigratable instead of shifting dissolve_free_huge_pages. --- include/linux/hugetlb.h | 19 +++++++++++++++---- include/linux/mm.h | 6 ++++-- mm/hugetlb.c | 9 +++++---- mm/memory-failure.c | 19 +++++++++++++++---- 4 files changed, 39 insertions(+), 14 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index a899bc76d677..0d137dab42b6 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -183,8 +183,9 @@ bool hugetlb_reserve_pages(struct inode *inode, long from, long to, long hugetlb_unreserve_pages(struct inode *inode, long start, long end, long freed); int isolate_hugetlb(struct page *page, struct list_head *list); -int get_hwpoison_huge_page(struct page *page, bool *hugetlb); -int get_huge_page_for_hwpoison(unsigned long pfn, int flags); +int get_hwpoison_huge_page(struct page *page, bool *hugetlb, bool unpoison); +int get_huge_page_for_hwpoison(unsigned long pfn, int flags, + bool *migratable_cleared); void putback_active_hugepage(struct page *page); void move_hugetlb_state(struct folio *old_folio, struct folio *new_folio, int reason); void free_huge_page(struct page *page); @@ -391,12 +392,13 @@ static inline int isolate_hugetlb(struct page *page, struct list_head *list) return -EBUSY; } -static inline int get_hwpoison_huge_page(struct page *page, bool *hugetlb) +static inline int get_hwpoison_huge_page(struct page *page, bool *hugetlb, bool unpoison) { return 0; } -static inline int get_huge_page_for_hwpoison(unsigned long pfn, int flags) +static inline int get_huge_page_for_hwpoison(unsigned long pfn, int flags, + bool *migratable_cleared) { return 0; } @@ -614,6 +616,10 @@ void folio_clear_hugetlb_##flname(struct folio *folio) \ } \ static inline void ClearHPage##uname(struct page *page) \ { clear_bit(HPG_##flname, &(page->private)); } + +#define TESTCLEARHPAGEFLAG(uname, flname) \ +static inline int TestClearHPage##uname(struct page *page) \ + { return test_and_clear_bit(HPG_##flname, &(page->private)); } #else #define TESTHPAGEFLAG(uname, flname) \ static inline bool \ @@ -635,6 +641,10 @@ folio_clear_hugetlb_##flname(struct folio *folio) \ { } \ static inline void ClearHPage##uname(struct page *page) \ { } + +#define TESTCLEARHPAGEFLAG(uname, flname) \ +static inline int TestClearHPage##uname(struct page *page) \ + { return 0; } #endif #define HPAGEFLAG(uname, flname) \ @@ -647,6 +657,7 @@ static inline void ClearHPage##uname(struct page *page) \ */ HPAGEFLAG(RestoreReserve, restore_reserve) HPAGEFLAG(Migratable, migratable) + TESTCLEARHPAGEFLAG(Migratable, migratable) HPAGEFLAG(Temporary, temporary) HPAGEFLAG(Freed, freed) HPAGEFLAG(VmemmapOptimized, vmemmap_optimized) diff --git a/include/linux/mm.h b/include/linux/mm.h index 58345f06a2f4..3da6283c9d30 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3246,9 +3246,11 @@ extern void shake_page(struct page *p); extern atomic_long_t num_poisoned_pages __read_mostly; extern int soft_offline_page(unsigned long pfn, int flags); #ifdef CONFIG_MEMORY_FAILURE -extern int __get_huge_page_for_hwpoison(unsigned long pfn, int flags); +extern int __get_huge_page_for_hwpoison(unsigned long pfn, int flags, + bool *migratable_cleared); #else -static inline int __get_huge_page_for_hwpoison(unsigned long pfn, int flags) +static inline int __get_huge_page_for_hwpoison(unsigned long pfn, int flags, + bool *migratable_cleared) { return 0; } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 931789a8f734..88d2dc756822 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -7292,7 +7292,7 @@ int isolate_hugetlb(struct page *page, struct list_head *list) return ret; } -int get_hwpoison_huge_page(struct page *page, bool *hugetlb) +int get_hwpoison_huge_page(struct page *page, bool *hugetlb, bool unpoison) { int ret = 0; @@ -7302,7 +7302,7 @@ int get_hwpoison_huge_page(struct page *page, bool *hugetlb) *hugetlb = true; if (HPageFreed(page)) ret = 0; - else if (HPageMigratable(page)) + else if (HPageMigratable(page) || unpoison) ret = get_page_unless_zero(page); else ret = -EBUSY; @@ -7311,12 +7311,13 @@ int get_hwpoison_huge_page(struct page *page, bool *hugetlb) return ret; } -int get_huge_page_for_hwpoison(unsigned long pfn, int flags) +int get_huge_page_for_hwpoison(unsigned long pfn, int flags, + bool *migratable_cleared) { int ret; spin_lock_irq(&hugetlb_lock); - ret = __get_huge_page_for_hwpoison(pfn, flags); + ret = __get_huge_page_for_hwpoison(pfn, flags, migratable_cleared); spin_unlock_irq(&hugetlb_lock); return ret; } diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 62cf1e0fbc8e..31589f2f5f07 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1250,7 +1250,7 @@ static int __get_hwpoison_page(struct page *page, unsigned long flags) int ret = 0; bool hugetlb = false; - ret = get_hwpoison_huge_page(head, &hugetlb); + ret = get_hwpoison_huge_page(head, &hugetlb, false); if (hugetlb) return ret; @@ -1340,7 +1340,7 @@ static int __get_unpoison_page(struct page *page) int ret = 0; bool hugetlb = false; - ret = get_hwpoison_huge_page(head, &hugetlb); + ret = get_hwpoison_huge_page(head, &hugetlb, true); if (hugetlb) return ret; @@ -1791,7 +1791,8 @@ void hugetlb_clear_page_hwpoison(struct page *hpage) * -EBUSY - the hugepage is busy (try to retry) * -EHWPOISON - the hugepage is already hwpoisoned */ -int __get_huge_page_for_hwpoison(unsigned long pfn, int flags) +int __get_huge_page_for_hwpoison(unsigned long pfn, int flags, + bool *migratable_cleared) { struct page *page = pfn_to_page(pfn); struct page *head = compound_head(page); @@ -1821,6 +1822,13 @@ int __get_huge_page_for_hwpoison(unsigned long pfn, int flags) goto out; } + /* + * Clearing HPageMigratable for hwpoisoned hugepages to prevent them + * from being migrated by memory hotremove. + */ + if (count_increased) + *migratable_cleared = TestClearHPageMigratable(head); + return ret; out: if (count_increased) @@ -1840,10 +1848,11 @@ static int try_memory_failure_hugetlb(unsigned long pfn, int flags, int *hugetlb struct page *p = pfn_to_page(pfn); struct page *head; unsigned long page_flags; + bool migratable_cleared = false; *hugetlb = 1; retry: - res = get_huge_page_for_hwpoison(pfn, flags); + res = get_huge_page_for_hwpoison(pfn, flags, &migratable_cleared); if (res == 2) { /* fallback to normal page handling */ *hugetlb = 0; return 0; @@ -1867,6 +1876,8 @@ static int try_memory_failure_hugetlb(unsigned long pfn, int flags, int *hugetlb if (hwpoison_filter(p)) { hugetlb_clear_page_hwpoison(head); + if (migratable_cleared) + SetHPageMigratable(head); unlock_page(head); if (res == 1) put_page(head); From patchwork Mon Oct 24 06:20:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 8171 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp280652wru; Sun, 23 Oct 2022 23:23:24 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6B+BwGE30mq7QgbjtujwxTdYnSeznnvn9YdEOxdPt9RceSGgzexvhU/G+oOkLq9hlmM/lt X-Received: by 2002:a05:6402:f1e:b0:461:cfd3:48c2 with SMTP id i30-20020a0564020f1e00b00461cfd348c2mr2345025eda.294.1666592604336; Sun, 23 Oct 2022 23:23:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666592604; cv=none; d=google.com; s=arc-20160816; b=RQLgqDoJoDiwJczAhGDAPT7LREaK0iuea33lyUp5Qd2FtoyoYP6DZ41MRBAtZIdzJo pXJ0edtLyTsrdV0TTcE0FeXqga8u8LQq7YnIU+l7L5hddS5xujakYmo+3eIeCxkmAeRR 17R10bzHbAqUXH7XOW6AHEolOcLIhVUUwgyubIKCzvAO6T7cVtRv2Dh/z9TafuNhSj/q fjobzHUqgClcBhwuEEZNCdAYUqNrnM5R8H5WYi5mjLriD6+WBto+/GWc0/FmWuFLS4/G YPZpzNoyl3cxpKkVcG7Umhs+EK3O7oITBqXrUg6NR5j4j0+JcCYyunmIl9fi2Zbk4126 jiLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Jyy+TtDTebM2ULRn5gfgfaG/rShZnN8sUyQZqnKLbrQ=; b=AlWMYEdQusu2CWxCXbdkX1+ygsDDaOdtzBonubur58DVNmwq+yGZHqD8yCqSLWNgNg 4OWQqGwjwvOLz4euptuTejBOrUKLIC8W9LNb0mewLwgk0rMEhAH52+V9SvMTcnLAoWFn oCUSznibtnE3kyhGcJz+eM72fvp+Uri2KwOSJT4QfadxEX/vRRWk6TWmSqJExu5Dckvf 7ugb61uvvKBt+7p5V+3oI89PXlepLzTV3oixI0RKV11hi6ZC0c9rcqK7VMaUNo0ZoWVS 2JIdSysPFvvwm+QCleEs31yRhl+MzCSzuRk3SuDbAp/koBnz91qQ7Bgz/OAsjp1yKlC/ L92Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=tHXsEaUy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z14-20020a05640240ce00b00461b8e2c7f4si3118035edb.548.2022.10.23.23.23.00; Sun, 23 Oct 2022 23:23:24 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=tHXsEaUy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230262AbiJXGUo (ORCPT + 99 others); Mon, 24 Oct 2022 02:20:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32990 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230209AbiJXGUj (ORCPT ); Mon, 24 Oct 2022 02:20:39 -0400 Received: from out0.migadu.com (out0.migadu.com [94.23.1.103]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CEE5446634 for ; Sun, 23 Oct 2022 23:20:36 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1666592434; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Jyy+TtDTebM2ULRn5gfgfaG/rShZnN8sUyQZqnKLbrQ=; b=tHXsEaUyTrQt+zgOBwT4prd/SfcMMeDbA7Qj+zM6EmkyEdN5SviEjUVNzFbLeGIeqcP9id RwiDKW0U/eF7fHi33f19lx+tic+gi3DLOlL/K++AW9m/W+GAU2Lg4t2bt1tBW5dt1Ik3Ia ylbhwZ4b8AHTL398aXbGSvd7Q9YRssQ= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , Miaohe Lin , David Hildenbrand , Mike Kravetz , Yang Shi , Oscar Salvador , Muchun Song , Jane Chu , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v7 2/4] mm/hwpoison: move definitions of num_poisoned_pages_* to memory-failure.c Date: Mon, 24 Oct 2022 15:20:10 +0900 Message-Id: <20221024062012.1520887-3-naoya.horiguchi@linux.dev> In-Reply-To: <20221024062012.1520887-1-naoya.horiguchi@linux.dev> References: <20221024062012.1520887-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_PASS, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747549006622595628?= X-GMAIL-MSGID: =?utf-8?q?1747549006622595628?= From: Naoya Horiguchi These interfaces will be used by drivers/base/memory.c by later patch, so as a preparatory work move them to more common header file visible to the file. Signed-off-by: Naoya Horiguchi Reviewed-by: Miaohe Lin --- ChangeLog v3 -> v6: - remove static in definition of num_poisoned_pages_inc() to fix build error. ChangeLog v2 -> v3: - added declaration of num_poisoned_pages_inc() in #ifdef CONFIG_MEMORY_FAILURE --- arch/parisc/kernel/pdt.c | 3 +-- include/linux/mm.h | 5 +++++ include/linux/swapops.h | 24 ++---------------------- mm/memory-failure.c | 10 ++++++++++ 4 files changed, 18 insertions(+), 24 deletions(-) diff --git a/arch/parisc/kernel/pdt.c b/arch/parisc/kernel/pdt.c index e391b175f5ec..fdc880e2575a 100644 --- a/arch/parisc/kernel/pdt.c +++ b/arch/parisc/kernel/pdt.c @@ -18,8 +18,7 @@ #include #include #include -#include -#include +#include #include #include diff --git a/include/linux/mm.h b/include/linux/mm.h index 3da6283c9d30..80d7c2987c3b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3248,12 +3248,17 @@ extern int soft_offline_page(unsigned long pfn, int flags); #ifdef CONFIG_MEMORY_FAILURE extern int __get_huge_page_for_hwpoison(unsigned long pfn, int flags, bool *migratable_cleared); +extern void num_poisoned_pages_inc(void); #else static inline int __get_huge_page_for_hwpoison(unsigned long pfn, int flags, bool *migratable_cleared) { return 0; } + +static inline void num_poisoned_pages_inc(void) +{ +} #endif #ifndef arch_memory_failure diff --git a/include/linux/swapops.h b/include/linux/swapops.h index a91dd08e107b..3e58a812399a 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -581,8 +581,6 @@ static inline int is_pmd_migration_entry(pmd_t pmd) #ifdef CONFIG_MEMORY_FAILURE -extern atomic_long_t num_poisoned_pages __read_mostly; - /* * Support for hardware poisoned pages */ @@ -610,17 +608,7 @@ static inline struct page *hwpoison_entry_to_page(swp_entry_t entry) return p; } -static inline void num_poisoned_pages_inc(void) -{ - atomic_long_inc(&num_poisoned_pages); -} - -static inline void num_poisoned_pages_sub(long i) -{ - atomic_long_sub(i, &num_poisoned_pages); -} - -#else /* CONFIG_MEMORY_FAILURE */ +#else static inline swp_entry_t make_hwpoison_entry(struct page *page) { @@ -636,15 +624,7 @@ static inline struct page *hwpoison_entry_to_page(swp_entry_t entry) { return NULL; } - -static inline void num_poisoned_pages_inc(void) -{ -} - -static inline void num_poisoned_pages_sub(long i) -{ -} -#endif /* CONFIG_MEMORY_FAILURE */ +#endif static inline int non_swap_entry(swp_entry_t entry) { diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 31589f2f5f07..3e0e20cac211 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -74,6 +74,16 @@ atomic_long_t num_poisoned_pages __read_mostly = ATOMIC_LONG_INIT(0); static bool hw_memory_failure __read_mostly = false; +inline void num_poisoned_pages_inc(void) +{ + atomic_long_inc(&num_poisoned_pages); +} + +static inline void num_poisoned_pages_sub(long i) +{ + atomic_long_sub(i, &num_poisoned_pages); +} + /* * Return values: * 1: the page is dissolved (if needed) and taken off from buddy, From patchwork Mon Oct 24 06:20:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 8170 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp280240wru; Sun, 23 Oct 2022 23:21:53 -0700 (PDT) X-Google-Smtp-Source: AMsMyM52fYAJxzfLGxUZRT6thMso/dUNf0nc6eH3G2RJ6lBFNqzvp2ARz9UV45WdQaJv4NQ4JvGN X-Received: by 2002:a17:906:db07:b0:77b:82cf:54af with SMTP id xj7-20020a170906db0700b0077b82cf54afmr27087535ejb.666.1666592513563; Sun, 23 Oct 2022 23:21:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666592513; cv=none; d=google.com; s=arc-20160816; b=P3EAGMyOQYcDtzoH0w+P0sMPNhrpRVg9Gu4JbwRIbbLEWPz4MvgyKNJlHUijIL/9rL 1qgfJtdFXAqucdW5/BXalQlnvFaPx+F8Ha4iIPL0lkpeGS6jJCrsTr66ixq5ukCVcEVo +kkXEtsx2rGEN+gvZeuRv053lg/cSB96CmLNdMJXRcnoAgnLNaM7i+127BKvnqZedZ3W hL2izB3catllj0s88f7Uor4K9eBei1JucJF+rKn62uT2Zn1lP3lL7IlayEHS8vtZKsyv WOOFopdsmBpaR0w9d64YocSK2uGeM/C0q6acWJzAm5DpxnQi8FM9Xfh0gzqByAguotEq k0Lw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Fdcd5K2Zm1oroc0cuGQtvJdbZtbqPLVuISUqhZii2k4=; b=mibUfsdxiEGhEpWGRHw95ypfaxtgaEfp4Zg4wKFWj+eWmI+mWgyBn2fgOwj6XU8Mb+ A68bv8F73leJTNxONmr+L5HRCcpeLLInr4b4IxXD9qGN8FhnfAhQipQVAhsHTP+7EPnh lru/ZtLE54Q2UKihKJ5MzbTKgkLrr/lTbm8VDfhyfuDoIZHb/l82ZxmL7XAJU9f+yf1Z uTcbQFMlaJY+sZFJ98j5YG7sST9+SHy7+ollaBXHAmFztwGN7ex+M4r3VP2Lo7UJVxn5 Og9e0+8xb8MNUGP3DbRwXsJ/Buj58W3MKk/m72jVH+aYHSLDK7zjmX+2ZuQOrl5UZ73v 0NFQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=c8loFVBO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id dn20-20020a17090794d400b007123952b00dsi27124889ejc.100.2022.10.23.23.21.30; Sun, 23 Oct 2022 23:21:53 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=c8loFVBO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230331AbiJXGUy (ORCPT + 99 others); Mon, 24 Oct 2022 02:20:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33250 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230230AbiJXGUn (ORCPT ); Mon, 24 Oct 2022 02:20:43 -0400 Received: from out0.migadu.com (out0.migadu.com [IPv6:2001:41d0:2:267::]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9713E48A0D for ; Sun, 23 Oct 2022 23:20:39 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1666592438; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Fdcd5K2Zm1oroc0cuGQtvJdbZtbqPLVuISUqhZii2k4=; b=c8loFVBOLyhb5hGT+4t/Yooug5C55tinrhnDsKGqIIIM4vYSOPcNMx5zi4B4DrVEiBrqiH vfmHhbA87DyOaEgpLbL9zzb+IZw/l9gNwipzBNu2LzZVUnPYzzQmE8JkDx1KpFkwQdUWLd O7G/zsS1barp3U72MMcvBgOjD8i0ns4= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , Miaohe Lin , David Hildenbrand , Mike Kravetz , Yang Shi , Oscar Salvador , Muchun Song , Jane Chu , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v7 3/4] mm/hwpoison: pass pfn to num_poisoned_pages_*() Date: Mon, 24 Oct 2022 15:20:11 +0900 Message-Id: <20221024062012.1520887-4-naoya.horiguchi@linux.dev> In-Reply-To: <20221024062012.1520887-1-naoya.horiguchi@linux.dev> References: <20221024062012.1520887-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_PASS, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747548911807438770?= X-GMAIL-MSGID: =?utf-8?q?1747548911807438770?= From: Naoya Horiguchi No functional change. Signed-off-by: Naoya Horiguchi Reviewed-by: Miaohe Lin --- ChangeLog v2 -> v3: - added declaration of num_poisoned_pages_inc() in #ifdef CONFIG_MEMORY_FAILURE --- arch/parisc/kernel/pdt.c | 2 +- include/linux/mm.h | 4 ++-- mm/memory-failure.c | 14 +++++++------- 3 files changed, 10 insertions(+), 10 deletions(-) diff --git a/arch/parisc/kernel/pdt.c b/arch/parisc/kernel/pdt.c index fdc880e2575a..80943a00e245 100644 --- a/arch/parisc/kernel/pdt.c +++ b/arch/parisc/kernel/pdt.c @@ -231,7 +231,7 @@ void __init pdc_pdt_init(void) /* mark memory page bad */ memblock_reserve(pdt_entry[i] & PAGE_MASK, PAGE_SIZE); - num_poisoned_pages_inc(); + num_poisoned_pages_inc(addr >> PAGE_SHIFT); } } diff --git a/include/linux/mm.h b/include/linux/mm.h index 80d7c2987c3b..278e24a0e3d3 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3248,7 +3248,7 @@ extern int soft_offline_page(unsigned long pfn, int flags); #ifdef CONFIG_MEMORY_FAILURE extern int __get_huge_page_for_hwpoison(unsigned long pfn, int flags, bool *migratable_cleared); -extern void num_poisoned_pages_inc(void); +extern void num_poisoned_pages_inc(unsigned long pfn); #else static inline int __get_huge_page_for_hwpoison(unsigned long pfn, int flags, bool *migratable_cleared) @@ -3256,7 +3256,7 @@ static inline int __get_huge_page_for_hwpoison(unsigned long pfn, int flags, return 0; } -static inline void num_poisoned_pages_inc(void) +static inline void num_poisoned_pages_inc(unsigned long pfn) { } #endif diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 3e0e20cac211..527ee0867742 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -74,12 +74,12 @@ atomic_long_t num_poisoned_pages __read_mostly = ATOMIC_LONG_INIT(0); static bool hw_memory_failure __read_mostly = false; -inline void num_poisoned_pages_inc(void) +inline void num_poisoned_pages_inc(unsigned long pfn) { atomic_long_inc(&num_poisoned_pages); } -static inline void num_poisoned_pages_sub(long i) +static inline void num_poisoned_pages_sub(unsigned long pfn, long i) { atomic_long_sub(i, &num_poisoned_pages); } @@ -125,7 +125,7 @@ static bool page_handle_poison(struct page *page, bool hugepage_or_freepage, boo if (release) put_page(page); page_ref_inc(page); - num_poisoned_pages_inc(); + num_poisoned_pages_inc(page_to_pfn(page)); return true; } @@ -1198,7 +1198,7 @@ static int action_result(unsigned long pfn, enum mf_action_page_type type, { trace_memory_failure_event(pfn, type, result); - num_poisoned_pages_inc(); + num_poisoned_pages_inc(pfn); pr_err("%#lx: recovery action for %s: %s\n", pfn, action_page_types[type], action_name[result]); @@ -1747,7 +1747,7 @@ static int hugetlb_set_page_hwpoison(struct page *hpage, struct page *page) llist_add(&raw_hwp->node, head); /* the first error event will be counted in action_result(). */ if (ret) - num_poisoned_pages_inc(); + num_poisoned_pages_inc(page_to_pfn(page)); } else { /* * Failed to save raw error info. We no longer trace all @@ -2421,7 +2421,7 @@ int unpoison_memory(unsigned long pfn) unlock_mutex: mutex_unlock(&mf_mutex); if (!ret || freeit) { - num_poisoned_pages_sub(count); + num_poisoned_pages_sub(pfn, count); unpoison_pr_info("Unpoison: Software-unpoisoned page %#lx\n", page_to_pfn(p), &unpoison_rs); } @@ -2637,5 +2637,5 @@ void clear_hwpoisoned_pages(struct page *memmap, int nr_pages) } } if (total) - num_poisoned_pages_sub(total); + num_poisoned_pages_sub(0, total); } From patchwork Mon Oct 24 06:20:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naoya Horiguchi X-Patchwork-Id: 8172 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp283987wru; Sun, 23 Oct 2022 23:34:56 -0700 (PDT) X-Google-Smtp-Source: AMsMyM60HEpTyX7QzUmQ2JEtEzWmm4jW0NFtSCKoj+R5KajTBeo1fCRqxz+G3ZHKyRUu8LOxfBQb X-Received: by 2002:a05:6402:50c:b0:461:bc01:1828 with SMTP id m12-20020a056402050c00b00461bc011828mr4307940edv.64.1666593296511; Sun, 23 Oct 2022 23:34:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666593296; cv=none; d=google.com; s=arc-20160816; b=aFBYhc5ANQ2rRKtS+fQ6sZKx/vpFUcQSo2QEfZA+cWoC0sELbZn22fWbh8WRWtmtA1 4TPgRZ8gbppRb5As1ztlnFFEl3i+U8lRRDLKnXpD/7oefggrL82UVh6OKxeFoQjaM+cl VlBphNjrtiAZTIVl55aHz6ROwBZumowWb/UFdJVNEDKENSU03wawcwgXxRHm/s6XysSo mXA1Q63Xne16NnsJF7j+TxhRD5UYh2PPm89VmShlx0kUMCcywx60c1rkwSKamJYaEh30 Ad2SBCcBAxtv+AOKEQ02+vQioXQg4n16vZod3vajzFqofI52B5NNAlz0hbUbiPITpsRp mzzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=azeIBq1aHvQQIRT2KHpv+IEd8FygdW8OOvfeZOl0uDQ=; b=YypTDUik4VEyE7yTPLuxNn9Gn2zjZBGEQu0Z+hyqVirU1E1KeqtwWUDw4soU9CMWjo vLC+io5ipzeDq14CANqtVrw+L91JIz2nZ/yChwsoEdbxKti6wTGAV+5N8dq0eYf0Dl+w SBfmpGRDnaIO55/X0tll7N9uO6UuoMzVwYm0GEPWoWnESTkhY8cXf++y2xL5XhDTxZNO woqjVAQncXm/c+AukfD9SAu6IqkVZqSf8zBR2naTEQ1g5zsNsBUUOY067VJmHt5wV3oz GBLxH3at0lC1vL6QgCYa1k1BbGeI3ns8DN9K5PiriMHB6t+WGdh5nqtr7iBwxERtQVO7 u9+g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=ZExR2aZX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p7-20020a170906498700b0073d8659db5csi24108414eju.966.2022.10.23.23.34.28; Sun, 23 Oct 2022 23:34:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=ZExR2aZX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230250AbiJXGU5 (ORCPT + 99 others); Mon, 24 Oct 2022 02:20:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33988 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230320AbiJXGUw (ORCPT ); Mon, 24 Oct 2022 02:20:52 -0400 Received: from out0.migadu.com (out0.migadu.com [94.23.1.103]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5FC314A138 for ; Sun, 23 Oct 2022 23:20:43 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1666592442; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=azeIBq1aHvQQIRT2KHpv+IEd8FygdW8OOvfeZOl0uDQ=; b=ZExR2aZXubcE08RKgeDxd6X5g23Ls5fgkhqMtWzXbmOF5UZbZePMqXKRmVDQd9EdpKQPdd wjet4XQfEj6OcV0k4e2xJiFASsyPsFPWsTGwn4+x8qw5trjoMPwtF5/T2EO6gAjhU/CeuE Yaw9cyR8nKg8iXVE37v0HedR5zoV2y8= From: Naoya Horiguchi To: linux-mm@kvack.org Cc: Andrew Morton , Miaohe Lin , David Hildenbrand , Mike Kravetz , Yang Shi , Oscar Salvador , Muchun Song , Jane Chu , Naoya Horiguchi , linux-kernel@vger.kernel.org Subject: [PATCH v7 4/4] mm/hwpoison: introduce per-memory_block hwpoison counter Date: Mon, 24 Oct 2022 15:20:12 +0900 Message-Id: <20221024062012.1520887-5-naoya.horiguchi@linux.dev> In-Reply-To: <20221024062012.1520887-1-naoya.horiguchi@linux.dev> References: <20221024062012.1520887-1-naoya.horiguchi@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_PASS, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747549732449540585?= X-GMAIL-MSGID: =?utf-8?q?1747549732449540585?= From: Naoya Horiguchi Currently PageHWPoison flag does not behave well when experiencing memory hotremove/hotplug. Any data field in struct page is unreliable when the associated memory is offlined, and the current mechanism can't tell whether a memory block is onlined because a new memory devices is installed or because previous failed offline operations are undone. Especially if there's a hwpoisoned memory, it's unclear what the best option is. So introduce a new mechanism to make struct memory_block remember that a memory block has hwpoisoned memory inside it. And make any online event fail if the onlining memory block contains hwpoison. struct memory_block is freed and reallocated over ACPI-based hotremove/hotplug, but not over sysfs-based hotremove/hotplug. So the new counter can distinguish these cases. Signed-off-by: Naoya Horiguchi Reported-by: kernel test robot Reviewed-by: Miaohe Lin --- ChangeLog v6 -> v7: - remove 'extern' in declarations of num_poisoned_pages_{inc,sub} because they are not necessary and definition have 'inline'. - remove unneeded declaration of memblk_nr_poison_{inc,sub}. ChangeLog v5 -> v6: - fix build errors over memblk_nr_poison_inc() and memblk_nr_poison_sub(), - pass "struct memory_block *" to memblk_nr_poison() instead of pfn, - removed clear_hwpoisoned_pages() and call num_poisoned_pages_sub() directly. - add static keyword to the definition of memblk_nr_poison(). - Mioahe added Reviewed-by for v5, but I have some non trivial changes in v6, so let me hold to add it. - unpoison_memory() properly cancels per-memblk hwpoison counter. ChangeLog v4 -> v5: - add Reported-by of lkp bot, - check both CONFIG_MEMORY_FAILURE and CONFIG_MEMORY_HOTPLUG in introduced #ifdefs, intending to fix "undefined reference" errors in aarch64. ChangeLog v3 -> v4: - fix build error (https://lore.kernel.org/linux-mm/202209231134.tnhKHRfg-lkp@intel.com/) by using memblk_nr_poison() to access to the member ->nr_hwpoison --- drivers/base/memory.c | 38 ++++++++++++++++++++++++++++++++++++++ include/linux/memory.h | 3 +++ include/linux/mm.h | 20 +++++++++++++++++++- mm/internal.h | 8 -------- mm/memory-failure.c | 36 +++++++++++------------------------- mm/sparse.c | 2 -- 6 files changed, 71 insertions(+), 36 deletions(-) diff --git a/drivers/base/memory.c b/drivers/base/memory.c index 9aa0da991cfb..fe98fb8d94e5 100644 --- a/drivers/base/memory.c +++ b/drivers/base/memory.c @@ -175,6 +175,15 @@ int memory_notify(unsigned long val, void *v) return blocking_notifier_call_chain(&memory_chain, val, v); } +#if defined(CONFIG_MEMORY_FAILURE) && defined(CONFIG_MEMORY_HOTPLUG) +static unsigned long memblk_nr_poison(struct memory_block *mem); +#else +static inline unsigned long memblk_nr_poison(struct memory_block *mem) +{ + return 0; +} +#endif + static int memory_block_online(struct memory_block *mem) { unsigned long start_pfn = section_nr_to_pfn(mem->start_section_nr); @@ -183,6 +192,9 @@ static int memory_block_online(struct memory_block *mem) struct zone *zone; int ret; + if (memblk_nr_poison(mem)) + return -EHWPOISON; + zone = zone_for_pfn_range(mem->online_type, mem->nid, mem->group, start_pfn, nr_pages); @@ -864,6 +876,7 @@ void remove_memory_block_devices(unsigned long start, unsigned long size) mem = find_memory_block_by_id(block_id); if (WARN_ON_ONCE(!mem)) continue; + num_poisoned_pages_sub(-1UL, memblk_nr_poison(mem)); unregister_memory_block_under_nodes(mem); remove_memory_block(mem); } @@ -1164,3 +1177,28 @@ int walk_dynamic_memory_groups(int nid, walk_memory_groups_func_t func, } return ret; } + +#if defined(CONFIG_MEMORY_FAILURE) && defined(CONFIG_MEMORY_HOTPLUG) +void memblk_nr_poison_inc(unsigned long pfn) +{ + const unsigned long block_id = pfn_to_block_id(pfn); + struct memory_block *mem = find_memory_block_by_id(block_id); + + if (mem) + atomic_long_inc(&mem->nr_hwpoison); +} + +void memblk_nr_poison_sub(unsigned long pfn, long i) +{ + const unsigned long block_id = pfn_to_block_id(pfn); + struct memory_block *mem = find_memory_block_by_id(block_id); + + if (mem) + atomic_long_sub(i, &mem->nr_hwpoison); +} + +static unsigned long memblk_nr_poison(struct memory_block *mem) +{ + return atomic_long_read(&mem->nr_hwpoison); +} +#endif diff --git a/include/linux/memory.h b/include/linux/memory.h index 463662ef7614..31343566c221 100644 --- a/include/linux/memory.h +++ b/include/linux/memory.h @@ -84,6 +84,9 @@ struct memory_block { unsigned long nr_vmemmap_pages; struct memory_group *group; /* group (if any) for this block */ struct list_head group_next; /* next block inside memory group */ +#if defined(CONFIG_MEMORY_FAILURE) && defined(CONFIG_MEMORY_HOTPLUG) + atomic_long_t nr_hwpoison; +#endif }; int arch_get_memory_phys_device(unsigned long start_pfn); diff --git a/include/linux/mm.h b/include/linux/mm.h index 278e24a0e3d3..816f566847e4 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3248,7 +3248,8 @@ extern int soft_offline_page(unsigned long pfn, int flags); #ifdef CONFIG_MEMORY_FAILURE extern int __get_huge_page_for_hwpoison(unsigned long pfn, int flags, bool *migratable_cleared); -extern void num_poisoned_pages_inc(unsigned long pfn); +void num_poisoned_pages_inc(unsigned long pfn); +void num_poisoned_pages_sub(unsigned long pfn, long i); #else static inline int __get_huge_page_for_hwpoison(unsigned long pfn, int flags, bool *migratable_cleared) @@ -3259,6 +3260,23 @@ static inline int __get_huge_page_for_hwpoison(unsigned long pfn, int flags, static inline void num_poisoned_pages_inc(unsigned long pfn) { } + +static inline void num_poisoned_pages_sub(unsigned long pfn, long i) +{ +} +#endif + +#if defined(CONFIG_MEMORY_FAILURE) && defined(CONFIG_MEMORY_HOTPLUG) +extern void memblk_nr_poison_inc(unsigned long pfn); +extern void memblk_nr_poison_sub(unsigned long pfn, long i); +#else +static inline void memblk_nr_poison_inc(unsigned long pfn) +{ +} + +static inline void memblk_nr_poison_sub(unsigned long pfn, long i) +{ +} #endif #ifndef arch_memory_failure diff --git a/mm/internal.h b/mm/internal.h index 4b44ced87fff..cb4c663a714e 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -708,14 +708,6 @@ extern u64 hwpoison_filter_flags_value; extern u64 hwpoison_filter_memcg; extern u32 hwpoison_filter_enable; -#ifdef CONFIG_MEMORY_FAILURE -void clear_hwpoisoned_pages(struct page *memmap, int nr_pages); -#else -static inline void clear_hwpoisoned_pages(struct page *memmap, int nr_pages) -{ -} -#endif - extern unsigned long __must_check vm_mmap_pgoff(struct file *, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long); diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 527ee0867742..3389dce6966e 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -77,11 +77,14 @@ static bool hw_memory_failure __read_mostly = false; inline void num_poisoned_pages_inc(unsigned long pfn) { atomic_long_inc(&num_poisoned_pages); + memblk_nr_poison_inc(pfn); } -static inline void num_poisoned_pages_sub(unsigned long pfn, long i) +inline void num_poisoned_pages_sub(unsigned long pfn, long i) { atomic_long_sub(i, &num_poisoned_pages); + if (pfn != -1UL) + memblk_nr_poison_sub(pfn, i); } /* @@ -1712,6 +1715,8 @@ static unsigned long __free_raw_hwp_pages(struct page *hpage, bool move_flag) if (move_flag) SetPageHWPoison(p->page); + else + num_poisoned_pages_sub(page_to_pfn(p->page), 1); kfree(p); count++; } @@ -2339,6 +2344,7 @@ int unpoison_memory(unsigned long pfn) int ret = -EBUSY; int freeit = 0; unsigned long count = 1; + bool huge = false; static DEFINE_RATELIMIT_STATE(unpoison_rs, DEFAULT_RATELIMIT_INTERVAL, DEFAULT_RATELIMIT_BURST); @@ -2387,6 +2393,7 @@ int unpoison_memory(unsigned long pfn) ret = get_hwpoison_page(p, MF_UNPOISON); if (!ret) { if (PageHuge(p)) { + huge = true; count = free_raw_hwp_pages(page, false); if (count == 0) { ret = -EBUSY; @@ -2402,6 +2409,7 @@ int unpoison_memory(unsigned long pfn) pfn, &unpoison_rs); } else { if (PageHuge(p)) { + huge = true; count = free_raw_hwp_pages(page, false); if (count == 0) { ret = -EBUSY; @@ -2421,7 +2429,8 @@ int unpoison_memory(unsigned long pfn) unlock_mutex: mutex_unlock(&mf_mutex); if (!ret || freeit) { - num_poisoned_pages_sub(pfn, count); + if (!huge) + num_poisoned_pages_sub(pfn, 1); unpoison_pr_info("Unpoison: Software-unpoisoned page %#lx\n", page_to_pfn(p), &unpoison_rs); } @@ -2616,26 +2625,3 @@ int soft_offline_page(unsigned long pfn, int flags) return ret; } - -void clear_hwpoisoned_pages(struct page *memmap, int nr_pages) -{ - int i, total = 0; - - /* - * A further optimization is to have per section refcounted - * num_poisoned_pages. But that would need more space per memmap, so - * for now just do a quick global check to speed up this routine in the - * absence of bad pages. - */ - if (atomic_long_read(&num_poisoned_pages) == 0) - return; - - for (i = 0; i < nr_pages; i++) { - if (PageHWPoison(&memmap[i])) { - total++; - ClearPageHWPoison(&memmap[i]); - } - } - if (total) - num_poisoned_pages_sub(0, total); -} diff --git a/mm/sparse.c b/mm/sparse.c index e5a8a3a0edd7..2779b419ef2a 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -926,8 +926,6 @@ void sparse_remove_section(struct mem_section *ms, unsigned long pfn, unsigned long nr_pages, unsigned long map_offset, struct vmem_altmap *altmap) { - clear_hwpoisoned_pages(pfn_to_page(pfn) + map_offset, - nr_pages - map_offset); section_deactivate(pfn, nr_pages, altmap); } #endif /* CONFIG_MEMORY_HOTPLUG */