From patchwork Fri Oct 21 20:01:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Luck, Tony" X-Patchwork-Id: 6960 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp882120wrr; Fri, 21 Oct 2022 13:09:19 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5oAc53O5M+unp8IhvojbN+e2W53/TpbhDf9nZPo1s3HXapRv0pm0vp/JBD6lKojKCIK5ET X-Received: by 2002:a05:6402:528a:b0:454:8613:6560 with SMTP id en10-20020a056402528a00b0045486136560mr19817572edb.252.1666382959293; Fri, 21 Oct 2022 13:09:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666382959; cv=none; d=google.com; s=arc-20160816; b=o0woHIhgJPExuR86Yh1EHOjsYOmlf7KESLASFsfFiCXtZhqbqQO4X/2Pxm64fSYsrz OSqb33h5zdgEXdZHWHmy8b1K+XG8BCBaiycHmUQgNvZJcwFtOMzU8rQ6/Z0GQ0fhsebN dpq0g3352RZtqGwOv5mQ2DchTBi5A1x5/nMGOFnObZWK2lgcYnZZYlM3fzML+bmH8WPs XTOkooUs+1DS9Pq0y3/EPZ6YMLc65d9uvdHMGmyDmS1S9/bVv6nTX8htwrRQGwM8XCGi OH6vNKgWMLPWFCDRxciFS7WDEMg0oNlAqOMeIXI0K5gu7X1E4uwDaPNPu/sjnHEyWmsh z6Jg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=C8HsyIbzksL7xddJtKJ3gj2MC9GwhhXQUiSjyDvYCvw=; b=hftxIcSZtBoaYwHXIgBftTxV+jPr6UkA0u3xNoVv/udiIYLN1Xt77g2eFqQafP5Wjv SWZ8b4YS7VChXFECoJqiE/lx4sviQ4veDNWn+vECXyjlroX5zjAg8xPxjMIPhlZ+06Mp XyPwuCftEtEU9a9W1HpdkW9nEDkuLITCKFXCkO6aN74AHChjd1XWEg6H/afaqnBWJde8 j+OHL12qjIv08HvfRoiMwVD/Q2nuf3X3jcydRqX3+VUedweo2tAfX2/tK94ATF3sDDHR yPuCslSDy+Q4VxIsqKpo2jgI85+bG0FMfdfzVxNzWEvHR2cUd0COe7I+0xeyhbbD9zoy nWRA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=J2IcpiDQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id sb14-20020a1709076d8e00b007820aa60dc3si22095439ejc.74.2022.10.21.13.08.52; Fri, 21 Oct 2022 13:09:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=J2IcpiDQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230169AbiJUUBk (ORCPT + 99 others); Fri, 21 Oct 2022 16:01:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47072 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229897AbiJUUBa (ORCPT ); Fri, 21 Oct 2022 16:01:30 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CFD3125E88D for ; Fri, 21 Oct 2022 13:01:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666382489; x=1697918489; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KaF9e0WVFrqnnol9CO30O+x7LQt9U5cOhvA5BN7/yRs=; b=J2IcpiDQ1b2GhBT6r13xInTu+9MtO3hQ5vZmuWCSUUCrWeUHzJ/F/tyZ ufmzAUUyamv9wjsytXy0K4nZeLwmyml/R6NUULsU8JfrE87+RZ369OzQf 7cFwP3JG5n80sv+UQ9b1/Tz2DmFY/5WhAgvXCZyq2K42TCg70dYiAQboz u5xR3DQ15ufMMjMO8LenHTbyTViqheJmCNwpQ/QoG6N6OtjeWEPAujzSC 5DBYr13Wh8kbtDWW01nwBZloomcaVuzQBe/hA0qXuXNZqwSax/HUozylE ontfT8MhV4T7pBtfPV0z0H2NlE4r3jRmmsg9UY2BxBd4YFhaMa5qwOJOS Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10507"; a="290401088" X-IronPort-AV: E=Sophos;i="5.95,203,1661842800"; d="scan'208";a="290401088" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2022 13:01:28 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10507"; a="633069088" X-IronPort-AV: E=Sophos;i="5.95,203,1661842800"; d="scan'208";a="633069088" Received: from agluck-desk3.sc.intel.com ([172.25.222.78]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2022 13:01:27 -0700 From: Tony Luck To: Naoya Horiguchi , Andrew Morton Cc: Miaohe Lin , Matthew Wilcox , Shuai Xue , Dan Williams , Michael Ellerman , Nicholas Piggin , Christophe Leroy , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, Tony Luck Subject: [PATCH v3 1/2] mm, hwpoison: Try to recover from copy-on write faults Date: Fri, 21 Oct 2022 13:01:19 -0700 Message-Id: <20221021200120.175753-2-tony.luck@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221021200120.175753-1-tony.luck@intel.com> References: <20221019170835.155381-1-tony.luck@intel.com> <20221021200120.175753-1-tony.luck@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_NONE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747137077376497017?= X-GMAIL-MSGID: =?utf-8?q?1747329178217471816?= If the kernel is copying a page as the result of a copy-on-write fault and runs into an uncorrectable error, Linux will crash because it does not have recovery code for this case where poison is consumed by the kernel. It is easy to set up a test case. Just inject an error into a private page, fork(2), and have the child process write to the page. I wrapped that neatly into a test at: git://git.kernel.org/pub/scm/linux/kernel/git/aegl/ras-tools.git just enable ACPI error injection and run: # ./einj_mem-uc -f copy-on-write Add a new copy_user_highpage_mc() function that uses copy_mc_to_kernel() on architectures where that is available (currently x86 and powerpc). When an error is detected during the page copy, return VM_FAULT_HWPOISON to caller of wp_page_copy(). This propagates up the call stack. Both x86 and powerpc have code in their fault handler to deal with this code by sending a SIGBUS to the application. Note that this patch avoids a system crash and signals the process that triggered the copy-on-write action. It does not take any action for the memory error that is still in the shared page. To handle that a call to memory_failure() is needed. But this cannot be done from wp_page_copy() because it holds mmap_lock(). Perhaps the architecture fault handlers can deal with this loose end in a subsequent patch? On Intel/x86 this loose end will often be handled automatically because the memory controller provides an additional notification of the h/w poison in memory, the handler for this will call memory_failure(). This isn't a 100% solution. If there are multiple errors, not all may be logged in this way. Reviewed-by: Dan Williams Signed-off-by: Tony Luck Reviewed-by: Naoya Horiguchi Reviewed-by: Alexander Potapenko --- Changes in V3: Dan Williams Rename copy_user_highpage_mc() to copy_mc_user_highpage() for consistency with Linus' discussion on names of functions that check for machine check. Write complete functions for the have/have-not copy_mc_to_kernel cases (so grep shows there are two versions) Change __wp_page_copy_user() to return 0 for success, negative for fail [I picked -EAGAIN for both non-EHWPOISON cases] Changes in V2: Naoya Horiguchi: 1) Use -EHWPOISON error code instead of minus one. 2) Poison path needs also to deal with old_page Tony Luck: Rewrote commit message Added some powerpc folks to Cc: list --- include/linux/highmem.h | 24 ++++++++++++++++++++++++ mm/memory.c | 30 ++++++++++++++++++++---------- 2 files changed, 44 insertions(+), 10 deletions(-) diff --git a/include/linux/highmem.h b/include/linux/highmem.h index e9912da5441b..a32c64681f03 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -319,6 +319,30 @@ static inline void copy_user_highpage(struct page *to, struct page *from, #endif +#ifdef copy_mc_to_kernel +static inline int copy_mc_user_highpage(struct page *to, struct page *from, + unsigned long vaddr, struct vm_area_struct *vma) +{ + unsigned long ret; + char *vfrom, *vto; + + vfrom = kmap_local_page(from); + vto = kmap_local_page(to); + ret = copy_mc_to_kernel(vto, vfrom, PAGE_SIZE); + kunmap_local(vto); + kunmap_local(vfrom); + + return ret; +} +#else +static inline int copy_mc_user_highpage(struct page *to, struct page *from, + unsigned long vaddr, struct vm_area_struct *vma) +{ + copy_user_highpage(to, from, vaddr, vma); + return 0; +} +#endif + #ifndef __HAVE_ARCH_COPY_HIGHPAGE static inline void copy_highpage(struct page *to, struct page *from) diff --git a/mm/memory.c b/mm/memory.c index f88c351aecd4..b6056eef2f72 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2848,10 +2848,16 @@ static inline int pte_unmap_same(struct vm_fault *vmf) return same; } -static inline bool __wp_page_copy_user(struct page *dst, struct page *src, - struct vm_fault *vmf) +/* + * Return: + * 0: copied succeeded + * -EHWPOISON: copy failed due to hwpoison in source page + * -EAGAIN: copied failed (some other reason) + */ +static inline int __wp_page_copy_user(struct page *dst, struct page *src, + struct vm_fault *vmf) { - bool ret; + int ret; void *kaddr; void __user *uaddr; bool locked = false; @@ -2860,8 +2866,9 @@ static inline bool __wp_page_copy_user(struct page *dst, struct page *src, unsigned long addr = vmf->address; if (likely(src)) { - copy_user_highpage(dst, src, addr, vma); - return true; + if (copy_mc_user_highpage(dst, src, addr, vma)) + return -EHWPOISON; + return 0; } /* @@ -2888,7 +2895,7 @@ static inline bool __wp_page_copy_user(struct page *dst, struct page *src, * and update local tlb only */ update_mmu_tlb(vma, addr, vmf->pte); - ret = false; + ret = -EAGAIN; goto pte_unlock; } @@ -2913,7 +2920,7 @@ static inline bool __wp_page_copy_user(struct page *dst, struct page *src, if (!likely(pte_same(*vmf->pte, vmf->orig_pte))) { /* The PTE changed under us, update local tlb */ update_mmu_tlb(vma, addr, vmf->pte); - ret = false; + ret = -EAGAIN; goto pte_unlock; } @@ -2932,7 +2939,7 @@ static inline bool __wp_page_copy_user(struct page *dst, struct page *src, } } - ret = true; + ret = 0; pte_unlock: if (locked) @@ -3104,6 +3111,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) pte_t entry; int page_copied = 0; struct mmu_notifier_range range; + int ret; delayacct_wpcopy_start(); @@ -3121,19 +3129,21 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) if (!new_page) goto oom; - if (!__wp_page_copy_user(new_page, old_page, vmf)) { + ret = __wp_page_copy_user(new_page, old_page, vmf); + if (ret) { /* * COW failed, if the fault was solved by other, * it's fine. If not, userspace would re-fault on * the same address and we will handle the fault * from the second attempt. + * The -EHWPOISON case will not be retried. */ put_page(new_page); if (old_page) put_page(old_page); delayacct_wpcopy_end(); - return 0; + return ret == -EHWPOISON ? VM_FAULT_HWPOISON : 0; } kmsan_copy_page_meta(new_page, old_page); } From patchwork Fri Oct 21 20:01:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Luck, Tony" X-Patchwork-Id: 6959 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4242:0:0:0:0:0 with SMTP id s2csp881668wrr; Fri, 21 Oct 2022 13:08:00 -0700 (PDT) X-Google-Smtp-Source: AMsMyM42G8tP+HtjlJ/80KCbyj/EfN1Rlivw+W0C13V37iXKJQhpfiuPLqaMJQGTpkG9u3+UOZ6y X-Received: by 2002:a17:907:62a1:b0:781:b320:90c0 with SMTP id nd33-20020a17090762a100b00781b32090c0mr16012938ejc.255.1666382880655; Fri, 21 Oct 2022 13:08:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666382880; cv=none; d=google.com; s=arc-20160816; b=FKj3ver8+nK97gVucnV6tApw7sBq57Uq1EuUnJyLMV7t5P2CtNvee82idGTYNClzbK +nUT0AlVXzUHdETLfWyb7ozng7XHw0+g2pahgJnoACKT1WHbG/ynd9aGKBJbI2cPBsUa 46XHxy5M2FMl+kNemJLDgZltosR6Eajh6RIyNWkMC13lP1P5rvsfpl5gKLmJ61CfuFY2 aJ1ClaEgayLVHyM37GxI7L3re/OdAS8nenTsHVhC1VYiNLq621YdMgVuB67L4CiVmXv6 ARM1q3lwuv4kFF2rpfK2a/6KvK6E99sz03SJuIdFhxEMQCsSCrs8KrVuCDMWcx+7kbGL 0S9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=DtzjFiVfsz6uRTaunxk9mK5vEo7zwX7ecg9qpP3wQuw=; b=YIXImSGH/L7qvd1FWuwscSVpUbiYdOsYV+4rWiKWIws+iIJGaxZotjFS/EPZ30UCqS bsmeyrioiSpLUmHxyT5HtUQgfhxEH8BUhm99xdeI3/qoV4cAFOy+PKdTX3FfiTsCWi6b taOj2RiTe2IEw2v70O11OCDSFmWRhPRU91BuTlauZ90oy9UDvMmGd96oJfdxzm3V1dCi LU7EmTpK26blN99tei+aoQMwlMVz7DtFbNrwaaVldRQB5GDSqFAAeLlWvhr8usGz2lmJ BJBF1P7wRC7zuJkCDEdYG1MnzgZKaKnsNqI5g0thB5HP56XOUYt7V7EIBYB0OzPg0MDi 8xOA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=WcBHh3Hc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id d10-20020a50cd4a000000b0046111befa1csi4823209edj.452.2022.10.21.13.07.35; Fri, 21 Oct 2022 13:08:00 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=WcBHh3Hc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230024AbiJUUBg (ORCPT + 99 others); Fri, 21 Oct 2022 16:01:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229872AbiJUUBa (ORCPT ); Fri, 21 Oct 2022 16:01:30 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0F60B25E891 for ; Fri, 21 Oct 2022 13:01:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666382490; x=1697918490; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=V6m+YOfy5sxTfMqFs+Sj5pHdQRZvXRPoJwkK0Nyet44=; b=WcBHh3Hct0txAAqvbYNLgbiUbR3F1xFiGvVtyJPdniUKRz4xcQdbKR1g tZveZV78S3XyYLhXPVrlJrwlOESJ0yMdZYXnPn7sjp2CJjTt8QNox5KcB A6ybOnZ38917SRiop7MtVLLI13vh0j1r7+0UwTXovdh4tUN9MXKFQzxkM 0hJBGMwxAI6t8ln4lD0YjDjuWas9YSrZPweuU9L9jNs4/zBYLsaZ8KXnh VI3Aq3rOLVbh5P4qmoSJFQytAzsP8QmAbgPGxUE/DJ0XwwctUcilJr4rb fvFOx82ySPOIruWEXR080ohDdOf7S7Lm9M2SUUyi82Xa6eu47KSdaHZvy A==; X-IronPort-AV: E=McAfee;i="6500,9779,10507"; a="290401090" X-IronPort-AV: E=Sophos;i="5.95,203,1661842800"; d="scan'208";a="290401090" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2022 13:01:28 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10507"; a="633069091" X-IronPort-AV: E=Sophos;i="5.95,203,1661842800"; d="scan'208";a="633069091" Received: from agluck-desk3.sc.intel.com ([172.25.222.78]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Oct 2022 13:01:27 -0700 From: Tony Luck To: Naoya Horiguchi , Andrew Morton Cc: Miaohe Lin , Matthew Wilcox , Shuai Xue , Dan Williams , Michael Ellerman , Nicholas Piggin , Christophe Leroy , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, Tony Luck Subject: [PATCH v3 2/2] mm, hwpoison: When copy-on-write hits poison, take page offline Date: Fri, 21 Oct 2022 13:01:20 -0700 Message-Id: <20221021200120.175753-3-tony.luck@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221021200120.175753-1-tony.luck@intel.com> References: <20221019170835.155381-1-tony.luck@intel.com> <20221021200120.175753-1-tony.luck@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_NONE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747329095532355117?= X-GMAIL-MSGID: =?utf-8?q?1747329095532355117?= Cannot call memory_failure() directly from the fault handler because mmap_lock (and others) are held. It is important, but not urgent, to mark the source page as h/w poisoned and unmap it from other tasks. Use memory_failure_queue() to request a call to memory_failure() for the page with the error. Also provide a stub version for CONFIG_MEMORY_FAILURE=n Signed-off-by: Tony Luck --- include/linux/mm.h | 5 ++++- mm/memory.c | 4 +++- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 8bbcccbc5565..03ced659eb58 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3268,7 +3268,6 @@ enum mf_flags { int mf_dax_kill_procs(struct address_space *mapping, pgoff_t index, unsigned long count, int mf_flags); extern int memory_failure(unsigned long pfn, int flags); -extern void memory_failure_queue(unsigned long pfn, int flags); extern void memory_failure_queue_kick(int cpu); extern int unpoison_memory(unsigned long pfn); extern int sysctl_memory_failure_early_kill; @@ -3277,8 +3276,12 @@ extern void shake_page(struct page *p); extern atomic_long_t num_poisoned_pages __read_mostly; extern int soft_offline_page(unsigned long pfn, int flags); #ifdef CONFIG_MEMORY_FAILURE +extern void memory_failure_queue(unsigned long pfn, int flags); extern int __get_huge_page_for_hwpoison(unsigned long pfn, int flags); #else +static inline void memory_failure_queue(unsigned long pfn, int flags) +{ +} static inline int __get_huge_page_for_hwpoison(unsigned long pfn, int flags) { return 0; diff --git a/mm/memory.c b/mm/memory.c index b6056eef2f72..eae242351726 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2866,8 +2866,10 @@ static inline int __wp_page_copy_user(struct page *dst, struct page *src, unsigned long addr = vmf->address; if (likely(src)) { - if (copy_mc_user_highpage(dst, src, addr, vma)) + if (copy_mc_user_highpage(dst, src, addr, vma)) { + memory_failure_queue(page_to_pfn(src), 0); return -EHWPOISON; + } return 0; }