From patchwork Wed Jan 25 01:57:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zach O'Keefe X-Patchwork-Id: 47996 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp43234wrn; Tue, 24 Jan 2023 17:58:47 -0800 (PST) X-Google-Smtp-Source: AMrXdXuue9GEjm9+/iv86hUmYcuOxlBS9A5OZCBW7oy1lyLuvMdiy3x1a1AxaGSm9l1uynHTV1MC X-Received: by 2002:a17:902:9002:b0:194:9c02:7619 with SMTP id a2-20020a170902900200b001949c027619mr31445223plp.29.1674611926871; Tue, 24 Jan 2023 17:58:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674611926; cv=none; d=google.com; s=arc-20160816; b=oQeDarYb3BlgHKr/wzKzVsXAX7IRrKpi/+8OxKmAxa0PHlWRl7/h0nXTZhBfVeWdQ6 NvX+zbNuFe0UypEayv+v2NFcdbHxOarxPbYPImkpK2yQP2u0E37D8fY9zkW6JAJc+7o+ 4IxMwB4JjBQ8JsmWxf+ornRu8FQnykuDMpTaAvZ4uJDAuhbQlJQ7vND3GFgaT+ETYj+v nEcUZH5Yd8BjCAkaWpihMjtZwvjgUeGZmmb01L+QuGtFRcgsxQQUvNSLXyWLOE5IQI19 iwhUuX11WqDEYEZh2tTBcjaGdXYXAuyMSJk7kcmwTPBadL401CzDzdQOSmYEapj03Ubk Jj4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:mime-version:date :dkim-signature; bh=fp9ccs/R/HqGAoRU6R/FJYwx892zglssKTnZXb6TNhs=; b=k2e563GvsDBnWPhMNuk1IZV3JO4v8I0ke3qrWoMVVNSLF3tywVDgIC5SwYO6DkrMTf mOBsuPJ0T9uvuCcFTlXLTsqN/oWiqyyA+pojXatNINYKCAJek4bq58aID8Fts3wqcFYx x+KYy2rba+cSsX/Ygpi7efY/NRLYcWD8MY8vm1wWnVfr4kfhNW0lwERkct39o3tmmfM9 PWnB8PRTD0qE4IwRxLBN7+Acy3p0bqbDaDueb8O3WFJPf+LqkJFcbksWc080Mjr4eVNw P6+OJpvSXtYL5pd7lRU+I00B488yp0WI1OrvCQN4cq0c6/tK0HJCprBGkzSI/smLai5S 2EiQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=WAqnNhti; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 13-20020a170902c24d00b00194968874f1si3996552plg.453.2023.01.24.17.58.35; Tue, 24 Jan 2023 17:58:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=WAqnNhti; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230160AbjAYB6Q (ORCPT + 99 others); Tue, 24 Jan 2023 20:58:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45202 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231286AbjAYB6O (ORCPT ); Tue, 24 Jan 2023 20:58:14 -0500 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AE54BCD for ; Tue, 24 Jan 2023 17:58:11 -0800 (PST) Received: by mail-pj1-x104a.google.com with SMTP id lk5-20020a17090b33c500b00228cb369d7aso280771pjb.5 for ; Tue, 24 Jan 2023 17:58:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=fp9ccs/R/HqGAoRU6R/FJYwx892zglssKTnZXb6TNhs=; b=WAqnNhtiG8WjUY8O+bk3uMlpTRdXAO1398g06DJIjO5tpLENLCDEFEWfl5MKQQ2cS4 FJmL4Hx1MgPkUvaWuxuEKoXfRmKWL2q9d5N75BjvVzLdMDt/6/LMiboSf2ZuostsQRoT 5oOPTgTiEDS0FKff/+yrSTTKn/j66fTfrr+Er9d8jkJiPj/NjiPkpckv7Sfa8Z4H5BlN +qW9JTz3wAKujy3DdbdwCaEfka7lpLk/bYQvXMvMgNfLHIjIFV2LvQDDSZdYttuhSOO5 tVfxl+1M1BxCZvN4M8o48BWAxGGoiHGmTA+PLRW0WNl344BBT0PzlqW+jQqTi6sTu+My HpVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=fp9ccs/R/HqGAoRU6R/FJYwx892zglssKTnZXb6TNhs=; b=3QujvQ+fz0XBqNSE0t9UJ34Y5Ss1ztLQY2PAjGKwJI6jtFKW0I5BE/QagwhSsqfuGI pp9Yx7Wy/83B2X36cleUechny506RuoSairPzGfjibCq4ljac2k7N4AHRr9CrTwDQ6zU mKCQOX9lToGXrmN9at0C+zESN3ypYFLdhdFNoCusVo+Lwg2uOlbduvIeu3UiP6ChYy// HaUV7/cMcLmAxPQG9669Z9Is7hfw9ObXw/yCYnbsgkWPpW80fM+iR4hUl1gU9/r2XCw1 NGt2H30//at7etQlPdPbjqiGBmFROr/L/2O9gppYX963DKyOtMh4a8fd8fdFp25pr5is M9IQ== X-Gm-Message-State: AFqh2koCf1bFjEHfofhtmGrQhLOp4WHLqAN8DgNBm6VHWIFHD75EcOj/ 8gztPSHDWMZRBNqyRikXXj0PnAjK5AeN X-Received: from zokeefe3.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1b6]) (user=zokeefe job=sendgmr) by 2002:a05:6a00:22ca:b0:58b:3d7f:746a with SMTP id f10-20020a056a0022ca00b0058b3d7f746amr3665212pfj.77.1674611891064; Tue, 24 Jan 2023 17:58:11 -0800 (PST) Date: Tue, 24 Jan 2023 17:57:37 -0800 Mime-Version: 1.0 X-Mailer: git-send-email 2.39.1.405.gd4c25cc71f-goog Message-ID: <20230125015738.912924-1-zokeefe@google.com> Subject: [PATCH 1/2] mm/MADV_COLLAPSE: set EAGAIN on unexpected page refcount From: "Zach O'Keefe" To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , Hugh Dickins , Yang Shi , "Zach O'Keefe" X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755957875970271137?= X-GMAIL-MSGID: =?utf-8?q?1755957875970271137?= During collapse, in a few places we check to see if a given small page has any unaccounted references. If the refcount on the page doesn't match our expectations, it must be there is an unknown user concurrently interested in the page, and so it's not safe to move the contents elsewhere. However, the unaccounted pins are likely an ephemeral state. In such a situation, make MADV_COLLAPSE set EAGAIN errno, indicating that collapse may succeed on retry. Fixes: 7d8faaf15545 ("mm/madvise: introduce MADV_COLLAPSE sync hugepage collapse") Reported-by: Hugh Dickins Signed-off-by: Zach O'Keefe Reviewed-by: Yang Shi Acked-by: Hugh Dickins --- mm/khugepaged.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index e23619bfecc4..fa38cae240b9 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2712,6 +2712,7 @@ static int madvise_collapse_errno(enum scan_result r) case SCAN_CGROUP_CHARGE_FAIL: return -EBUSY; /* Resource temporary unavailable - trying again might succeed */ + case SCAN_PAGE_COUNT: case SCAN_PAGE_LOCK: case SCAN_PAGE_LRU: case SCAN_DEL_PAGE_LRU: From patchwork Wed Jan 25 01:57:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zach O'Keefe X-Patchwork-Id: 47997 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp43274wrn; Tue, 24 Jan 2023 17:58:54 -0800 (PST) X-Google-Smtp-Source: AMrXdXuewevSWV22j9TKnjX1ZFrF4KlLzOCdNkVrpQdzHnvdJjjGEcdYCVlQ5f9ulkT5TA5HOK6k X-Received: by 2002:a62:3385:0:b0:589:d850:7ea5 with SMTP id z127-20020a623385000000b00589d8507ea5mr27801055pfz.6.1674611934390; Tue, 24 Jan 2023 17:58:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1674611934; cv=none; d=google.com; s=arc-20160816; b=UUB4wLynKZZh2GAZ9dGB/H1/NcN9LKDLNavFgliHpNl0EpSLJhgS0JIoNVtU+7j71a O1ONeKMq7wuR9fhBjtHtDM1R9cvSM5yOnWe814C4GKn+YxXJ+ZnYHUFyr0yGU00y4DRz JSxRWl9GlKF5ODi5dEBooYT1PEDJaF2TibfJIqjTdYPcSM3D+rh94IS9P35HrUYWqgIC mfA7RZ57cdeCCSynTwMJhw+2gvf7z5dP91X9KgmCVlXwMNuFNBaTNaiBKZAFXhG0+jLg lcm6OPll/ntkFG7wd/PPV+gLEABb2bYz3FjAKRaS/gfa0qxfliYyHnADngjYGfdH3z4P gPsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=CLeT8Nf/jj+XeEyGWZaHUysjcWhO2WN556Qt8YbZFTA=; b=Z5J5dDBs3065aGZYXN8w5tgsEmvf9UKuizG6dZxjqt8Kl8GE7HRLSZkGTIMmBhnJ12 qXiNvXeOYA64UelJzNMEzShKhJyGWxAZNmKP4IpMsLUnwEpRF3S2eGgJ9Q4RkNRT0omE VGdyB4Q+mlrYbSUh2tNaREUoKtHwyBhv1CSB4gHSG2g2lKWZYJ+waifq0puZ2rzCTJA7 BsvIDinebKgCqcIYBOaMJuUlH0uiCa0TZLpFsAJawKxByisAIfMSInn9OFO/+4NLN1sj 5Pz/AscYH2H5DNRjVfSYFysyj9eTSS/td9Y/rlYw9Cp7FHIoHaDL76kPkDHNGGqcafkP PIuA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=bXlVQRum; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q6-20020aa79606000000b0058e2403f011si4334631pfg.56.2023.01.24.17.58.42; Tue, 24 Jan 2023 17:58:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=bXlVQRum; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232084AbjAYB6S (ORCPT + 99 others); Tue, 24 Jan 2023 20:58:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45208 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232306AbjAYB6Q (ORCPT ); Tue, 24 Jan 2023 20:58:16 -0500 Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7051538023 for ; Tue, 24 Jan 2023 17:58:13 -0800 (PST) Received: by mail-pg1-x54a.google.com with SMTP id 193-20020a6305ca000000b004cece0d0d64so7697156pgf.13 for ; Tue, 24 Jan 2023 17:58:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=CLeT8Nf/jj+XeEyGWZaHUysjcWhO2WN556Qt8YbZFTA=; b=bXlVQRumzm40VAjDslHGOuknPuAdE9sb0L4akt62AYYiNfF1nl/PE9a3K4+mhxfxbU qlOjlx9HYpNZ7nlKZHVgHB0Q1n9WvvwyURl+Khn4oVXSDFDI5stAZNLD/GV7mYuhnAmX 4fP31lq2dBuIKqP7/dpZqBG1jQWdaE4UbMixSINcYMwJN6K9z5usFxW1G1RYeQzjzLmE IuTvdJPPP3texNImaofSkc7EevfsFsTYsDDWMHluYgcK4K8lzmtm33VRfk8XHHm/C1AO JzwN7PIDjhXbMVtdks4HwD6HiFKY6CPRKEfjrxQ0G2uWsS+SXuQvOj6jyB/oyIkb1ojY XkMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=CLeT8Nf/jj+XeEyGWZaHUysjcWhO2WN556Qt8YbZFTA=; b=EkiXB9Kvw16qJ5dui8fH29xlhoPTxrB3LMOGIazRwOG5i4OR/+hjIRleQGzF6eEZWe 6+bgFEHvKVviMMOJN5mjo13qqkrXgJ0Xxw9gU5qNj50ZRa9bvP3hxBUDOfOxIefZwVs3 SUfG6Twi0FR7CA0TLoex3LA6EQhJ1VX1dFhQHyZaOTwig7C/whEefgu7iv54RKGCdKee a9B3RDSmNI19uc8WR8eKtYRWOU/mT4pP33XKly/63dqH279Zvj/LyL+Cmt3yubIHweOY YPzosY0EGygzN8kIVD4erYixXyQVaw3Di+io2UIgwg7lkpyJ8KkODvfDmF1ozIR+x/yK rFtw== X-Gm-Message-State: AFqh2kojUvQOGBS+aZCzyKSJ648U5UpVqD1VnWPGwEAfnKZKrAmaaxG5 QngvbRFhHybHlv03chy8KNvLVMYstkOM X-Received: from zokeefe3.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1b6]) (user=zokeefe job=sendgmr) by 2002:a62:6d04:0:b0:578:9709:615f with SMTP id i4-20020a626d04000000b005789709615fmr3520547pfc.45.1674611892894; Tue, 24 Jan 2023 17:58:12 -0800 (PST) Date: Tue, 24 Jan 2023 17:57:38 -0800 In-Reply-To: <20230125015738.912924-1-zokeefe@google.com> Mime-Version: 1.0 References: <20230125015738.912924-1-zokeefe@google.com> X-Mailer: git-send-email 2.39.1.405.gd4c25cc71f-goog Message-ID: <20230125015738.912924-2-zokeefe@google.com> Subject: [PATCH 2/2] mm/MADV_COLLAPSE: catch !none !huge !bad pmd lookups From: "Zach O'Keefe" To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , Hugh Dickins , Yang Shi , "Zach O'Keefe" , stable@vger.kernel.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1755957883578026063?= X-GMAIL-MSGID: =?utf-8?q?1755957883578026063?= In commit 34488399fa08 ("mm/madvise: add file and shmem support to MADV_COLLAPSE") we make the following change to find_pmd_or_thp_or_none(): - if (!pmd_present(pmde)) - return SCAN_PMD_NULL; + if (pmd_none(pmde)) + return SCAN_PMD_NONE; This was for-use by MADV_COLLAPSE file/shmem codepaths, where MADV_COLLAPSE might identify a pte-mapped hugepage, only to have khugepaged race-in, free the pte table, and clear the pmd. Such codepaths include: A) If we find a suitably-aligned compound page of order HPAGE_PMD_ORDER already in the pagecache. B) In retract_page_tables(), if we fail to grab mmap_lock for the target mm/address. In these cases, collapse_pte_mapped_thp() really does expect a none (not just !present) pmd, and we want to suitably identify that case separate from the case where no pmd is found, or it's a bad-pmd (of course, many things could happen once we drop mmap_lock, and the pmd could plausibly undergo multiple transitions due to intervening fault, split, etc). Regardless, the code is prepared install a huge-pmd only when the existing pmd entry is either a genuine pte-table-mapping-pmd, or the none-pmd. However, the commit introduces a logical hole; namely, that we've allowed !none- && !huge- && !bad-pmds to be classified as genuine pte-table-mapping-pmds. One such example that could leak through are swap entries. The pmd values aren't checked again before use in pte_offset_map_lock(), which is expecting nothing less than a genuine pte-table-mapping-pmd. We want to put back the !pmd_present() check (below the pmd_none() check), but need to be careful to deal with subtleties in pmd transitions and treatments by various arch. The issue is that __split_huge_pmd_locked() temporarily clears the present bit (or otherwise marks the entry as invalid), but pmd_present() and pmd_trans_huge() still need to return true while the pmd is in this transitory state. For example, x86's pmd_present() also checks the _PAGE_PSE , riscv's version also checks the _PAGE_LEAF bit, and arm64 also checks a PMD_PRESENT_INVALID bit. Covering all 4 cases for x86 (all checks done on the same pmd value): 1) pmd_present() && pmd_trans_huge() All we actually know here is that the PSE bit is set. Either: a) We aren't racing with __split_huge_page(), and PRESENT or PROTNONE is set. => huge-pmd b) We are currently racing with __split_huge_page(). The danger here is that we proceed as-if we have a huge-pmd, but really we are looking at a pte-mapping-pmd. So, what is the risk of this danger? The only relevant path is: madvise_collapse() -> collapse_pte_mapped_thp() Where we might just incorrectly report back "success", when really the memory isn't pmd-backed. This is fine, since split could happen immediately after (actually) successful madvise_collapse(). So, it should be safe to just assume huge-pmd here. 2) pmd_present() && !pmd_trans_huge() Either: a) PSE not set and either PRESENT or PROTNONE is. => pte-table-mapping pmd (or PROT_NONE) b) devmap. This routine can be called immediately after unlocking/locking mmap_lock -- or called with no locks held (see khugepaged_scan_mm_slot()), so previous VMA checks have since been invalidated. 3) !pmd_present() && pmd_trans_huge() Not possible. 4) !pmd_present() && !pmd_trans_huge() Neither PRESENT nor PROTNONE set => not present I've checked all archs that implement pmd_trans_huge() (arm64, riscv, powerpc, longarch, x86, mips, s390) and this logic roughly translates (though devmap treatment is unique to x86 and powerpc, and (3) doesn't necessarily hold in general -- but that doesn't matter since !pmd_present() always takes failure path). Also, add a comment above find_pmd_or_thp_or_none() to help future travelers reason about the validity of the code; namely, the possible mutations that might happen out from under us, depending on how mmap_lock is held (if at all). Fixes: 34488399fa08 ("mm/madvise: add file and shmem support to MADV_COLLAPSE") Reported-by: Hugh Dickins Signed-off-by: Zach O'Keefe Cc: stable@vger.kernel.org --- Request that this be pulled into stable since it's theoretically possible (though I have no reproducer) that while mmap_lock is dropped, racing thp migration installs a pmd migration entry which then has a path to be consumed, unchecked, by pte_offset_map(). --- mm/khugepaged.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index fa38cae240b9..7ea668bbea70 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -941,6 +941,10 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, return SCAN_SUCCEED; } +/* + * See pmd_trans_unstable() for how the result may change out from + * underneath us, even if we hold mmap_lock in read. + */ static int find_pmd_or_thp_or_none(struct mm_struct *mm, unsigned long address, pmd_t **pmd) @@ -959,8 +963,12 @@ static int find_pmd_or_thp_or_none(struct mm_struct *mm, #endif if (pmd_none(pmde)) return SCAN_PMD_NONE; + if (!pmd_present(pmde)) + return SCAN_PMD_NULL; if (pmd_trans_huge(pmde)) return SCAN_PMD_MAPPED; + if (pmd_devmap(pmd)) + return SCAN_PMD_NULL; if (pmd_bad(pmde)) return SCAN_PMD_NULL; return SCAN_SUCCEED;