From patchwork Fri Jun 2 09:29:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 102416 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp904662vqr; Fri, 2 Jun 2023 02:44:12 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5IyStkJu2CpcSo6tTdY8FHW0uFbpXV0b18rZyR/4hWojMPPMJsMZFEXedfDPZHorXOBW+Y X-Received: by 2002:a05:6a00:2496:b0:651:9ca6:de40 with SMTP id c22-20020a056a00249600b006519ca6de40mr4097552pfv.3.1685699052404; Fri, 02 Jun 2023 02:44:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685699052; cv=none; d=google.com; s=arc-20160816; b=J5RwqKfWVm/iWfdmPJ4KmbT4aIp7UPjFiVFnVStlHvapTaDViVgB6TyQ8uOK7aKtBs YoAkBycfoVBzGXL7s+eAKHRfCZOijXSn5RMfLAUeqKsqnj/+xYX0RGupuZ8e1d4MWgnq 6vF5yMBthmXaPjGhdjYhVP9Y7FMVECh2TiXg7YwZagGrzq64um/TkNqmcPYXaGFElqIj X36jwdC0WQhEzEVUyW3AOs97c56pMbYYZUQtBM/1wnX/2UENHPTCbXMmmLQewhXHnY4R oHQAr+5c32oJ8dDG1G/es9qsfo4CXXCwIRHdHa2LRrwJbPrGpURrVlUsFO5s51tGWvug SzKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=fR4yNwzvaAJnisMVw8RkhRtyeaH2J5bwe2DxjFB1zZ8=; b=Hrt2Gz9dAA/+dlUCt3Y616DN//a4UP4UeVIwgdbruluAercTz2kwkTA3yr0wTn1zop Rzw3zS+jZYWDxJT9yQRqBj4Bi9UpAgUZK05yGdJQNk1+/Pt24f5rvDCUP193uYNn/xY/ q27XG4j8wSVk/SO9vKhGrej8gNZVOidYpQq1lmLuSck2Xw6BQUNp6TPD0u2yUgH+c7GG 7B1RMQwu4+cRkRTrhewib9C2HPAGreR+u0qJR5Ue3xkPyfSpCN2wHsSvVw9Y7TdysKFD pyY7ysGi5l6MUPSg7X9Kfy0osjQBDEQqhQfa/DAhUaRCVBlg3w03SULweZ2Uqm4QI3Pz mNrw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q13-20020aa7960d000000b0063d48d82e85si518606pfg.15.2023.06.02.02.44.00; Fri, 02 Jun 2023 02:44:12 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234244AbjFBJbN (ORCPT + 99 others); Fri, 2 Jun 2023 05:31:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47480 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234686AbjFBJah (ORCPT ); Fri, 2 Jun 2023 05:30:37 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id E899BE5A for ; Fri, 2 Jun 2023 02:30:05 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 19B1E12FC; Fri, 2 Jun 2023 02:30:51 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C874E3F7BD; Fri, 2 Jun 2023 02:30:03 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , SeongJae Park , Christoph Hellwig Cc: Ryan Roberts , "Matthew Wilcox (Oracle)" , "Kirill A. Shutemov" , Lorenzo Stoakes , Uladzislau Rezki , Zi Yan , Yu Zhao , Mike Rapoport , linux-kernel@vger.kernel.org, linux-mm@kvack.org, damon@lists.linux.dev, Christoph Hellwig Subject: [PATCH v3 1/4] mm: vmalloc must set pte via arch code Date: Fri, 2 Jun 2023 10:29:46 +0100 Message-Id: <20230602092949.545577-2-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230602092949.545577-1-ryan.roberts@arm.com> References: <20230602092949.545577-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767583569350961017?= X-GMAIL-MSGID: =?utf-8?q?1767583569350961017?= It is bad practice to directly set pte entries within a pte table. Instead all modifications must go through arch-provided helpers such as set_pte_at() to give the arch code visibility and allow it to check (and potentially modify) the operation. Fixes: 3e9a9e256b1e ("mm: add a vmap_pfn function") Signed-off-by: Ryan Roberts Reviewed-by: Zi Yan Acked-by: Lorenzo Stoakes Reviewed-by: Christoph Hellwig Reviewed-by: Uladzislau Rezki (Sony) Reviewed-by: Mike Rapoport (IBM) --- mm/vmalloc.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 9683573f1225..48202ec5f79a 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2899,10 +2899,16 @@ struct vmap_pfn_data { static int vmap_pfn_apply(pte_t *pte, unsigned long addr, void *private) { struct vmap_pfn_data *data = private; + unsigned long pfn = data->pfns[data->idx]; + pte_t ptent; - if (WARN_ON_ONCE(pfn_valid(data->pfns[data->idx]))) + if (WARN_ON_ONCE(pfn_valid(pfn))) return -EINVAL; - *pte = pte_mkspecial(pfn_pte(data->pfns[data->idx++], data->prot)); + + ptent = pte_mkspecial(pfn_pte(pfn, data->prot)); + set_pte_at(&init_mm, addr, pte, ptent); + + data->idx++; return 0; } From patchwork Fri Jun 2 09:29:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 102421 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp905533vqr; Fri, 2 Jun 2023 02:46:17 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5a/gzFAvYopHQWayZqUNIA5N745CRazV1/XwceFHffbnz/eLGwal0ManClUI8xdoNrCFyT X-Received: by 2002:a17:90a:5982:b0:255:38ed:9dcd with SMTP id l2-20020a17090a598200b0025538ed9dcdmr2164434pji.0.1685699176835; Fri, 02 Jun 2023 02:46:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685699176; cv=none; d=google.com; s=arc-20160816; b=LF4ZpU3WAtZV7I36o5VD8dvYaMt+v+xRZcrJvNm9DtKYAdrWRdeVlUeC6+Dbfr8Q58 OP/RC2f14hI1ca966dyrpzef+ayf0NtvRHeHa/rmqJe2TVl5kZmtkRHncmhzuPTumRhy Sb8JsuvG057xXtZScU+0kRLDQqCBnKCtTvViNKf2/961Z7+TiVm6pFyLmY82zE1VRCTZ WRlnsNCBh61o13ZgoiM1TdCZCAKZL4zjhnyRh/J2AvpypKUQy/ImMm19nmAOhu69ZGWS 7Vk/HRD3Z4H9MggQyXk6iF8rcd7DKUAdaFYWRwuePs6k/pwulXUeEoAPvb1cfdv0ncMd vxzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=Kot1/bspnGlxwv0XLlB/lHLb+4n0KmxZddm5AnOwnXM=; b=JHwd8mk9zMdlE7HbI6N/2btoRfZWvDsG+PLtR6BdFl572rIbzIPZjMB8Y6CnbJKFWa DrjL16+8+frZOiJVbacLROldr7hrvQJsKNjL8m6yw+mYTNbWGgChDjzQUiB53ooBUY8g MtsjnwWgQI29xSTNJAMfjSYynfKq2CMTqHJGc2tvzM7Xr+uTKVaJeuNdqqkphLFNo0E9 zSkanL+OQwwwqW9Wa5uLkW0ZeKkoTe+2Uegte5o4eKCOsdPqYn+9NMTpVqIepInKwneP 8PPfFmyb2isfJ3gvM5ym3UyN8AONACwlvGA1h3y5sXOWwnRc4hM1FkrZ007YEAMzv7Ql rwZA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id oj11-20020a17090b4d8b00b002473e331f90si817043pjb.46.2023.06.02.02.46.03; Fri, 02 Jun 2023 02:46:16 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234967AbjFBJbT (ORCPT + 99 others); Fri, 2 Jun 2023 05:31:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47496 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234751AbjFBJak (ORCPT ); Fri, 2 Jun 2023 05:30:40 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id D6334E68 for ; Fri, 2 Jun 2023 02:30:07 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 13E121764; Fri, 2 Jun 2023 02:30:53 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DD7893F7BD; Fri, 2 Jun 2023 02:30:05 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , SeongJae Park , Christoph Hellwig Cc: Ryan Roberts , "Matthew Wilcox (Oracle)" , "Kirill A. Shutemov" , Lorenzo Stoakes , Uladzislau Rezki , Zi Yan , Yu Zhao , Mike Rapoport , linux-kernel@vger.kernel.org, linux-mm@kvack.org, damon@lists.linux.dev Subject: [PATCH v3 2/4] mm/damon/ops-common: atomically test and clear young on ptes and pmds Date: Fri, 2 Jun 2023 10:29:47 +0100 Message-Id: <20230602092949.545577-3-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230602092949.545577-1-ryan.roberts@arm.com> References: <20230602092949.545577-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767583699817025198?= X-GMAIL-MSGID: =?utf-8?q?1767583699817025198?= It is racy to non-atomically read a pte, then clear the young bit, then write it back as this could discard dirty information. Further, it is bad practice to directly set a pte entry within a table. Instead clearing young must go through the arch-provided helper, ptep_test_and_clear_young() to ensure it is modified atomically and to give the arch code visibility and allow it to check (and potentially modify) the operation. Fixes: 3f49584b262c ("mm/damon: implement primitives for the virtual memory address spaces"). Signed-off-by: Ryan Roberts Reviewed-by: Zi Yan Reviewed-by: SeongJae Park Reviewed-by: Mike Rapoport (IBM) --- mm/damon/ops-common.c | 16 ++++++---------- mm/damon/ops-common.h | 4 ++-- mm/damon/paddr.c | 4 ++-- mm/damon/vaddr.c | 4 ++-- 4 files changed, 12 insertions(+), 16 deletions(-) diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c index cc63cf953636..acc264b97903 100644 --- a/mm/damon/ops-common.c +++ b/mm/damon/ops-common.c @@ -37,7 +37,7 @@ struct folio *damon_get_folio(unsigned long pfn) return folio; } -void damon_ptep_mkold(pte_t *pte, struct mm_struct *mm, unsigned long addr) +void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *vma, unsigned long addr) { bool referenced = false; struct folio *folio = damon_get_folio(pte_pfn(*pte)); @@ -45,13 +45,11 @@ void damon_ptep_mkold(pte_t *pte, struct mm_struct *mm, unsigned long addr) if (!folio) return; - if (pte_young(*pte)) { + if (ptep_test_and_clear_young(vma, addr, pte)) referenced = true; - *pte = pte_mkold(*pte); - } #ifdef CONFIG_MMU_NOTIFIER - if (mmu_notifier_clear_young(mm, addr, addr + PAGE_SIZE)) + if (mmu_notifier_clear_young(vma->vm_mm, addr, addr + PAGE_SIZE)) referenced = true; #endif /* CONFIG_MMU_NOTIFIER */ @@ -62,7 +60,7 @@ void damon_ptep_mkold(pte_t *pte, struct mm_struct *mm, unsigned long addr) folio_put(folio); } -void damon_pmdp_mkold(pmd_t *pmd, struct mm_struct *mm, unsigned long addr) +void damon_pmdp_mkold(pmd_t *pmd, struct vm_area_struct *vma, unsigned long addr) { #ifdef CONFIG_TRANSPARENT_HUGEPAGE bool referenced = false; @@ -71,13 +69,11 @@ void damon_pmdp_mkold(pmd_t *pmd, struct mm_struct *mm, unsigned long addr) if (!folio) return; - if (pmd_young(*pmd)) { + if (pmdp_test_and_clear_young(vma, addr, pmd)) referenced = true; - *pmd = pmd_mkold(*pmd); - } #ifdef CONFIG_MMU_NOTIFIER - if (mmu_notifier_clear_young(mm, addr, addr + HPAGE_PMD_SIZE)) + if (mmu_notifier_clear_young(vma->vm_mm, addr, addr + HPAGE_PMD_SIZE)) referenced = true; #endif /* CONFIG_MMU_NOTIFIER */ diff --git a/mm/damon/ops-common.h b/mm/damon/ops-common.h index 14f4bc69f29b..18d837d11bce 100644 --- a/mm/damon/ops-common.h +++ b/mm/damon/ops-common.h @@ -9,8 +9,8 @@ struct folio *damon_get_folio(unsigned long pfn); -void damon_ptep_mkold(pte_t *pte, struct mm_struct *mm, unsigned long addr); -void damon_pmdp_mkold(pmd_t *pmd, struct mm_struct *mm, unsigned long addr); +void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *vma, unsigned long addr); +void damon_pmdp_mkold(pmd_t *pmd, struct vm_area_struct *vma, unsigned long addr); int damon_cold_score(struct damon_ctx *c, struct damon_region *r, struct damos *s); diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 467b99166b43..5b3a3463d078 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -24,9 +24,9 @@ static bool __damon_pa_mkold(struct folio *folio, struct vm_area_struct *vma, while (page_vma_mapped_walk(&pvmw)) { addr = pvmw.address; if (pvmw.pte) - damon_ptep_mkold(pvmw.pte, vma->vm_mm, addr); + damon_ptep_mkold(pvmw.pte, vma, addr); else - damon_pmdp_mkold(pvmw.pmd, vma->vm_mm, addr); + damon_pmdp_mkold(pvmw.pmd, vma, addr); } return true; } diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index 1fec16d7263e..37994fb6120c 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -311,7 +311,7 @@ static int damon_mkold_pmd_entry(pmd_t *pmd, unsigned long addr, } if (pmd_trans_huge(*pmd)) { - damon_pmdp_mkold(pmd, walk->mm, addr); + damon_pmdp_mkold(pmd, walk->vma, addr); spin_unlock(ptl); return 0; } @@ -323,7 +323,7 @@ static int damon_mkold_pmd_entry(pmd_t *pmd, unsigned long addr, pte = pte_offset_map_lock(walk->mm, pmd, addr, &ptl); if (!pte_present(*pte)) goto out; - damon_ptep_mkold(pte, walk->mm, addr); + damon_ptep_mkold(pte, walk->vma, addr); out: pte_unmap_unlock(pte, ptl); return 0; From patchwork Fri Jun 2 09:29:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 102423 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp906706vqr; Fri, 2 Jun 2023 02:49:19 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ72XC60prwAbi3rQ6CbNOoRIqauOobEOV7I5KdQt9v2vk/RZb0wMAV5HFDR5yTMbun41k2d X-Received: by 2002:a05:6358:4315:b0:123:3ea3:2bd2 with SMTP id r21-20020a056358431500b001233ea32bd2mr8790474rwc.32.1685699358762; Fri, 02 Jun 2023 02:49:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685699358; cv=none; d=google.com; s=arc-20160816; b=0v5aM7wru53coh49r1cDTEzCKd4TLVx+WZrz1aLD+KolY0yFvyvwppNwArVYMk5mdi cPUXpk8ksI5oT7wSnY/L8PSBgho+O9Gcy/uI4K4OYOCK0uMbvK3sz+40732ZrihFqYqR MWMNhCQn3K5JOx00MLDjbgzB8tK8PfyQm0sSwMqUlKJ3KzJ8ULVGGXYAo/brBql3WMKF MrPcQbyAOvHDibbgwOnKKq19+OAeR4pu5EiM/wekWzHkAoMbR8TkayVBcwTB0ZkKHYvC AI0TAgXV6m349jYYIfnUgrxWjE0JV7WYWBmTnIVPKY1UQHokYjfrMf7hVZtXNkthhSLP Ntog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=VKRNb17rmx4CrscdLqv68+7Swb2k3gEeCrh0aTpL6lk=; b=twP7lxIa9CBqxmU3Ajf38WCyzaaeABiRyeE2ymBTm1OTkeabIY4UcdYJfbovTng1Ff GZLphBWEHV3xvCrnweFWCbiqT6R0UpVbrHL5twC9SFassjNQ2E1HG4EhLQHDCZYtXZKi Tn82CU9Tm4aZrcxVNqAl9f87V0guizYY0YHELZb9yTHsriUlxtZ6uz4V/ZitNmmXxqpu /LLW8tu5v7CTgVgi2UCcjMbLmhWspNzzcevE58yu14wWArHl/bePJ8mzlClpQW42zuPM c4OFe9vfF51yPCSkwwGNhXMHvlk+jNCQOkk/JCaDWy92cDLfovtsnJo6U2QU6JKJgNF3 Acmg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z11-20020a17090ab10b00b00256c8abe403si2466626pjq.167.2023.06.02.02.49.04; Fri, 02 Jun 2023 02:49:18 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235109AbjFBJbX (ORCPT + 99 others); Fri, 2 Jun 2023 05:31:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47506 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234695AbjFBJal (ORCPT ); Fri, 2 Jun 2023 05:30:41 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id CD8B910F3 for ; Fri, 2 Jun 2023 02:30:09 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0D42D176A; Fri, 2 Jun 2023 02:30:55 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D73083F7BD; Fri, 2 Jun 2023 02:30:07 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , SeongJae Park , Christoph Hellwig Cc: Ryan Roberts , "Matthew Wilcox (Oracle)" , "Kirill A. Shutemov" , Lorenzo Stoakes , Uladzislau Rezki , Zi Yan , Yu Zhao , Mike Rapoport , linux-kernel@vger.kernel.org, linux-mm@kvack.org, damon@lists.linux.dev Subject: [PATCH v3 3/4] mm/damon/ops-common: Refactor to use {pte|pmd}p_clear_young_notify() Date: Fri, 2 Jun 2023 10:29:48 +0100 Message-Id: <20230602092949.545577-4-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230602092949.545577-1-ryan.roberts@arm.com> References: <20230602092949.545577-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767583890677331626?= X-GMAIL-MSGID: =?utf-8?q?1767583890677331626?= With the fix in place to atomically test and clear young on ptes and pmds, simplify the code to handle the clearing for both the primary mmu and the mmu notifier with a single API call. Signed-off-by: Ryan Roberts Acked-by: Yu Zhao Reviewed-by: SeongJae Park --- mm/damon/ops-common.c | 22 ++-------------------- 1 file changed, 2 insertions(+), 20 deletions(-) diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c index acc264b97903..d4ab81229136 100644 --- a/mm/damon/ops-common.c +++ b/mm/damon/ops-common.c @@ -39,21 +39,12 @@ struct folio *damon_get_folio(unsigned long pfn) void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *vma, unsigned long addr) { - bool referenced = false; struct folio *folio = damon_get_folio(pte_pfn(*pte)); if (!folio) return; - if (ptep_test_and_clear_young(vma, addr, pte)) - referenced = true; - -#ifdef CONFIG_MMU_NOTIFIER - if (mmu_notifier_clear_young(vma->vm_mm, addr, addr + PAGE_SIZE)) - referenced = true; -#endif /* CONFIG_MMU_NOTIFIER */ - - if (referenced) + if (ptep_clear_young_notify(vma, addr, pte)) folio_set_young(folio); folio_set_idle(folio); @@ -63,21 +54,12 @@ void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *vma, unsigned long addr void damon_pmdp_mkold(pmd_t *pmd, struct vm_area_struct *vma, unsigned long addr) { #ifdef CONFIG_TRANSPARENT_HUGEPAGE - bool referenced = false; struct folio *folio = damon_get_folio(pmd_pfn(*pmd)); if (!folio) return; - if (pmdp_test_and_clear_young(vma, addr, pmd)) - referenced = true; - -#ifdef CONFIG_MMU_NOTIFIER - if (mmu_notifier_clear_young(vma->vm_mm, addr, addr + HPAGE_PMD_SIZE)) - referenced = true; -#endif /* CONFIG_MMU_NOTIFIER */ - - if (referenced) + if (pmdp_clear_young_notify(vma, addr, pmd)) folio_set_young(folio); folio_set_idle(folio); From patchwork Fri Jun 2 09:29:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 102414 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp903942vqr; Fri, 2 Jun 2023 02:42:25 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4JRRuzUM87WLhnJRsEY3pRrfg0B2ehAc5FigoQpyEJtppgXwv3xOneAOyDNk684ZVFdche X-Received: by 2002:a92:d581:0:b0:33b:151a:e29f with SMTP id a1-20020a92d581000000b0033b151ae29fmr8643459iln.11.1685698945626; Fri, 02 Jun 2023 02:42:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685698945; cv=none; d=google.com; s=arc-20160816; b=n1Ld5CGXxTGbnQQCnHFvVPl1GO+Tt5U1GjIlC3dTJLrC8EpFGPjF+Mk0ARrEkRMymm hh/KiWEosu9jPu0m/aU5Kh+ibUk215Yw3fUAexqzX8NMvLToCi1kIg4gbGoXI7GcnPAV tBeXTcy9fg0hU+BQyc6QHsn5Go8TDALhncMIOIA7izTTFVswVw+kr0MAVpO7YBsamBiK qS4VR41e4ZbEV5NMIZC+7o7mIsFrxw3mbr0BJv7ZX6sQc0gdyYcLWxDf8LcCJJR7VRE1 s8mConZvfF7E8uJka97KICs/B4C1P5m02ak+xpt2Jz+ik6rs6eFI4w/wRBfVXsKnzIuC dDVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=vZ8R5MX0YBLxMMX9FnLRmjkoG+ZGaABcPpi932QFLAY=; b=PJgPS9ig6gQBIjA/09+it5I04h+ta/XtQE4WApt7T2belmT6NNjsKcsWVHJoJ6VLYm J9JrwyZWDMwsNiAKupYhdKwc/W9rHMLwEII/b0kgHA63BKyVxgMb0uXykxKi7mLLFsfG LBRlN3iTbuTSnUS50XUpr1j4z6XuvZQZFqUoqeqsrg7VWvUEFEWjcxNG2/reSEHQjNWB Adt2g538ASA4xNgkvHJLpSvwL+LHgumw0Phfn4KID4ZlKg8LCd0BDZpX964lpK/VN9HK lf7FmNFr4wZEwUv2/Tc/khDPtVK1QObQSK5SmWi9fpnlucBmP/hCxkn5iCzAXytEcXeQ Qxdg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id v189-20020a6389c6000000b0053f2601e5aasi724660pgd.195.2023.06.02.02.42.13; Fri, 02 Jun 2023 02:42:25 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235033AbjFBJb1 (ORCPT + 99 others); Fri, 2 Jun 2023 05:31:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47512 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235012AbjFBJam (ORCPT ); Fri, 2 Jun 2023 05:30:42 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 149F910F7 for ; Fri, 2 Jun 2023 02:30:12 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0728A176B; Fri, 2 Jun 2023 02:30:57 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D0E1D3F7BD; Fri, 2 Jun 2023 02:30:09 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , SeongJae Park , Christoph Hellwig Cc: Ryan Roberts , "Matthew Wilcox (Oracle)" , "Kirill A. Shutemov" , Lorenzo Stoakes , Uladzislau Rezki , Zi Yan , Yu Zhao , Mike Rapoport , linux-kernel@vger.kernel.org, linux-mm@kvack.org, damon@lists.linux.dev Subject: [PATCH v3 4/4] mm: Fix failure to unmap pte on highmem systems Date: Fri, 2 Jun 2023 10:29:49 +0100 Message-Id: <20230602092949.545577-5-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230602092949.545577-1-ryan.roberts@arm.com> References: <20230602092949.545577-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767583457690167480?= X-GMAIL-MSGID: =?utf-8?q?1767583457690167480?= The loser of a race to service a pte for a device private entry in the swap path previously unlocked the ptl, but failed to unmap the pte. This only affects highmem systems since unmapping a pte is a noop on non-highmem systems. Fixes: 16ce101db85d ("mm/memory.c: fix race when faulting a device private page") Signed-off-by: Ryan Roberts Reviewed-by: Zi Yan Reviewed-by: Mike Rapoport (IBM) --- mm/memory.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index f69fbc251198..ed429e20a1bb 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3728,10 +3728,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) vmf->page = pfn_swap_entry_to_page(entry); vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd, vmf->address, &vmf->ptl); - if (unlikely(!pte_same(*vmf->pte, vmf->orig_pte))) { - spin_unlock(vmf->ptl); - goto out; - } + if (unlikely(!pte_same(*vmf->pte, vmf->orig_pte))) + goto unlock; /* * Get a page reference while we know the page can't be