Message ID | 20230626171430.3167004-5-ryan.roberts@arm.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp7639159vqr; Mon, 26 Jun 2023 10:26:57 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4MnvWHAkuydJQh0u2ggcXfkrwtOiXwPYmBPcx1mJ581uVTlNDGBPLqgFshqG+6JSl+wmF6 X-Received: by 2002:a05:6512:449:b0:4f8:674c:415b with SMTP id y9-20020a056512044900b004f8674c415bmr13671671lfk.42.1687800417379; Mon, 26 Jun 2023 10:26:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687800417; cv=none; d=google.com; s=arc-20160816; b=gUWUCShbwQMSrOiBWtHAGKYbXgwrOGaYCcsnPKdmX+NeCLzjIJtfh8di1hBUIgvG3j zLeIjLmYJR8rU5h0rza2EjFhj0Incp41vh57h2Jq/lvcukEuCC6htk+gRVuXHwUPcR3P 8hqZ+52FC6rmFlUtMEPMn0/hjfrcijwc/VMXdhqxkXnRoYCx7+VSMB9qvmP7627cUQ30 va4n+Rfgd6gM2uN96tzkQp8I5DbxuA1tN/9A+aOfhJvrGnG1EIK27lQRnj0eD9awOCw3 jV8UG3snPvMqfUqXqUqjjSIorUutT0IYhSh8zTpakhY9FEyFLVM3WfYmvTImynT7pJo/ O8bA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=JK++ZwPDFjFzL1t38tJLvARD4YCY0yS3OUbZtrWhuTw=; fh=H2MVjBlipHVEN6kEAh1RDhnPLB9jpPNjGExTmo1/EvA=; b=foXgrl8TeJ9Hgg97Q0XApt0GXR267fPb/cvGbPtS3ZZUNIFwtJ21mYqPsIhMUX/Jzr XdHpEZNxxoxqgWiF5eQ1Cc4gOtPeDkLxHMrECEtOZDOqS3phnupYhjgJ5Vv9AzdrbUkz kBrdzqmPEAU710SWypYdVWB+SdHYMP0jaxi4ChMehmpTaD4/Z2H1LgKt8NjGqwFHCmah RYdZpWbdbARaWUFKMkjL46x0QYjUUW9ZjxDvz/eQF+p0eTpQNvkaz1Scv9RuWrt96Bxs g9fNasPcMRd005vTVY8gV8XgWlea8zz5lxhMj1SQQPYWvK+8C2Jss8ZZnhfPCiRVUaBW LghQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h21-20020aa7c615000000b0051bed973bb2si2982975edq.455.2023.06.26.10.26.33; Mon, 26 Jun 2023 10:26:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231318AbjFZRPs (ORCPT <rfc822;filip.gregor98@gmail.com> + 99 others); Mon, 26 Jun 2023 13:15:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51502 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231314AbjFZRPB (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Mon, 26 Jun 2023 13:15:01 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id A42F910E0; Mon, 26 Jun 2023 10:14:53 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7835F1480; Mon, 26 Jun 2023 10:15:37 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8F06B3F663; Mon, 26 Jun 2023 10:14:50 -0700 (PDT) From: Ryan Roberts <ryan.roberts@arm.com> To: Andrew Morton <akpm@linux-foundation.org>, "Matthew Wilcox (Oracle)" <willy@infradead.org>, "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>, Yin Fengwei <fengwei.yin@intel.com>, David Hildenbrand <david@redhat.com>, Yu Zhao <yuzhao@google.com>, Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will@kernel.org>, Geert Uytterhoeven <geert@linux-m68k.org>, Christian Borntraeger <borntraeger@linux.ibm.com>, Sven Schnelle <svens@linux.ibm.com>, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>, "H. Peter Anvin" <hpa@zytor.com> Cc: Ryan Roberts <ryan.roberts@arm.com>, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-alpha@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-ia64@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-s390@vger.kernel.org Subject: [PATCH v1 04/10] mm: Implement folio_add_new_anon_rmap_range() Date: Mon, 26 Jun 2023 18:14:24 +0100 Message-Id: <20230626171430.3167004-5-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230626171430.3167004-1-ryan.roberts@arm.com> References: <20230626171430.3167004-1-ryan.roberts@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769787010461861650?= X-GMAIL-MSGID: =?utf-8?q?1769787010461861650?= |
Series |
variable-order, large folios for anonymous memory
|
|
Commit Message
Ryan Roberts
June 26, 2023, 5:14 p.m. UTC
Like folio_add_new_anon_rmap() but batch-rmaps a range of pages
belonging to a folio, for effciency savings. All pages are accounted as
small pages.
Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
---
include/linux/rmap.h | 2 ++
mm/rmap.c | 43 +++++++++++++++++++++++++++++++++++++++++++
2 files changed, 45 insertions(+)
Comments
On Mon, Jun 26, 2023 at 11:14 AM Ryan Roberts <ryan.roberts@arm.com> wrote: > > Like folio_add_new_anon_rmap() but batch-rmaps a range of pages > belonging to a folio, for effciency savings. All pages are accounted as > small pages. > > Signed-off-by: Ryan Roberts <ryan.roberts@arm.com> > --- > include/linux/rmap.h | 2 ++ > mm/rmap.c | 43 +++++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 45 insertions(+) > > diff --git a/include/linux/rmap.h b/include/linux/rmap.h > index a3825ce81102..15433a3d0cbf 100644 > --- a/include/linux/rmap.h > +++ b/include/linux/rmap.h > @@ -196,6 +196,8 @@ void page_add_new_anon_rmap(struct page *, struct vm_area_struct *, > unsigned long address); > void folio_add_new_anon_rmap(struct folio *, struct vm_area_struct *, > unsigned long address); > +void folio_add_new_anon_rmap_range(struct folio *folio, struct page *page, > + int nr, struct vm_area_struct *vma, unsigned long address); We should update folio_add_new_anon_rmap() to support large() && !folio_test_pmd_mappable() folios instead. I double checked all places currently using folio_add_new_anon_rmap(), and as expected, none actually allocates large() && !folio_test_pmd_mappable() and maps it one by one, which makes the cases simpler, i.e., if (!large()) // the existing basepage case else if (!folio_test_pmd_mappable()) // our new case else // the existing THP case > void page_add_file_rmap(struct page *, struct vm_area_struct *, > bool compound); > void folio_add_file_rmap_range(struct folio *, struct page *, unsigned int nr, > diff --git a/mm/rmap.c b/mm/rmap.c > index 1d8369549424..4050bcea7ae7 100644 > --- a/mm/rmap.c > +++ b/mm/rmap.c > @@ -1305,6 +1305,49 @@ void folio_add_new_anon_rmap(struct folio *folio, struct vm_area_struct *vma, > __page_set_anon_rmap(folio, &folio->page, vma, address, 1); > } > > +/** > + * folio_add_new_anon_rmap_range - Add mapping to a set of pages within a new > + * anonymous potentially large folio. > + * @folio: The folio containing the pages to be mapped > + * @page: First page in the folio to be mapped > + * @nr: Number of pages to be mapped > + * @vma: the vm area in which the mapping is added > + * @address: the user virtual address of the first page to be mapped > + * > + * Like folio_add_new_anon_rmap() but batch-maps a range of pages within a folio > + * using non-THP accounting. Like folio_add_new_anon_rmap(), the inc-and-test is > + * bypassed and the folio does not have to be locked. All pages in the folio are > + * individually accounted. > + * > + * As the folio is new, it's assumed to be mapped exclusively by a single > + * process. > + */ > +void folio_add_new_anon_rmap_range(struct folio *folio, struct page *page, > + int nr, struct vm_area_struct *vma, unsigned long address) > +{ > + int i; > + > + VM_BUG_ON_VMA(address < vma->vm_start || > + address + (nr << PAGE_SHIFT) > vma->vm_end, vma); BTW, VM_BUG_ON* shouldn't be used in new code: Documentation/process/coding-style.rst
On 27/06/2023 08:08, Yu Zhao wrote: > On Mon, Jun 26, 2023 at 11:14 AM Ryan Roberts <ryan.roberts@arm.com> wrote: >> >> Like folio_add_new_anon_rmap() but batch-rmaps a range of pages >> belonging to a folio, for effciency savings. All pages are accounted as >> small pages. >> >> Signed-off-by: Ryan Roberts <ryan.roberts@arm.com> >> --- >> include/linux/rmap.h | 2 ++ >> mm/rmap.c | 43 +++++++++++++++++++++++++++++++++++++++++++ >> 2 files changed, 45 insertions(+) >> >> diff --git a/include/linux/rmap.h b/include/linux/rmap.h >> index a3825ce81102..15433a3d0cbf 100644 >> --- a/include/linux/rmap.h >> +++ b/include/linux/rmap.h >> @@ -196,6 +196,8 @@ void page_add_new_anon_rmap(struct page *, struct vm_area_struct *, >> unsigned long address); >> void folio_add_new_anon_rmap(struct folio *, struct vm_area_struct *, >> unsigned long address); >> +void folio_add_new_anon_rmap_range(struct folio *folio, struct page *page, >> + int nr, struct vm_area_struct *vma, unsigned long address); > > We should update folio_add_new_anon_rmap() to support large() && > !folio_test_pmd_mappable() folios instead. > > I double checked all places currently using folio_add_new_anon_rmap(), > and as expected, none actually allocates large() && > !folio_test_pmd_mappable() and maps it one by one, which makes the > cases simpler, i.e., > if (!large()) > // the existing basepage case > else if (!folio_test_pmd_mappable()) > // our new case > else > // the existing THP case I don't have a strong opinion either way. Happy to go with this suggestion. But the reason I did it as a new function was because I was following the pattern in [1] which adds a new folio_add_file_rmap_range() function. [1] https://lore.kernel.org/linux-mm/20230315051444.3229621-35-willy@infradead.org/ > >> void page_add_file_rmap(struct page *, struct vm_area_struct *, >> bool compound); >> void folio_add_file_rmap_range(struct folio *, struct page *, unsigned int nr, >> diff --git a/mm/rmap.c b/mm/rmap.c >> index 1d8369549424..4050bcea7ae7 100644 >> --- a/mm/rmap.c >> +++ b/mm/rmap.c >> @@ -1305,6 +1305,49 @@ void folio_add_new_anon_rmap(struct folio *folio, struct vm_area_struct *vma, >> __page_set_anon_rmap(folio, &folio->page, vma, address, 1); >> } >> >> +/** >> + * folio_add_new_anon_rmap_range - Add mapping to a set of pages within a new >> + * anonymous potentially large folio. >> + * @folio: The folio containing the pages to be mapped >> + * @page: First page in the folio to be mapped >> + * @nr: Number of pages to be mapped >> + * @vma: the vm area in which the mapping is added >> + * @address: the user virtual address of the first page to be mapped >> + * >> + * Like folio_add_new_anon_rmap() but batch-maps a range of pages within a folio >> + * using non-THP accounting. Like folio_add_new_anon_rmap(), the inc-and-test is >> + * bypassed and the folio does not have to be locked. All pages in the folio are >> + * individually accounted. >> + * >> + * As the folio is new, it's assumed to be mapped exclusively by a single >> + * process. >> + */ >> +void folio_add_new_anon_rmap_range(struct folio *folio, struct page *page, >> + int nr, struct vm_area_struct *vma, unsigned long address) >> +{ >> + int i; >> + >> + VM_BUG_ON_VMA(address < vma->vm_start || >> + address + (nr << PAGE_SHIFT) > vma->vm_end, vma); > > BTW, VM_BUG_ON* shouldn't be used in new code: > Documentation/process/coding-style.rst Thanks, sorry about that. Was copy-pasting from folio_add_new_anon_rmap().
On 6/27/23 15:08, Yu Zhao wrote: > On Mon, Jun 26, 2023 at 11:14 AM Ryan Roberts <ryan.roberts@arm.com> wrote: >> >> Like folio_add_new_anon_rmap() but batch-rmaps a range of pages >> belonging to a folio, for effciency savings. All pages are accounted as >> small pages. >> >> Signed-off-by: Ryan Roberts <ryan.roberts@arm.com> >> --- >> include/linux/rmap.h | 2 ++ >> mm/rmap.c | 43 +++++++++++++++++++++++++++++++++++++++++++ >> 2 files changed, 45 insertions(+) >> >> diff --git a/include/linux/rmap.h b/include/linux/rmap.h >> index a3825ce81102..15433a3d0cbf 100644 >> --- a/include/linux/rmap.h >> +++ b/include/linux/rmap.h >> @@ -196,6 +196,8 @@ void page_add_new_anon_rmap(struct page *, struct vm_area_struct *, >> unsigned long address); >> void folio_add_new_anon_rmap(struct folio *, struct vm_area_struct *, >> unsigned long address); >> +void folio_add_new_anon_rmap_range(struct folio *folio, struct page *page, >> + int nr, struct vm_area_struct *vma, unsigned long address); > > We should update folio_add_new_anon_rmap() to support large() && > !folio_test_pmd_mappable() folios instead. > > I double checked all places currently using folio_add_new_anon_rmap(), > and as expected, none actually allocates large() && > !folio_test_pmd_mappable() and maps it one by one, which makes the > cases simpler, i.e., > if (!large()) > // the existing basepage case > else if (!folio_test_pmd_mappable()) > // our new case > else > // the existing THP case I suppose we can merge the new case and existing THP case. Regards Yin, Fengwei > >> void page_add_file_rmap(struct page *, struct vm_area_struct *, >> bool compound); >> void folio_add_file_rmap_range(struct folio *, struct page *, unsigned int nr, >> diff --git a/mm/rmap.c b/mm/rmap.c >> index 1d8369549424..4050bcea7ae7 100644 >> --- a/mm/rmap.c >> +++ b/mm/rmap.c >> @@ -1305,6 +1305,49 @@ void folio_add_new_anon_rmap(struct folio *folio, struct vm_area_struct *vma, >> __page_set_anon_rmap(folio, &folio->page, vma, address, 1); >> } >> >> +/** >> + * folio_add_new_anon_rmap_range - Add mapping to a set of pages within a new >> + * anonymous potentially large folio. >> + * @folio: The folio containing the pages to be mapped >> + * @page: First page in the folio to be mapped >> + * @nr: Number of pages to be mapped >> + * @vma: the vm area in which the mapping is added >> + * @address: the user virtual address of the first page to be mapped >> + * >> + * Like folio_add_new_anon_rmap() but batch-maps a range of pages within a folio >> + * using non-THP accounting. Like folio_add_new_anon_rmap(), the inc-and-test is >> + * bypassed and the folio does not have to be locked. All pages in the folio are >> + * individually accounted. >> + * >> + * As the folio is new, it's assumed to be mapped exclusively by a single >> + * process. >> + */ >> +void folio_add_new_anon_rmap_range(struct folio *folio, struct page *page, >> + int nr, struct vm_area_struct *vma, unsigned long address) >> +{ >> + int i; >> + >> + VM_BUG_ON_VMA(address < vma->vm_start || >> + address + (nr << PAGE_SHIFT) > vma->vm_end, vma); > > BTW, VM_BUG_ON* shouldn't be used in new code: > Documentation/process/coding-style.rst
On 6/27/23 16:09, Ryan Roberts wrote: > On 27/06/2023 08:08, Yu Zhao wrote: >> On Mon, Jun 26, 2023 at 11:14 AM Ryan Roberts <ryan.roberts@arm.com> wrote: >>> >>> Like folio_add_new_anon_rmap() but batch-rmaps a range of pages >>> belonging to a folio, for effciency savings. All pages are accounted as >>> small pages. >>> >>> Signed-off-by: Ryan Roberts <ryan.roberts@arm.com> >>> --- >>> include/linux/rmap.h | 2 ++ >>> mm/rmap.c | 43 +++++++++++++++++++++++++++++++++++++++++++ >>> 2 files changed, 45 insertions(+) >>> >>> diff --git a/include/linux/rmap.h b/include/linux/rmap.h >>> index a3825ce81102..15433a3d0cbf 100644 >>> --- a/include/linux/rmap.h >>> +++ b/include/linux/rmap.h >>> @@ -196,6 +196,8 @@ void page_add_new_anon_rmap(struct page *, struct vm_area_struct *, >>> unsigned long address); >>> void folio_add_new_anon_rmap(struct folio *, struct vm_area_struct *, >>> unsigned long address); >>> +void folio_add_new_anon_rmap_range(struct folio *folio, struct page *page, >>> + int nr, struct vm_area_struct *vma, unsigned long address); >> >> We should update folio_add_new_anon_rmap() to support large() && >> !folio_test_pmd_mappable() folios instead. >> >> I double checked all places currently using folio_add_new_anon_rmap(), >> and as expected, none actually allocates large() && >> !folio_test_pmd_mappable() and maps it one by one, which makes the >> cases simpler, i.e., >> if (!large()) >> // the existing basepage case >> else if (!folio_test_pmd_mappable()) >> // our new case >> else >> // the existing THP case > > I don't have a strong opinion either way. Happy to go with this suggestion. But > the reason I did it as a new function was because I was following the pattern in > [1] which adds a new folio_add_file_rmap_range() function. > > [1] https://lore.kernel.org/linux-mm/20230315051444.3229621-35-willy@infradead.org/ Oh. There is different here: For page cache, large folio could be created by previous file access. But later file access by other process just need map partial large folio. In this case, we need _range for filemap. But for anonymous, I suppose we always map whole folio in. So I agree with Yu. We don't need _range for folio_add_new_anon_rmap(). Thanks. Regards Yin, Fengwei > > >> >>> void page_add_file_rmap(struct page *, struct vm_area_struct *, >>> bool compound); >>> void folio_add_file_rmap_range(struct folio *, struct page *, unsigned int nr, >>> diff --git a/mm/rmap.c b/mm/rmap.c >>> index 1d8369549424..4050bcea7ae7 100644 >>> --- a/mm/rmap.c >>> +++ b/mm/rmap.c >>> @@ -1305,6 +1305,49 @@ void folio_add_new_anon_rmap(struct folio *folio, struct vm_area_struct *vma, >>> __page_set_anon_rmap(folio, &folio->page, vma, address, 1); >>> } >>> >>> +/** >>> + * folio_add_new_anon_rmap_range - Add mapping to a set of pages within a new >>> + * anonymous potentially large folio. >>> + * @folio: The folio containing the pages to be mapped >>> + * @page: First page in the folio to be mapped >>> + * @nr: Number of pages to be mapped >>> + * @vma: the vm area in which the mapping is added >>> + * @address: the user virtual address of the first page to be mapped >>> + * >>> + * Like folio_add_new_anon_rmap() but batch-maps a range of pages within a folio >>> + * using non-THP accounting. Like folio_add_new_anon_rmap(), the inc-and-test is >>> + * bypassed and the folio does not have to be locked. All pages in the folio are >>> + * individually accounted. >>> + * >>> + * As the folio is new, it's assumed to be mapped exclusively by a single >>> + * process. >>> + */ >>> +void folio_add_new_anon_rmap_range(struct folio *folio, struct page *page, >>> + int nr, struct vm_area_struct *vma, unsigned long address) >>> +{ >>> + int i; >>> + >>> + VM_BUG_ON_VMA(address < vma->vm_start || >>> + address + (nr << PAGE_SHIFT) > vma->vm_end, vma); >> >> BTW, VM_BUG_ON* shouldn't be used in new code: >> Documentation/process/coding-style.rst > > Thanks, sorry about that. Was copy-pasting from folio_add_new_anon_rmap(). >
On 28/06/2023 03:20, Yin Fengwei wrote: > > > On 6/27/23 16:09, Ryan Roberts wrote: >> On 27/06/2023 08:08, Yu Zhao wrote: >>> On Mon, Jun 26, 2023 at 11:14 AM Ryan Roberts <ryan.roberts@arm.com> wrote: >>>> >>>> Like folio_add_new_anon_rmap() but batch-rmaps a range of pages >>>> belonging to a folio, for effciency savings. All pages are accounted as >>>> small pages. >>>> >>>> Signed-off-by: Ryan Roberts <ryan.roberts@arm.com> >>>> --- >>>> include/linux/rmap.h | 2 ++ >>>> mm/rmap.c | 43 +++++++++++++++++++++++++++++++++++++++++++ >>>> 2 files changed, 45 insertions(+) >>>> >>>> diff --git a/include/linux/rmap.h b/include/linux/rmap.h >>>> index a3825ce81102..15433a3d0cbf 100644 >>>> --- a/include/linux/rmap.h >>>> +++ b/include/linux/rmap.h >>>> @@ -196,6 +196,8 @@ void page_add_new_anon_rmap(struct page *, struct vm_area_struct *, >>>> unsigned long address); >>>> void folio_add_new_anon_rmap(struct folio *, struct vm_area_struct *, >>>> unsigned long address); >>>> +void folio_add_new_anon_rmap_range(struct folio *folio, struct page *page, >>>> + int nr, struct vm_area_struct *vma, unsigned long address); >>> >>> We should update folio_add_new_anon_rmap() to support large() && >>> !folio_test_pmd_mappable() folios instead. >>> >>> I double checked all places currently using folio_add_new_anon_rmap(), >>> and as expected, none actually allocates large() && >>> !folio_test_pmd_mappable() and maps it one by one, which makes the >>> cases simpler, i.e., >>> if (!large()) >>> // the existing basepage case >>> else if (!folio_test_pmd_mappable()) >>> // our new case >>> else >>> // the existing THP case >> >> I don't have a strong opinion either way. Happy to go with this suggestion. But >> the reason I did it as a new function was because I was following the pattern in >> [1] which adds a new folio_add_file_rmap_range() function. >> >> [1] https://lore.kernel.org/linux-mm/20230315051444.3229621-35-willy@infradead.org/ > Oh. There is different here: > For page cache, large folio could be created by previous file access. But later > file access by other process just need map partial large folio. In this case, we need > _range for filemap. > > But for anonymous, I suppose we always map whole folio in. So I agree with Yu. We > don't need _range for folio_add_new_anon_rmap(). Thanks. Yes that makes sense - thanks. I'll merge the new case into folio_add_new_anon_rmap() for v2. > > > Regards > Yin, Fengwei > >> >> >>> >>>> void page_add_file_rmap(struct page *, struct vm_area_struct *, >>>> bool compound); >>>> void folio_add_file_rmap_range(struct folio *, struct page *, unsigned int nr, >>>> diff --git a/mm/rmap.c b/mm/rmap.c >>>> index 1d8369549424..4050bcea7ae7 100644 >>>> --- a/mm/rmap.c >>>> +++ b/mm/rmap.c >>>> @@ -1305,6 +1305,49 @@ void folio_add_new_anon_rmap(struct folio *folio, struct vm_area_struct *vma, >>>> __page_set_anon_rmap(folio, &folio->page, vma, address, 1); >>>> } >>>> >>>> +/** >>>> + * folio_add_new_anon_rmap_range - Add mapping to a set of pages within a new >>>> + * anonymous potentially large folio. >>>> + * @folio: The folio containing the pages to be mapped >>>> + * @page: First page in the folio to be mapped >>>> + * @nr: Number of pages to be mapped >>>> + * @vma: the vm area in which the mapping is added >>>> + * @address: the user virtual address of the first page to be mapped >>>> + * >>>> + * Like folio_add_new_anon_rmap() but batch-maps a range of pages within a folio >>>> + * using non-THP accounting. Like folio_add_new_anon_rmap(), the inc-and-test is >>>> + * bypassed and the folio does not have to be locked. All pages in the folio are >>>> + * individually accounted. >>>> + * >>>> + * As the folio is new, it's assumed to be mapped exclusively by a single >>>> + * process. >>>> + */ >>>> +void folio_add_new_anon_rmap_range(struct folio *folio, struct page *page, >>>> + int nr, struct vm_area_struct *vma, unsigned long address) >>>> +{ >>>> + int i; >>>> + >>>> + VM_BUG_ON_VMA(address < vma->vm_start || >>>> + address + (nr << PAGE_SHIFT) > vma->vm_end, vma); >>> >>> BTW, VM_BUG_ON* shouldn't be used in new code: >>> Documentation/process/coding-style.rst >> >> Thanks, sorry about that. Was copy-pasting from folio_add_new_anon_rmap(). >>
diff --git a/include/linux/rmap.h b/include/linux/rmap.h index a3825ce81102..15433a3d0cbf 100644 --- a/include/linux/rmap.h +++ b/include/linux/rmap.h @@ -196,6 +196,8 @@ void page_add_new_anon_rmap(struct page *, struct vm_area_struct *, unsigned long address); void folio_add_new_anon_rmap(struct folio *, struct vm_area_struct *, unsigned long address); +void folio_add_new_anon_rmap_range(struct folio *folio, struct page *page, + int nr, struct vm_area_struct *vma, unsigned long address); void page_add_file_rmap(struct page *, struct vm_area_struct *, bool compound); void folio_add_file_rmap_range(struct folio *, struct page *, unsigned int nr, diff --git a/mm/rmap.c b/mm/rmap.c index 1d8369549424..4050bcea7ae7 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1305,6 +1305,49 @@ void folio_add_new_anon_rmap(struct folio *folio, struct vm_area_struct *vma, __page_set_anon_rmap(folio, &folio->page, vma, address, 1); } +/** + * folio_add_new_anon_rmap_range - Add mapping to a set of pages within a new + * anonymous potentially large folio. + * @folio: The folio containing the pages to be mapped + * @page: First page in the folio to be mapped + * @nr: Number of pages to be mapped + * @vma: the vm area in which the mapping is added + * @address: the user virtual address of the first page to be mapped + * + * Like folio_add_new_anon_rmap() but batch-maps a range of pages within a folio + * using non-THP accounting. Like folio_add_new_anon_rmap(), the inc-and-test is + * bypassed and the folio does not have to be locked. All pages in the folio are + * individually accounted. + * + * As the folio is new, it's assumed to be mapped exclusively by a single + * process. + */ +void folio_add_new_anon_rmap_range(struct folio *folio, struct page *page, + int nr, struct vm_area_struct *vma, unsigned long address) +{ + int i; + + VM_BUG_ON_VMA(address < vma->vm_start || + address + (nr << PAGE_SHIFT) > vma->vm_end, vma); + __folio_set_swapbacked(folio); + + if (folio_test_large(folio)) { + /* increment count (starts at 0) */ + atomic_set(&folio->_nr_pages_mapped, nr); + } + + for (i = 0; i < nr; i++) { + /* increment count (starts at -1) */ + atomic_set(&page->_mapcount, 0); + __page_set_anon_rmap(folio, page, vma, address, 1); + page++; + address += PAGE_SIZE; + } + + __lruvec_stat_mod_folio(folio, NR_ANON_MAPPED, nr); + +} + /** * folio_add_file_rmap_range - add pte mapping to page range of a folio * @folio: The folio to add the mapping to