[v4,0/7] Split a folio to any lower order folios

  From: Zi Yan <ziy@nvidia.com>

Hi all,

File folio supports any order and multi-size THP is upstreamed[1], so both
file and anonymous folios can be >0 order. Currently, split_huge_page()
only splits a huge page to order-0 pages, but splitting to orders higher than
0 is going to better utilize large folios. In addition, Large Block
Sizes in XFS support would benefit from it[2]. This patchset adds support for
splitting a large folio to any lower order folios and uses it during file
folio truncate operations.

For Patch 6, Hugh did not like my approach to minimize the number of
folios for truncate[3]. I would like to get more feedback, especially
from FS people, on it to decide whether to keep it or not.

The patchset is on top of mm-everything-2024-02-13-01-26.

Changelog 
===

Since v3
---
1. Excluded shmem folios and pagecache folios without FS support from
splitting to any order (per Hugh Dickins).
2. Allowed splitting anonymous large folio to any lower order since
multi-size THP is upstreamed.
3. Adapted selftests code to new framework.

Since v2
---
1. Fixed an issue in __split_page_owner() introduced during my rebase

Since v1
---
1. Changed split_page_memcg() and split_page_owner() parameter to use order
2. Used folio_test_pmd_mappable() in place of the equivalent code

Details
===

* Patch 1 changes split_page_memcg() to use order instead of nr_pages
* Patch 2 changes split_page_owner() to use order instead of nr_pages
* Patch 3 and 4 add new_order parameter split_page_memcg() and
  split_page_owner() and prepare for upcoming changes.
* Patch 5 adds split_huge_page_to_list_to_order() to split a huge page
  to any lower order. The original split_huge_page_to_list() calls
  split_huge_page_to_list_to_order() with new_order = 0.
* Patch 6 uses split_huge_page_to_list_to_order() in large pagecache folio
  truncation instead of split the large folio all the way down to order-0.
* Patch 7 adds a test API to debugfs and test cases in
  split_huge_page_test selftests.

Comments and/or suggestions are welcome.

[1] https://lore.kernel.org/all/20231207161211.2374093-1-ryan.roberts@arm.com/
[2] https://lore.kernel.org/linux-mm/qzbcjn4gcyxla4gwuj6smlnwknz2wvo5wrjctin6eengjfqjei@lzkxv3iy6bol/
[3] https://lore.kernel.org/linux-mm/9dd96da-efa2-5123-20d4-4992136ef3ad@google.com/

Zi Yan (7):
  mm/memcg: use order instead of nr in split_page_memcg()
  mm/page_owner: use order instead of nr in split_page_owner()
  mm: memcg: make memcg huge page split support any order split.
  mm: page_owner: add support for splitting to any order in split
    page_owner.
  mm: thp: split huge page to any lower order pages (except order-1).
  mm: truncate: split huge page cache page to a non-zero order if
    possible.
  mm: huge_memory: enable debugfs to split huge pages to any order.

 include/linux/huge_mm.h                       |  21 +-
 include/linux/memcontrol.h                    |   4 +-
 include/linux/page_owner.h                    |  10 +-
 mm/huge_memory.c                              | 149 +++++++++---
 mm/memcontrol.c                               |  10 +-
 mm/page_alloc.c                               |   8 +-
 mm/page_owner.c                               |   8 +-
 mm/truncate.c                                 |  21 +-
 .../selftests/mm/split_huge_page_test.c       | 223 +++++++++++++++++-
 9 files changed, 382 insertions(+), 72 deletions(-)
  

On 13.02.24 22:55, Zi Yan wrote:
> From: Zi Yan <ziy@nvidia.com>
> 
> Hi all,
> 
> File folio supports any order and multi-size THP is upstreamed[1], so both
> file and anonymous folios can be >0 order. Currently, split_huge_page()
> only splits a huge page to order-0 pages, but splitting to orders higher than
> 0 is going to better utilize large folios. In addition, Large Block
> Sizes in XFS support would benefit from it[2]. This patchset adds support for
> splitting a large folio to any lower order folios and uses it during file
> folio truncate operations.
> 
> For Patch 6, Hugh did not like my approach to minimize the number of
> folios for truncate[3]. I would like to get more feedback, especially
> from FS people, on it to decide whether to keep it or not.

I'm curious, would it make sense to exclude the "more" controversial 
parts (i.e., patch #6) for now, and focus on the XFS use case only?
  

On 13 Feb 2024, at 17:21, David Hildenbrand wrote:

> On 13.02.24 22:55, Zi Yan wrote:
>> From: Zi Yan <ziy@nvidia.com>
>>
>> Hi all,
>>
>> File folio supports any order and multi-size THP is upstreamed[1], so both
>> file and anonymous folios can be >0 order. Currently, split_huge_page()
>> only splits a huge page to order-0 pages, but splitting to orders higher than
>> 0 is going to better utilize large folios. In addition, Large Block
>> Sizes in XFS support would benefit from it[2]. This patchset adds support for
>> splitting a large folio to any lower order folios and uses it during file
>> folio truncate operations.
>>
>> For Patch 6, Hugh did not like my approach to minimize the number of
>> folios for truncate[3]. I would like to get more feedback, especially
>> from FS people, on it to decide whether to keep it or not.
>
> I'm curious, would it make sense to exclude the "more" controversial parts (i.e., patch #6) for now, and focus on the XFS use case only?

Sure. Patch 6 was there to make use of split_huge_page_to_list_to_order().
Now we have multi-size THP and XFS use cases, it can be dropped.

--
Best Regards,
Yan, Zi
  

On 13/02/2024 22:31, Zi Yan wrote:
> On 13 Feb 2024, at 17:21, David Hildenbrand wrote:
> 
>> On 13.02.24 22:55, Zi Yan wrote:
>>> From: Zi Yan <ziy@nvidia.com>
>>>
>>> Hi all,
>>>
>>> File folio supports any order and multi-size THP is upstreamed[1], so both
>>> file and anonymous folios can be >0 order. Currently, split_huge_page()
>>> only splits a huge page to order-0 pages, but splitting to orders higher than
>>> 0 is going to better utilize large folios. In addition, Large Block
>>> Sizes in XFS support would benefit from it[2]. This patchset adds support for
>>> splitting a large folio to any lower order folios and uses it during file
>>> folio truncate operations.
>>>
>>> For Patch 6, Hugh did not like my approach to minimize the number of
>>> folios for truncate[3]. I would like to get more feedback, especially
>>> from FS people, on it to decide whether to keep it or not.
>>
>> I'm curious, would it make sense to exclude the "more" controversial parts (i.e., patch #6) for now, and focus on the XFS use case only?
> 
> Sure. Patch 6 was there to make use of split_huge_page_to_list_to_order().
> Now we have multi-size THP and XFS use cases, it can be dropped.

What are your plans for how to determine when to split THP and to what order? I
don't see anything in this series that would split anon THP to non-zero order?

We have talked about using hints from user space in the past (e.g.  mremap,
munmap, madvise, etc). But chrome has a use case where it temporarily mprotects
a single (4K) page as part of garbage collection (IIRC). If you eagerly split on
that hint, you will have lost the benefits of the large folio when it later
mprotects back to the original setting.

I guess David will suggest this would be a good use case for the khugepaged-lite
machanism we have been talking about. I dunno - it seems wasteful to split then
collapse again.

Or perhaps you're considering doing something clever in deferred split?

> 
> --
> Best Regards,
> Yan, Zi
  

On 14.02.24 11:50, Ryan Roberts wrote:
> On 13/02/2024 22:31, Zi Yan wrote:
>> On 13 Feb 2024, at 17:21, David Hildenbrand wrote:
>>
>>> On 13.02.24 22:55, Zi Yan wrote:
>>>> From: Zi Yan <ziy@nvidia.com>
>>>>
>>>> Hi all,
>>>>
>>>> File folio supports any order and multi-size THP is upstreamed[1], so both
>>>> file and anonymous folios can be >0 order. Currently, split_huge_page()
>>>> only splits a huge page to order-0 pages, but splitting to orders higher than
>>>> 0 is going to better utilize large folios. In addition, Large Block
>>>> Sizes in XFS support would benefit from it[2]. This patchset adds support for
>>>> splitting a large folio to any lower order folios and uses it during file
>>>> folio truncate operations.
>>>>
>>>> For Patch 6, Hugh did not like my approach to minimize the number of
>>>> folios for truncate[3]. I would like to get more feedback, especially
>>>> from FS people, on it to decide whether to keep it or not.
>>>
>>> I'm curious, would it make sense to exclude the "more" controversial parts (i.e., patch #6) for now, and focus on the XFS use case only?
>>
>> Sure. Patch 6 was there to make use of split_huge_page_to_list_to_order().
>> Now we have multi-size THP and XFS use cases, it can be dropped.
> 
> What are your plans for how to determine when to split THP and to what order? I
> don't see anything in this series that would split anon THP to non-zero order?
> 
> We have talked about using hints from user space in the past (e.g.  mremap,
> munmap, madvise, etc). But chrome has a use case where it temporarily mprotects
> a single (4K) page as part of garbage collection (IIRC). If you eagerly split on
> that hint, you will have lost the benefits of the large folio when it later
> mprotects back to the original setting.

Not only that, splitting will make some of these operations more 
expensive, possibly with no actual benefit.

> 
> I guess David will suggest this would be a good use case for the khugepaged-lite
> machanism we have been talking about. I dunno - it seems wasteful to split then
> collapse again.

I agree. mprotect() and even madvise(), ... might not be good candidates 
for splitting. mremap() likely is, if the folio is mapped exclusively. 
MADV_DONTNEED/munmap()/mlock() might be good candidates (again, if 
mapped exclusively). This will need a lot of thought I'm afraid (as you 
say, deferred splitting is another example).
  

On 14 Feb 2024, at 5:55, David Hildenbrand wrote:

> On 14.02.24 11:50, Ryan Roberts wrote:
>> On 13/02/2024 22:31, Zi Yan wrote:
>>> On 13 Feb 2024, at 17:21, David Hildenbrand wrote:
>>>
>>>> On 13.02.24 22:55, Zi Yan wrote:
>>>>> From: Zi Yan <ziy@nvidia.com>
>>>>>
>>>>> Hi all,
>>>>>
>>>>> File folio supports any order and multi-size THP is upstreamed[1], so both
>>>>> file and anonymous folios can be >0 order. Currently, split_huge_page()
>>>>> only splits a huge page to order-0 pages, but splitting to orders higher than
>>>>> 0 is going to better utilize large folios. In addition, Large Block
>>>>> Sizes in XFS support would benefit from it[2]. This patchset adds support for
>>>>> splitting a large folio to any lower order folios and uses it during file
>>>>> folio truncate operations.
>>>>>
>>>>> For Patch 6, Hugh did not like my approach to minimize the number of
>>>>> folios for truncate[3]. I would like to get more feedback, especially
>>>>> from FS people, on it to decide whether to keep it or not.
>>>>
>>>> I'm curious, would it make sense to exclude the "more" controversial parts (i.e., patch #6) for now, and focus on the XFS use case only?
>>>
>>> Sure. Patch 6 was there to make use of split_huge_page_to_list_to_order().
>>> Now we have multi-size THP and XFS use cases, it can be dropped.
>>
>> What are your plans for how to determine when to split THP and to what order? I
>> don't see anything in this series that would split anon THP to non-zero order?
>>
>> We have talked about using hints from user space in the past (e.g.  mremap,
>> munmap, madvise, etc). But chrome has a use case where it temporarily mprotects
>> a single (4K) page as part of garbage collection (IIRC). If you eagerly split on
>> that hint, you will have lost the benefits of the large folio when it later
>> mprotects back to the original setting.
>
> Not only that, splitting will make some of these operations more expensive, possibly with no actual benefit.
>
>>
>> I guess David will suggest this would be a good use case for the khugepaged-lite
>> machanism we have been talking about. I dunno - it seems wasteful to split then
>> collapse again.
>
> I agree. mprotect() and even madvise(), ... might not be good candidates for splitting. mremap() likely is, if the folio is mapped exclusively. MADV_DONTNEED/munmap()/mlock() might be good candidates (again, if mapped exclusively). This will need a lot of thought I'm afraid (as you say, deferred splitting is another example).

My initial use was for splitting 1GB THP to 2MB THP, but 1GB THP is not upstream
yet. So for now, this might only be used by XFS. For anonymous large folios,
we will use this when we find a justified use case. What I can think of is
when a PMD-mapped THP happens to be split and the resulting order can be a HW/SW
favored order, like 64KB or 32KB (to be able to use contig PTE), we split
to that order, otherwise, we still split to order-0.

--
Best Regards,
Yan, Zi
  

Hi Pankaj,

On 13 Feb 2024, at 16:55, Zi Yan wrote:

> From: Zi Yan <ziy@nvidia.com>
>
> Hi all,
>
> File folio supports any order and multi-size THP is upstreamed[1], so both
> file and anonymous folios can be >0 order. Currently, split_huge_page()
> only splits a huge page to order-0 pages, but splitting to orders higher than
> 0 is going to better utilize large folios. In addition, Large Block
> Sizes in XFS support would benefit from it[2]. This patchset adds support for

Just talked to Matthew about his order-1 pagecache folio, I am planning to
grab that into this one, so that I can remove the restriction in my patches
and you guys do not need to do that in your patchset. Let me know if it works
for you.

--
Best Regards,
Yan, Zi
  

On 16 Feb 2024, at 5:06, Pankaj Raghav (Samsung) wrote:

> Hi Zi Yan,
>
> On Tue, Feb 13, 2024 at 04:55:13PM -0500, Zi Yan wrote:
>> From: Zi Yan <ziy@nvidia.com>
>>
>> Hi all,
>>
>> File folio supports any order and multi-size THP is upstreamed[1], so both
>> file and anonymous folios can be >0 order. Currently, split_huge_page()
>> only splits a huge page to order-0 pages, but splitting to orders higher than
>> 0 is going to better utilize large folios. In addition, Large Block
>> Sizes in XFS support would benefit from it[2]. This patchset adds support for
>> splitting a large folio to any lower order folios and uses it during file
>> folio truncate operations.
>
> I added your patches on top of my patches, but removed patch 6 and I
> added this instead:
>
> diff --git a/mm/truncate.c b/mm/truncate.c
> index 725b150e47ac..dd07e2e327a8 100644
> --- a/mm/truncate.c
> +++ b/mm/truncate.c
> @@ -239,7 +239,8 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end)
>                 folio_invalidate(folio, offset, length);
>         if (!folio_test_large(folio))
>                 return true;
> -       if (split_folio(folio) == 0)
> +       if (split_folio_to_order(folio,
> +                                mapping_min_folio_order(folio->mapping)) == 0)
>                 return true;
>         if (folio_test_dirty(folio))
>                 return false;
>
> I ran genric/476 fstest[1] with SOAK_DURATION set to 360 seconds. This
> test uses fstress to do a lot of writes, truncate operations, etc. I ran
> this on XFS with **64k block size on a 4k page size system**.
>
> I recorded the vm event for split page and this was the result I got:
>
> Before your patches:
> root@debian:~/xfstests# cat /proc/vmstat  | grep split
> thp_split_page 0
> thp_split_page_failed 5819
>
> After your patches:
> root@debian:~/xfstests# cat /proc/vmstat  | grep split
> thp_split_page 5846
> thp_split_page_failed 20
>
> Your patch series definitely helps with splitting the folios while still
> maintaining the min_folio_order that LBS requires.

Sounds great! Thanks for testing.

>
> We are still discussing how to quantify this benefit in terms of some
> metric with this support. If you have some ideas here, let me know.

From my understanding, the benefit will come from that page cache folio
size is bigger with LBS (plus this patchset) after truncate. I assume any
benchmark testing read/write throughput after truncate operations might
be helpful.

>
> I will run the whole xfstests tonight to check for any regressions.

Can you use the update patches from: https://github.com/x-y-z/linux-1gb-thp/tree/split_thp_to_any_order_v5-mm-everything-2024-02-16-01-35? It contains
changes and fixes based on the feedback from this version. I am planning
to send this new version out soon.

>
> --
> Pankaj
>
> [1] https://github.com/kdave/xfstests/blob/master/tests/generic/476


--
Best Regards,
Yan, Zi