Message ID | 20230727204624.1942372-5-usama.arif@bytedance.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:918b:0:b0:3e4:2afc:c1 with SMTP id s11csp14035vqg; Thu, 27 Jul 2023 14:11:03 -0700 (PDT) X-Google-Smtp-Source: APBJJlGE+rvRpFsOL042M+AsQdRzI9brDIyScqBTX8LwKRspiPWnH3jUfu/WOEnQb/o/J63yqprB X-Received: by 2002:a17:902:ab5a:b0:1b8:ae12:5610 with SMTP id ij26-20020a170902ab5a00b001b8ae125610mr443664plb.7.1690492263549; Thu, 27 Jul 2023 14:11:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1690492263; cv=none; d=google.com; s=arc-20160816; b=OSorLCkltWcIGc6wFkfPda+C6ExujIV6+2LxKFlPmJdQnLVtPdu5dB56fSkyzaIJ9H dWHumEgEbmyPqhOANnQqsWTkHCijOizfgi49ldLI1sCGMjPiTVtZPbkd3br46KuJw+yG ICGkjPVM8cRw4Xf4kaDlu0pK8RQX9i7Mpjc/fsXjTkbt9dxy5GHgZ6/9wkGQEOaCdpYX fXaYmOfqcp6NXJ74oU/7udT5WAMO3yeV0hPAcLTeDuvbjErucVnKg2uF6sagh2dbk6LI QHe9NNjTJRlg/5vb9WDUel+Ncx/1AmOLyNR1rj/GWzU7wcqglwF12wEPhYSF9v58pIL2 txwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=JQe+LnlORYwelNDP2OagFjwF2/T/1IMvjhocyHCh0HM=; fh=6ywE3cadAEHt3CqWKbi2najT+v0c7si89uhbZ0SfPzA=; b=Hduycj+0F6MrP/4fzfSAJrCgHxbi7YUKcwZ9KbCLotGAh6gMe7WNO++GpKHn0hLmfE 468LtcKC81MI63IjI4nZ0dnQe2aKXRZEOPdASolM04LcdvL1ey8dulp63rnLVhK3qK2o 5o3Vo3r3lGpQvMbsxCB5C8eJv4aWFIAox5Ao1f0MQxlN7239n3Tfi7d4ZWNRlY3asTid kwHt8k7DmQ6Agz6H0HAu1kFzaRxZ52LHzjToU9tZcxXC21Fnx2Fn2STCyJs/f/xKMD4I v2mjlJ8R19e3SaYGTsI8MATqv44mV9d5DR+plK3uVMm9Mewr2f33/dMmN0GxyeVyegFQ zZrA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=gVT8uA85; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k11-20020a170902c40b00b001aafb271d13si1926651plk.235.2023.07.27.14.10.50; Thu, 27 Jul 2023 14:11:03 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=gVT8uA85; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232849AbjG0UrN (ORCPT <rfc822;kloczko.tomasz@gmail.com> + 99 others); Thu, 27 Jul 2023 16:47:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34410 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232213AbjG0UrB (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Thu, 27 Jul 2023 16:47:01 -0400 Received: from mail-lj1-x234.google.com (mail-lj1-x234.google.com [IPv6:2a00:1450:4864:20::234]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B87852D45 for <linux-kernel@vger.kernel.org>; Thu, 27 Jul 2023 13:46:36 -0700 (PDT) Received: by mail-lj1-x234.google.com with SMTP id 38308e7fff4ca-2b701e1ca63so21959931fa.1 for <linux-kernel@vger.kernel.org>; Thu, 27 Jul 2023 13:46:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1690490795; x=1691095595; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JQe+LnlORYwelNDP2OagFjwF2/T/1IMvjhocyHCh0HM=; b=gVT8uA85Q/2RybadFL1WMGqwI8huoiPfSEh3Ga80yhJTkx2wqYddCczcvHucvrLvi3 8LyY62f1ojKZNCIJ1RJJ6AJm/H/D5kfULuGeQgjkEhp/sdQz5z18I7dL/szLmlIfJpYD u61EVQdgSt8wbKxI6DB5hsAcwBuLB2hgvn0jrCvRww9Z0lGEXVNLysQBc2Fe36p46uCT jl2OXbNIY2R4lV5NY2Ex7WgCxOm/ALG64TE59hb/8mRc6i3LDTDk1ah1xfi+hOTG8MYd 4LLgZYdnHkKk+A17Ls6+GXL7nzPHfRkZeRmwu0Uv3hqpWoCBuyANT80yrLWtGgfm1gge gtcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690490795; x=1691095595; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JQe+LnlORYwelNDP2OagFjwF2/T/1IMvjhocyHCh0HM=; b=fIwHg8/52HS2L+/h6TlO/UClI7myuMGNqms80m595MZbW/ka/rULtjlJm4N8KnXJ6m MhRGlKPptJGbH58GvKiXwHeggjCEFm95YAfcP9grwnmuc+c5qI6sYJgJhHqpa2cmRV0x l/5ZsxvA5tnDIuClAJOW54O1Ol2/Ry8ozTb+V5+J/SB+cJavogtQUv+vY0IQDCzcZZA/ 4+xfIMcAg7ABkXtU5i5bHG5psl5Ua+7d5syCXILErkGSPWnsPzfNDFIWPLJaWYQgsP1A KjdR1fr/KgwqdX6DnS0my2vIQDb6OFUCQ9hOhXxwFEXikMyPez0jcxDQZGqJ780RhOIB V28A== X-Gm-Message-State: ABy/qLZe596ymFhynY4Aqp2Gw3heu6nQkznfmI9tmOT7B04RD0ji9BKv d49NuY0uYgUsPPhUCy1XZ6V6fw== X-Received: by 2002:a2e:3003:0:b0:2b5:974f:385 with SMTP id w3-20020a2e3003000000b002b5974f0385mr129660ljw.9.1690490795035; Thu, 27 Jul 2023 13:46:35 -0700 (PDT) Received: from localhost.localdomain ([2a02:6b6a:b465:0:7e3e:db9e:70fa:9ccb]) by smtp.gmail.com with ESMTPSA id k1-20020a7bc401000000b003fc06169abdsm2701400wmi.2.2023.07.27.13.46.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Jul 2023 13:46:34 -0700 (PDT) From: Usama Arif <usama.arif@bytedance.com> To: linux-mm@kvack.org, muchun.song@linux.dev, mike.kravetz@oracle.com, rppt@kernel.org Cc: linux-kernel@vger.kernel.org, fam.zheng@bytedance.com, liangma@liangbit.com, simon.evans@bytedance.com, punit.agrawal@bytedance.com, Usama Arif <usama.arif@bytedance.com> Subject: [v1 4/6] memblock: introduce MEMBLOCK_RSRV_NOINIT flag Date: Thu, 27 Jul 2023 21:46:22 +0100 Message-Id: <20230727204624.1942372-5-usama.arif@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230727204624.1942372-1-usama.arif@bytedance.com> References: <20230727204624.1942372-1-usama.arif@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1772609615761804595 X-GMAIL-MSGID: 1772609615761804595 |
Series |
mm/memblock: Skip prep and initialization of struct pages freed later by HVO
|
|
Commit Message
Usama Arif
July 27, 2023, 8:46 p.m. UTC
For reserved memory regions marked with this flag,
reserve_bootmem_region is not called during memmap_init_reserved_pages.
This can be used to avoid struct page initialization for
regions which won't need them, for e.g. hugepages with
HVO enabled.
Signed-off-by: Usama Arif <usama.arif@bytedance.com>
---
include/linux/memblock.h | 7 +++++++
mm/memblock.c | 32 ++++++++++++++++++++++++++------
2 files changed, 33 insertions(+), 6 deletions(-)
Comments
Hi, On 7/27/23 23:46, Usama Arif wrote: > For reserved memory regions marked with this flag, > reserve_bootmem_region is not called during memmap_init_reserved_pages. > This can be used to avoid struct page initialization for > regions which won't need them, for e.g. hugepages with > HVO enabled. > > Signed-off-by: Usama Arif <usama.arif@bytedance.com> > --- > include/linux/memblock.h | 7 +++++++ > mm/memblock.c | 32 ++++++++++++++++++++++++++------ > 2 files changed, 33 insertions(+), 6 deletions(-) > > diff --git a/include/linux/memblock.h b/include/linux/memblock.h > index f71ff9f0ec81..7f9d06c08592 100644 > --- a/include/linux/memblock.h > +++ b/include/linux/memblock.h > @@ -47,6 +47,7 @@ enum memblock_flags { > MEMBLOCK_MIRROR = 0x2, /* mirrored region */ > MEMBLOCK_NOMAP = 0x4, /* don't add to kernel direct mapping */ > MEMBLOCK_DRIVER_MANAGED = 0x8, /* always detected via a driver */ > + MEMBLOCK_RSRV_NOINIT = 0x10, /* don't call reserve_bootmem_region for this region */ > }; > > /** > @@ -125,6 +126,7 @@ int memblock_clear_hotplug(phys_addr_t base, phys_addr_t size); > int memblock_mark_mirror(phys_addr_t base, phys_addr_t size); > int memblock_mark_nomap(phys_addr_t base, phys_addr_t size); > int memblock_clear_nomap(phys_addr_t base, phys_addr_t size); > +int memblock_rsrv_mark_noinit(phys_addr_t base, phys_addr_t size); > > void memblock_free_all(void); > void memblock_free(void *ptr, size_t size); > @@ -259,6 +261,11 @@ static inline bool memblock_is_nomap(struct memblock_region *m) > return m->flags & MEMBLOCK_NOMAP; > } > > +static inline bool memblock_is_noinit(struct memblock_region *m) > +{ > + return m->flags & MEMBLOCK_RSRV_NOINIT; > +} > + > static inline bool memblock_is_driver_managed(struct memblock_region *m) > { > return m->flags & MEMBLOCK_DRIVER_MANAGED; > diff --git a/mm/memblock.c b/mm/memblock.c > index 4fd431d16ef2..3a15708af3b6 100644 > --- a/mm/memblock.c > +++ b/mm/memblock.c > @@ -997,6 +997,22 @@ int __init_memblock memblock_clear_nomap(phys_addr_t base, phys_addr_t size) > return memblock_setclr_flag(base, size, 0, MEMBLOCK_NOMAP, 0); > } > > +/** > + * memblock_rsrv_mark_noinit - Mark a reserved memory region with flag MEMBLOCK_RSRV_NOINIT. > + * @base: the base phys addr of the region > + * @size: the size of the region > + * > + * For memory regions marked with %MEMBLOCK_RSRV_NOINIT, reserve_bootmem_region > + * is not called during memmap_init_reserved_pages, hence struct pages are not > + * initialized for this region. > + * > + * Return: 0 on success, -errno on failure. > + */ > +int __init_memblock memblock_rsrv_mark_noinit(phys_addr_t base, phys_addr_t size) > +{ > + return memblock_setclr_flag(base, size, 1, MEMBLOCK_RSRV_NOINIT, 1); > +} > + > static bool should_skip_region(struct memblock_type *type, > struct memblock_region *m, > int nid, int flags) > @@ -2113,13 +2129,17 @@ static void __init memmap_init_reserved_pages(void) > memblock_set_node(start, end, &memblock.reserved, nid); > } > > - /* initialize struct pages for the reserved regions */ > + /* > + * initialize struct pages for reserved regions that don't have > + * the MEMBLOCK_RSRV_NOINIT flag set > + */ > for_each_reserved_mem_region(region) { > - nid = memblock_get_region_node(region); > - start = region->base; > - end = start + region->size; > - > - reserve_bootmem_region(start, end, nid); > + if (!memblock_is_noinit(region)) { > + nid = memblock_get_region_node(region); > + start = region->base; > + end = start + region->size; > + reserve_bootmem_region(start, end, nid); > + } > } > } > There's code like: static inline void free_vmemmap_page(struct page *page) { if (PageReserved(page)) free_bootmem_page(page); else __free_page(page); } which depends on the PageReserved being in vmempages pages, so I think you can't skip that part? --Mika
On 28/07/2023 05:30, Mika Penttilä wrote: > Hi, > > On 7/27/23 23:46, Usama Arif wrote: > >> For reserved memory regions marked with this flag, >> reserve_bootmem_region is not called during memmap_init_reserved_pages. >> This can be used to avoid struct page initialization for >> regions which won't need them, for e.g. hugepages with >> HVO enabled. >> >> Signed-off-by: Usama Arif <usama.arif@bytedance.com> >> --- >> include/linux/memblock.h | 7 +++++++ >> mm/memblock.c | 32 ++++++++++++++++++++++++++------ >> 2 files changed, 33 insertions(+), 6 deletions(-) >> >> diff --git a/include/linux/memblock.h b/include/linux/memblock.h >> index f71ff9f0ec81..7f9d06c08592 100644 >> --- a/include/linux/memblock.h >> +++ b/include/linux/memblock.h >> @@ -47,6 +47,7 @@ enum memblock_flags { >> MEMBLOCK_MIRROR = 0x2, /* mirrored region */ >> MEMBLOCK_NOMAP = 0x4, /* don't add to kernel direct >> mapping */ >> MEMBLOCK_DRIVER_MANAGED = 0x8, /* always detected via a >> driver */ >> + MEMBLOCK_RSRV_NOINIT = 0x10, /* don't call >> reserve_bootmem_region for this region */ >> }; >> /** >> @@ -125,6 +126,7 @@ int memblock_clear_hotplug(phys_addr_t base, >> phys_addr_t size); >> int memblock_mark_mirror(phys_addr_t base, phys_addr_t size); >> int memblock_mark_nomap(phys_addr_t base, phys_addr_t size); >> int memblock_clear_nomap(phys_addr_t base, phys_addr_t size); >> +int memblock_rsrv_mark_noinit(phys_addr_t base, phys_addr_t size); >> void memblock_free_all(void); >> void memblock_free(void *ptr, size_t size); >> @@ -259,6 +261,11 @@ static inline bool memblock_is_nomap(struct >> memblock_region *m) >> return m->flags & MEMBLOCK_NOMAP; >> } >> +static inline bool memblock_is_noinit(struct memblock_region *m) >> +{ >> + return m->flags & MEMBLOCK_RSRV_NOINIT; >> +} >> + >> static inline bool memblock_is_driver_managed(struct memblock_region >> *m) >> { >> return m->flags & MEMBLOCK_DRIVER_MANAGED; >> diff --git a/mm/memblock.c b/mm/memblock.c >> index 4fd431d16ef2..3a15708af3b6 100644 >> --- a/mm/memblock.c >> +++ b/mm/memblock.c >> @@ -997,6 +997,22 @@ int __init_memblock >> memblock_clear_nomap(phys_addr_t base, phys_addr_t size) >> return memblock_setclr_flag(base, size, 0, MEMBLOCK_NOMAP, 0); >> } >> +/** >> + * memblock_rsrv_mark_noinit - Mark a reserved memory region with >> flag MEMBLOCK_RSRV_NOINIT. >> + * @base: the base phys addr of the region >> + * @size: the size of the region >> + * >> + * For memory regions marked with %MEMBLOCK_RSRV_NOINIT, >> reserve_bootmem_region >> + * is not called during memmap_init_reserved_pages, hence struct >> pages are not >> + * initialized for this region. >> + * >> + * Return: 0 on success, -errno on failure. >> + */ >> +int __init_memblock memblock_rsrv_mark_noinit(phys_addr_t base, >> phys_addr_t size) >> +{ >> + return memblock_setclr_flag(base, size, 1, MEMBLOCK_RSRV_NOINIT, 1); >> +} >> + >> static bool should_skip_region(struct memblock_type *type, >> struct memblock_region *m, >> int nid, int flags) >> @@ -2113,13 +2129,17 @@ static void __init >> memmap_init_reserved_pages(void) >> memblock_set_node(start, end, &memblock.reserved, nid); >> } >> - /* initialize struct pages for the reserved regions */ >> + /* >> + * initialize struct pages for reserved regions that don't have >> + * the MEMBLOCK_RSRV_NOINIT flag set >> + */ >> for_each_reserved_mem_region(region) { >> - nid = memblock_get_region_node(region); >> - start = region->base; >> - end = start + region->size; >> - >> - reserve_bootmem_region(start, end, nid); >> + if (!memblock_is_noinit(region)) { >> + nid = memblock_get_region_node(region); >> + start = region->base; >> + end = start + region->size; >> + reserve_bootmem_region(start, end, nid); >> + } >> } >> } > > There's code like: > > static inline void free_vmemmap_page(struct page *page) > { > if (PageReserved(page)) > free_bootmem_page(page); > else > __free_page(page); > } > > which depends on the PageReserved being in vmempages pages, so I think > you can't skip that part? > free_vmemmap_page_list (free_vmemmap_page) is called on struct pages (refer to as [1]) that point to memory *which contains* the struct pages (refer to as [2]) for the hugepage. The above if (!memblock_is_noinit(region)) to not reserve_bootmem_region is called for the struct pages [2] for the hugepage. struct pages [1] are not changed with my patch. As an experiment if I run the diff at the bottom with and without these patches I get the same log "HugeTLB: reserved pages 4096, normal pages 0", which means those struct pages are treated the same without and without these patches. (Its 4096 as 262144 struct pages [2] per hugepage * 64 bytes per struct page / PAGE_SIZE = 4096 struct pages [1] ) Also should have mentioned in cover letter, I used cat /proc/meminfo to make sure it was working as expected. Reserving 500 1G hugepages with and without these patches when hugetlb_free_vmemmap=on MemTotal: 536207112 kB (511.4G) when hugetlb_free_vmemmap=off MemTotal: 528015112 kB (503G) The expectation is that for 500 1G hugepages, when using HVO we have a saving of 16380K*500=~8GB which is what we see with and without those patches (511.4G - 503G). These patches didnt affect these numbers. diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index b5b7834e0f42..bc0ec90552b7 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -208,6 +208,8 @@ static int vmemmap_remap_range(unsigned long start, unsigned long end, return 0; } +static int i = 0, j = 0; + /* * Free a vmemmap page. A vmemmap page can be allocated from the memblock * allocator or buddy allocator. If the PG_reserved flag is set, it means @@ -216,10 +218,14 @@ static int vmemmap_remap_range(unsigned long start, unsigned long end, */ static inline void free_vmemmap_page(struct page *page) { - if (PageReserved(page)) + if (PageReserved(page)) { + i++; free_bootmem_page(page); - else + } + else { + j++; __free_page(page); + } } /* Free a list of the vmemmap pages */ @@ -380,6 +386,7 @@ static int vmemmap_remap_free(unsigned long start, unsigned long end, free_vmemmap_page_list(&vmemmap_pages); + pr_err("reserved pages %u, normal pages %u", i, j); return ret; } > --Mika > >
On 7/28/23 16:47, Usama Arif wrote: > > > On 28/07/2023 05:30, Mika Penttilä wrote: >> Hi, >> >> On 7/27/23 23:46, Usama Arif wrote: >> >>> For reserved memory regions marked with this flag, >>> reserve_bootmem_region is not called during memmap_init_reserved_pages. >>> This can be used to avoid struct page initialization for >>> regions which won't need them, for e.g. hugepages with >>> HVO enabled. >>> >>> Signed-off-by: Usama Arif <usama.arif@bytedance.com> >>> --- >>> include/linux/memblock.h | 7 +++++++ >>> mm/memblock.c | 32 ++++++++++++++++++++++++++------ >>> 2 files changed, 33 insertions(+), 6 deletions(-) >>> >>> diff --git a/include/linux/memblock.h b/include/linux/memblock.h >>> index f71ff9f0ec81..7f9d06c08592 100644 >>> --- a/include/linux/memblock.h >>> +++ b/include/linux/memblock.h >>> @@ -47,6 +47,7 @@ enum memblock_flags { >>> MEMBLOCK_MIRROR = 0x2, /* mirrored region */ >>> MEMBLOCK_NOMAP = 0x4, /* don't add to kernel direct >>> mapping */ >>> MEMBLOCK_DRIVER_MANAGED = 0x8, /* always detected via a >>> driver */ >>> + MEMBLOCK_RSRV_NOINIT = 0x10, /* don't call >>> reserve_bootmem_region for this region */ >>> }; >>> /** >>> @@ -125,6 +126,7 @@ int memblock_clear_hotplug(phys_addr_t base, >>> phys_addr_t size); >>> int memblock_mark_mirror(phys_addr_t base, phys_addr_t size); >>> int memblock_mark_nomap(phys_addr_t base, phys_addr_t size); >>> int memblock_clear_nomap(phys_addr_t base, phys_addr_t size); >>> +int memblock_rsrv_mark_noinit(phys_addr_t base, phys_addr_t size); >>> void memblock_free_all(void); >>> void memblock_free(void *ptr, size_t size); >>> @@ -259,6 +261,11 @@ static inline bool memblock_is_nomap(struct >>> memblock_region *m) >>> return m->flags & MEMBLOCK_NOMAP; >>> } >>> +static inline bool memblock_is_noinit(struct memblock_region *m) >>> +{ >>> + return m->flags & MEMBLOCK_RSRV_NOINIT; >>> +} >>> + >>> static inline bool memblock_is_driver_managed(struct >>> memblock_region *m) >>> { >>> return m->flags & MEMBLOCK_DRIVER_MANAGED; >>> diff --git a/mm/memblock.c b/mm/memblock.c >>> index 4fd431d16ef2..3a15708af3b6 100644 >>> --- a/mm/memblock.c >>> +++ b/mm/memblock.c >>> @@ -997,6 +997,22 @@ int __init_memblock >>> memblock_clear_nomap(phys_addr_t base, phys_addr_t size) >>> return memblock_setclr_flag(base, size, 0, MEMBLOCK_NOMAP, 0); >>> } >>> +/** >>> + * memblock_rsrv_mark_noinit - Mark a reserved memory region with >>> flag MEMBLOCK_RSRV_NOINIT. >>> + * @base: the base phys addr of the region >>> + * @size: the size of the region >>> + * >>> + * For memory regions marked with %MEMBLOCK_RSRV_NOINIT, >>> reserve_bootmem_region >>> + * is not called during memmap_init_reserved_pages, hence struct >>> pages are not >>> + * initialized for this region. >>> + * >>> + * Return: 0 on success, -errno on failure. >>> + */ >>> +int __init_memblock memblock_rsrv_mark_noinit(phys_addr_t base, >>> phys_addr_t size) >>> +{ >>> + return memblock_setclr_flag(base, size, 1, >>> MEMBLOCK_RSRV_NOINIT, 1); >>> +} >>> + >>> static bool should_skip_region(struct memblock_type *type, >>> struct memblock_region *m, >>> int nid, int flags) >>> @@ -2113,13 +2129,17 @@ static void __init >>> memmap_init_reserved_pages(void) >>> memblock_set_node(start, end, &memblock.reserved, nid); >>> } >>> - /* initialize struct pages for the reserved regions */ >>> + /* >>> + * initialize struct pages for reserved regions that don't have >>> + * the MEMBLOCK_RSRV_NOINIT flag set >>> + */ >>> for_each_reserved_mem_region(region) { >>> - nid = memblock_get_region_node(region); >>> - start = region->base; >>> - end = start + region->size; >>> - >>> - reserve_bootmem_region(start, end, nid); >>> + if (!memblock_is_noinit(region)) { >>> + nid = memblock_get_region_node(region); >>> + start = region->base; >>> + end = start + region->size; >>> + reserve_bootmem_region(start, end, nid); >>> + } >>> } >>> } >> >> There's code like: >> >> static inline void free_vmemmap_page(struct page *page) >> { >> if (PageReserved(page)) >> free_bootmem_page(page); >> else >> __free_page(page); >> } >> >> which depends on the PageReserved being in vmempages pages, so I >> think you can't skip that part? >> > > free_vmemmap_page_list (free_vmemmap_page) is called on struct pages > (refer to as [1]) that point to memory *which contains* the struct > pages (refer to as [2]) for the hugepage. The above if > (!memblock_is_noinit(region)) to not reserve_bootmem_region is called > for the struct pages [2] for the hugepage. struct pages [1] are not > changed with my patch. > > As an experiment if I run the diff at the bottom with and without > these patches I get the same log "HugeTLB: reserved pages 4096, normal > pages 0", which means those struct pages are treated the same without > and without these patches. (Its 4096 as 262144 struct pages [2] per > hugepage * 64 bytes per struct page / PAGE_SIZE = 4096 struct pages [1] ) > > Also should have mentioned in cover letter, I used cat /proc/meminfo > to make sure it was working as expected. Reserving 500 1G hugepages > with and without these patches when hugetlb_free_vmemmap=on > MemTotal: 536207112 kB (511.4G) > > when hugetlb_free_vmemmap=off > MemTotal: 528015112 kB (503G) > > > The expectation is that for 500 1G hugepages, when using HVO we have a > saving of 16380K*500=~8GB which is what we see with and without those > patches (511.4G - 503G). These patches didnt affect these numbers. > > You are right, thanks for the explanation! --Mika > > diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c > index b5b7834e0f42..bc0ec90552b7 100644 > --- a/mm/hugetlb_vmemmap.c > +++ b/mm/hugetlb_vmemmap.c > @@ -208,6 +208,8 @@ static int vmemmap_remap_range(unsigned long > start, unsigned long end, > return 0; > } > > +static int i = 0, j = 0; > + > /* > * Free a vmemmap page. A vmemmap page can be allocated from the > memblock > * allocator or buddy allocator. If the PG_reserved flag is set, it > means > @@ -216,10 +218,14 @@ static int vmemmap_remap_range(unsigned long > start, unsigned long end, > */ > static inline void free_vmemmap_page(struct page *page) > { > - if (PageReserved(page)) > + if (PageReserved(page)) { > + i++; > free_bootmem_page(page); > - else > + } > + else { > + j++; > __free_page(page); > + } > } > > /* Free a list of the vmemmap pages */ > @@ -380,6 +386,7 @@ static int vmemmap_remap_free(unsigned long start, > unsigned long end, > > free_vmemmap_page_list(&vmemmap_pages); > > + pr_err("reserved pages %u, normal pages %u", i, j); > return ret; > } > > > > > >> --Mika >> >> >
On Thu, Jul 27, 2023 at 09:46:22PM +0100, Usama Arif wrote: > For reserved memory regions marked with this flag, > reserve_bootmem_region is not called during memmap_init_reserved_pages. > This can be used to avoid struct page initialization for > regions which won't need them, for e.g. hugepages with > HVO enabled. > > Signed-off-by: Usama Arif <usama.arif@bytedance.com> > --- > include/linux/memblock.h | 7 +++++++ > mm/memblock.c | 32 ++++++++++++++++++++++++++------ > 2 files changed, 33 insertions(+), 6 deletions(-) > > diff --git a/include/linux/memblock.h b/include/linux/memblock.h > index f71ff9f0ec81..7f9d06c08592 100644 > --e a/include/linux/memblock.h > +++ b/include/linux/memblock.h > @@ -47,6 +47,7 @@ enum memblock_flags { > MEMBLOCK_MIRROR = 0x2, /* mirrored region */ > MEMBLOCK_NOMAP = 0x4, /* don't add to kernel direct mapping */ > MEMBLOCK_DRIVER_MANAGED = 0x8, /* always detected via a driver */ > + MEMBLOCK_RSRV_NOINIT = 0x10, /* don't call reserve_bootmem_region for this region */ The comment should reflect what it does, not how. > }; > > /** > @@ -125,6 +126,7 @@ int memblock_clear_hotplug(phys_addr_t base, phys_addr_t size); > int memblock_mark_mirror(phys_addr_t base, phys_addr_t size); > int memblock_mark_nomap(phys_addr_t base, phys_addr_t size); > int memblock_clear_nomap(phys_addr_t base, phys_addr_t size); > +int memblock_rsrv_mark_noinit(phys_addr_t base, phys_addr_t size); Please spell out reserved here. > void memblock_free_all(void); > void memblock_free(void *ptr, size_t size); > @@ -259,6 +261,11 @@ static inline bool memblock_is_nomap(struct memblock_region *m) > return m->flags & MEMBLOCK_NOMAP; > } > > +static inline bool memblock_is_noinit(struct memblock_region *m) > +{ > + return m->flags & MEMBLOCK_RSRV_NOINIT; > +} > + > static inline bool memblock_is_driver_managed(struct memblock_region *m) > { > return m->flags & MEMBLOCK_DRIVER_MANAGED; > diff --git a/mm/memblock.c b/mm/memblock.c > index 4fd431d16ef2..3a15708af3b6 100644 > --- a/mm/memblock.c > +++ b/mm/memblock.c > @@ -997,6 +997,22 @@ int __init_memblock memblock_clear_nomap(phys_addr_t base, phys_addr_t size) > return memblock_setclr_flag(base, size, 0, MEMBLOCK_NOMAP, 0); > } > > +/** > + * memblock_rsrv_mark_noinit - Mark a reserved memory region with flag MEMBLOCK_RSRV_NOINIT. > + * @base: the base phys addr of the region > + * @size: the size of the region > + * > + * For memory regions marked with %MEMBLOCK_RSRV_NOINIT, reserve_bootmem_region > + * is not called during memmap_init_reserved_pages, hence struct pages are not > + * initialized for this region. Here as well, the part of how is much less important. Here you should emphasize that struct pages for MEMBLOCK_RSRV_NOINIT regions are not initialized. > + * > + * Return: 0 on success, -errno on failure. > + */ > +int __init_memblock memblock_rsrv_mark_noinit(phys_addr_t base, phys_addr_t size) > +{ > + return memblock_setclr_flag(base, size, 1, MEMBLOCK_RSRV_NOINIT, 1); > +} > + > static bool should_skip_region(struct memblock_type *type, > struct memblock_region *m, > int nid, int flags) > @@ -2113,13 +2129,17 @@ static void __init memmap_init_reserved_pages(void) > memblock_set_node(start, end, &memblock.reserved, nid); > } > > - /* initialize struct pages for the reserved regions */ > + /* > + * initialize struct pages for reserved regions that don't have > + * the MEMBLOCK_RSRV_NOINIT flag set > + */ > for_each_reserved_mem_region(region) { > - nid = memblock_get_region_node(region); > - start = region->base; > - end = start + region->size; > - > - reserve_bootmem_region(start, end, nid); > + if (!memblock_is_noinit(region)) { > + nid = memblock_get_region_node(region); > + start = region->base; > + end = start + region->size; Please keep the empty line here > + reserve_bootmem_region(start, end, nid); > + } > } > } > > -- > 2.25.1 >
diff --git a/include/linux/memblock.h b/include/linux/memblock.h index f71ff9f0ec81..7f9d06c08592 100644 --- a/include/linux/memblock.h +++ b/include/linux/memblock.h @@ -47,6 +47,7 @@ enum memblock_flags { MEMBLOCK_MIRROR = 0x2, /* mirrored region */ MEMBLOCK_NOMAP = 0x4, /* don't add to kernel direct mapping */ MEMBLOCK_DRIVER_MANAGED = 0x8, /* always detected via a driver */ + MEMBLOCK_RSRV_NOINIT = 0x10, /* don't call reserve_bootmem_region for this region */ }; /** @@ -125,6 +126,7 @@ int memblock_clear_hotplug(phys_addr_t base, phys_addr_t size); int memblock_mark_mirror(phys_addr_t base, phys_addr_t size); int memblock_mark_nomap(phys_addr_t base, phys_addr_t size); int memblock_clear_nomap(phys_addr_t base, phys_addr_t size); +int memblock_rsrv_mark_noinit(phys_addr_t base, phys_addr_t size); void memblock_free_all(void); void memblock_free(void *ptr, size_t size); @@ -259,6 +261,11 @@ static inline bool memblock_is_nomap(struct memblock_region *m) return m->flags & MEMBLOCK_NOMAP; } +static inline bool memblock_is_noinit(struct memblock_region *m) +{ + return m->flags & MEMBLOCK_RSRV_NOINIT; +} + static inline bool memblock_is_driver_managed(struct memblock_region *m) { return m->flags & MEMBLOCK_DRIVER_MANAGED; diff --git a/mm/memblock.c b/mm/memblock.c index 4fd431d16ef2..3a15708af3b6 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -997,6 +997,22 @@ int __init_memblock memblock_clear_nomap(phys_addr_t base, phys_addr_t size) return memblock_setclr_flag(base, size, 0, MEMBLOCK_NOMAP, 0); } +/** + * memblock_rsrv_mark_noinit - Mark a reserved memory region with flag MEMBLOCK_RSRV_NOINIT. + * @base: the base phys addr of the region + * @size: the size of the region + * + * For memory regions marked with %MEMBLOCK_RSRV_NOINIT, reserve_bootmem_region + * is not called during memmap_init_reserved_pages, hence struct pages are not + * initialized for this region. + * + * Return: 0 on success, -errno on failure. + */ +int __init_memblock memblock_rsrv_mark_noinit(phys_addr_t base, phys_addr_t size) +{ + return memblock_setclr_flag(base, size, 1, MEMBLOCK_RSRV_NOINIT, 1); +} + static bool should_skip_region(struct memblock_type *type, struct memblock_region *m, int nid, int flags) @@ -2113,13 +2129,17 @@ static void __init memmap_init_reserved_pages(void) memblock_set_node(start, end, &memblock.reserved, nid); } - /* initialize struct pages for the reserved regions */ + /* + * initialize struct pages for reserved regions that don't have + * the MEMBLOCK_RSRV_NOINIT flag set + */ for_each_reserved_mem_region(region) { - nid = memblock_get_region_node(region); - start = region->base; - end = start + region->size; - - reserve_bootmem_region(start, end, nid); + if (!memblock_is_noinit(region)) { + nid = memblock_get_region_node(region); + start = region->base; + end = start + region->size; + reserve_bootmem_region(start, end, nid); + } } }