From patchwork Sat Oct 28 21:15:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Gomez X-Patchwork-Id: 159301 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:d641:0:b0:403:3b70:6f57 with SMTP id cy1csp1401511vqb; Sat, 28 Oct 2023 14:18:09 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFPDambeTJV5Uv4nEGmCybtdFiEDb15rrq0Sx9l1WllnmL/HZCG+RjwOfY7c+4OaCrJb1Yo X-Received: by 2002:a05:6a21:4988:b0:15e:9c50:73ee with SMTP id ax8-20020a056a21498800b0015e9c5073eemr7230470pzc.53.1698527888988; Sat, 28 Oct 2023 14:18:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1698527888; cv=none; d=google.com; s=arc-20160816; b=bnaEr0gGkFi5Sq9EJPcrHS/q/BeuYs8KvRayk9f31PQgsqEr3qmMr9YBomQD4lpkAQ Vv6XtGliH88kzMYKCZgC8LkfOVoRFCC99vqw5VKFNYDH/dvXGE7XdPeR8Vq2QvsGvpVT QbcaKRV1L+mvgbxZkGQWKuhMGCpER09LRhYEpmVQislTUd7J9uIGZXFgycrLCHSQEISI iKv9FZTktQTjszw4GjgG22s56ZSLg4MTwObxjGkh4Hl0ULs8wuvb0/tQDUAK5Gu87+AC eqBJxk4WbODDl42IpzlWw6jNrGtnU2DhYAw3TNyNMhnIRRqwa3kC8PCGvenY1mKKDcq/ nPwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:cms-type:mime-version :content-transfer-encoding:content-language:accept-language :in-reply-to:message-id:date:thread-index:thread-topic:subject:cc:to :from:dkim-signature:dkim-filter; bh=V58CpUMglUVRv2d2n4PjwOPxGbR6MAVB1lgi43BKbik=; fh=uxBAUgUSkBYxjd1CTtcj7MO+vdwExfJsOb4I3IkdCcM=; b=WEzqiAeBjxfLQMBzKnR7YFvdRGkJFBmrISEYkoAWgVxkrfkDzDz2N0CsqLxcqFsHsw PX75roVKNY6MqiZHO1bOaAfgipJmLkvv4uqTGbXZtgfhfs6c0YsS1tG70tfyKF99TfiA Ft8WKQDoej5uM1X8x+F5LWB7dpxMqu2P7ZW2xvmUgi2F+l1iAJ1ldajMD9mrTvwKNOeN 4+S0twq/gg88M/KrJXrDxcwRvtKb6/5VjLYnxRWigsCB9B1bF3+ocaTzPgt34gBej3lE XweePhKKohyg/58mTKuKf+vVH5aS/GS6k8OiMnrrmDBOiLO4BpubEJseztsFRqDVU3gv TC3A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@samsung.com header.s=mail20170921 header.b=Ug90+BkE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=samsung.com Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id j2-20020a63e742000000b005b91518ab92si2740721pgk.671.2023.10.28.14.18.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 28 Oct 2023 14:18:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@samsung.com header.s=mail20170921 header.b=Ug90+BkE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=samsung.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id E51E18055ACB; Sat, 28 Oct 2023 14:18:05 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230103AbjJ1VQr (ORCPT + 29 others); Sat, 28 Oct 2023 17:16:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44546 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230064AbjJ1VQ1 (ORCPT ); Sat, 28 Oct 2023 17:16:27 -0400 Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 72E5D19E for ; Sat, 28 Oct 2023 14:15:54 -0700 (PDT) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20231028211553euoutp02bad875fb88c13ddbe8613cd5cb4cf336~SYf4K7h5K0890208902euoutp02J for ; Sat, 28 Oct 2023 21:15:53 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20231028211553euoutp02bad875fb88c13ddbe8613cd5cb4cf336~SYf4K7h5K0890208902euoutp02J DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1698527753; bh=V58CpUMglUVRv2d2n4PjwOPxGbR6MAVB1lgi43BKbik=; h=From:To:CC:Subject:Date:In-Reply-To:References:From; b=Ug90+BkEzSkTyQzxO6IX69s40XfH1kin4j5qyV0qRip82QwCoST7VXdZhGtirfp/C eCUwW7HRdmW9RoJGBubVMRkje6dh2LhFr2aY5RD5wPKF7i+0tilH+Z95hxaUGhQlqG PwLLloFWfV3YsuaTmdl68/3RahrJys8XbK1q3o3E= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20231028211552eucas1p2368f287afd825b44877b59399542f661~SYf3yR5P21087910879eucas1p2Q; Sat, 28 Oct 2023 21:15:52 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id A3.20.11320.80A7D356; Sat, 28 Oct 2023 22:15:52 +0100 (BST) Received: from eusmtrp1.samsung.com (unknown [182.198.249.138]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20231028211551eucas1p1552b7695f12c27f4ea1b92ecb6259b31~SYf3Amogw0616106161eucas1p1Y; Sat, 28 Oct 2023 21:15:51 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp1.samsung.com (KnoxPortal) with ESMTP id 20231028211551eusmtrp166725da83a27a1657bdc91947630f5f0~SYf3AFyW50755507555eusmtrp1e; Sat, 28 Oct 2023 21:15:51 +0000 (GMT) X-AuditID: cbfec7f4-993ff70000022c38-94-653d7a08c743 Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 04.52.10549.70A7D356; Sat, 28 Oct 2023 22:15:51 +0100 (BST) Received: from CAMSVWEXC01.scsc.local (unknown [106.1.227.71]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20231028211551eusmtip146fad08e90fd824bcd219c14210d9955~SYf206cR30467404674eusmtip1U; Sat, 28 Oct 2023 21:15:51 +0000 (GMT) Received: from CAMSVWEXC02.scsc.local (2002:6a01:e348::6a01:e348) by CAMSVWEXC01.scsc.local (2002:6a01:e347::6a01:e347) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 28 Oct 2023 22:15:51 +0100 Received: from CAMSVWEXC02.scsc.local ([::1]) by CAMSVWEXC02.scsc.local ([fe80::3c08:6c51:fa0a:6384%13]) with mapi id 15.00.1497.012; Sat, 28 Oct 2023 22:15:51 +0100 From: Daniel Gomez To: "minchan@kernel.org" , "senozhatsky@chromium.org" , "axboe@kernel.dk" , "djwong@kernel.org" , "willy@infradead.org" , "hughd@google.com" , "akpm@linux-foundation.org" , "mcgrof@kernel.org" , "linux-kernel@vger.kernel.org" , "linux-block@vger.kernel.org" , "linux-xfs@vger.kernel.org" , "linux-fsdevel@vger.kernel.org" , "linux-mm@kvack.org" CC: "gost.dev@samsung.com" , Pankaj Raghav , Daniel Gomez Subject: [RFC PATCH 10/11] shmem: add large folio support to the write path Thread-Topic: [RFC PATCH 10/11] shmem: add large folio support to the write path Thread-Index: AQHaCePt75l+dUBUfkS9wehNfGwv9w== Date: Sat, 28 Oct 2023 21:15:50 +0000 Message-ID: <20231028211518.3424020-11-da.gomez@samsung.com> In-Reply-To: <20231028211518.3424020-1-da.gomez@samsung.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [106.110.32.103] MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01Sa0hTYRjmOzs7O65WpyntU7uuJHK6mRWcwCIp8FAQXcio6LLypJWaba2L Rs0wbEMsNaFdKi95aS2mq0zMGc1srSlltrSFhjYtjFLTynRYm2eF/573ed7nfd7348NZfDs7 BD+ceoKWpUqThRgXrXn2+2Uknr6ajrrTE07qTUaMvNN5GSPb3DPI3u+5KGlxich6ix0l2+r0 GNll/MMm6zy1HLLjSi8gy38McMhH1SUYOT6qx9byKJ2yFaWKzArqXmU41daioMwGFUaZv+dz qOfXxlFq2DxvM76LG5NAJx8+Scska/Zzk765+pC0/tDTqjdsJbg7Ww0CcEisgJ5uD1sNuDif qARwqETvL0YAbHmRgzLFMIBV7V3oP0tldpu/qwJAzS8j539XzjeVX2n2KtlWFlPcBnBU7eT4 /BixFDbYzZOWIELHho2ll4BPYBHnoLFPOxkSSGyEat1LxIeDiK1QVTXsnYR7sRjmmfg+GiXC oNPowXw0j4iBFucOHx3ghR9+PsR8GBBzYc/tMQ4zXQBd7psIc8IsWKKrZzF4Npyo68YYHAFb 2t2AwVHwQVmD/2Qh9Fwq9m8phh2FVzEGi2B58ZfJOTzvTLvGPflekJgIgJ/euTiMeT0ccjzz BwfCftt9Pz8HOgpy0CtApJ2yn3ZKhnZKhnZKRhFADUBAK+QpibQ8OpU+JZZLU+SK1ETxwWMp ZuD9bY4J20gtqOgfElsBggMrgDhLGMRjxcbQfF6C9Ew6LTu2T6ZIpuVWEIqjQgEvLGE+zScS pSfoozSdRsv+qQgeEKJE5i//ql6W4dwcqNuQsPSGctH0n9kr25EDTdNb8euZUR8zsyLinzsD t29aXbBlYLdh59vfYUe1NVW2EE2kQSM595pXai/r3VXKFYWmVUeT7vsOW0ZElct9vDk/Djmo 2vteYNmwCpfsF9BNSwp7zkpuGanqGGmpq1H5+HxSrEyTaY5+ldt89q648NCRkxe4ow3TbtWb ihokkcFdktqHrWOfLzwZYiODfU8/lnXk2bIuJq1akC6yFECVZ1tdfOg60Q7HI8Hgq25Rsbtp bOOgfmFn5/jcgdz8dl7rptrglmufyDjX4qzYRnNwRb+8e+RIdZZhZob1zx7PD5FJsqjGmhdn EqLyJOmycJZMLv0LX4liiNwDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrMKsWRmVeSWpSXmKPExsVy+t/xu7rsVbapBofPmlrMWb+GzWL13X42 i8tP+Cyefupjsdh7S9tiz96TLBaXd81hs7i35j+rxa4/O9gtbkx4ymix7Ot7dovdGxexWfz+ MYfNgddjdsNFFo8Fm0o9Nq/Q8rh8ttRj06pONo9Nnyaxe5yY8ZvF4/MmuQCOKD2bovzSklSF jPziElulaEMLIz1DSws9IxNLPUNj81grI1MlfTublNSczLLUIn27BL2Md7eeMRW8kq7ovMLa wLhWrIuRk0NCwERiRftl1i5GLg4hgaWMEg0HX7FAJGQkNn65ygphC0v8udbFBlH0kVFiWvcN qI4zjBJvZp6HclYySrzomMwI0sImoCmx7+QmdpCEiMBsVonDizvAEswCdRJrns0C2yEs4C3R Nfs8E4gtIhAkcXjpUqBJHEC2nsTE9UIgYRYBVYmra/6wgYR5BWwk9l4NBzGFBHIl+tsyQSo4 gaL3v21nA7EZBWQlHq38xQ6xSFzi1pP5TBAPCEgs2XOeGcIWlXj5+B/UYzoSZ68/YYSwDSS2 Lt0H9bySxJ+OhVAH60ncmDqFDcLWlli28DXYHF4BQYmTM5+wTGCUnoVk3SwkLbOQtMxC0rKA kWUVo0hqaXFuem6xoV5xYm5xaV66XnJ+7iZGYGraduzn5h2M81591DvEyMTBeIhRgoNZSYSX 2dEmVYg3JbGyKrUoP76oNCe1+BCjKTCEJjJLiSbnA5NjXkm8oZmBqaGJmaWBqaWZsZI4r2dB R6KQQHpiSWp2ampBahFMHxMHp1QDU0CAsoTS1hAGmS/nH3i36u929eEWimr/tZDT+c+W9zdk +j7s2BBvKC2z605S8rkvx6zP7W76dXdpiD1nSUCCzzKz5+//acvJXYzLU9dRFOwRv7rpzSVr prif27hWlFeseN8dwbHrtGKr7et7BevnuVody/ztGbJti0zUyv+27/oS65/7LOU+3VBofMyU ba9bTIDa0tPsS0+auyyOkO79Fc3ie2qNyC/p6H9n7FOfzFvkdt3+P+uf9zcMHmnJeP6yecp3 L4fX6NlNa4Z0JrPtjH7Mn3uDbx282qlQsc/b0zl292vXxYLP/rz7eylyVlNTK091Z+fPjF6e pqvmcyRzN6l/04ttmFnDl7CJ8zeDEktxRqKhFnNRcSIABtzmANYDAAA= X-CMS-MailID: 20231028211551eucas1p1552b7695f12c27f4ea1b92ecb6259b31 X-Msg-Generator: CA X-RootMTR: 20231028211551eucas1p1552b7695f12c27f4ea1b92ecb6259b31 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20231028211551eucas1p1552b7695f12c27f4ea1b92ecb6259b31 References: <20230919135536.2165715-1-da.gomez@samsung.com> <20231028211518.3424020-1-da.gomez@samsung.com> X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Sat, 28 Oct 2023 14:18:06 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1781035580009351884 X-GMAIL-MSGID: 1781035580009351884 Current work in progress. Large folios in the fallocate path makes regress fstests generic/285 and generic/436. Add large folio support for shmem write path matching the same high order preference mechanism used for iomap buffered IO path as used in __filemap_get_folio(). Add shmem_mapping_size_order to get a hint for the order of the folio based on the file size which takes care of the mapping requirements. Swap does not support high order folios for now, so make it order 0 in case swap is enabled. Add the __GFP_COMP flag for high order folios except when huge is enabled. This fixes a memory leak when allocating high order folios. Signed-off-by: Daniel Gomez --- mm/shmem.c | 49 ++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 48 insertions(+), 1 deletion(-) diff --git a/mm/shmem.c b/mm/shmem.c index fc7605da4316..eb314927be78 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1621,6 +1621,9 @@ static struct folio *shmem_alloc_folio(gfp_t gfp, struct shmem_inode_info *info, pgoff_t ilx; struct page *page; + if ((order != 0) && !(gfp & VM_HUGEPAGE)) + gfp |= __GFP_COMP; + mpol = shmem_get_pgoff_policy(info, index, order, &ilx); page = alloc_pages_mpol(gfp, order, mpol, ilx, numa_node_id()); mpol_cond_put(mpol); @@ -1628,17 +1631,56 @@ static struct folio *shmem_alloc_folio(gfp_t gfp, struct shmem_inode_info *info, return page_rmappable_folio(page); } +/** + * shmem_mapping_size_order - Get maximum folio order for the given file size. + * @mapping: Target address_space. + * @index: The page index. + * @size: The suggested size of the folio to create. + * + * This returns a high order for folios (when supported) based on the file size + * which the mapping currently allows at the given index. The index is relevant + * due to alignment considerations the mapping might have. The returned order + * may be less than the size passed. + * + * Like __filemap_get_folio order calculation. + * + * Return: The order. + */ +static inline unsigned int +shmem_mapping_size_order(struct address_space *mapping, pgoff_t index, + size_t size, struct shmem_sb_info *sbinfo) +{ + unsigned int order = ilog2(size); + + if ((order <= PAGE_SHIFT) || + (!mapping_large_folio_support(mapping) || !sbinfo->noswap)) + return 0; + + order -= PAGE_SHIFT; + + /* If we're not aligned, allocate a smaller folio */ + if (index & ((1UL << order) - 1)) + order = __ffs(index); + + order = min_t(size_t, order, MAX_PAGECACHE_ORDER); + + /* Order-1 not supported due to THP dependency */ + return (order == 1) ? 0 : order; +} + static struct folio *shmem_alloc_and_add_folio(gfp_t gfp, struct inode *inode, pgoff_t index, struct mm_struct *fault_mm, size_t len) { struct address_space *mapping = inode->i_mapping; struct shmem_inode_info *info = SHMEM_I(inode); - unsigned int order = 0; + unsigned int order = shmem_mapping_size_order(mapping, index, len, + SHMEM_SB(inode->i_sb)); struct folio *folio; long pages; int error; +neworder: if (gfp & VM_HUGEPAGE) { pages = HPAGE_PMD_NR; index = round_down(index, HPAGE_PMD_NR); @@ -1721,6 +1763,11 @@ static struct folio *shmem_alloc_and_add_folio(gfp_t gfp, unlock: folio_unlock(folio); folio_put(folio); + if (order != 0) { + if (--order == 1) + order = 0; + goto neworder; + } return ERR_PTR(error); }