From patchwork Fri Feb 2 16:15:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 19856 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:9bc1:b0:106:209c:c626 with SMTP id op1csp550645dyc; Fri, 2 Feb 2024 08:40:32 -0800 (PST) X-Google-Smtp-Source: AGHT+IGT4Q/lVzGqFkQ/lq0N9XQi2Z8W4wgslIqVqyhlGK2E1QmHczTpvNGkDxEsB21PbaTYA7vj X-Received: by 2002:a05:6358:3a0a:b0:176:7f77:a2b2 with SMTP id g10-20020a0563583a0a00b001767f77a2b2mr6070833rwe.5.1706892032364; Fri, 02 Feb 2024 08:40:32 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706892032; cv=pass; d=google.com; s=arc-20160816; b=XYuZM5K1SABKxsew6DSuALHTGZ65JrRqCBUFzegiuiEpn1UFkhahHOdg4rs+2ekHfV j9V16OmpFS26tPbP9spSgP/E2H2zrrJG6YQ9VfHAffnOZcPGLaBCwqvhBPw0OjDu8/kM 1V4kzy4N0nNjvHTSBKqmEZBGecXuiuhzEPgTC+14kkKj374XdKSXib07liULKo/QOjFN MCrteGdybv5m6mJTo71Zn9nxgFZhoQ1DCishA/B2Ig9dzRn5KhfAbq5Vyu9pkgbwRwA1 yvzAx4+APVSDurpb5Rcsh6V6lz03mzRXx/Kq556A3YGf337kdjs8lm8eyZXarFH2InFN h9XA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:reply-to:message-id:date:subject :cc:to:from:feedback-id:dkim-signature:dkim-signature; bh=sbJRcJVhKwu3xlXdpvrqMPMBrNMkSAU2s2K/uCxEzpo=; fh=wqUYHphvopxq6Ci25Irbc6BnaScvMvEkrl7vbpIZrBU=; b=0Nc6+dKRxtkkgfk0/u4pOGh+XMIGkKmSHQndp8OCqaY4Gp0k4Vygt66j9oCsalXfCz p/Gn+HkQ8b3NQreujC9qRgd3wORhsN+4/Yfaw9amOw9ihkaETSeT0IqGYCD0t8QulrQe 0XGR4waHowJrtD8cU/1mqUDKdYKvRTjMJc5F3gumImZue8WL2SfMcNrRbiQCNqqyda9V KC+BZYQPzw1to4tA4iObpAUudeAaDbzhm8/4mfL2WQVHYXLsuWlzBz19nVQHtOBm3bgm X9UqPtMBmj5ElWo5SAz0ArZY5IhegCr1YGiA/w22akXRB7vV/9WdAHssCnMsxhzHq2jf vzCw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@sent.com header.s=fm3 header.b=v+j7jFd0; dkim=pass header.i=@messagingengine.com header.s=fm3 header.b=ab9ItWyz; arc=pass (i=1 spf=pass spfdomain=sent.com dkim=pass dkdomain=sent.com dkim=pass dkdomain=messagingengine.com dmarc=pass fromdomain=sent.com); spf=pass (google.com: domain of linux-kernel+bounces-50117-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-50117-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=sent.com X-Forwarded-Encrypted: i=1; AJvYcCXvxdrfOuB+vYcYuHWsAcKcVv/WAdqrnjDJycb9rgbeNKNCNyV/+NuwuQeOd5ztCzGKJfaB9NYGbZvxknQWZFXaD0FokA== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id d2-20020a631d42000000b005cdf4899a56si1813108pgm.658.2024.02.02.08.40.31 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Feb 2024 08:40:32 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-50117-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@sent.com header.s=fm3 header.b=v+j7jFd0; dkim=pass header.i=@messagingengine.com header.s=fm3 header.b=ab9ItWyz; arc=pass (i=1 spf=pass spfdomain=sent.com dkim=pass dkdomain=sent.com dkim=pass dkdomain=messagingengine.com dmarc=pass fromdomain=sent.com); spf=pass (google.com: domain of linux-kernel+bounces-50117-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-50117-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=sent.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 97056B223A4 for ; Fri, 2 Feb 2024 16:17:07 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DF6621487D8; Fri, 2 Feb 2024 16:16:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sent.com header.i=@sent.com header.b="v+j7jFd0"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="ab9ItWyz" Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C0577182BB for ; Fri, 2 Feb 2024 16:16:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=66.111.4.26 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706890579; cv=none; b=Vh/eeo9vUVaATxzB2ZSm0IpbdJnxjOo94kFkFo/+VM4cJQkZxhaYwnmqfYSQZVBd+qxV8qXD32dTzVY8kl7uY6NThqLp+C1fV4rtmTI08aNK9fxPaTltdNSd9m9nvJnuJ+QBWtx5i/WmGICwMeOmpxopwdBNNWa4Mqe72MCLZKE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706890579; c=relaxed/simple; bh=9U1mw1yVdLk+BExwcI7S+DGgNlo8VhiS206YK4nSCvY=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type; b=ffRlq0R81Sksgu37oBfCLGYHwR//NmnFoqCdHFi8kb/Yk6wbI0d5z3coNKy3fnpw/smbTrsaOLEuPHgsujv6eELMnGN2K7A/qVdoAbW3vywSw81SlhBXLrNlloC5MBLBzwKK9hEC96824UokQVSd3PZsvtfBAmCJSWNaCZO7TZw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=sent.com; spf=pass smtp.mailfrom=sent.com; dkim=pass (2048-bit key) header.d=sent.com header.i=@sent.com header.b=v+j7jFd0; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=ab9ItWyz; arc=none smtp.client-ip=66.111.4.26 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=sent.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sent.com Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id BC04B5C00D2; Fri, 2 Feb 2024 11:16:15 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Fri, 02 Feb 2024 11:16:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:message-id:mime-version:reply-to :reply-to:subject:subject:to:to; s=fm3; t=1706890575; x= 1706976975; bh=sbJRcJVhKwu3xlXdpvrqMPMBrNMkSAU2s2K/uCxEzpo=; b=v +j7jFd0c9xykrWBxQKHlC4Lqv0qDlusH0gSEzy613M7wiNQfNgJbMMliEDhadZ4D wznDUqevKp+erQefUvuQmR8nPV3azE5AOBEtIqZWSFOi9De/f+ODhs/dC7+L67WY DJza7+BsbL22h2J9kBBpm4Di1lFLOG1oleBrlf85ZzKsLEBcscTU8B7NbaW9kpVd 0iwrzSqaxAQ3fd+qB5WZo7qBuyl74l5ffCJ35FRx0XSIopnKjvttFkI5027uCy6n fcwIjDYVM6lmy27TeH6pF2ZXxwVyYZ0W40gNcHorP4NDWeQkzPABKMpnW3ffWToa tqyGQEuVCAi8UZQGRauGA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:message-id:mime-version:reply-to:reply-to :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm3; t=1706890575; x=1706976975; bh=s bJRcJVhKwu3xlXdpvrqMPMBrNMkSAU2s2K/uCxEzpo=; b=ab9ItWyzWsppQVL6B kO8yizKcNw9Lm/oTtpaUXiqXCNJfxb9arOTnXjqPyKrmwR5l9tSBHbk2S+E6K91A 99Lw5ZxzXiqCzK23Dt2CH2sMDhmQtf56numXYBKZZO2mgJUKc/Xs774gY+ptKyDT 5t/be3bkes1T1rjhy/BQ/Ayj5frlChhltWILSaixrwSrX40Psp5VA1oRxwY9t2d2 irWpVI+4vbkHWfbRuanednL6+p18fyLueDomWY8ZS7fwsm4q4ffaQm701MjvzRoy xrfS/NbOVD/OC8va3EqfXH8ko+n+8oM5xZ2olY0L+6fVF3BMEbDkjA0WBxVBJMqL RliTg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrfedugedgkeehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofhrgggtgfesthekredtredtjeenucfhrhhomhepkghiucgj rghnuceoiihirdihrghnsehsvghnthdrtghomheqnecuggftrfgrthhtvghrnhepkefhle duhffhjedttdetudejtdeuieevueetuddtuedthffffffgueffhfegjeffnecuffhomhgr ihhnpehkvghrnhgvlhdrohhrghdpfhholhhiohhsrdhmmhdptghomhhprggtthhiohhnrd hmmhdpshhplhhithdrmhhmnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehm rghilhhfrhhomhepiihirdihrghnsehsvghnthdrtghomh X-ME-Proxy: Feedback-ID: iccd040f4:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 2 Feb 2024 11:16:11 -0500 (EST) From: Zi Yan To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Zi Yan , "Huang, Ying" , Ryan Roberts , Andrew Morton , "Matthew Wilcox (Oracle)" , David Hildenbrand , "Yin, Fengwei" , Yu Zhao , Vlastimil Babka , "Kirill A . Shutemov" , Johannes Weiner , Baolin Wang , Kemeng Shi , Mel Gorman , Rohan Puri , Mcgrof Chamberlain , Adam Manzanares , "Vishal Moola (Oracle)" Subject: [PATCH v3 0/3] Enable >0 order folio memory compaction Date: Fri, 2 Feb 2024 11:15:51 -0500 Message-ID: <20240202161554.565023-1-zi.yan@sent.com> X-Mailer: git-send-email 2.43.0 Reply-To: Zi Yan Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789806019829407541 X-GMAIL-MSGID: 1789806019829407541 From: Zi Yan Hi all, This patchset enables >0 order folio memory compaction, which is one of the prerequisitions for large folio support[1]. It includes the fix[4] for V2 and is on top of mm-everything-2024-01-29-07-19. I am aware of that split free pages is necessary for folio migration in compaction, since if >0 order free pages are never split and no order-0 free page is scanned, compaction will end prematurely due to migration returns -ENOMEM. Free page split becomes a must instead of an optimization. lkp ncompare results for default LRU (-no-mglru) and CONFIG_LRU_GEN are shown at the bottom (on a 8-CPU (Intel Xeon E5-2650 v4 @ 2.20GHz) 16G VM). In sum, most of vm-scalability applications do not see performance change, and the others see ~4% to ~26% performance boost under default LRU and ~2% to ~6% performance boost under CONFIG_LRU_GEN. Changelog === From V2 [3]: 1. Added missing free page count in fast isolation path. This fixed the weird performance outcome. From V1 [2]: 1. Used folio_test_large() instead of folio_order() > 0. (per Matthew Wilcox) 2. Fixed code rebase error. (per Baolin Wang) 3. Used list_split_init() instead of list_split(). (per Ryan Boberts) 4. Added free_pages_prepare_fpi_none() to avoid duplicate free page code in compaction_free(). 5. Dropped source page order sorting patch. From RFC [1]: 1. Enabled >0 order folio compaction in the first patch by splitting all to-be-migrated folios. (per Huang, Ying) 2. Stopped isolating compound pages with order greater than cc->order to avoid wasting effort, since cc->order gives a hint that no free pages with order greater than it exist, thus migrating the compound pages will fail. (per Baolin Wang) 3. Retained the folio check within lru lock. (per Baolin Wang) 4. Made isolate_freepages_block() generate order-sorted multi lists. (per Johannes Weiner) Overview === To support >0 order folio compaction, the patchset changes how free pages used for migration are kept during compaction. Free pages used to be split into order-0 pages that are post allocation processed (i.e., PageBuddy flag cleared, page order stored in page->private is zeroed, and page reference is set to 1). Now all free pages are kept in a MAX_ORDER+1 array of page lists based on their order without post allocation process. When migrate_pages() asks for a new page, one of the free pages, based on the requested page order, is then processed and given out. Feel free to give comments and ask questions. Thanks. vm-scalability results on CONFIG_LRU_GEN === ========================================================================================= compiler/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-13/defconfig/debian/300s/qemu-vm/mmap-xread-seq-mt/vm-scalability commit: 6.8.0-rc1-mm-everything-2024-01-29-07-19+ 6.8.0-rc1-split-folio-in-compaction+ 6.8.0-rc1-folio-migration-in-compaction+ 6.8.0-rc1-folio-migration-free-page-split+ 6.8.0-rc1-mm-eve 6.8.0-rc1-split-folio-in-co 6.8.0-rc1-folio-migration-i 6.8.0-rc1-folio-migration-f ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 15107616 +3.2% 15590339 +1.3% 15297619 +3.0% 15567998 vm-scalability.throughput ========================================================================================= compiler/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-13/defconfig/debian/300s/qemu-vm/mmap-pread-seq/vm-scalability commit: 6.8.0-rc1-mm-everything-2024-01-29-07-19+ 6.8.0-rc1-split-folio-in-compaction+ 6.8.0-rc1-folio-migration-in-compaction+ 6.8.0-rc1-folio-migration-free-page-split+ 6.8.0-rc1-mm-eve 6.8.0-rc1-split-folio-in-co 6.8.0-rc1-folio-migration-i 6.8.0-rc1-folio-migration-f ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 12611785 +1.8% 12832919 +0.9% 12724223 +1.6% 12812682 vm-scalability.throughput ========================================================================================= compiler/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-13/defconfig/debian/300s/qemu-vm/lru-file-readtwice/vm-scalability commit: 6.8.0-rc1-mm-everything-2024-01-29-07-19+ 6.8.0-rc1-split-folio-in-compaction+ 6.8.0-rc1-folio-migration-in-compaction+ 6.8.0-rc1-folio-migration-free-page-split+ 6.8.0-rc1-mm-eve 6.8.0-rc1-split-folio-in-co 6.8.0-rc1-folio-migration-i 6.8.0-rc1-folio-migration-f ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 9833393 +5.7% 10390190 +3.0% 10126606 +5.9% 10408804 vm-scalability.throughput ========================================================================================= compiler/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-13/defconfig/debian/300s/qemu-vm/lru-file-mmap-read/vm-scalability commit: 6.8.0-rc1-mm-everything-2024-01-29-07-19+ 6.8.0-rc1-split-folio-in-compaction+ 6.8.0-rc1-folio-migration-in-compaction+ 6.8.0-rc1-folio-migration-free-page-split+ 6.8.0-rc1-mm-eve 6.8.0-rc1-split-folio-in-co 6.8.0-rc1-folio-migration-i 6.8.0-rc1-folio-migration-f ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 7034709 ± 3% +2.9% 7241429 +3.2% 7256680 ± 2% +3.9% 7308375 vm-scalability.throughput vm-scalability results on default LRU (with -no-mglru suffix) === ========================================================================================= compiler/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-13/defconfig/debian/300s/qemu-vm/mmap-xread-seq-mt/vm-scalability commit: 6.8.0-rc1-mm-everything-2024-01-29-07-19-no-mglru+ 6.8.0-rc1-split-folio-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-free-page-split-no-mglru+ 6.8.0-rc1-mm-eve 6.8.0-rc1-split-folio-in-co 6.8.0-rc1-folio-migration-i 6.8.0-rc1-folio-migration-f ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 14401491 +3.7% 14940270 +2.4% 14748626 +4.0% 14975716 vm-scalability.throughput ========================================================================================= compiler/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-13/defconfig/debian/300s/qemu-vm/mmap-pread-seq/vm-scalability commit: 6.8.0-rc1-mm-everything-2024-01-29-07-19-no-mglru+ 6.8.0-rc1-split-folio-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-free-page-split-no-mglru+ 6.8.0-rc1-mm-eve 6.8.0-rc1-split-folio-in-co 6.8.0-rc1-folio-migration-i 6.8.0-rc1-folio-migration-f ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 11407497 +5.1% 11989632 -0.5% 11349272 +4.8% 11957423 vm-scalability.throughput ========================================================================================= compiler/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-13/defconfig/debian/300s/qemu-vm/mmap-pread-seq-mt/vm-scalability commit: 6.8.0-rc1-mm-everything-2024-01-29-07-19-no-mglru+ 6.8.0-rc1-split-folio-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-free-page-split-no-mglru+ 6.8.0-rc1-mm-eve 6.8.0-rc1-split-folio-in-co 6.8.0-rc1-folio-migration-i 6.8.0-rc1-folio-migration-f ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 11348474 +3.3% 11719453 -1.2% 11208759 +3.7% 11771926 vm-scalability.throughput ========================================================================================= compiler/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-13/defconfig/debian/300s/qemu-vm/lru-file-readtwice/vm-scalability commit: 6.8.0-rc1-mm-everything-2024-01-29-07-19-no-mglru+ 6.8.0-rc1-split-folio-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-free-page-split-no-mglru+ 6.8.0-rc1-mm-eve 6.8.0-rc1-split-folio-in-co 6.8.0-rc1-folio-migration-i 6.8.0-rc1-folio-migration-f ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 8065614 ± 3% +7.7% 8686626 ± 2% +5.0% 8467577 ± 4% +11.8% 9016077 ± 2% vm-scalability.throughput ========================================================================================= compiler/kconfig/rootfs/runtime/tbox_group/test/testcase: gcc-13/defconfig/debian/300s/qemu-vm/lru-file-mmap-read/vm-scalability commit: 6.8.0-rc1-mm-everything-2024-01-29-07-19-no-mglru+ 6.8.0-rc1-split-folio-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-in-compaction-no-mglru+ 6.8.0-rc1-folio-migration-free-page-split-no-mglru+ 6.8.0-rc1-mm-eve 6.8.0-rc1-split-folio-in-co 6.8.0-rc1-folio-migration-i 6.8.0-rc1-folio-migration-f ---------------- --------------------------- --------------------------- --------------------------- %stddev %change %stddev %change %stddev %change %stddev \ | \ | \ | \ 6438422 ± 2% +27.5% 8206734 ± 2% +10.6% 7118390 +26.2% 8127192 ± 4% vm-scalability.throughput [1] https://lore.kernel.org/linux-mm/20230912162815.440749-1-zi.yan@sent.com/ [2] https://lore.kernel.org/linux-mm/20231113170157.280181-1-zi.yan@sent.com/ [3] https://lore.kernel.org/linux-mm/20240123034636.1095672-1-zi.yan@sent.com/ [4] https://lore.kernel.org/linux-mm/23BA8CC1-1014-4D09-9C33-938638E13C01@nvidia.com/ Zi Yan (3): mm/compaction: enable compacting >0 order folios. mm/compaction: add support for >0 order folio memory compaction. mm/compaction: optimize >0 order folio compaction with free page split. mm/compaction.c | 219 ++++++++++++++++++++++++++++++++++-------------- mm/internal.h | 9 +- mm/page_alloc.c | 6 ++ 3 files changed, 170 insertions(+), 64 deletions(-) Reviewed-by: Baolin Wang Tested-by: Baolin Wang