From patchwork Thu May 25 12:53:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 99018 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp348062vqr; Thu, 25 May 2023 06:01:33 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5a5vXWN5/YpNEUEMtbLxvAPTHqgfBdgalK1nQxddmrZ/ibjTGp7BF9+hwGg+hQqmrkw/r1 X-Received: by 2002:a17:902:b204:b0:1af:eb1c:80ba with SMTP id t4-20020a170902b20400b001afeb1c80bamr1535552plr.8.1685019693442; Thu, 25 May 2023 06:01:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685019693; cv=none; d=google.com; s=arc-20160816; b=Lsz8zRzJMGrNH3etqaFMK9f8uLa56gVWACHr02ItSwrpFPHYBVhv+n7BuoSx4k4g9b LbHwie0SA0xHa4wTg1FuEdcWm/XW1zxZgiXfkqEIf+NPwsl57ie84ioyTc1Lk/7i6eez 5xaxeHMq4DRiqvZAZbP5s+Q9at4lilRiMs0t3Cr0X//fYuTHz+/NtRpNtGAnGIn+TXxM QcvbJ7lcuTHCkVYDuJnUapkEsXcpkCF69Nkpo7TZyYXavZC/T4ny+IalTrJp/vcP5haG 9xiD0VfbgvtI6o7W7wsPwMtRC6RcsIPxEyjtKsxd7YyHIwcmKFx2izVfke+7BBvbR59H 0tZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=3zql6VrCbwQTXhnkE3JZRhUnjSXniNlSH3VFs4iHJ90=; b=lWlpIhR0mTzFb3JP4l5QCr+gxF1Dto2I74JXtl+SYDA9+KTmfzwHVe68Z+JoZJ/lwI eJJIvWXqOsFN7bGZwzMLEvKBTwuEADe2tKjTUWELnzWkimq2wL7tutxGB9k/tZm+9c1t LFBCa3GFgie3QdfxSrvUjo5n6UNJ2hGjvqQGl/L3YYRyLkXaVzn81Rywz3asQHgeo1iI eAJD+8F4GKPa+LFmnRb73uPZLZPxov3WbysI2MlXZg9pSFs97WHt1OUXk4qOduPA3gWW i3dKNcOXhcqAsdZdgsKxoNEgvUFODpdvnT6zmRGN8LRuR9RtA0Lj5F7gIrzGLhoBt9lM +sVw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id n4-20020a170903110400b001ab23cbe13bsi1537835plh.104.2023.05.25.06.01.17; Thu, 25 May 2023 06:01:33 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236025AbjEYMyT (ORCPT + 99 others); Thu, 25 May 2023 08:54:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37130 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231263AbjEYMyS (ORCPT ); Thu, 25 May 2023 08:54:18 -0400 Received: from out30-124.freemail.mail.aliyun.com (out30-124.freemail.mail.aliyun.com [115.124.30.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2EB7B135 for ; Thu, 25 May 2023 05:54:16 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R101e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046050;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=6;SR=0;TI=SMTPD_---0VjSNUk1_1685019251; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VjSNUk1_1685019251) by smtp.aliyun-inc.com; Thu, 25 May 2023 20:54:12 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: mgorman@techsingularity.net, vbabka@suse.cz, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/6] mm: compaction: drop the redundant page validation in update_pageblock_skip() Date: Thu, 25 May 2023 20:53:56 +0800 Message-Id: <5142e15b9295fe8c447dbb39b7907a20177a1413.1685018752.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE,UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766871209850345222?= X-GMAIL-MSGID: =?utf-8?q?1766871209850345222?= The caller has validated the page before calling pdate_pageblock_skip(), thus drop the redundant page validation in update_pageblock_skip(). Signed-off-by: Baolin Wang Acked-by: Vlastimil Babka --- mm/compaction.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index 163e2ec70aff..426bb6ce070b 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -436,9 +436,6 @@ static void update_pageblock_skip(struct compact_control *cc, if (cc->no_set_skip_hint) return; - if (!page) - return; - set_pageblock_skip(page); /* Update where async and sync compaction should restart */ From patchwork Thu May 25 12:53:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 99013 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp347408vqr; Thu, 25 May 2023 06:00:54 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7n1bBPPW8jgtSzTkLNhSMY47eJSWhHwtKX+bT91+0TMA7LEwWd6XljA9amdH+bdmCTfqGB X-Received: by 2002:a17:90a:4092:b0:255:69e5:2f6c with SMTP id l18-20020a17090a409200b0025569e52f6cmr1771543pjg.26.1685019653348; Thu, 25 May 2023 06:00:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685019653; cv=none; d=google.com; s=arc-20160816; b=rwLsW2TJJ+CLQeOwHVHyT9Pkm3fg3REYTMXWLlnsyhtn1oVSJCu0uodB/xQy8184XI Z7aLjxK7vd8mLvrGvfkdDW/Jnvd096xEtjnW1wvpa5Pv5eO2MnWL//U/fX1kzTGgVryc Q5TElEe0IXGlrYyJDAA87wV4nnqmHLxWcbBwfh+wlnt0TfyuRdy8BXfnUK5o0mWcsXaG HVFu62FjxIG4jRjSGxedyx/AJ4Bh05cfziVMnFtBWt6bHuVhskvSlcF7AV3gUl+AOkZD P+gzEV/UifZjE8qby6fu6wADsQkYATF3nLRaDgvD5bCWMLi7vkzuIM0CqYDBcNrBRrQP 8ZXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=mreEMDSeY2JC7ArjqMMVi/9sS2N/VZgQ7hVIQpjkd4M=; b=ilDUR9Mf4MvxpnnGmKpDn8Y8/4lPGmjIZo+tOmDFT+xSTAt6ao4yLbmWPInCSaRHlQ s8BrrhfPiMjC/31Bf8tnu1464RTa+GA+gzzCrTJrs4i7HlfDmaQSVZqGLD5Q/wtIt766 YzLmGu10x09qVmHv0c/SZoRAHSfLUKaFA5ADrDeF+8HezJ+qi8HUq6pkHdU3rguoKP6v cH4Ooz9yFhdrADavVIPG4BzahMjWhP3jXHzWSova7OHRr3Pu6u/3PfM+E+rRE7ihJOzi ajOemtwLQYk3vla0P+wxUzln27juS4KX9QbBHQUX3rNiHQBXrW/B5s8qjzR/nnVXoZdR akkg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id c19-20020a17090abf1300b00250db0f5689si1439767pjs.179.2023.05.25.06.00.39; Thu, 25 May 2023 06:00:53 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240158AbjEYMyU (ORCPT + 99 others); Thu, 25 May 2023 08:54:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37134 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231895AbjEYMyS (ORCPT ); Thu, 25 May 2023 08:54:18 -0400 Received: from out30-110.freemail.mail.aliyun.com (out30-110.freemail.mail.aliyun.com [115.124.30.110]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9D241187 for ; Thu, 25 May 2023 05:54:16 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R621e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045168;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=6;SR=0;TI=SMTPD_---0VjSJ3fp_1685019252; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VjSJ3fp_1685019252) by smtp.aliyun-inc.com; Thu, 25 May 2023 20:54:13 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: mgorman@techsingularity.net, vbabka@suse.cz, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/6] mm: compaction: change fast_isolate_freepages() to void type Date: Thu, 25 May 2023 20:53:57 +0800 Message-Id: <759fca20b22ebf4c81afa30496837b9e0fb2e53b.1685018752.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE,UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766871168191826684?= X-GMAIL-MSGID: =?utf-8?q?1766871168191826684?= No caller cares about the return value of fast_isolate_freepages(), void it. Signed-off-by: Baolin Wang Acked-by: Vlastimil Babka --- mm/compaction.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index 426bb6ce070b..3737c6591bfb 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1435,8 +1435,7 @@ static int next_search_order(struct compact_control *cc, int order) return order; } -static unsigned long -fast_isolate_freepages(struct compact_control *cc) +static void fast_isolate_freepages(struct compact_control *cc) { unsigned int limit = max(1U, freelist_scan_limit(cc) >> 1); unsigned int nr_scanned = 0; @@ -1449,7 +1448,7 @@ fast_isolate_freepages(struct compact_control *cc) /* Full compaction passes in a negative order */ if (cc->order <= 0) - return cc->free_pfn; + return; /* * If starting the scan, use a deeper search and use the highest @@ -1588,11 +1587,10 @@ fast_isolate_freepages(struct compact_control *cc) cc->total_free_scanned += nr_scanned; if (!page) - return cc->free_pfn; + return; low_pfn = page_to_pfn(page); fast_isolate_around(cc, low_pfn); - return low_pfn; } /* From patchwork Thu May 25 12:53:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 99017 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp348080vqr; Thu, 25 May 2023 06:01:34 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5B4qxM6fiVXiGHWiiCYueo+3gBzGz8MA0sfc0PK1L9OVGM9eagp2OzISC/m4dMTAi8J9uP X-Received: by 2002:a4a:614b:0:b0:54f:49ad:1c8f with SMTP id u11-20020a4a614b000000b0054f49ad1c8fmr8655217ooe.1.1685019694625; Thu, 25 May 2023 06:01:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685019694; cv=none; d=google.com; s=arc-20160816; b=u7VewEjVX54R2/v5e67KczL//b5PvMtCLfngUOmoyNn+beUec6PuIa36MHfTlFgvaC RdJk8OoHEynZm0qxemEjcZHNCsOGbmOnN/acKUnMHc3DyR4Q7O8MJErnGJAwBeM7Bw3A iCuCMA7Nfv55Xa2kjJX4poLwuHVuqeoDuMxrVVivHmEvGeZFRop70UdU0CGaBBobBvTv 72F/MrVTgPs9dY4YuL+xRW9AJHq9vTWe1VS49cxUJ7OY0/KrDVosD+wC5h+ZJLtjfyk+ Z9g9vmFbf2bUhCd9HINYuikK1vjHOLgt8AP4c7DukHqx18+eV4t6gdA0vJ406H9pirOV cprg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=9PEP4Qe0owgriL+os+ebwHktoCpGXaTA8NLjInkUkIw=; b=qftLKiFD+qNX6j9gY7Nfzk40d+4cSogwWQO6vySPvVO/auXcsVrAgq0beIIz9fekOh 5Yz23mRNf6SVjCLu6nB7+oG9HJMCpavhDoHdZrBfqyNZw8B6cXU6PuizKzqj9SJFtx0c PXQnCg+itlaa2ujTS7X3qCP78AAZaCtX0RzRDKni4/61lcJ9V/MnXiVYpzHfXcewOyGI Es3tTnMOTL44UKg5Hn3zpaLN3vPKKHpJbu0Z5jVVNEQ9IF3MtQOsr3GlahLi1Y1zq1Hi eIOZWjZ9UPfrZtsPKg2vs46/GzAWDRjSa0JQF/HjzHfYNUjf5xmuuJeNhgIvr8ew8CMU 6qkA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y4-20020a4aea24000000b005476b953946si572155ood.37.2023.05.25.06.01.20; Thu, 25 May 2023 06:01:34 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240960AbjEYMy0 (ORCPT + 99 others); Thu, 25 May 2023 08:54:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37142 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236065AbjEYMyT (ORCPT ); Thu, 25 May 2023 08:54:19 -0400 Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4B46D183 for ; Thu, 25 May 2023 05:54:18 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R861e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046049;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=6;SR=0;TI=SMTPD_---0VjSJ3gf_1685019253; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VjSJ3gf_1685019253) by smtp.aliyun-inc.com; Thu, 25 May 2023 20:54:14 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: mgorman@techsingularity.net, vbabka@suse.cz, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/6] mm: compaction: skip more fully scanned pageblock Date: Thu, 25 May 2023 20:53:58 +0800 Message-Id: X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE,UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766871210847564347?= X-GMAIL-MSGID: =?utf-8?q?1766871210847564347?= In fast_isolate_around(), it assumes the pageblock is fully scanned if cc->nr_freepages < cc->nr_migratepages after trying to isolate some free pages, and will set skip flag to avoid scanning in future. However this can miss setting the skip flag for a fully scanned pageblock (returned 'start_pfn' is equal to 'end_pfn') in the case where cc->nr_freepages is larger than cc->nr_migratepages. So using the returned 'start_pfn' from isolate_freepages_block() and 'end_pfn' to decide if a pageblock is fully scanned makes more sense. It can also cover the case where cc->nr_freepages < cc->nr_migratepages, which means the 'start_pfn' is usually equal to 'end_pfn' except some uncommon fatal error occurs after non-strict mode isolation. Signed-off-by: Baolin Wang Acked-by: Vlastimil Babka --- mm/compaction.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/compaction.c b/mm/compaction.c index 3737c6591bfb..1e5183f39ca9 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1411,7 +1411,7 @@ fast_isolate_around(struct compact_control *cc, unsigned long pfn) isolate_freepages_block(cc, &start_pfn, end_pfn, &cc->freepages, 1, false); /* Skip this pageblock in the future as it's full or nearly full */ - if (cc->nr_freepages < cc->nr_migratepages) + if (start_pfn == end_pfn) set_pageblock_skip(page); return; From patchwork Thu May 25 12:53:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 99015 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp347558vqr; Thu, 25 May 2023 06:01:01 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6EtnaxQJF3PsFWtTOqKMxm6W5xuJ18LE/UhOn/Dn0CUchOvlg/78IeBJQBOOEH10sb/y2Q X-Received: by 2002:a05:6a00:2382:b0:64c:e899:dcd1 with SMTP id f2-20020a056a00238200b0064ce899dcd1mr8390418pfc.5.1685019661424; Thu, 25 May 2023 06:01:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685019661; cv=none; d=google.com; s=arc-20160816; b=VdyHRU3S1Jjgkx+nPHCxv30dXoAKsdKl3slQ4bd4fUneaChV/WpHogEirSVQWR5FuQ JApbcSqdd0UMinMTpseKDCUKnQdYV5ZAYjggG+CxbLnFOcTtGCY/pvSkB2Wu7yKSPZSr uqFuUCce1+tTCYZEXxX3IXo6OD3bnRqiCVZBd9RS6wJ8vkzgEAmX1dtTgDxK7bjlX3Iq jOnXaHOiM/wdsOQgws3JKUXiWi3vR6CfWqRq4mZKalX7m0+gGqBtIbUQSlISxeVgX1AE RR0VckakkQinJEgvGtQl0BiPLX+o7B2OnErAILcpxKxccDJfcdLl6B57lW1QYBSRwhEf HCcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=G6LTpZvzHfBwjs0sQGnuwspELdetmSx1jF49jzAUX2k=; b=wp+8fpNsd6KXT2KVDfOS65wqbAA8cexxa4SjKIfAMsR+PawXN4fgFskj9cf5tWK33O 0TY3LCnmaysT3qQL0NQlzP/1PNsNlpluZYXaU3ZG5rEW5FobgLCSJmc3DOZ3CzsOBvJn qlFucV/1ApBGffsUu2BChZ8FcZp/UCqRMECuuLkxEskp6mtFLWsRs3WAXrdEZ/9DQStO QlzFxZunjQOSDMNr5CaKNXjzEKb7+gJCzisbV1uKnpi29xmNl344Ji7NSXIcq4m57GB1 mquJ/Wi8X5dQaSXAVsaUayZMp/jcCJz/DKl8k+mmklCfOTgPfkAGOKwUmzlfSXkj2huj SjsQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h62-20020a638341000000b005192652814bsi1144405pge.541.2023.05.25.06.00.48; Thu, 25 May 2023 06:01:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240973AbjEYMy1 (ORCPT + 99 others); Thu, 25 May 2023 08:54:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37152 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239813AbjEYMyU (ORCPT ); Thu, 25 May 2023 08:54:20 -0400 Received: from out30-100.freemail.mail.aliyun.com (out30-100.freemail.mail.aliyun.com [115.124.30.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C0A44183 for ; Thu, 25 May 2023 05:54:19 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R121e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045170;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=6;SR=0;TI=SMTPD_---0VjSJ3hf_1685019254; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VjSJ3hf_1685019254) by smtp.aliyun-inc.com; Thu, 25 May 2023 20:54:15 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: mgorman@techsingularity.net, vbabka@suse.cz, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 4/6] mm: compaction: only set skip flag if cc->no_set_skip_hint is false Date: Thu, 25 May 2023 20:53:59 +0800 Message-Id: <0eb2cd2407ffb259ae6e3071e10f70f2d41d0f3e.1685018752.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE,UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766871176809074202?= X-GMAIL-MSGID: =?utf-8?q?1766871176809074202?= To keep the same logic as test_and_set_skip(), only set the skip flag if cc->no_set_skip_hint is false, which makes code more reasonable. Signed-off-by: Baolin Wang Acked-by: Vlastimil Babka --- mm/compaction.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/compaction.c b/mm/compaction.c index 1e5183f39ca9..65d8d9223acc 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1223,7 +1223,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, * rescanned twice in a row. */ if (low_pfn == end_pfn && (!nr_isolated || cc->finish_pageblock)) { - if (valid_page && !skip_updated) + if (!cc->no_set_skip_hint && valid_page && !skip_updated) set_pageblock_skip(valid_page); update_cached_migrate(cc, low_pfn); } From patchwork Thu May 25 12:54:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 99009 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp346945vqr; Thu, 25 May 2023 06:00:23 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4aYpmE3f3PTTj6m3GOxWZYReHXnmuQmvHm1OlF5XcqQQ5lPo4GU9LNrbGMHB+B2LlVlKzS X-Received: by 2002:a17:90a:a88b:b0:252:aa5e:e54f with SMTP id h11-20020a17090aa88b00b00252aa5ee54fmr1748288pjq.13.1685019622689; Thu, 25 May 2023 06:00:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685019622; cv=none; d=google.com; s=arc-20160816; b=fuGQ5qsFvt8uEqNWoFi9tQxBfELVf4xHxjmaI4MqpDZuTX2v4Ui7lgW7pP0QPjz+Us YqnV1GGHYuRm4Pu1lIHZBKUXNt000Ir5jMkYnzG05x2hkFECwr3yu3lKHJgyRp2DWxg8 a3aBFdb/0rWNCVuMfdrjDHZjp1aGDaat9cC1DtR0NfNHZCaKUpW4TdVIHWo//D4OPEVf GubKnBl0C5Ybf0XJKvlxCJCf/sdM2KliwEP6JRiHGFEgvk26rEY4i9WcNBWE3qpJ/Ddz 3qh6oZvkHzl80/bKYgwRodHen/6gsVaDgo9SSOA3tkVxMY/xqJqfXRAaz4USQkyMc3zw ilLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=SdT5R3m9rZCokN9uO6H95AhoqA+livNaJXRoPZKNaD8=; b=jKzkv0kiLlZorqgF8lFSPO+Jv+V6c70t7OATbur3coEYKT3C24DSUxD7tJ6OWVIabd okq/lES9lwU0oRi3vhfY4Nf6x8xXJtcif/ywpMmzyQ4sm2PpsD51xE3riGvSo0Mixpji oTcT16u3ofrWQFxK1VRXnSrLSox9G3J+7axEehm0RzCmfKj8T5X+F2+oEjXO1mXlp22o 4pP6tTvQ++kcYlfp9tBLGWgPpsJ8vjNsOPUwyiC2yu6wuIqxM0zd924PyQKn4WMhTlmV QzTYE59Jr8pYn/hq69DjSSUrCQcE47Y6k5GNur8rstcBUDTBzlevE15TRRpNfl5ywmq3 Vhcw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id fh5-20020a17090b034500b0024e3c22a7dbsi2427665pjb.113.2023.05.25.06.00.08; Thu, 25 May 2023 06:00:22 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241078AbjEYMya (ORCPT + 99 others); Thu, 25 May 2023 08:54:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37164 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231895AbjEYMyV (ORCPT ); Thu, 25 May 2023 08:54:21 -0400 Received: from out30-124.freemail.mail.aliyun.com (out30-124.freemail.mail.aliyun.com [115.124.30.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 97C5D187 for ; Thu, 25 May 2023 05:54:20 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R781e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045168;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=6;SR=0;TI=SMTPD_---0VjSMoYJ_1685019255; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VjSMoYJ_1685019255) by smtp.aliyun-inc.com; Thu, 25 May 2023 20:54:16 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: mgorman@techsingularity.net, vbabka@suse.cz, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 5/6] mm: compaction: add trace event for fast freepages isolation Date: Thu, 25 May 2023 20:54:00 +0800 Message-Id: <78d2932d0160d122c15372aceb3f2c45460a17fc.1685018752.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE,UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766871135748557893?= X-GMAIL-MSGID: =?utf-8?q?1766871135748557893?= The fast_isolate_freepages() can also isolate freepages, but we can not know the fast isolation efficiency to understand the fast isolation pressure. So add a trace event to show some numbers to help to understand the efficiency for fast freepages isolation. Signed-off-by: Baolin Wang Acked-by: Vlastimil Babka --- include/trace/events/compaction.h | 11 +++++++++++ mm/compaction.c | 6 +++++- 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/include/trace/events/compaction.h b/include/trace/events/compaction.h index 3313eb83c117..2b2a975efd20 100644 --- a/include/trace/events/compaction.h +++ b/include/trace/events/compaction.h @@ -64,6 +64,17 @@ DEFINE_EVENT(mm_compaction_isolate_template, mm_compaction_isolate_freepages, TP_ARGS(start_pfn, end_pfn, nr_scanned, nr_taken) ); +DEFINE_EVENT(mm_compaction_isolate_template, mm_compaction_fast_isolate_freepages, + + TP_PROTO( + unsigned long start_pfn, + unsigned long end_pfn, + unsigned long nr_scanned, + unsigned long nr_taken), + + TP_ARGS(start_pfn, end_pfn, nr_scanned, nr_taken) +); + #ifdef CONFIG_COMPACTION TRACE_EVENT(mm_compaction_migratepages, diff --git a/mm/compaction.c b/mm/compaction.c index 65d8d9223acc..eccec84dae82 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1438,7 +1438,7 @@ static int next_search_order(struct compact_control *cc, int order) static void fast_isolate_freepages(struct compact_control *cc) { unsigned int limit = max(1U, freelist_scan_limit(cc) >> 1); - unsigned int nr_scanned = 0; + unsigned int nr_scanned = 0, total_isolated = 0; unsigned long low_pfn, min_pfn, highest = 0; unsigned long nr_isolated = 0; unsigned long distance; @@ -1537,6 +1537,7 @@ static void fast_isolate_freepages(struct compact_control *cc) set_page_private(page, order); nr_isolated = 1 << order; nr_scanned += nr_isolated - 1; + total_isolated += nr_isolated; cc->nr_freepages += nr_isolated; list_add_tail(&page->lru, &cc->freepages); count_compact_events(COMPACTISOLATED, nr_isolated); @@ -1557,6 +1558,9 @@ static void fast_isolate_freepages(struct compact_control *cc) limit = max(1U, limit >> 1); } + trace_mm_compaction_fast_isolate_freepages(min_pfn, cc->free_pfn, + nr_scanned, total_isolated); + if (!page) { cc->fast_search_fail++; if (scan_start) { From patchwork Thu May 25 12:54:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 99012 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp347241vqr; Thu, 25 May 2023 06:00:40 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5WsLSmOvGt+j3mWZcdI+gMf+tv2Ub9dm1ryhCUBxovmRzSeXm8XgieigEICZCX1lnHY/1S X-Received: by 2002:a17:90a:c7cf:b0:255:cddf:a0b8 with SMTP id gf15-20020a17090ac7cf00b00255cddfa0b8mr1580946pjb.47.1685019639736; Thu, 25 May 2023 06:00:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685019639; cv=none; d=google.com; s=arc-20160816; b=KqL6H8Al9OorhBe1d6I/GzCMVUptsJxKJHVEVc6R89iRN2SMsTlgRkvnK2+N7Ksh0M bU02mGYlNj7RNchWBtjTCaoKpS5D0RNDqeMxvuYUN/pUQzSx0noc1vsytr16XVSeBIGD QvhQkikTYUTGsqcj25cce1NFKAIlAdd0XS0M2ixwRagyvSBBjaHo1/bWCQGLdIZgJzTa RCBHW/vX7TH4uSqd+mjjqmoTqwGWGw9HAFkzogp7hY4Z9PgK9Fh+8ScHXii2Hi8YzRvm /Lpiz/uK08YlNceGMjVwPSu4EkHKuviFDtF3aTrbkzTHAqvig3jk0GsJXYQZrnlKK1p4 NYDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=Mn3fhMFcaTYsz7UQ9NrJ1DjTrAq5Xs3DKzZTvyhCQfA=; b=jDku580Va0BgV2jKNnEAGVSV3eo68GXBguUJclU3iWiOe4zm2fj2uudZEXpwBTMZMg eutNU7wcUT2GcwSmDjvaKEHvvU+GgyZBY6PEO/uKRvvxpjExmk22DriOsrfYdI2skULn 99gTbLlDJQJbBc4exu/5Irr/GtIR2412VdRdQ+mVaxDsPMWaTkxzNcQM4La0855sKLpv XCEDWyYezTpAFHoruJzeNZtdfwW4u8tCh5w+e9yefwF/xroGNbTT+WpIC12mkrgr/TA6 +/12gzd8KWqidV5CTIwya7YKovGUHlhEQYIe3OK7ljQAYDoCDFrl8KAiCJWwP0sgYbtM VF6w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l11-20020a17090a598b00b00252f4ca40c4si3636165pji.34.2023.05.25.06.00.24; Thu, 25 May 2023 06:00:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241105AbjEYMyc (ORCPT + 99 others); Thu, 25 May 2023 08:54:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37164 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240536AbjEYMyW (ORCPT ); Thu, 25 May 2023 08:54:22 -0400 Received: from out30-97.freemail.mail.aliyun.com (out30-97.freemail.mail.aliyun.com [115.124.30.97]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E6FB3189 for ; Thu, 25 May 2023 05:54:20 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R711e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046059;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=6;SR=0;TI=SMTPD_---0VjSJ3iS_1685019256; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VjSJ3iS_1685019256) by smtp.aliyun-inc.com; Thu, 25 May 2023 20:54:17 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: mgorman@techsingularity.net, vbabka@suse.cz, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 6/6] mm: compaction: skip fast freepages isolation if enough freepages are isolated Date: Thu, 25 May 2023 20:54:01 +0800 Message-Id: X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE,UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1766871153711952948?= X-GMAIL-MSGID: =?utf-8?q?1766871153711952948?= I've observed that fast isolation often isolates more pages than cc->migratepages, and the excess freepages will be released back to the buddy system. So skip fast freepages isolation if enough freepages are isolated to save some CPU cycles. Signed-off-by: Baolin Wang Acked-by: Vlastimil Babka --- mm/compaction.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mm/compaction.c b/mm/compaction.c index eccec84dae82..3ade4c095ed2 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1550,6 +1550,10 @@ static void fast_isolate_freepages(struct compact_control *cc) spin_unlock_irqrestore(&cc->zone->lock, flags); + /* Skip fast search if enough freepages isolated */ + if (cc->nr_freepages >= cc->nr_migratepages) + break; + /* * Smaller scan on next order so the total scan is related * to freelist_scan_limit.