From patchwork Wed Apr 5 16:18:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 79796 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp443295vqo; Wed, 5 Apr 2023 09:38:59 -0700 (PDT) X-Google-Smtp-Source: AKy350Z8KG0iiHNmX4INzBhOyc1hBE0MHrNw20tdaiQ+mcH1W8J6XNf1g5qgh+Bwlftw180+ekqb X-Received: by 2002:aa7:8f29:0:b0:627:e49a:8714 with SMTP id y9-20020aa78f29000000b00627e49a8714mr6487698pfr.9.1680712739560; Wed, 05 Apr 2023 09:38:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680712739; cv=none; d=google.com; s=arc-20160816; b=mPWNu3+N6lPmOK8veBC4LcgCEq8HD2DBja/+9cC9UrYYHbms54EDHmxQoLSkiacLxa BYFmJiprl9uU6ZOwBYtQTnqYChTkSLDYDlmuxdNdrc0YP1U1T/gEZlGeJKqJaQS86ddO qF1979x4GZqauAdGqA3zrz5Yha0SxgyKLOYCSQEOMUidkZvYrT/MdKPOB37Yog8DlzNm UReEoZjrVMDUw5toxcuo2XjzILCni22lI1StLuYzf9q7h0WIRccQhY/Kd93cgJLNEpxp 6U4zQPCOMpa/VbGgjeKudDyaZhGDly59zz77pMOd7S1Y+RXyimCpnhyr2yuhGRJkvw/I LlpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=1To4EWudWGMS35VthxJEVPn4ml+5FfmSSEC6TcGeIwk=; b=QHVzVv2iYxNU+gRi8i252sR2wwaYVuKSVemjWlD+wT5x+4StuYvnLNlxQnL0a3JwMy bWS47zwi1LBddN+o3WEAAFcCrM6t7z8JQMqkj9P39+La0omcEB6HMefQCjCAfA1Kl4YF BLQ/Zdr8Bz8THLuj89RJ0oDpwtPZ29vnFFexT0NvdwvAb48WJJim7BX9hvLmddpMhhtz UPmdPB1C4kdvbCDXWemqWd3Y7wicvH4Xs4rHNGNXHxIi7eYS1FSRHZXoe6Ok1AchiV7H inBMBpG+1o035oP/GZzJGFM+7sz8H3erv2UQkxIGraPmijcKzh1V1DTlNDmDzrycNDHx iMIA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=WIri22gc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z9-20020aa79589000000b00627f6643c07si748727pfj.185.2023.04.05.09.38.47; Wed, 05 Apr 2023 09:38:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=WIri22gc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231191AbjDEQTi (ORCPT + 99 others); Wed, 5 Apr 2023 12:19:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51076 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230125AbjDEQTc (ORCPT ); Wed, 5 Apr 2023 12:19:32 -0400 Received: from mail-pg1-x52f.google.com (mail-pg1-x52f.google.com [IPv6:2607:f8b0:4864:20::52f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6181F18F for ; Wed, 5 Apr 2023 09:19:09 -0700 (PDT) Received: by mail-pg1-x52f.google.com with SMTP id 41be03b00d2f7-503e712b4a1so91786a12.0 for ; Wed, 05 Apr 2023 09:19:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1680711549; x=1683303549; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=1To4EWudWGMS35VthxJEVPn4ml+5FfmSSEC6TcGeIwk=; b=WIri22gcv76zZgGKYCx26KszTqldyT8X6yf2bGElCpnSwQnDbraJXBaEcsLvbViTwi Wf6OppbwwjoohEEzK1A5/liDV7LqBW3U+pWSgPcI2sAOtGe7SGvMHM7NlKuLxBnkEJVu 4p2OeLHdMa5o7kJzkXZhPrwpx/f6A0w/qLBod8JTmiUwJa7ynfvRx72WCUCdhDaD6jqZ ss7ZbhS12KjvH8BLVL2FAslvEWx/EvgaU8lvilFwH731oKKW+7eBCSgxFPjUlSOzeTyh UvsT7nM66ZrHANZ3zhZJBAHvN/qa4qxHe/9M3c1h/W180TPinfY65ggffPvVI2HdKL8E jPDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680711549; x=1683303549; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=1To4EWudWGMS35VthxJEVPn4ml+5FfmSSEC6TcGeIwk=; b=Mn9TRLWAnS+7KORvQqYh7EmF65thdyt/Bg1i1xuEN8ZaNGklCgU6q/zv9FgyG/s9Ac JveMTf6t4wFcPnCNPTV90n14SSUKwGG6lTslLbQv8f1M1Ick8+JZWMULCdI8H2mPLkYp sy+rtDPNv946fVA0aQ/Sx+aHekIW+zWe2DQSb5T/Vuz3XWy0BBmgzgpmN4BJFnF9GGVX OIIrWVDez1owJFkWgmgJGJQBPnRmjyhK0DvM4Dx2DC22GWqPtFgQo/UDfSB1PYPit+SP q/TN3GLL7OktnoHLE1Q30pFVt555ddBJjdWiJgAW+O68M0JzKlmGfzUa3mMhv8TF1pWU qTog== X-Gm-Message-State: AAQBX9cMLcp73hqQONRKs7oDZLw55z/Wx+xRbxmyBhDWzf0bMZwuYXRT G92sN/sjEGLFZPr/RnsFwmJr/Q== X-Received: by 2002:a05:6a21:6da0:b0:e4:9a37:2707 with SMTP id wl32-20020a056a216da000b000e49a372707mr2934273pzb.5.1680711548776; Wed, 05 Apr 2023 09:19:08 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.244]) by smtp.gmail.com with ESMTPSA id x24-20020a62fb18000000b00582f222f088sm10878011pfm.47.2023.04.05.09.19.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Apr 2023 09:19:08 -0700 (PDT) From: Qi Zheng To: akpm@linux-foundation.org, willy@infradead.org, lstoakes@gmail.com Cc: mgorman@suse.de, vbabka@suse.cz, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Qi Zheng Subject: [PATCH v2 1/2] mm: swap: fix performance regression on sparsetruncate-tiny Date: Thu, 6 Apr 2023 00:18:53 +0800 Message-Id: <20230405161854.6931-1-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) MIME-Version: 1.0 X-Spam-Status: No, score=-0.2 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762355041828924835?= X-GMAIL-MSGID: =?utf-8?q?1762355041828924835?= The ->percpu_pvec_drained was originally introduced by commit d9ed0d08b6c6 ("mm: only drain per-cpu pagevecs once per pagevec usage") to drain per-cpu pagevecs only once per pagevec usage. But after converting the swap code to be more folio-based, the commit c2bc16817aa0 ("mm/swap: add folio_batch_move_lru()") breaks this logic, which would cause ->percpu_pvec_drained to be reset to false, that means per-cpu pagevecs will be drained multiple times per pagevec usage. In theory, there should be no functional changes when converting code to be more folio-based. We should call folio_batch_reinit() in folio_batch_move_lru() instead of folio_batch_init(). And to verify that we still need ->percpu_pvec_drained, I ran mmtests/sparsetruncate-tiny and got the following data: baseline with baseline/ patch/ Min Time 326.00 ( 0.00%) 328.00 ( -0.61%) 1st-qrtle Time 334.00 ( 0.00%) 336.00 ( -0.60%) 2nd-qrtle Time 338.00 ( 0.00%) 341.00 ( -0.89%) 3rd-qrtle Time 343.00 ( 0.00%) 347.00 ( -1.17%) Max-1 Time 326.00 ( 0.00%) 328.00 ( -0.61%) Max-5 Time 327.00 ( 0.00%) 330.00 ( -0.92%) Max-10 Time 328.00 ( 0.00%) 331.00 ( -0.91%) Max-90 Time 350.00 ( 0.00%) 357.00 ( -2.00%) Max-95 Time 395.00 ( 0.00%) 390.00 ( 1.27%) Max-99 Time 508.00 ( 0.00%) 434.00 ( 14.57%) Max Time 547.00 ( 0.00%) 476.00 ( 12.98%) Amean Time 344.61 ( 0.00%) 345.56 * -0.28%* Stddev Time 30.34 ( 0.00%) 19.51 ( 35.69%) CoeffVar Time 8.81 ( 0.00%) 5.65 ( 35.87%) BAmean-99 Time 342.38 ( 0.00%) 344.27 ( -0.55%) BAmean-95 Time 338.58 ( 0.00%) 341.87 ( -0.97%) BAmean-90 Time 336.89 ( 0.00%) 340.26 ( -1.00%) BAmean-75 Time 335.18 ( 0.00%) 338.40 ( -0.96%) BAmean-50 Time 332.54 ( 0.00%) 335.42 ( -0.87%) BAmean-25 Time 329.30 ( 0.00%) 332.00 ( -0.82%) From the above it can be seen that we get similar data to when ->percpu_pvec_drained was introduced, so we still need it. Let's call folio_batch_reinit() in folio_batch_move_lru() to restore the original logic. Fixes: c2bc16817aa0 ("mm/swap: add folio_batch_move_lru()") Signed-off-by: Qi Zheng Reviewed-by: Matthew Wilcox (Oracle) Acked-by: Mel Gorman --- Changlog in v1 to v2: - revise commit message and add test data mm/swap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/swap.c b/mm/swap.c index 57cb01b042f6..423199ee8478 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -222,7 +222,7 @@ static void folio_batch_move_lru(struct folio_batch *fbatch, move_fn_t move_fn) if (lruvec) unlock_page_lruvec_irqrestore(lruvec, flags); folios_put(fbatch->folios, folio_batch_count(fbatch)); - folio_batch_init(fbatch); + folio_batch_reinit(fbatch); } static void folio_batch_add_and_move(struct folio_batch *fbatch, From patchwork Wed Apr 5 16:18:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 79794 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp441092vqo; Wed, 5 Apr 2023 09:35:21 -0700 (PDT) X-Google-Smtp-Source: AKy350YXpfJEuDOrmXNEecBDZ5ZrJtRVmQykROSuOZKRlarwf5Ri7niqYUMWcjlPSWigHDRhwQU3 X-Received: by 2002:a17:906:868c:b0:947:4828:4399 with SMTP id g12-20020a170906868c00b0094748284399mr3617327ejx.12.1680712521034; Wed, 05 Apr 2023 09:35:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680712521; cv=none; d=google.com; s=arc-20160816; b=z2+PZHMMNk1O+vh+TRNdYZidymEpuWcnY0QNIMA6QqXiSDDRsu7UpDfnuXdFeU58LY W7e70HmUhl19MMi9kuxNlgt5X13VmeCmxqFsevz4YPBriNW7dnQ0EdRXubgXwB5GftbN 6h+lwFSVLKsPtvK+oHFHGU2KkbnEs5wPwjHOXMiuhtVpE9HL3WLUekDvOk0rN70Whod9 Y2K2PyOoQ/pqpIlbCMM0yd5v/wVQSZ8SMpZbbGsClyJRTXVb+IZm/6/ESkRQtMAcF9XJ HmxqNh/9qo12jnZckAm8fLFIfMsHit1fsWko+tO1fkdR9FiTUzsHPOLzt6tYoACRXaH/ oe/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=hIf6WOekKcX5cYk5n6UfHyTuwolCuQ5aOTxYYMf+HPA=; b=pKHod/VuUM+s3/YnFO8TnurDDiyvG11gtjNrSh5iTPQ67+TObL3qfD2UkKQVy+pkbV 82AVRRT18NK6AUqbaHGVaFx75jwj7N3xR5Okfmu+/R4LsrtsucmN0Q64HyoXDwf9n9mY VJsPhxorhPdtxQcX7rk1BqZ/TP6vHihYUB7b6saAa+llZhoE1fN22v8yMb3pnwxJev+G zkEGrtC6rReV2hH4EcKaKhnGVPZrbr5dwBqDNM0XFXkBOWTrYXBSIsQ8TXFGbkhZyAjJ /OMJK/squqBAFIK8AqmpbjW5xqeaOV9BsSkPFB4DvkGqD8SsM17BsHm83PAN7ULLWd48 AIMg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=GyF6XruE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bb1-20020a1709070a0100b0094995ed7ce3si665862ejc.351.2023.04.05.09.34.57; Wed, 05 Apr 2023 09:35:21 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance.com header.s=google header.b=GyF6XruE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231441AbjDEQTo (ORCPT + 99 others); Wed, 5 Apr 2023 12:19:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230335AbjDEQTi (ORCPT ); Wed, 5 Apr 2023 12:19:38 -0400 Received: from mail-pj1-x1029.google.com (mail-pj1-x1029.google.com [IPv6:2607:f8b0:4864:20::1029]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2EB3C10D1 for ; Wed, 5 Apr 2023 09:19:13 -0700 (PDT) Received: by mail-pj1-x1029.google.com with SMTP id 98e67ed59e1d1-23b2fe6c4f9so120294a91.0 for ; Wed, 05 Apr 2023 09:19:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1680711552; x=1683303552; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hIf6WOekKcX5cYk5n6UfHyTuwolCuQ5aOTxYYMf+HPA=; b=GyF6XruEcbEScRnuqs6RQtdnVAffeAwDNhR2YCAmw+Qc8S0WRFoL3Sxb4xjqjwQnQq OJ8Jiu9y2bolJdZp3mVpP2NSYELRJRHnJhmmEf704erywyOH/2UMAWbPe1zUpU1HuWGv VSOVAZddOLhrbspdDkzCd879KiYIXO2SObxHTSB/0VdvDjNYdSg/GRsAtG8hhds2RC8i 0YQxpgt1eQwUExDD4y4em5xkDQ4Ja2RDE4QesV1V72RAvOfmTpQ6TgOhMRmSY1LBV0Jw Y3K4AtG7RcIN10tIjIywe0UgWQ8Pfvq8uXWnf35XFo8HTAEdN8X6iyZOXwZycLtwVXuD 6flw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680711552; x=1683303552; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hIf6WOekKcX5cYk5n6UfHyTuwolCuQ5aOTxYYMf+HPA=; b=xKD0lQ788K8sheYee8YrIRiweT4DYkdmXk5ulTYLBj3E6QiFAkw1hF//otLfo5MSvI gse17g+wa2sSbhI7qfGJn/IPxqXbz18ViHa82JmZhBDE+YGpmJtVU640u7VCmfoO9DV/ q7yYwIhdVsQEN2LZrAp06TaU9m33HEadDRJNLcPnoexhj7dBNPOWHOAEB3FUxFlnWhto q0vO5CH5Gbcmvu5fTt6eme6tKOYCQ1Z+y2/ApgoqAXLZQWEb8LusdtRn7J/2kh7VYCud D+bCsQVqc/G5T4RFbDPe6YCHaAP9KrTQPjow4gIzb4awvqSciv7BrEQQPqijge/bUNzA +lnQ== X-Gm-Message-State: AAQBX9dgE7eJMWtdijOiW1zBYN9is4g9zoZk6SIJ4WgKCwWfagWYMCpe NyNbrYfWTuxdG6n/KyaRXMrmCw== X-Received: by 2002:a05:6a00:139d:b0:62d:e32a:8b5a with SMTP id t29-20020a056a00139d00b0062de32a8b5amr2835918pfg.2.1680711552583; Wed, 05 Apr 2023 09:19:12 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.244]) by smtp.gmail.com with ESMTPSA id x24-20020a62fb18000000b00582f222f088sm10878011pfm.47.2023.04.05.09.19.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Apr 2023 09:19:12 -0700 (PDT) From: Qi Zheng To: akpm@linux-foundation.org, willy@infradead.org, lstoakes@gmail.com Cc: mgorman@suse.de, vbabka@suse.cz, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Qi Zheng Subject: [PATCH v2 2/2] mm: mlock: use folios_put() in mlock_folio_batch() Date: Thu, 6 Apr 2023 00:18:54 +0800 Message-Id: <20230405161854.6931-2-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20230405161854.6931-1-zhengqi.arch@bytedance.com> References: <20230405161854.6931-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.2 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762354812256074941?= X-GMAIL-MSGID: =?utf-8?q?1762354812256074941?= Since we have updated mlock to use folios, it's better to call folios_put() instead of calling release_pages() directly. Signed-off-by: Qi Zheng Acked-by: Mel Gorman --- mm/mlock.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/mlock.c b/mm/mlock.c index 617469fce96d..40b43f8740df 100644 --- a/mm/mlock.c +++ b/mm/mlock.c @@ -206,7 +206,7 @@ static void mlock_folio_batch(struct folio_batch *fbatch) if (lruvec) unlock_page_lruvec_irq(lruvec); - release_pages(fbatch->folios, fbatch->nr); + folios_put(fbatch->folios, folio_batch_count(fbatch)); folio_batch_reinit(fbatch); }