From patchwork Thu Aug 10 10:33:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13445 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b824:0:b0:3f2:4152:657d with SMTP id z4csp338549vqi; Thu, 10 Aug 2023 04:09:07 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFiWnU2NYsHQmlTPbT45dnmu2gMogT9WCYxJu4lHZj8EwyhPLPmXD3zcFqzWigCcHm8qZrP X-Received: by 2002:a05:6402:385:b0:51d:95ac:22ed with SMTP id o5-20020a056402038500b0051d95ac22edmr2396310edv.1.1691665747103; Thu, 10 Aug 2023 04:09:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691665747; cv=none; d=google.com; s=arc-20160816; b=IMHg+eNSBKG2RpQi8c9rwwUsab4C6de+XTI7Qku+ai+hoIpoMv2/B7iHdvfX7SYvIX UROsmqJ0gaE9Cs5qcqECzYct2Po6ZcaF0xj8F9ldoEQtUCD/W8brpi1NDOEOt/qzXG5R 4tlG8vME0Pji+DCgOR1glszEEQQa8HZNWLRy6XhAL8Lbfe8nui+Dgz6YCtc0AO8mrXom eZ9zgBDaKOIo8H2JRpGg02y0JQCFfD6VzrsgJtdP2gC3H8FNqD/iCszZjj52KA9bj4sK 7bh3nJmcd75wjgBClV2vUlqWFCGakF2w+x8T6yd9W1j5NxMT8GRusZNmkMpwIsdOWTUv fWmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from; bh=3WSe5bChNNZzp5pgrhTbo7ZbVUijl9UYAECkb7P534g=; fh=y3Sw7VG+9eLsYabwWUWsSnvNS303XOkFtMtRzGTCOfU=; b=EpOUXBGEucX0XY4x+eJ+hAv//i56bzL/kydX7GODJcY2Nxzx5pCnY3uiHzKkpNVTnE Ota261C3wsr4b0auMlWIUIuJmvJNIagk/fl5nlCiVtupXxAsqWKkEc7aPxrsl4QeM4YO 4ZnTNjw+975ik/Igvs8R2JN2JyZV8gpzo4t3LPUudsJQ+4VWnzqS+n/nx8GBQBhgE8K7 Im/DWNKPwRPy8Lnv+JEHoc1gSS0UBHz283+ZcCBm3heSLPkyXNhmZimGyF02kN3+Twdh XHiYNR5qeJZE7WEi/VRpfuRUp7qAF4+Y2q7x1H6WwM6wo9nxyRikX/CQd0FffxZLQB1D oxIw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z26-20020aa7cf9a000000b005236537c1b8si1161268edx.109.2023.08.10.04.08.42; Thu, 10 Aug 2023 04:09:07 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233612AbjHJKds (ORCPT + 99 others); Thu, 10 Aug 2023 06:33:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44730 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229890AbjHJKdr (ORCPT ); Thu, 10 Aug 2023 06:33:47 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 689ADF2 for ; Thu, 10 Aug 2023 03:33:46 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4A960D75; Thu, 10 Aug 2023 03:34:28 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 637153F6C4; Thu, 10 Aug 2023 03:33:43 -0700 (PDT) From: Ryan Roberts To: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra , Christian Borntraeger , Sven Schnelle , Arnd Bergmann , "Matthew Wilcox (Oracle)" , David Hildenbrand , Yu Zhao , "Kirill A. Shutemov" , Yin Fengwei , Yang Shi , "Huang, Ying" , Zi Yan Cc: Ryan Roberts , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v1 0/4] Optimize mmap_exit for large folios Date: Thu, 10 Aug 2023 11:33:28 +0100 Message-Id: <20230810103332.3062143-1-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773840102649626604 X-GMAIL-MSGID: 1773840102649626604 Hi All, This is a series to improve performance of process teardown, taking advantage of the fact that large folios are increasingly regularly pte-mapped in user space; supporting filesystems already use large folios for pagecache memory, and large folios for anonymous memory are (hopefully) on the horizon. See last patch for performance numbers, including measurements that show this approach doesn't regress (and actually improves a little bit) when all folios are small. The basic approach is to accumulate contiguous ranges of pages in the mmu_gather structure (instead of storing each individual page pointer), then take advantage of this internal format to efficiently batch rmap removal, swapcache removal and page release - see the commit messages for more details. This series replaces the previous approach I took at [1], which was much smaller in scope, only attempting to batch rmap removal for anon pages. Feedback was that I should do something more general that would also batch-remove pagecache pages from the rmap. But while designing that, I found it was also possible to improve swapcache removal and page release. Hopefully I haven't gone too far the other way now! Note that patch 1 is unchanged from that originl series. This series is based on mm-unstable (ad3232df3e41). I'm going to be out on holiday from the end of today, returning on 29th August. So responses will likely be patchy, as I'm terrified of posting to list from my phone! [1] https://lore.kernel.org/linux-mm/20230727141837.3386072-1-ryan.roberts@arm.com/ Thanks, Ryan Ryan Roberts (4): mm: Implement folio_remove_rmap_range() mm/mmu_gather: generalize mmu_gather rmap removal mechanism mm/mmu_gather: Remove encoded_page infrastructure mm/mmu_gather: Store and process pages in contig ranges arch/s390/include/asm/tlb.h | 9 +-- include/asm-generic/tlb.h | 49 +++++++------- include/linux/mm.h | 11 +++- include/linux/mm_types.h | 34 +--------- include/linux/rmap.h | 2 + include/linux/swap.h | 6 +- mm/memory.c | 24 ++++--- mm/mmu_gather.c | 112 +++++++++++++++++++++++--------- mm/rmap.c | 125 +++++++++++++++++++++++++++--------- mm/swap.c | 99 ++++++++++++++++++++++++++-- mm/swap_state.c | 11 ++-- 11 files changed, 333 insertions(+), 149 deletions(-) --- 2.25.1