From patchwork Wed Nov 9 20:30:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Torvalds X-Patchwork-Id: 17810 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp548268wru; Wed, 9 Nov 2022 12:35:43 -0800 (PST) X-Google-Smtp-Source: AMsMyM62OwrI2Op62fRtZ7wBigdPrJmlwrYkNbH2/eHcNhbfWDQvsHjSNeTXB+zbWMC7oQmbOT0G X-Received: by 2002:a17:90b:3e81:b0:213:f86b:f94b with SMTP id rj1-20020a17090b3e8100b00213f86bf94bmr52329124pjb.134.1668026143177; Wed, 09 Nov 2022 12:35:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668026143; cv=none; d=google.com; s=arc-20160816; b=mMGQPFzYWgcIam7SUunBht6ALmFR1VtEuMZx6S7u5P7RIVQ0vvm/agWlS9cUqAZDnG KKoyvDcWHxcWFJA9vTp0rtJ5GKuT9DK8sqKDHebhAhiLt4VRxydG7hYrfLdOPuJLzbz0 5qeR1ulPgl5J73BRzIW9x54+fMZrEAWI6zl5mWq9ZZxE5kaTG//hBYVwhu+GG+P+qooZ NfI0EYPdK3m39QwEV5pQplh92pzeRj0DCHnNw2+04zDSMyuCyn04vsOwuDlIZhoZZIUw RcriTLr1dWkiGsW43/+nohbKiaAfw9D6GddaeuQO/e4tyvV6D1mD5YAoy3AfVXe075ni J3vQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=UkzXwuIPer6m1qfUHwUxQj1lq9ihfcZ5svVTS/LLtro=; b=aSoZ/5JJo8UeSb9NtqEi4rdB3/kDkaCBKYvsAS2BoZafvt7z2fRxFuY6pVP1pfcLbz 9qGleb7PNZHFimuvnT7onKZYdnSzoLsQmeyT5zhlBBCBxY2Yb1iUlNKGz+YUA16d1rW5 9uHd0krK7XoHZPLEfxeqZWlC1mswXLbqT96vD7/l0UDhephiNvWW9J8L17V3tVa1cT0W rJyvhZETGnWQnHdiQrrwx9FIRujfFQotuvjHRoBOqelS51x2G1XqNvDZ8/+06Z+K1ClG 4WS0J6L6A45+qsioa9vot9YVszWh1WckZMbWERAzzihrZnXiL5L57yob5FrmTuL/WlVI 6qww== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux-foundation.org header.s=korg header.b=xAEEZ1gj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s15-20020a65690f000000b0046ebaed8e45si18983990pgq.62.2022.11.09.12.35.29; Wed, 09 Nov 2022 12:35:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux-foundation.org header.s=korg header.b=xAEEZ1gj; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231466AbiKIUa6 (ORCPT + 99 others); Wed, 9 Nov 2022 15:30:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37230 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231362AbiKIUa4 (ORCPT ); Wed, 9 Nov 2022 15:30:56 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 166ADB4B4 for ; Wed, 9 Nov 2022 12:30:55 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A4EA761CCC for ; Wed, 9 Nov 2022 20:30:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EFF72C433D6; Wed, 9 Nov 2022 20:30:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1668025854; bh=jfWc6kn68kAyUZF8S+7X4sveku1vOvDNUz/xy6wAir4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=xAEEZ1gj7l/8wAdz869fjwgcq9wqP+Dk9DLHIoqpv7MpspKrUddN8gJBa9flxuM9G Vrz2Zp2MCnXa6CWwwcVkFSlbSQHlIrFSXzHlH1b8ADAoxLYHHvXiB9fJOStTLi+dBv CJTPJDnlQrjEPOrNEJgV2mPcJCmjITlERtnVSK7U= From: Linus Torvalds To: Hugh Dickins , Johannes Weiner , Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 2/4] mm: teach release_pages() to take an array of encoded page pointers too Date: Wed, 9 Nov 2022 12:30:49 -0800 Message-Id: <20221109203051.1835763-2-torvalds@linux-foundation.org> X-Mailer: git-send-email 2.38.1.284.gfd9468d787 In-Reply-To: <20221109203051.1835763-1-torvalds@linux-foundation.org> References: <20221109203051.1835763-1-torvalds@linux-foundation.org> MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749052181119906021?= X-GMAIL-MSGID: =?utf-8?q?1749052181119906021?= release_pages() already could take either an array of page pointers, or an array of folio pointers. Expand it to also accept an array of encoded page pointers, which is what both the existing mlock() use and the upcoming mmu_gather use of encoded page pointers wants. Note that release_pages() won't actually use, or react to, any extra encoded bits. Instead, this is very much a case of "I have walked the array of encoded pages and done everything the extra bits tell me to do, now release it all". Also, while the "either page or folio pointers" dual use was handled with a cast of the pointer in "release_folios()", this takes a slightly different approach and uses the "transparent union" attribute to describe the set of arguments to the function: https://gcc.gnu.org/onlinedocs/gcc/Common-Type-Attributes.html which has been supported by gcc forever, but the kernel hasn't used before. That allows us to avoid using various wrappers with casts, and just use the same function regardless of use. Acked-by: Johannes Weiner Acked-by: Hugh Dickins Signed-off-by: Linus Torvalds Reviewed-by: David Hildenbrand --- include/linux/mm.h | 21 +++++++++++++++++++-- mm/swap.c | 16 ++++++++++++---- 2 files changed, 31 insertions(+), 6 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 8bbcccbc5565..d9fb5c3e3045 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1179,7 +1179,24 @@ static inline void folio_put_refs(struct folio *folio, int refs) __folio_put(folio); } -void release_pages(struct page **pages, int nr); +/** + * release_pages - release an array of pages or folios + * + * This just releases a simple array of multiple pages, and + * accepts various different forms of said page array: either + * a regular old boring array of pages, an array of folios, or + * an array of encoded page pointers. + * + * The transparent union syntax for this kind of "any of these + * argument types" is all kinds of ugly, so look away. + */ +typedef union { + struct page **pages; + struct folio **folios; + struct encoded_page **encoded_pages; +} release_pages_arg __attribute__ ((__transparent_union__)); + +void release_pages(release_pages_arg, int nr); /** * folios_put - Decrement the reference count on an array of folios. @@ -1195,7 +1212,7 @@ void release_pages(struct page **pages, int nr); */ static inline void folios_put(struct folio **folios, unsigned int nr) { - release_pages((struct page **)folios, nr); + release_pages(folios, nr); } static inline void put_page(struct page *page) diff --git a/mm/swap.c b/mm/swap.c index 955930f41d20..596ed226ddb8 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -968,22 +968,30 @@ void lru_cache_disable(void) /** * release_pages - batched put_page() - * @pages: array of pages to release + * @arg: array of pages to release * @nr: number of pages * - * Decrement the reference count on all the pages in @pages. If it + * Decrement the reference count on all the pages in @arg. If it * fell to zero, remove the page from the LRU and free it. + * + * Note that the argument can be an array of pages, encoded pages, + * or folio pointers. We ignore any encoded bits, and turn any of + * them into just a folio that gets free'd. */ -void release_pages(struct page **pages, int nr) +void release_pages(release_pages_arg arg, int nr) { int i; + struct encoded_page **encoded = arg.encoded_pages; LIST_HEAD(pages_to_free); struct lruvec *lruvec = NULL; unsigned long flags = 0; unsigned int lock_batch; for (i = 0; i < nr; i++) { - struct folio *folio = page_folio(pages[i]); + struct folio *folio; + + /* Turn any of the argument types into a folio */ + folio = page_folio(encoded_page_ptr(encoded[i])); /* * Make sure the IRQ-safe lock-holding time does not get