From patchwork Wed Nov 9 20:30:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Torvalds X-Patchwork-Id: 17807 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp547956wru; Wed, 9 Nov 2022 12:34:56 -0800 (PST) X-Google-Smtp-Source: AMsMyM6ug16d4KSN8PuIfFCkyrdyItW30dOY6QC8AB0ae21vNj2cU3YOEmqNHZHcmZNWyYhPhdkD X-Received: by 2002:a17:906:11cc:b0:7ad:e178:de59 with SMTP id o12-20020a17090611cc00b007ade178de59mr1748609eja.449.1668026095932; Wed, 09 Nov 2022 12:34:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668026095; cv=none; d=google.com; s=arc-20160816; b=1CSf4v21u1GhXU8Ousfuc0RRFvbjAf4/orNdkKu5P24VqjUXOIu8Jp+1L/4v8RV7IR mrKNmvMwXtfm0KY+itakH0XuPd8WjCa4YiCIodBHjDdHQVtzTJ7iPAZsZsUhNSmBDZpA duI6s6RsCOzOkPILJ1SjbAOn2p441qU4ZZ1G4NLFwuaPMRD/SrnkNpQ7AdiYKTW/3XVR tz5qssT3sWkv3nS+0jOFRkR81HPjknaSeq8HA392H3JxDWXfUJEIfOeU2Vj6yeeM3xOL d8SFwtL3uMzH9RjmVydOegHpd2+re1AUuJKzR8BPpHp8Eg/ortLpLjNgRJMLe4sHZLb9 vHeQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=UAFQiOpzZMBKv2AQjP7YaqcakaLTBmk9LN48Ag/gOS8=; b=tHcQhi/iTTfs96cdZaizccEeQzbj44DWHPItG1tviyGYC61XfUNohwi+cEV9bErjIb XfWd81aix2YUjolx0v2+XOsmH3PPMiwPINUHSKFMIuFsTTcWFwpx5K6AnxugJi3ZnJTl EHJQ8FKTipG2szgQ5D5hTWaIWjJ6LdH/pTLoH9mZOXGgVs5fYnRUtkR9VQUNIkEE8FMQ vCSGSAKFnb5CYjaPKkLA/fNak1E2Q0VSa8Dfy4btmGG6Asb2o/mt/cvGt8o0v76VLn2e k1XjY9mhhKzbU8p2Gp2dZcA72FZ44ydsrGtwUoi6IYrzjCDs73+0y+IHd6BfnMvmlOmb bkiw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux-foundation.org header.s=korg header.b="QDtR/oMN"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id gb12-20020a170907960c00b007ae1d040bffsi17028155ejc.223.2022.11.09.12.34.32; Wed, 09 Nov 2022 12:34:55 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux-foundation.org header.s=korg header.b="QDtR/oMN"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231544AbiKIUbD (ORCPT + 99 others); Wed, 9 Nov 2022 15:31:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37228 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230443AbiKIUa4 (ORCPT ); Wed, 9 Nov 2022 15:30:56 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2FDD019018 for ; Wed, 9 Nov 2022 12:30:55 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id CFF24B82004 for ; Wed, 9 Nov 2022 20:30:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6098AC433C1; Wed, 9 Nov 2022 20:30:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1668025852; bh=penp5eGjlAzwEMHIlYfO2UthsDqmi4JE3eNVxwebGdU=; h=From:To:Cc:Subject:Date:From; b=QDtR/oMNFrNnuSC8rkqWQuyDgSXMdz7cE6sGNbUSY63i7rPywtVF2bs/QeWo1hR6D E+ZSTT3gQ1V/p4cp0tg271dSgkqpaQYCwj4Hw+IDg1mhCSpAbyrGxAwL6Aja826eaJ t6IFNeMUW7uI0RdE4bX2KpSOHjrhnD9sK5lLsPGY= From: Linus Torvalds To: Hugh Dickins , Johannes Weiner , Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Alexander Gordeev Subject: [PATCH 1/4] mm: introduce 'encoded' page pointers with embedded extra bits Date: Wed, 9 Nov 2022 12:30:48 -0800 Message-Id: <20221109203051.1835763-1-torvalds@linux-foundation.org> X-Mailer: git-send-email 2.38.1.284.gfd9468d787 MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748958273313708342?= X-GMAIL-MSGID: =?utf-8?q?1749052131440558640?= We already have this notion in parts of the MM code (see the mlock code with the LRU_PAGE and NEW_PAGE bits), but I'm going to introduce a new case, and I refuse to do the same thing we've done before where we just put bits in the raw pointer and say it's still a normal pointer. So this introduces a 'struct encoded_page' pointer that cannot be used for anything else than to encode a real page pointer and a couple of extra bits in the low bits. That way the compiler can trivially track the state of the pointer and you just explicitly encode and decode the extra bits. Note that this makes the alignment of 'struct page' explicit even for the case where CONFIG_HAVE_ALIGNED_STRUCT_PAGE is not set. That is entirely redundant in almost all cases, since the page structure already contains several word-sized entries. However, on m68k, the alignment of even 32-bit data is just 16 bits, and as such in theory the alignment of 'struct page' could be too. So let's just make it very very explicit that the alignment needs to be at least 32 bits, giving us a guarantee of two unused low bits in the pointer. Now, in practice, our page struct array is aligned much more than that anyway, even on m68k, and our existing code in mm/mlock.c obviously already depended on that. But since the whole point of this change is to be careful about the type system when hiding extra bits in the pointer, let's also be explicit about the assumptions we make. NOTE! This is being very careful in another way too: it has a build-time assertion that the 'flags' added to the page pointer actually fit in the two bits. That means that this helper must be inlined, and can only be used in contexts where the compiler can statically determine that the value fits in the available bits. Link: https://lore.kernel.org/all/Y2tKixpO4RO6DgW5@tuxmaker.boeblingen.de.ibm.com/ Cc: Alexander Gordeev Acked-by: Johannes Weiner Acked-by: Hugh Dickins Signed-off-by: Linus Torvalds Reviewed-by: David Hildenbrand --- include/linux/mm_types.h | 34 +++++++++++++++++++++++++++++++++- 1 file changed, 33 insertions(+), 1 deletion(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 500e536796ca..0a38fcb08d85 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -67,7 +67,7 @@ struct mem_cgroup; #ifdef CONFIG_HAVE_ALIGNED_STRUCT_PAGE #define _struct_page_alignment __aligned(2 * sizeof(unsigned long)) #else -#define _struct_page_alignment +#define _struct_page_alignment __aligned(sizeof(unsigned long)) #endif struct page { @@ -241,6 +241,38 @@ struct page { #endif } _struct_page_alignment; +/** + * struct encoded_page - a nonexistent type marking this pointer + * + * An 'encoded_page' pointer is a pointer to a regular 'struct page', but + * with the low bits of the pointer indicating extra context-dependent + * information. Not super-common, but happens in mmu_gather and mlock + * handling, and this acts as a type system check on that use. + * + * We only really have two guaranteed bits in general, although you could + * play with 'struct page' alignment (see CONFIG_HAVE_ALIGNED_STRUCT_PAGE) + * for more. + * + * Use the supplied helper functions to endcode/decode the pointer and bits. + */ +struct encoded_page; +#define ENCODE_PAGE_BITS 3ul +static __always_inline struct encoded_page *encode_page(struct page *page, unsigned long flags) +{ + BUILD_BUG_ON(flags > ENCODE_PAGE_BITS); + return (struct encoded_page *)(flags | (unsigned long)page); +} + +static inline unsigned long encoded_page_flags(struct encoded_page *page) +{ + return ENCODE_PAGE_BITS & (unsigned long)page; +} + +static inline struct page *encoded_page_ptr(struct encoded_page *page) +{ + return (struct page *)(~ENCODE_PAGE_BITS & (unsigned long)page); +} + /** * struct folio - Represents a contiguous set of bytes. * @flags: Identical to the page flags.