From patchwork Wed Jul 12 04:39:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 118824 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a6b2:0:b0:3e4:2afc:c1 with SMTP id c18csp919601vqm; Tue, 11 Jul 2023 22:17:39 -0700 (PDT) X-Google-Smtp-Source: APBJJlEvE1W8YhSOGfusS/ehXU/lA2kQKwkglKO3U8eg4QKlzQB+nkw4DIGfSpRRweJZ0xBHHW16 X-Received: by 2002:a05:6a00:b93:b0:668:82fe:16f1 with SMTP id g19-20020a056a000b9300b0066882fe16f1mr22586676pfj.1.1689139059517; Tue, 11 Jul 2023 22:17:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689139059; cv=none; d=google.com; s=arc-20160816; b=rih5xeB2AdG/I46kR38OFLsbaXL7rl1JZdo1iBHImm3jWeAe2YnEQFvl5a2bmlSHId lHAf3DQI8uHCcHPjFYyzOCzuqqJsXh3Q1myybnw/PsGfEryh6DV1zW8QA0Hmy5NwESfa bOY/5D9Tu2v/7ZBVtk1cZR+H1v/kc2Aehd1BWS0oKk9BrVmRO7IjQTRrblrNmX2u3lsE j6gwrASWUcqAvoMncghLa+EMRg7o+qOU5SWFeFI15L0nqAgaQDY5ZkT4CTAKik3KNzYO ZaoAuEmCIJN0M0GV4SwT+UZ0wnIilTv1ZPCMjr6iL75wfEEVuhaQupp/ENz2O+89GFmI zPvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:message-id:in-reply-to :subject:cc:to:from:date:dkim-signature; bh=2t2oc+yeedMzdUFXYKBxRHbKcZMzrhHMUkscNN4bTko=; fh=F9IVpUHia72ABF8rrQzDtAcJe241JKL7UyOA6EvSYz4=; b=u/6i3VovbuphAbEcMYPKUsDbcIfX9AMf/5a6AwJZ88yk98TN5tILejqojHGrth4bpU mOM2rCRABMnw8FLInpyZ+Xdnju35oQ6mc/hqo2LzfFZaT1W2I0rf+zHZ2Zcec3PmlxKN Bg3Hsk/kNHClxhIZxScUerATHmJVW3CbkrRO9whmiEJ1AzA0iqy5/gcX8yCkgQeezem6 caPwpsjahFlhOsi7RwsBq4UUEpEYoTpAE4qapmeHPCw7B5JuR0wKdl2Pa8fKAkRwnOGI eFMcUygOsYFCteRFZ98J7rIbrtKU9b+sHzFoxuuFfQhrX0pyg8JNjSx5LD1nIltuisuJ M6cg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=XYfimDGo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s22-20020a635256000000b0055ae207577csi2531302pgl.650.2023.07.11.22.17.26; Tue, 11 Jul 2023 22:17:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=XYfimDGo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231533AbjGLEj6 (ORCPT + 99 others); Wed, 12 Jul 2023 00:39:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53964 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231255AbjGLEj4 (ORCPT ); Wed, 12 Jul 2023 00:39:56 -0400 Received: from mail-yw1-x112c.google.com (mail-yw1-x112c.google.com [IPv6:2607:f8b0:4864:20::112c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 76678195 for ; Tue, 11 Jul 2023 21:39:54 -0700 (PDT) Received: by mail-yw1-x112c.google.com with SMTP id 00721157ae682-5703d12ab9aso76389277b3.2 for ; Tue, 11 Jul 2023 21:39:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689136793; x=1691728793; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=2t2oc+yeedMzdUFXYKBxRHbKcZMzrhHMUkscNN4bTko=; b=XYfimDGolFEty51TCKUyAZmm69LKogtUNWLIyMCuw81Si1ZjaboZJMyonyZe8BlA1H 4dw7ywAuqviGzR+cqYGZtxV+ytY3jiQkgFAvjXIkhqrYEAwOpEMsZRaoOyR5rEd3GDD+ 6LnUt02lOICM7P8CDg5UfhiUAjvip1TQsKWkNZqXUfOXl95Q2+l607lxyvF7WfoMEkup n8srQ8rKA7PHkhNMQEN57jpX0Q3gk0ZBR2Liws8TUdU4okAi97PytXAsy84/ZZk7I96p LrZbc0ZAQKs4VNt1L3qD7duolPC2jOZozxLhx0DiCnOdCct9HdR1WwipR08docsFv3CN 0s6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689136793; x=1691728793; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=2t2oc+yeedMzdUFXYKBxRHbKcZMzrhHMUkscNN4bTko=; b=Fb1tgEEfZET9Tz5YkiCQqdq2ti9vjzR5GnB7E8ka3KlO6teEglWtzZqholIlsqwRwS KT+DTmbJWSL9ICgYccIdbaaYzlxaN4ho/DsU8y9u4HJxd3aMrgQe/u99zlRtgBkXdU2w WbBH60Yo5HqOqU0NTuKEY/lnmt4aKBAZohUIBmDmRXMFIc3RIUGt/FGIuXsLGARzYbCG nmZmcXDm24kSu36lxoLaNdzErsPGnrMhcpdqZHt33J7u1tH3HIwygcMH7PLw5x/fWJ3M Hf3pUnBIgKENMne+r6Te3ev8y+IkhkzVHFwCZMLoj5++iffhZc33fBA6PmLCXtMk+pQX iWBw== X-Gm-Message-State: ABy/qLZmDYTOdqNDb1SeqsS2YKR9c/l4IjFv0sX6BzHmfHr51Ut4YKbh S7SDzzRxZ8kC9a8WeuiPBiqe/A== X-Received: by 2002:a81:a187:0:b0:579:efb4:7d08 with SMTP id y129-20020a81a187000000b00579efb47d08mr15922621ywg.27.1689136793558; Tue, 11 Jul 2023 21:39:53 -0700 (PDT) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id f130-20020a816a88000000b005702bfb19bfsm974843ywc.130.2023.07.11.21.39.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jul 2023 21:39:53 -0700 (PDT) Date: Tue, 11 Jul 2023 21:39:48 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@ripple.attlocal.net To: Andrew Morton cc: Mike Kravetz , Mike Rapoport , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Suren Baghdasaryan , Qi Zheng , Yang Shi , Mel Gorman , Peter Xu , Peter Zijlstra , Will Deacon , Yu Zhao , Alistair Popple , Ralph Campbell , Ira Weiny , Steven Price , SeongJae Park , Lorenzo Stoakes , Huang Ying , Naoya Horiguchi , Christophe Leroy , Zack Rusin , Jason Gunthorpe , Axel Rasmussen , Anshuman Khandual , Pasha Tatashin , Miaohe Lin , Minchan Kim , Christoph Hellwig , Song Liu , Thomas Hellstrom , Russell King , "David S. Miller" , Michael Ellerman , "Aneesh Kumar K.V" , Heiko Carstens , Christian Borntraeger , Claudio Imbrenda , Alexander Gordeev , Gerald Schaefer , Vasily Gorbik , Jann Horn , Vishal Moola , Vlastimil Babka , Zi Yan , linux-arm-kernel@lists.infradead.org, sparclinux@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v3 08/13] mm/pgtable: add pte_free_defer() for pgtable as page In-Reply-To: <7cd843a9-aa80-14f-5eb2-33427363c20@google.com> Message-ID: <78e921b0-b681-a1b0-dc20-44c9efa4ef3c@google.com> References: <7cd843a9-aa80-14f-5eb2-33427363c20@google.com> MIME-Version: 1.0 X-Spam-Status: No, score=-17.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL,USER_IN_DEF_SPF_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1771190678414705928 X-GMAIL-MSGID: 1771190678414705928 Add the generic pte_free_defer(), to call pte_free() via call_rcu(). pte_free_defer() will be called inside khugepaged's retract_page_tables() loop, where allocating extra memory cannot be relied upon. This version suits all those architectures which use an unfragmented page for one page table (none of whose pte_free()s use the mm arg which was passed to it). Signed-off-by: Hugh Dickins --- include/linux/mm_types.h | 4 ++++ include/linux/pgtable.h | 2 ++ mm/pgtable-generic.c | 20 ++++++++++++++++++++ 3 files changed, 26 insertions(+) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index de10fc797c8e..17a7868f00bd 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -144,6 +144,10 @@ struct page { struct { /* Page table pages */ unsigned long _pt_pad_1; /* compound_head */ pgtable_t pmd_huge_pte; /* protected by page->ptl */ + /* + * A PTE page table page might be freed by use of + * rcu_head: which overlays those two fields above. + */ unsigned long _pt_pad_2; /* mapping */ union { struct mm_struct *pt_mm; /* x86 pgds only */ diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 7f2db400f653..9fa34be65159 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -112,6 +112,8 @@ static inline void pte_unmap(pte_t *pte) } #endif +void pte_free_defer(struct mm_struct *mm, pgtable_t pgtable); + /* Find an entry in the second-level page table.. */ #ifndef pmd_offset static inline pmd_t *pmd_offset(pud_t *pud, unsigned long address) diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index b9a0c2137cc1..fa9d4d084291 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -13,6 +13,7 @@ #include #include #include +#include #include /* @@ -230,6 +231,25 @@ pmd_t pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long address, return pmd; } #endif + +/* arch define pte_free_defer in asm/pgalloc.h for its own implementation */ +#ifndef pte_free_defer +static void pte_free_now(struct rcu_head *head) +{ + struct page *page; + + page = container_of(head, struct page, rcu_head); + pte_free(NULL /* mm not passed and not used */, (pgtable_t)page); +} + +void pte_free_defer(struct mm_struct *mm, pgtable_t pgtable) +{ + struct page *page; + + page = pgtable; + call_rcu(&page->rcu_head, pte_free_now); +} +#endif /* pte_free_defer */ #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ #if defined(CONFIG_GUP_GET_PXX_LOW_HIGH) && \