From patchwork Mon Jul 3 13:53:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 115379 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f45:0:b0:3ea:f831:8777 with SMTP id v5csp550654vqx; Mon, 3 Jul 2023 07:07:42 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ66l6DgNuD2VFWpmAtkUJl283wWIOGrLhdcqXcp+9au0kew9LP3nS/KHDBUzSj/XpPTvCDP X-Received: by 2002:a05:6830:913:b0:6b8:7653:dd66 with SMTP id v19-20020a056830091300b006b87653dd66mr13066501ott.0.1688393262256; Mon, 03 Jul 2023 07:07:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688393262; cv=none; d=google.com; s=arc-20160816; b=KUOI6JBq2TkZxISnrH3OTlYKjhQUIcGEW3uYE28TrlnjpeugUnvV3YIjNsPcT1DK1P PYZbAOuRF1Z9hWVTHfbhFwEfMwdWS2K/edBOyqm1WL8Ns0n72ihfnKOhx3ojRdUJDfFn jBzc02Ibj861saZO04quCg1Ljr7aeA80hoAk81RqPw6b2qCO/rmlJuLnfUHoECW83x3s 3tS/2j1qAzRwvTN5yTrsNO33b57ISyVTsQ6COw2WYqpceUbPhgXt0x7Kr0RLBQEh1DH2 kwev9xgdfZ5XaET8FS6TFy+xJUvkWJOHjTOVoOiRfN6TRC2SQOSz5VeE0CG9C74c4SyC C1Cg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=0Xs0/r8JxC0IODzuzWxhN6nl3kLwEjLKvoCF58989ko=; fh=W2gTym8Tes+4XZMJyNdq9yGC4n1yJ+vTSxJnRC1NAwA=; b=AFcCFBweN8yFAl+/BXA2qtEX/Dyt7bkFIvVS7LwFFAdNN8l6b9TUyyTv4aWlLtTkqv CmA4mh1bws+/dz0I/ChbImDv0lObyk/HbVanHUjvF90CsiopP5lsKTYM3+VrboKptp4s NReWExSt71i9zOyzjRc0Lvhpgo2e2IMSGrBDVoNK8Aj1TFhOXcGp0lOxE7LMEq8AQYtc /eKwY+IAa8YbXCRTlRTMQ4McPp4gKZ3X8lZTek8GY/w+LY2x6bYyFdHR5RcrNovOACe7 lLp1rDAQtgVV8hh2NTa6Fid/VnRxfPaBINqxExbiRxWxe8qP3IScxaNOEu80WdeGrW5H ka+Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k25-20020a635a59000000b00553813c2df0si18068673pgm.513.2023.07.03.07.07.22; Mon, 03 Jul 2023 07:07:42 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229973AbjGCNyY (ORCPT + 99 others); Mon, 3 Jul 2023 09:54:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35888 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230094AbjGCNyQ (ORCPT ); Mon, 3 Jul 2023 09:54:16 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 38C171716 for ; Mon, 3 Jul 2023 06:53:53 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6FD552F4; Mon, 3 Jul 2023 06:54:35 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id EBD643F73F; Mon, 3 Jul 2023 06:53:50 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Matthew Wilcox , "Kirill A. Shutemov" , Yin Fengwei , David Hildenbrand , Yu Zhao , Catalin Marinas , Will Deacon , Anshuman Khandual , Yang Shi Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 5/5] arm64: mm: Override arch_wants_pte_order() Date: Mon, 3 Jul 2023 14:53:30 +0100 Message-Id: <20230703135330.1865927-6-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230703135330.1865927-1-ryan.roberts@arm.com> References: <20230703135330.1865927-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770408653687393009?= X-GMAIL-MSGID: =?utf-8?q?1770408653687393009?= Define an arch-specific override of arch_wants_pte_order() so that when FLEXIBLE_THP is enabled, large folios will be allocated for anonymous memory with an order that is compatible with arm64's contpte mappings. arch_wants_pte_order() returns an order according to the following policy: For the unhinted case, when THP is not requested for the vma, don't allow anything bigger than 64K. This means we don't waste too much memory. Additionally, for 4K pages this is the contpte size, and for 16K, this is (usually) the HPA size when the uarch feature is implemented. For the hinted case, when THP is requested for the vma, allow the contpte size for all page size configurations; 64K for 4K, 2M for 16K and 2M for 64K. Additionally, the THP and NOTHP order constants are defined using Kconfig so it is possible to override them at build time. Signed-off-by: Ryan Roberts --- arch/arm64/Kconfig | 12 ++++++++++++ arch/arm64/include/asm/pgtable.h | 4 ++++ arch/arm64/mm/mmu.c | 8 ++++++++ 3 files changed, 24 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 343e1e1cae10..689c5bf13dc1 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -281,6 +281,18 @@ config ARM64_CONT_PMD_SHIFT default 5 if ARM64_16K_PAGES default 4 +config ARM64_PTE_ORDER_NOTHP + int + default 0 if ARM64_64K_PAGES # 64K (1 page) + default 2 if ARM64_16K_PAGES # 64K (4 pages; benefits from HPA where HW supports it) + default 4 if ARM64_4K_PAGES # 64K (16 pages; eligible for contpte-mapping) + +config ARM64_PTE_ORDER_THP + int + default 5 if ARM64_64K_PAGES # 2M (32 pages; eligible for contpte-mapping) + default 7 if ARM64_16K_PAGES # 2M (128 pages; eligible for contpte-mapping) + default 4 if ARM64_4K_PAGES # 64K (16 pages; eligible for contpte-mapping) + config ARCH_MMAP_RND_BITS_MIN default 14 if ARM64_64K_PAGES default 16 if ARM64_16K_PAGES diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 6fd012663a01..8463d5f9f307 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -1117,6 +1117,10 @@ extern pte_t ptep_modify_prot_start(struct vm_area_struct *vma, extern void ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t old_pte, pte_t new_pte); + +#define arch_wants_pte_order arch_wants_pte_order +extern int arch_wants_pte_order(struct vm_area_struct *vma); + #endif /* !__ASSEMBLY__ */ #endif /* __ASM_PGTABLE_H */ diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index af6bc8403ee4..8556c4a9b507 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1481,3 +1481,11 @@ void ptep_modify_prot_commit(struct vm_area_struct *vma, unsigned long addr, pte { set_pte_at(vma->vm_mm, addr, ptep, pte); } + +int arch_wants_pte_order(struct vm_area_struct *vma) +{ + if (hugepage_vma_check(vma, vma->vm_flags, false, true, true)) + return CONFIG_ARM64_PTE_ORDER_THP; + else + return CONFIG_ARM64_PTE_ORDER_NOTHP; +}