From patchwork Wed Dec 20 04:00:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 181438 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:24d3:b0:fb:cd0c:d3e with SMTP id r19csp2403171dyi; Tue, 19 Dec 2023 20:01:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IHJIZBxCiZ+KHRZFWtkCY80cyuJfFWO44scWdNa/COLM8WXL4DnkLgzKTGkmRjCGWbQDtL1 X-Received: by 2002:a05:620a:88b:b0:781:149f:4e65 with SMTP id b11-20020a05620a088b00b00781149f4e65mr47531qka.39.1703044893732; Tue, 19 Dec 2023 20:01:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703044893; cv=none; d=google.com; s=arc-20160816; b=dfHro0afivpDH7Lxj8WwVB0LyIDvIJUu2VSV6l9bBoAzariPJvYpVg/RQMY/kCLmbs MWRqY0JbLmbZej+rZCaf8Y2d2IpNg7abMYoF4wKJxdJ0qQiwPu3km0KkfqJ5qMKhdeBe dRLodvVOL/pRJ0gBfkgHPmVH49iOSRSA8QpH4AK2CTPnhpVaFScDndAlLnznw6UJKEfE ljGW+CsthmZE19+lFgZnWOn2IBoQI8KJtqJTZySoT+dU7u2GZy9vg9Sh+PwhKLOM8IpH 5ZJ4WHOze2qFU0GdDvZwR2AnAFNaj+mf3p7HN1RzK6FwbDJrTGVcPSIBcLvpO7psEyt0 2vaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=wTWOqyfMsQnqPpUH1dExxKEsGutDkMUJttQEhdAsH5o=; fh=COwL00CAVvf6HClNn9uW/acbdshq5DvvB3VM/yZ8sXw=; b=m3yHk6qQSN+oSLi26nUIAPPLkjMN4IlNhAffcasZMxHVIXR+cLqWian496eJIWTPQ7 K+SxIDNzkk+H9g/D5437lweiqoY1y9gFkEeTf6szT2wGEJ4R75MZRfxYi5+uE7KSkr3l IxWNb5gQ3k+0I85WnKsD8QzFVnnBsboKEEOd5u6/JWvF++rOdSeiTMYDQ2PuOX9OBvFq xzFqUkPZKTqXx2yP3m7SmEEiC0YXAZb7AiUVJPG9tSpQ6ZZwCCM93YwacotAtgrVKlcV 2xlk/bAmO5HBFtD5UUchxrDm0O1KWWaxpUwVvj6de61CTXcx1PXSwDduamjXRNowFc6s N7sg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=rKsaJcrs; spf=pass (google.com: domain of linux-kernel+bounces-6342-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-6342-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id s12-20020a05620a29cc00b0077dc3ad1272si28839984qkp.299.2023.12.19.20.01.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Dec 2023 20:01:33 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-6342-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=rKsaJcrs; spf=pass (google.com: domain of linux-kernel+bounces-6342-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-6342-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 73CB21C2507B for ; Wed, 20 Dec 2023 04:01:33 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id EE8FB125AB; Wed, 20 Dec 2023 04:01:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="rKsaJcrs" X-Original-To: linux-kernel@vger.kernel.org Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 05D41BE5B for ; Wed, 20 Dec 2023 04:01:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kinseyho.bounces.google.com Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-1d3eaaaa5a1so2647365ad.1 for ; Tue, 19 Dec 2023 20:01:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1703044864; x=1703649664; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=wTWOqyfMsQnqPpUH1dExxKEsGutDkMUJttQEhdAsH5o=; b=rKsaJcrs2dUuxSvmCFXLFOs5pp/EKpnYlSytbzq40qviYj5P8OE4xtUbMdScUirWtS 8cPz3NvuxxGNJ1XvEaF9o8H3HWRK4ZvBtrnqu0qy1VT3aMdCqUBOnYs4MPz30CMKmLjd g+xQ9bsoOS5a5ckqIoyV9K1KhTEiCb5ucyNBkDRKTDlUshmj2DwIkcB4yKotYAPMPLzA FfWHpt+N/6h9o0pNMWELO2yQNG7lKsOeUUOHKYlvDrJS4Lt1Vf/YQ/EcHwF0Yh7WXQSl npgnLR6TuAvicKqcomM3A77Tr1G2ySVVRYkcHzc89JLGgQGMmJANk+kdR38rymlD/kSd IOpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703044864; x=1703649664; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=wTWOqyfMsQnqPpUH1dExxKEsGutDkMUJttQEhdAsH5o=; b=QCqPOErpm6udlZuL+zfkiJbNF8Rd2siQ2L9ChPFJIt89+B1TW+1e9b/7LfhSYGyCDZ jhLSuoyzNjfSZLEToXBlG7rBcCOEIQMng02OEFXzra8SaDVkkQxH53MoDqhuBKXMOQHl DOZPlVjyANPtDmQdzncBdC0RCa6Tg3TFadia3ShG90KiFlptBx52XyD0BtuNjw9nK2Ad TXPdstIkCVOhtO1j22GzPME4NabeZD3G2mmHt1rVLnK95d9+pIezuID/R3hwEtL8fp7y 0CpDFs/wIZdWLNll9NYC/IDOvj6PDJ+DmizkuRllc8i4lhL8hymAk5MsUG9woYjZ6bkY 17aA== X-Gm-Message-State: AOJu0YxKVkilAiYReGDbpfcH+tJlT+FaRy4NZ4840Yo5BWiu05qKqrhQ C+x/t3ntt0BqOT9oXvqbJj3VYgf8V6dDbw== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a17:902:da85:b0:1d0:4246:90b1 with SMTP id j5-20020a170902da8500b001d0424690b1mr110279plx.0.1703044864390; Tue, 19 Dec 2023 20:01:04 -0800 (PST) Date: Wed, 20 Dec 2023 04:00:34 +0000 In-Reply-To: <20231220040037.883811-1-kinseyho@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20231220040037.883811-1-kinseyho@google.com> X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20231220040037.883811-2-kinseyho@google.com> Subject: [PATCH mm-unstable v1 1/4] mm/mglru: add CONFIG_ARCH_HAS_HW_PTE_YOUNG From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, yuzhao@google.com, Kinsey Ho , Aneesh Kumar K V X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785772002462717741 X-GMAIL-MSGID: 1785772002462717741 Some architectures are able to set the accessed bit in PTEs when PTEs are used as part of linear address translations. Add CONFIG_ARCH_HAS_HW_PTE_YOUNG for such architectures to be able to override arch_has_hw_pte_young(). Signed-off-by: Kinsey Ho Co-developed-by: Aneesh Kumar K V Signed-off-by: Aneesh Kumar K V --- arch/Kconfig | 8 ++++++++ arch/arm64/Kconfig | 1 + arch/x86/Kconfig | 1 + arch/x86/include/asm/pgtable.h | 6 ------ include/linux/pgtable.h | 2 +- 5 files changed, 11 insertions(+), 7 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index f4b210ab0612..8c8901f80586 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -1470,6 +1470,14 @@ config DYNAMIC_SIGFRAME config HAVE_ARCH_NODE_DEV_GROUP bool +config ARCH_HAS_HW_PTE_YOUNG + bool + help + Architectures that select this option are capable of setting the + accessed bit in PTE entries when using them as part of linear address + translations. Architectures that require runtime check should select + this option and override arch_has_hw_pte_young(). + config ARCH_HAS_NONLEAF_PMD_YOUNG bool help diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 7b071a00425d..12d611f3da5d 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -36,6 +36,7 @@ config ARM64 select ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE select ARCH_HAS_PTE_DEVMAP select ARCH_HAS_PTE_SPECIAL + select ARCH_HAS_HW_PTE_YOUNG select ARCH_HAS_SETUP_DMA_OPS select ARCH_HAS_SET_DIRECT_MAP select ARCH_HAS_SET_MEMORY diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 1566748f16c4..04941a1ffc0a 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -88,6 +88,7 @@ config X86 select ARCH_HAS_PMEM_API if X86_64 select ARCH_HAS_PTE_DEVMAP if X86_64 select ARCH_HAS_PTE_SPECIAL + select ARCH_HAS_HW_PTE_YOUNG select ARCH_HAS_NONLEAF_PMD_YOUNG if PGTABLE_LEVELS > 2 select ARCH_HAS_UACCESS_FLUSHCACHE if X86_64 select ARCH_HAS_COPY_MC if X86_64 diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 57bab91bbf50..08b5cb22d9a6 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -1679,12 +1679,6 @@ static inline bool arch_has_pfn_modify_check(void) return boot_cpu_has_bug(X86_BUG_L1TF); } -#define arch_has_hw_pte_young arch_has_hw_pte_young -static inline bool arch_has_hw_pte_young(void) -{ - return true; -} - #define arch_check_zapped_pte arch_check_zapped_pte void arch_check_zapped_pte(struct vm_area_struct *vma, pte_t pte); diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index af7639c3b0a3..9ecc20fa6269 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -375,7 +375,7 @@ static inline bool arch_has_hw_nonleaf_pmd_young(void) */ static inline bool arch_has_hw_pte_young(void) { - return false; + return IS_ENABLED(CONFIG_ARCH_HAS_HW_PTE_YOUNG); } #endif From patchwork Wed Dec 20 04:00:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 181440 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:24d3:b0:fb:cd0c:d3e with SMTP id r19csp2403417dyi; Tue, 19 Dec 2023 20:02:12 -0800 (PST) X-Google-Smtp-Source: AGHT+IErQSGS+cUr4+W07GsiTiArYtQ0HVUrTMJRZ4A9huKIitajz2FmhI2uQ1vJi+b4zJCWY5JF X-Received: by 2002:a17:902:c942:b0:1d3:c604:538f with SMTP id i2-20020a170902c94200b001d3c604538fmr2868079pla.3.1703044932684; Tue, 19 Dec 2023 20:02:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703044932; cv=none; d=google.com; s=arc-20160816; b=VJzoLpNEX1dq4Ph9OH6EfdutNXsJVXvlvwDVce/VHyphufHm3DXWp+lXwStUmG0Szh 8BFb3Dt8EE57g9WxsZvMKDrtnTPk1qqgUh6aZIeUUtmTGFdADyXc2p0QscnZNOxqwt0p 6MmM2xThG6mWQoL188s7Qx0x7znCcnLEj2PrZRbvuX+56xQZHCac2mqK738ETl+BKw9o xv+wyUwhS/XD4UsKBTHmFF8tNZu8MVFvNgYfS2OorSxiFsVFSB8QqbU1sgzDauWjPlc+ iNgJ1cS7HOtf8Vc6NurLC7o08JOIhgEhEzuyabVU2C+uAV0yYyJL6cpkbm1Dou1mz07l EQ/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=5cCTRKeNQiBByqMOyIwmk53PSKP7NuCrv77vrh17FQc=; fh=COwL00CAVvf6HClNn9uW/acbdshq5DvvB3VM/yZ8sXw=; b=ej9zvKZ9sS486a22KuUeuW+qHo3KYbum2lugRWed+XvhKA3iWvA5WcmYf5FzWQt7aT gSdJu2/ibANcMfOLwr7J8hbhcZ3bqcsC1G6Hft1ATjuotMM1E6fHEXg+cqacD+p6z+hj mBZns1xkqYfeLLzCuPw1dKU5rHAoiZ+fBp1iwviR5v0+cVnS8cuS2EKz1bA39LZNhBjN CJ3okFAk7dPfZW3+bKvNdpgC4w0RFx/wjs/CFPM/9nj9elVGregXbJAgeGgLrED3GFjq HiX5aMwP6x5pfEa6xSgPGoh6WQjN13XRU05y6tqunJAv1kK+t9DRb9B5RwKVOQNe1sQm 2Ilw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=fJc8iZId; spf=pass (google.com: domain of linux-kernel+bounces-6344-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-6344-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id j14-20020a170903024e00b001d1ce46f067si20968404plh.485.2023.12.19.20.02.12 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Dec 2023 20:02:12 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-6344-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=fJc8iZId; spf=pass (google.com: domain of linux-kernel+bounces-6344-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-6344-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 69FD12885E2 for ; Wed, 20 Dec 2023 04:02:12 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1C633168D8; Wed, 20 Dec 2023 04:01:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="fJc8iZId" X-Original-To: linux-kernel@vger.kernel.org Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D16F3CA47 for ; Wed, 20 Dec 2023 04:01:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kinseyho.bounces.google.com Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-5c641d55e27so254661a12.1 for ; Tue, 19 Dec 2023 20:01:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1703044866; x=1703649666; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=5cCTRKeNQiBByqMOyIwmk53PSKP7NuCrv77vrh17FQc=; b=fJc8iZIdZl9HkdUI8BAL7NOgzOe0af+pva5Ybikv4pBZ9lxqL58p7k71TRUEdNo5yV Xfqqcyff0qQMxA3gwY3o4V8i/pCXINIbWpjCavnCerkYyINFlWT1Ggfddv0ridiPWRKL 85V/CsSRQ1R7aswBSJl9lNP2cwa6/w+FESSMqIfUnan32aIfUc+0t/szzQdxbGPEtTur OT3IwjUH0ohu+aU23hcP3s8qI4XPzO8HkbJKbIB6PkolJDfVrXqH1w8Sy+ewhICrSsJx 6dc/wunyZ29pfLBaon0IxLMpf0CkRBpAPf4oDLcnIhOFdbvWpg5oS0qDIBhy5K+eWnwv Afwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703044866; x=1703649666; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=5cCTRKeNQiBByqMOyIwmk53PSKP7NuCrv77vrh17FQc=; b=s45E4BgRj1jsAfZUP5voqV9vraEkp5v+Qj1XpKcGIPhzWc7jtTbGzcVmwkb+qDFq7T OC5PJATn5IZplAfTOnHpArZm6WxsHAArqMGDc6677qD9+sozNNtWu4c7FjrTOjv1JYQe KrFyDGgqlcX6fnYkSLmo0O5WzAjEkoyp3qn7FNjk4fuMMwgJdNCRMD4AzZTwiFic6Ohf vJIFLb3ulM+FHHiufC8JtkH6ix1Uo6F2fmovkY6BfKAZrCzgkFUcmoo8Mb7GCwq6wYsB wZg0eCuNuWbpKBYSGnemHHVdqAxQg/0r/hjsHP+ha+YVXhanvw58mLPBKedvw9W+dnpg toEw== X-Gm-Message-State: AOJu0Yzc4vm3YSLFDgj7elg5NuOYA0wF4eWdyaTTukA5G4LcORr+7fNL zT+sh+TSi4PYHrPco3MgsHpZWVRNG0bt2q/6BCTC X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a63:e446:0:b0:5cd:9278:f8de with SMTP id i6-20020a63e446000000b005cd9278f8demr15130pgk.0.1703044866166; Tue, 19 Dec 2023 20:01:06 -0800 (PST) Date: Wed, 20 Dec 2023 04:00:35 +0000 In-Reply-To: <20231220040037.883811-1-kinseyho@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20231220040037.883811-1-kinseyho@google.com> X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20231220040037.883811-3-kinseyho@google.com> Subject: [PATCH mm-unstable v1 2/4] mm/mglru: add CONFIG_LRU_GEN_WALKS_MMU From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, yuzhao@google.com, Kinsey Ho , Aneesh Kumar K V X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785772043444881311 X-GMAIL-MSGID: 1785772043444881311 Add CONFIG_LRU_GEN_WALKS_MMU such that if disabled, the code that walks page tables to promote pages into the youngest generation will not be built. Also improves code readability by adding two helper functions get_mm_state() and get_next_mm(). Signed-off-by: Kinsey Ho Co-developed-by: Aneesh Kumar K V Signed-off-by: Aneesh Kumar K V --- include/linux/memcontrol.h | 2 +- include/linux/mm_types.h | 12 ++- include/linux/mmzone.h | 2 + kernel/fork.c | 2 +- mm/Kconfig | 4 + mm/vmscan.c | 192 ++++++++++++++++++++++++------------- 6 files changed, 139 insertions(+), 75 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 5de775e6cdd9..20ff87f8e001 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -330,7 +330,7 @@ struct mem_cgroup { struct deferred_split deferred_split_queue; #endif -#ifdef CONFIG_LRU_GEN +#ifdef CONFIG_LRU_GEN_WALKS_MMU /* per-memcg mm_struct list */ struct lru_gen_mm_list mm_list; #endif diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index a66534c78c4d..552fa2d11c57 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -958,7 +958,7 @@ struct mm_struct { */ unsigned long ksm_zero_pages; #endif /* CONFIG_KSM */ -#ifdef CONFIG_LRU_GEN +#ifdef CONFIG_LRU_GEN_WALKS_MMU struct { /* this mm_struct is on lru_gen_mm_list */ struct list_head list; @@ -973,7 +973,7 @@ struct mm_struct { struct mem_cgroup *memcg; #endif } lru_gen; -#endif /* CONFIG_LRU_GEN */ +#endif /* CONFIG_LRU_GEN_WALKS_MMU */ } __randomize_layout; /* @@ -1011,6 +1011,10 @@ struct lru_gen_mm_list { spinlock_t lock; }; +#endif /* CONFIG_LRU_GEN */ + +#ifdef CONFIG_LRU_GEN_WALKS_MMU + void lru_gen_add_mm(struct mm_struct *mm); void lru_gen_del_mm(struct mm_struct *mm); #ifdef CONFIG_MEMCG @@ -1036,7 +1040,7 @@ static inline void lru_gen_use_mm(struct mm_struct *mm) WRITE_ONCE(mm->lru_gen.bitmap, -1); } -#else /* !CONFIG_LRU_GEN */ +#else /* !CONFIG_LRU_GEN_WALKS_MMU */ static inline void lru_gen_add_mm(struct mm_struct *mm) { @@ -1060,7 +1064,7 @@ static inline void lru_gen_use_mm(struct mm_struct *mm) { } -#endif /* CONFIG_LRU_GEN */ +#endif /* CONFIG_LRU_GEN_WALKS_MMU */ struct vma_iterator { struct ma_state mas; diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 2efd3be484fd..bc3f63ec4291 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -640,9 +640,11 @@ struct lruvec { #ifdef CONFIG_LRU_GEN /* evictable pages divided into generations */ struct lru_gen_folio lrugen; +#ifdef CONFIG_LRU_GEN_WALKS_MMU /* to concurrently iterate lru_gen_mm_list */ struct lru_gen_mm_state mm_state; #endif +#endif /* CONFIG_LRU_GEN */ #ifdef CONFIG_MEMCG struct pglist_data *pgdat; #endif diff --git a/kernel/fork.c b/kernel/fork.c index 93924392a5c3..56cf276432c8 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -2946,7 +2946,7 @@ pid_t kernel_clone(struct kernel_clone_args *args) get_task_struct(p); } - if (IS_ENABLED(CONFIG_LRU_GEN) && !(clone_flags & CLONE_VM)) { + if (IS_ENABLED(CONFIG_LRU_GEN_WALKS_MMU) && !(clone_flags & CLONE_VM)) { /* lock the task to synchronize with memcg migration */ task_lock(p); lru_gen_add_mm(p->mm); diff --git a/mm/Kconfig b/mm/Kconfig index 8f8b02e9c136..c98076dec5fb 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1274,6 +1274,10 @@ config LRU_GEN_STATS from evicted generations for debugging purpose. This option has a per-memcg and per-node memory overhead. + +config LRU_GEN_WALKS_MMU + def_bool y + depends on LRU_GEN && ARCH_HAS_HW_PTE_YOUNG # } config ARCH_SUPPORTS_PER_VMA_LOCK diff --git a/mm/vmscan.c b/mm/vmscan.c index b4ca3563bcf4..aa7ea09ffb4c 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2671,13 +2671,14 @@ static void get_item_key(void *item, int *key) key[1] = hash >> BLOOM_FILTER_SHIFT; } -static bool test_bloom_filter(struct lruvec *lruvec, unsigned long seq, void *item) +static bool test_bloom_filter(struct lru_gen_mm_state *mm_state, unsigned long seq, + void *item) { int key[2]; unsigned long *filter; int gen = filter_gen_from_seq(seq); - filter = READ_ONCE(lruvec->mm_state.filters[gen]); + filter = READ_ONCE(mm_state->filters[gen]); if (!filter) return true; @@ -2686,13 +2687,14 @@ static bool test_bloom_filter(struct lruvec *lruvec, unsigned long seq, void *it return test_bit(key[0], filter) && test_bit(key[1], filter); } -static void update_bloom_filter(struct lruvec *lruvec, unsigned long seq, void *item) +static void update_bloom_filter(struct lru_gen_mm_state *mm_state, unsigned long seq, + void *item) { int key[2]; unsigned long *filter; int gen = filter_gen_from_seq(seq); - filter = READ_ONCE(lruvec->mm_state.filters[gen]); + filter = READ_ONCE(mm_state->filters[gen]); if (!filter) return; @@ -2704,12 +2706,12 @@ static void update_bloom_filter(struct lruvec *lruvec, unsigned long seq, void * set_bit(key[1], filter); } -static void reset_bloom_filter(struct lruvec *lruvec, unsigned long seq) +static void reset_bloom_filter(struct lru_gen_mm_state *mm_state, unsigned long seq) { unsigned long *filter; int gen = filter_gen_from_seq(seq); - filter = lruvec->mm_state.filters[gen]; + filter = mm_state->filters[gen]; if (filter) { bitmap_clear(filter, 0, BIT(BLOOM_FILTER_SHIFT)); return; @@ -2717,13 +2719,15 @@ static void reset_bloom_filter(struct lruvec *lruvec, unsigned long seq) filter = bitmap_zalloc(BIT(BLOOM_FILTER_SHIFT), __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN); - WRITE_ONCE(lruvec->mm_state.filters[gen], filter); + WRITE_ONCE(mm_state->filters[gen], filter); } /****************************************************************************** * mm_struct list ******************************************************************************/ +#ifdef CONFIG_LRU_GEN_WALKS_MMU + static struct lru_gen_mm_list *get_mm_list(struct mem_cgroup *memcg) { static struct lru_gen_mm_list mm_list = { @@ -2740,6 +2744,29 @@ static struct lru_gen_mm_list *get_mm_list(struct mem_cgroup *memcg) return &mm_list; } +static struct lru_gen_mm_state *get_mm_state(struct lruvec *lruvec) +{ + return &lruvec->mm_state; +} + +static struct mm_struct *get_next_mm(struct lru_gen_mm_walk *walk) +{ + int key; + struct mm_struct *mm; + struct pglist_data *pgdat = lruvec_pgdat(walk->lruvec); + struct lru_gen_mm_state *mm_state = get_mm_state(walk->lruvec); + + mm = list_entry(mm_state->head, struct mm_struct, lru_gen.list); + key = pgdat->node_id % BITS_PER_TYPE(mm->lru_gen.bitmap); + + if (!walk->force_scan && !test_bit(key, &mm->lru_gen.bitmap)) + return NULL; + + clear_bit(key, &mm->lru_gen.bitmap); + + return mmget_not_zero(mm) ? mm : NULL; +} + void lru_gen_add_mm(struct mm_struct *mm) { int nid; @@ -2755,10 +2782,11 @@ void lru_gen_add_mm(struct mm_struct *mm) for_each_node_state(nid, N_MEMORY) { struct lruvec *lruvec = get_lruvec(memcg, nid); + struct lru_gen_mm_state *mm_state = get_mm_state(lruvec); /* the first addition since the last iteration */ - if (lruvec->mm_state.tail == &mm_list->fifo) - lruvec->mm_state.tail = &mm->lru_gen.list; + if (mm_state->tail == &mm_list->fifo) + mm_state->tail = &mm->lru_gen.list; } list_add_tail(&mm->lru_gen.list, &mm_list->fifo); @@ -2784,14 +2812,15 @@ void lru_gen_del_mm(struct mm_struct *mm) for_each_node(nid) { struct lruvec *lruvec = get_lruvec(memcg, nid); + struct lru_gen_mm_state *mm_state = get_mm_state(lruvec); /* where the current iteration continues after */ - if (lruvec->mm_state.head == &mm->lru_gen.list) - lruvec->mm_state.head = lruvec->mm_state.head->prev; + if (mm_state->head == &mm->lru_gen.list) + mm_state->head = mm_state->head->prev; /* where the last iteration ended before */ - if (lruvec->mm_state.tail == &mm->lru_gen.list) - lruvec->mm_state.tail = lruvec->mm_state.tail->next; + if (mm_state->tail == &mm->lru_gen.list) + mm_state->tail = mm_state->tail->next; } list_del_init(&mm->lru_gen.list); @@ -2834,10 +2863,30 @@ void lru_gen_migrate_mm(struct mm_struct *mm) } #endif +#else /* !CONFIG_LRU_GEN_WALKS_MMU */ + +static struct lru_gen_mm_list *get_mm_list(struct mem_cgroup *memcg) +{ + return NULL; +} + +static struct lru_gen_mm_state *get_mm_state(struct lruvec *lruvec) +{ + return NULL; +} + +static struct mm_struct *get_next_mm(struct lru_gen_mm_walk *walk) +{ + return NULL; +} + +#endif + static void reset_mm_stats(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, bool last) { int i; int hist; + struct lru_gen_mm_state *mm_state = get_mm_state(lruvec); lockdep_assert_held(&get_mm_list(lruvec_memcg(lruvec))->lock); @@ -2845,44 +2894,20 @@ static void reset_mm_stats(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, hist = lru_hist_from_seq(walk->max_seq); for (i = 0; i < NR_MM_STATS; i++) { - WRITE_ONCE(lruvec->mm_state.stats[hist][i], - lruvec->mm_state.stats[hist][i] + walk->mm_stats[i]); + WRITE_ONCE(mm_state->stats[hist][i], + mm_state->stats[hist][i] + walk->mm_stats[i]); walk->mm_stats[i] = 0; } } if (NR_HIST_GENS > 1 && last) { - hist = lru_hist_from_seq(lruvec->mm_state.seq + 1); + hist = lru_hist_from_seq(mm_state->seq + 1); for (i = 0; i < NR_MM_STATS; i++) - WRITE_ONCE(lruvec->mm_state.stats[hist][i], 0); + WRITE_ONCE(mm_state->stats[hist][i], 0); } } -static bool should_skip_mm(struct mm_struct *mm, struct lru_gen_mm_walk *walk) -{ - int type; - unsigned long size = 0; - struct pglist_data *pgdat = lruvec_pgdat(walk->lruvec); - int key = pgdat->node_id % BITS_PER_TYPE(mm->lru_gen.bitmap); - - if (!walk->force_scan && !test_bit(key, &mm->lru_gen.bitmap)) - return true; - - clear_bit(key, &mm->lru_gen.bitmap); - - for (type = !walk->can_swap; type < ANON_AND_FILE; type++) { - size += type ? get_mm_counter(mm, MM_FILEPAGES) : - get_mm_counter(mm, MM_ANONPAGES) + - get_mm_counter(mm, MM_SHMEMPAGES); - } - - if (size < MIN_LRU_BATCH) - return true; - - return !mmget_not_zero(mm); -} - static bool iterate_mm_list(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, struct mm_struct **iter) { @@ -2891,7 +2916,7 @@ static bool iterate_mm_list(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, struct mm_struct *mm = NULL; struct mem_cgroup *memcg = lruvec_memcg(lruvec); struct lru_gen_mm_list *mm_list = get_mm_list(memcg); - struct lru_gen_mm_state *mm_state = &lruvec->mm_state; + struct lru_gen_mm_state *mm_state = get_mm_state(lruvec); /* * mm_state->seq is incremented after each iteration of mm_list. There @@ -2929,11 +2954,7 @@ static bool iterate_mm_list(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, mm_state->tail = mm_state->head->next; walk->force_scan = true; } - - mm = list_entry(mm_state->head, struct mm_struct, lru_gen.list); - if (should_skip_mm(mm, walk)) - mm = NULL; - } while (!mm); + } while (!(mm = get_next_mm(walk))); done: if (*iter || last) reset_mm_stats(lruvec, walk, last); @@ -2941,7 +2962,7 @@ static bool iterate_mm_list(struct lruvec *lruvec, struct lru_gen_mm_walk *walk, spin_unlock(&mm_list->lock); if (mm && first) - reset_bloom_filter(lruvec, walk->max_seq + 1); + reset_bloom_filter(mm_state, walk->max_seq + 1); if (*iter) mmput_async(*iter); @@ -2956,7 +2977,7 @@ static bool iterate_mm_list_nowalk(struct lruvec *lruvec, unsigned long max_seq) bool success = false; struct mem_cgroup *memcg = lruvec_memcg(lruvec); struct lru_gen_mm_list *mm_list = get_mm_list(memcg); - struct lru_gen_mm_state *mm_state = &lruvec->mm_state; + struct lru_gen_mm_state *mm_state = get_mm_state(lruvec); spin_lock(&mm_list->lock); @@ -3469,6 +3490,7 @@ static void walk_pmd_range(pud_t *pud, unsigned long start, unsigned long end, DECLARE_BITMAP(bitmap, MIN_LRU_BATCH); unsigned long first = -1; struct lru_gen_mm_walk *walk = args->private; + struct lru_gen_mm_state *mm_state = get_mm_state(walk->lruvec); VM_WARN_ON_ONCE(pud_leaf(*pud)); @@ -3520,7 +3542,7 @@ static void walk_pmd_range(pud_t *pud, unsigned long start, unsigned long end, walk_pmd_range_locked(pud, addr, vma, args, bitmap, &first); } - if (!walk->force_scan && !test_bloom_filter(walk->lruvec, walk->max_seq, pmd + i)) + if (!walk->force_scan && !test_bloom_filter(mm_state, walk->max_seq, pmd + i)) continue; walk->mm_stats[MM_NONLEAF_FOUND]++; @@ -3531,7 +3553,7 @@ static void walk_pmd_range(pud_t *pud, unsigned long start, unsigned long end, walk->mm_stats[MM_NONLEAF_ADDED]++; /* carry over to the next generation */ - update_bloom_filter(walk->lruvec, walk->max_seq + 1, pmd + i); + update_bloom_filter(mm_state, walk->max_seq + 1, pmd + i); } walk_pmd_range_locked(pud, -1, vma, args, bitmap, &first); @@ -3738,16 +3760,25 @@ static bool try_to_inc_min_seq(struct lruvec *lruvec, bool can_swap) return success; } -static void inc_max_seq(struct lruvec *lruvec, bool can_swap, bool force_scan) +static bool inc_max_seq(struct lruvec *lruvec, unsigned long max_seq, + bool can_swap, bool force_scan) { + bool success; int prev, next; int type, zone; struct lru_gen_folio *lrugen = &lruvec->lrugen; restart: + if (max_seq < READ_ONCE(lrugen->max_seq)) + return false; + spin_lock_irq(&lruvec->lru_lock); VM_WARN_ON_ONCE(!seq_is_valid(lruvec)); + success = max_seq == lrugen->max_seq; + if (!success) + goto unlock; + for (type = ANON_AND_FILE - 1; type >= 0; type--) { if (get_nr_gens(lruvec, type) != MAX_NR_GENS) continue; @@ -3791,8 +3822,10 @@ static void inc_max_seq(struct lruvec *lruvec, bool can_swap, bool force_scan) WRITE_ONCE(lrugen->timestamps[next], jiffies); /* make sure preceding modifications appear */ smp_store_release(&lrugen->max_seq, lrugen->max_seq + 1); - +unlock: spin_unlock_irq(&lruvec->lru_lock); + + return success; } static bool try_to_inc_max_seq(struct lruvec *lruvec, unsigned long max_seq, @@ -3802,14 +3835,16 @@ static bool try_to_inc_max_seq(struct lruvec *lruvec, unsigned long max_seq, struct lru_gen_mm_walk *walk; struct mm_struct *mm = NULL; struct lru_gen_folio *lrugen = &lruvec->lrugen; + struct lru_gen_mm_state *mm_state = get_mm_state(lruvec); VM_WARN_ON_ONCE(max_seq > READ_ONCE(lrugen->max_seq)); + if (!mm_state) + return inc_max_seq(lruvec, max_seq, can_swap, force_scan); + /* see the comment in iterate_mm_list() */ - if (max_seq <= READ_ONCE(lruvec->mm_state.seq)) { - success = false; - goto done; - } + if (max_seq <= READ_ONCE(mm_state->seq)) + return false; /* * If the hardware doesn't automatically set the accessed bit, fallback @@ -3839,8 +3874,10 @@ static bool try_to_inc_max_seq(struct lruvec *lruvec, unsigned long max_seq, walk_mm(lruvec, mm, walk); } while (mm); done: - if (success) - inc_max_seq(lruvec, can_swap, force_scan); + if (success) { + success = inc_max_seq(lruvec, max_seq, can_swap, force_scan); + WARN_ON_ONCE(!success); + } return success; } @@ -3964,6 +4001,7 @@ void lru_gen_look_around(struct page_vma_mapped_walk *pvmw) struct mem_cgroup *memcg = folio_memcg(folio); struct pglist_data *pgdat = folio_pgdat(folio); struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat); + struct lru_gen_mm_state *mm_state = get_mm_state(lruvec); DEFINE_MAX_SEQ(lruvec); int old_gen, new_gen = lru_gen_from_seq(max_seq); @@ -4042,8 +4080,8 @@ void lru_gen_look_around(struct page_vma_mapped_walk *pvmw) mem_cgroup_unlock_pages(); /* feedback from rmap walkers to page table walkers */ - if (suitable_to_scan(i, young)) - update_bloom_filter(lruvec, max_seq, pvmw->pmd); + if (mm_state && suitable_to_scan(i, young)) + update_bloom_filter(mm_state, max_seq, pvmw->pmd); } /****************************************************************************** @@ -5219,6 +5257,7 @@ static void lru_gen_seq_show_full(struct seq_file *m, struct lruvec *lruvec, int type, tier; int hist = lru_hist_from_seq(seq); struct lru_gen_folio *lrugen = &lruvec->lrugen; + struct lru_gen_mm_state *mm_state = get_mm_state(lruvec); for (tier = 0; tier < MAX_NR_TIERS; tier++) { seq_printf(m, " %10d", tier); @@ -5244,6 +5283,9 @@ static void lru_gen_seq_show_full(struct seq_file *m, struct lruvec *lruvec, seq_putc(m, '\n'); } + if (!mm_state) + return; + seq_puts(m, " "); for (i = 0; i < NR_MM_STATS; i++) { const char *s = " "; @@ -5251,10 +5293,10 @@ static void lru_gen_seq_show_full(struct seq_file *m, struct lruvec *lruvec, if (seq == max_seq && NR_HIST_GENS == 1) { s = "LOYNFA"; - n = READ_ONCE(lruvec->mm_state.stats[hist][i]); + n = READ_ONCE(mm_state->stats[hist][i]); } else if (seq != max_seq && NR_HIST_GENS > 1) { s = "loynfa"; - n = READ_ONCE(lruvec->mm_state.stats[hist][i]); + n = READ_ONCE(mm_state->stats[hist][i]); } seq_printf(m, " %10lu%c", n, s[i]); @@ -5523,6 +5565,7 @@ void lru_gen_init_lruvec(struct lruvec *lruvec) int i; int gen, type, zone; struct lru_gen_folio *lrugen = &lruvec->lrugen; + struct lru_gen_mm_state *mm_state = get_mm_state(lruvec); lrugen->max_seq = MIN_NR_GENS + 1; lrugen->enabled = lru_gen_enabled(); @@ -5533,7 +5576,8 @@ void lru_gen_init_lruvec(struct lruvec *lruvec) for_each_gen_type_zone(gen, type, zone) INIT_LIST_HEAD(&lrugen->folios[gen][type][zone]); - lruvec->mm_state.seq = MIN_NR_GENS; + if (mm_state) + mm_state->seq = MIN_NR_GENS; } #ifdef CONFIG_MEMCG @@ -5552,28 +5596,38 @@ void lru_gen_init_pgdat(struct pglist_data *pgdat) void lru_gen_init_memcg(struct mem_cgroup *memcg) { - INIT_LIST_HEAD(&memcg->mm_list.fifo); - spin_lock_init(&memcg->mm_list.lock); + struct lru_gen_mm_list *mm_list = get_mm_list(memcg); + + if (!mm_list) + return; + + INIT_LIST_HEAD(&mm_list->fifo); + spin_lock_init(&mm_list->lock); } void lru_gen_exit_memcg(struct mem_cgroup *memcg) { int i; int nid; + struct lru_gen_mm_list *mm_list = get_mm_list(memcg); - VM_WARN_ON_ONCE(!list_empty(&memcg->mm_list.fifo)); + VM_WARN_ON_ONCE(mm_list && !list_empty(&mm_list->fifo)); for_each_node(nid) { struct lruvec *lruvec = get_lruvec(memcg, nid); + struct lru_gen_mm_state *mm_state = get_mm_state(lruvec); VM_WARN_ON_ONCE(memchr_inv(lruvec->lrugen.nr_pages, 0, sizeof(lruvec->lrugen.nr_pages))); lruvec->lrugen.list.next = LIST_POISON1; + if (!mm_state) + continue; + for (i = 0; i < NR_BLOOM_FILTERS; i++) { - bitmap_free(lruvec->mm_state.filters[i]); - lruvec->mm_state.filters[i] = NULL; + bitmap_free(mm_state->filters[i]); + mm_state->filters[i] = NULL; } } } From patchwork Wed Dec 20 04:00:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 181439 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:24d3:b0:fb:cd0c:d3e with SMTP id r19csp2403316dyi; Tue, 19 Dec 2023 20:01:55 -0800 (PST) X-Google-Smtp-Source: AGHT+IGmLOaXBbmKMAXBs4fgo0NFcTVpqfClVhI6qNtEJ+ujla3q3asSXgCPPVfIothmFqTxIP2A X-Received: by 2002:a05:600c:5250:b0:40d:2c0d:784f with SMTP id fc16-20020a05600c525000b0040d2c0d784fmr986513wmb.75.1703044914965; Tue, 19 Dec 2023 20:01:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703044914; cv=none; d=google.com; s=arc-20160816; b=KQN20LRQIV5ya+IVmyZcQ4o8wo1ct/QY7ACNROQJdkTIHvA6ZZ2ABK3/RkklUubgGz 1n8OwUVRgX7XL4IOVwOlTl5yCch/OvP9raGg3e6YNv/e/XIUBzOd5I7b3LSUcpHeNIod IjQiHVaaJFUiTGFTdIiutFhjEBM1UIc4NjztKGmeTnisA1TNJJ6lTaGFNbhI2lc7jND9 000aYSFV+nfa4daZArZzQkxTs3G7y3k2TojDqvjoqvidQ9+e3PFMlswJZ9+gROj2QHaf Q5Abz25EfiCG6oMIeFB6fTsOEvDyHaRBwJzGNfUfbk/HbcNxQyYEwEJBRjoEttAFcM+V TtOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=eq0C6POTAn+zScnOr/3HnKmDnLhtNh9lvwfd8Bw3a74=; fh=COwL00CAVvf6HClNn9uW/acbdshq5DvvB3VM/yZ8sXw=; b=ryRN44HKK2isUUe3TXmvTgAvBy6EfJIj51b1HU/nudptNSvSYnva2l8zFaTFUG9WrI h1YN0PwQ4GH+AwvoDh0qTjP36U8cpmSa72mJPh5Xrp3V/GbHj72AWM1f0wsMp9lgAPKC fJBHDgL3vIeGVjMJ0IRgqOZjUDmv4ZFMGyo6SU7s98/PDRHN/eOVjjtJvGuddMvqvfvZ 2paPjuHuVWff6Kli8HVbHwbE4zIbf7eCtf2iL7bhShtIZ+bp8G/L0ylu9+BsCeL1teNg QQsiJnGenm05Kmg7bKT8FkuLHlPbVfYFIyW7BB+dq5JO7V4sSxgR332GLZ3tOodseqk4 6GGg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=vclXodO9; spf=pass (google.com: domain of linux-kernel+bounces-6343-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-6343-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id d16-20020a170906371000b00a235075a1e7si2553197ejc.493.2023.12.19.20.01.54 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Dec 2023 20:01:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-6343-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=vclXodO9; spf=pass (google.com: domain of linux-kernel+bounces-6343-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-6343-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 58F801F25AB5 for ; Wed, 20 Dec 2023 04:01:54 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 309F5156F0; Wed, 20 Dec 2023 04:01:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="vclXodO9" X-Original-To: linux-kernel@vger.kernel.org Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 18DC2125BC for ; Wed, 20 Dec 2023 04:01:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kinseyho.bounces.google.com Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-1d3e38ba362so6849445ad.2 for ; Tue, 19 Dec 2023 20:01:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1703044868; x=1703649668; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=eq0C6POTAn+zScnOr/3HnKmDnLhtNh9lvwfd8Bw3a74=; b=vclXodO9JJaBo0qFKvwHDdyJbC826Ubcielk0dMd+V7wHdNN2LgBIWEVMLeG7CtbeN f1eTbC45FNoV/81yi9G8COtiUcTy/RJUSL5T8FUg5G1LD/4amhAYXQx8Id2RYYPeHpAA /VOOjnfqWBZr4g5PCurrhzZTexm/+L1+j3XMJovYSDAt08XGqepS7BwzDbeFZQAsFN8x 7VtuTXTgCrWpitedSx4v2K9Y1artfixlZf2X+9+iK5b9Ln/VP8QP0YkN/ns9oPen2A6u rnq5867jIB232mmR34vszdF2OYlaI7EVaa81TQXombBsMSYqj+2tnLz/cyLDFnxlmUrz ZZew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703044868; x=1703649668; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=eq0C6POTAn+zScnOr/3HnKmDnLhtNh9lvwfd8Bw3a74=; b=JhhB+aezBqikwQWz3vlJ0/Ki16wxVao0cxUw2FvrybozH6URJLnp19ixPpSpx21r1f gSPMq2fOwkqiB7VUs4meSl7GFW+0tjiZgL7FmB3zG+oHUwss8ogqEoqc3Fe3J4FYXfon zhiadDVzAAB8SX9uiRxod5tPHeP8p8rgOQMU8OqmU2NvU8tLrDGY/zV4knUQ3IbuKJQT /MVfvO5jLtUqyCJP0FQZFRBsVTj4xZ/Ch71RhEseqIn6Rlp6mXDZ9inWq1YU0N55ySoX Ny4E3MI669GG08FwwxEu2W07uJ0cwe9gr0yOinesna0+ONe0hOO1Ghk2PHUZliJfC8DC YMdA== X-Gm-Message-State: AOJu0YyXRxvanwJ265uPODWGqmhG9bniwMYCv2ZJrA7Kg2wA1xSitEiu 85bou80KTwRqKTJRcYXar/ROeYX0enUrtg== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:aa7:8486:0:b0:6d8:a6fd:dc98 with SMTP id u6-20020aa78486000000b006d8a6fddc98mr88954pfn.6.1703044868243; Tue, 19 Dec 2023 20:01:08 -0800 (PST) Date: Wed, 20 Dec 2023 04:00:36 +0000 In-Reply-To: <20231220040037.883811-1-kinseyho@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20231220040037.883811-1-kinseyho@google.com> X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20231220040037.883811-4-kinseyho@google.com> Subject: [PATCH mm-unstable v1 3/4] mm/mglru: remove CONFIG_MEMCG From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, yuzhao@google.com, Kinsey Ho , Aneesh Kumar K V X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785772024858642970 X-GMAIL-MSGID: 1785772024858642970 Remove CONFIG_MEMCG in a refactoring to improve code readability at the cost of a few bytes in struct lru_gen_folio per node when CONFIG_MEMCG=n. Signed-off-by: Kinsey Ho Co-developed-by: Aneesh Kumar K V Signed-off-by: Aneesh Kumar K V --- include/linux/mm_types.h | 4 --- include/linux/mmzone.h | 26 ++-------------- mm/vmscan.c | 67 +++++++++++++--------------------------- 3 files changed, 23 insertions(+), 74 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 552fa2d11c57..55b7121809ff 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -1017,9 +1017,7 @@ struct lru_gen_mm_list { void lru_gen_add_mm(struct mm_struct *mm); void lru_gen_del_mm(struct mm_struct *mm); -#ifdef CONFIG_MEMCG void lru_gen_migrate_mm(struct mm_struct *mm); -#endif static inline void lru_gen_init_mm(struct mm_struct *mm) { @@ -1050,11 +1048,9 @@ static inline void lru_gen_del_mm(struct mm_struct *mm) { } -#ifdef CONFIG_MEMCG static inline void lru_gen_migrate_mm(struct mm_struct *mm) { } -#endif static inline void lru_gen_init_mm(struct mm_struct *mm) { diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index bc3f63ec4291..28665e1b8475 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -440,14 +440,12 @@ struct lru_gen_folio { atomic_long_t refaulted[NR_HIST_GENS][ANON_AND_FILE][MAX_NR_TIERS]; /* whether the multi-gen LRU is enabled */ bool enabled; -#ifdef CONFIG_MEMCG /* the memcg generation this lru_gen_folio belongs to */ u8 gen; /* the list segment this lru_gen_folio belongs to */ u8 seg; /* per-node lru_gen_folio list for global reclaim */ struct hlist_nulls_node list; -#endif }; enum { @@ -493,11 +491,6 @@ struct lru_gen_mm_walk { bool force_scan; }; -void lru_gen_init_lruvec(struct lruvec *lruvec); -void lru_gen_look_around(struct page_vma_mapped_walk *pvmw); - -#ifdef CONFIG_MEMCG - /* * For each node, memcgs are divided into two generations: the old and the * young. For each generation, memcgs are randomly sharded into multiple bins @@ -555,6 +548,8 @@ struct lru_gen_memcg { }; void lru_gen_init_pgdat(struct pglist_data *pgdat); +void lru_gen_init_lruvec(struct lruvec *lruvec); +void lru_gen_look_around(struct page_vma_mapped_walk *pvmw); void lru_gen_init_memcg(struct mem_cgroup *memcg); void lru_gen_exit_memcg(struct mem_cgroup *memcg); @@ -563,19 +558,6 @@ void lru_gen_offline_memcg(struct mem_cgroup *memcg); void lru_gen_release_memcg(struct mem_cgroup *memcg); void lru_gen_soft_reclaim(struct mem_cgroup *memcg, int nid); -#else /* !CONFIG_MEMCG */ - -#define MEMCG_NR_GENS 1 - -struct lru_gen_memcg { -}; - -static inline void lru_gen_init_pgdat(struct pglist_data *pgdat) -{ -} - -#endif /* CONFIG_MEMCG */ - #else /* !CONFIG_LRU_GEN */ static inline void lru_gen_init_pgdat(struct pglist_data *pgdat) @@ -590,8 +572,6 @@ static inline void lru_gen_look_around(struct page_vma_mapped_walk *pvmw) { } -#ifdef CONFIG_MEMCG - static inline void lru_gen_init_memcg(struct mem_cgroup *memcg) { } @@ -616,8 +596,6 @@ static inline void lru_gen_soft_reclaim(struct mem_cgroup *memcg, int nid) { } -#endif /* CONFIG_MEMCG */ - #endif /* CONFIG_LRU_GEN */ struct lruvec { diff --git a/mm/vmscan.c b/mm/vmscan.c index aa7ea09ffb4c..351a0b5043c0 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4097,13 +4097,6 @@ enum { MEMCG_LRU_YOUNG, }; -#ifdef CONFIG_MEMCG - -static int lru_gen_memcg_seg(struct lruvec *lruvec) -{ - return READ_ONCE(lruvec->lrugen.seg); -} - static void lru_gen_rotate_memcg(struct lruvec *lruvec, int op) { int seg; @@ -4150,6 +4143,8 @@ static void lru_gen_rotate_memcg(struct lruvec *lruvec, int op) spin_unlock_irqrestore(&pgdat->memcg_lru.lock, flags); } +#ifdef CONFIG_MEMCG + void lru_gen_online_memcg(struct mem_cgroup *memcg) { int gen; @@ -4217,18 +4212,11 @@ void lru_gen_soft_reclaim(struct mem_cgroup *memcg, int nid) struct lruvec *lruvec = get_lruvec(memcg, nid); /* see the comment on MEMCG_NR_GENS */ - if (lru_gen_memcg_seg(lruvec) != MEMCG_LRU_HEAD) + if (READ_ONCE(lruvec->lrugen.seg) != MEMCG_LRU_HEAD) lru_gen_rotate_memcg(lruvec, MEMCG_LRU_HEAD); } -#else /* !CONFIG_MEMCG */ - -static int lru_gen_memcg_seg(struct lruvec *lruvec) -{ - return 0; -} - -#endif +#endif /* CONFIG_MEMCG */ /****************************************************************************** * the eviction @@ -4776,7 +4764,7 @@ static int shrink_one(struct lruvec *lruvec, struct scan_control *sc) if (mem_cgroup_below_low(NULL, memcg)) { /* see the comment on MEMCG_NR_GENS */ - if (lru_gen_memcg_seg(lruvec) != MEMCG_LRU_TAIL) + if (READ_ONCE(lruvec->lrugen.seg) != MEMCG_LRU_TAIL) return MEMCG_LRU_TAIL; memcg_memory_event(memcg, MEMCG_LOW); @@ -4799,12 +4787,10 @@ static int shrink_one(struct lruvec *lruvec, struct scan_control *sc) return 0; /* one retry if offlined or too small */ - return lru_gen_memcg_seg(lruvec) != MEMCG_LRU_TAIL ? + return READ_ONCE(lruvec->lrugen.seg) != MEMCG_LRU_TAIL ? MEMCG_LRU_TAIL : MEMCG_LRU_YOUNG; } -#ifdef CONFIG_MEMCG - static void shrink_many(struct pglist_data *pgdat, struct scan_control *sc) { int op; @@ -4896,20 +4882,6 @@ static void lru_gen_shrink_lruvec(struct lruvec *lruvec, struct scan_control *sc blk_finish_plug(&plug); } -#else /* !CONFIG_MEMCG */ - -static void shrink_many(struct pglist_data *pgdat, struct scan_control *sc) -{ - BUILD_BUG(); -} - -static void lru_gen_shrink_lruvec(struct lruvec *lruvec, struct scan_control *sc) -{ - BUILD_BUG(); -} - -#endif - static void set_initial_priority(struct pglist_data *pgdat, struct scan_control *sc) { int priority; @@ -5560,6 +5532,18 @@ static const struct file_operations lru_gen_ro_fops = { * initialization ******************************************************************************/ +void lru_gen_init_pgdat(struct pglist_data *pgdat) +{ + int i, j; + + spin_lock_init(&pgdat->memcg_lru.lock); + + for (i = 0; i < MEMCG_NR_GENS; i++) { + for (j = 0; j < MEMCG_NR_BINS; j++) + INIT_HLIST_NULLS_HEAD(&pgdat->memcg_lru.fifo[i][j], i); + } +} + void lru_gen_init_lruvec(struct lruvec *lruvec) { int i; @@ -5582,18 +5566,6 @@ void lru_gen_init_lruvec(struct lruvec *lruvec) #ifdef CONFIG_MEMCG -void lru_gen_init_pgdat(struct pglist_data *pgdat) -{ - int i, j; - - spin_lock_init(&pgdat->memcg_lru.lock); - - for (i = 0; i < MEMCG_NR_GENS; i++) { - for (j = 0; j < MEMCG_NR_BINS; j++) - INIT_HLIST_NULLS_HEAD(&pgdat->memcg_lru.fifo[i][j], i); - } -} - void lru_gen_init_memcg(struct mem_cgroup *memcg) { struct lru_gen_mm_list *mm_list = get_mm_list(memcg); @@ -5653,14 +5625,17 @@ late_initcall(init_lru_gen); static void lru_gen_age_node(struct pglist_data *pgdat, struct scan_control *sc) { + BUILD_BUG(); } static void lru_gen_shrink_lruvec(struct lruvec *lruvec, struct scan_control *sc) { + BUILD_BUG(); } static void lru_gen_shrink_node(struct pglist_data *pgdat, struct scan_control *sc) { + BUILD_BUG(); } #endif /* CONFIG_LRU_GEN */ From patchwork Wed Dec 20 04:00:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 181441 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:24d3:b0:fb:cd0c:d3e with SMTP id r19csp2403420dyi; Tue, 19 Dec 2023 20:02:13 -0800 (PST) X-Google-Smtp-Source: AGHT+IFcGYuIaMuZXy1N+ZPfIL/VQAJGydS7JnIJEEPWt4+Y3JdDXOBH+XZNi3GyoQbO5VWCo2IP X-Received: by 2002:a05:6870:cd84:b0:1fb:23:6feb with SMTP id xb4-20020a056870cd8400b001fb00236febmr21303838oab.39.1703044933266; Tue, 19 Dec 2023 20:02:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1703044933; cv=none; d=google.com; s=arc-20160816; b=fmDMcdmtNEb0lWlr6YM5tgqI46IMCyHXE97sahH45M3I2jHi1D1tCuBknUbZdhXwkd 6MR+ejaOp4wUm9Zq3pqHKhIPXRk3XjDRjkiHzQgVmpqieSZGCVFQ6EPQEgcf0YGApIWp N84yeLN2Hs9Vll5EHHknHGmI25o9Y9sVYobOUeWVj0Q+Q0fll0KyaMp4YPHIiBhFln/p edK/HMinmMtbnA2HpQeuBXVfyN1msyz1JL5eCeAjqJP5t1ZP9Aq1EvbEIYT35SUlkKBg L15kJHgWiDUd4+0GziopDns6ZPhTvF8IJT+VqAzqL1bzmcngjnTUGmW2kiaMT8Y8Pfki EyIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=mFAbLGESrfkdyY7V53iLSJc53insEPkBL2A8CkUV1gg=; fh=COwL00CAVvf6HClNn9uW/acbdshq5DvvB3VM/yZ8sXw=; b=KEosb+QORt2a1fmPcq+kas/I0V2RBk3RWuMfiK3Yeybr10cM1Rp2ozWnoiJRzS80Ub aH783MHlVtJLX9A1dQwBvmdm1MfcxsTGeW9/xL5iPAeA3LTV8RyCFQ/5NiJBSJerDtY/ PGIlji+JNHjVQeB0V5j+AQFF8IRkMqAqvcKXvZIMKpMvHG6m9omlPmI8tOJCNMmFDTH0 NS6JWnN8dUiUS5c02goq5HC73IVKAiUxCIuyi2eRzgjJCQTAdSwMWlDQtgcT57Tb86Ha Wfm+G17qCFZHj64NN2IRRwDhEwr+OhznyS2CeO9NzTaAuwcauxI1hAfXpkzq0UM4MzAl l0YQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=VQARkHFT; spf=pass (google.com: domain of linux-kernel+bounces-6345-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-6345-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id ck17-20020a056a02091100b005cd7c92f4f2si7359056pgb.671.2023.12.19.20.02.13 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Dec 2023 20:02:13 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-6345-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=VQARkHFT; spf=pass (google.com: domain of linux-kernel+bounces-6345-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-6345-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id E6F292885E9 for ; Wed, 20 Dec 2023 04:02:12 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 389C6168DF; Wed, 20 Dec 2023 04:01:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="VQARkHFT" X-Original-To: linux-kernel@vger.kernel.org Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 34A1413ACF for ; Wed, 20 Dec 2023 04:01:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kinseyho.bounces.google.com Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-dbccc6dfa6cso453550276.1 for ; Tue, 19 Dec 2023 20:01:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1703044870; x=1703649670; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=mFAbLGESrfkdyY7V53iLSJc53insEPkBL2A8CkUV1gg=; b=VQARkHFTbBsbnkb47yfQjmXLWmQuIZYPhGXzKpMpuNzEvbD6Kpuxd56H0XRbjNxJti l2EEL5qoe8YV0cyWWy3jo3gBQMtSsMJ/ykltKHbomgHRdZOA6o+pEL4NlgPJw0gDmu8r 3RtzYavOLYVLGfaimgxmrjOcpQaxuLZdi37O09ct1p1bEjHziu/juSBwoZRc8KDZCNie n44iAVPmIxhQzm76F4uWDrTL0MRnTgQaPhM/pFkWwxAlcCi+MZ39Lx7CzrDvPfaspHRC gbppTldhOfjG7sK/dwKgPIdBomB4ZUuMaogXszFzqNrlkYqblp9SDbLXTwGEmxT0zebo k5hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703044870; x=1703649670; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=mFAbLGESrfkdyY7V53iLSJc53insEPkBL2A8CkUV1gg=; b=h0ugH6aaFnkIe52fYQ5FwS1hSBG3FLV+PKlgd1pbEK569tqBVf4vvEb7a/QAm9NJxM Fl1FtLjqpYVnpXflMSGZKfbiQDAqdDWRrI+kLbeiIYCfsn681RqOeXeFGeSOB9L6zn1w ymKi4CQddpTYcSulpmptL6NeGpdjzHOibx7kTZkL7dZk6R37cHCtlFIe06O5WJN6e8/+ UiYTFUNEXEv9dc6TakqZfIsTh0JS5ByPEYS8lGa050gQlnURu/tB/nltr3P3YUrbaKjF svUAZ3Y2Xkko19bIu/klcpqoBIKwIky4AXXBmsargOPfpMFwvFYMG8ga4wb/ERjQX6FF 28cQ== X-Gm-Message-State: AOJu0YzJKGyuxmqbK9Z0s+EygTWZ8fUgYSLqVV3Slf1kbmZt8pHtdNUn nsuvamcs0zdHknn4JVOL/ZEJZHoZ9+7NhO4COFJc X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a05:6902:27c5:b0:dbc:c5f8:ae16 with SMTP id ec5-20020a05690227c500b00dbcc5f8ae16mr806580ybb.5.1703044870249; Tue, 19 Dec 2023 20:01:10 -0800 (PST) Date: Wed, 20 Dec 2023 04:00:37 +0000 In-Reply-To: <20231220040037.883811-1-kinseyho@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20231220040037.883811-1-kinseyho@google.com> X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20231220040037.883811-5-kinseyho@google.com> Subject: [PATCH mm-unstable v1 4/4] mm/mglru: remove CONFIG_TRANSPARENT_HUGEPAGE From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, yuzhao@google.com, Kinsey Ho , Aneesh Kumar K V X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785772044170245839 X-GMAIL-MSGID: 1785772044170245839 Improve code readability by removing CONFIG_TRANSPARENT_HUGEPAGE, since the compiler should be able to automatically optimize out the code that promotes THPs during page table walks. No functional changes. Signed-off-by: Kinsey Ho Co-developed-by: Aneesh Kumar K V Signed-off-by: Aneesh Kumar K V --- mm/vmscan.c | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 351a0b5043c0..ceba905e5630 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -3273,7 +3273,6 @@ static unsigned long get_pte_pfn(pte_t pte, struct vm_area_struct *vma, unsigned return pfn; } -#if defined(CONFIG_TRANSPARENT_HUGEPAGE) || defined(CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG) static unsigned long get_pmd_pfn(pmd_t pmd, struct vm_area_struct *vma, unsigned long addr) { unsigned long pfn = pmd_pfn(pmd); @@ -3291,7 +3290,6 @@ static unsigned long get_pmd_pfn(pmd_t pmd, struct vm_area_struct *vma, unsigned return pfn; } -#endif static struct folio *get_pfn_folio(unsigned long pfn, struct mem_cgroup *memcg, struct pglist_data *pgdat, bool can_swap) @@ -3394,7 +3392,6 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end, return suitable_to_scan(total, young); } -#if defined(CONFIG_TRANSPARENT_HUGEPAGE) || defined(CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG) static void walk_pmd_range_locked(pud_t *pud, unsigned long addr, struct vm_area_struct *vma, struct mm_walk *args, unsigned long *bitmap, unsigned long *first) { @@ -3472,12 +3469,6 @@ static void walk_pmd_range_locked(pud_t *pud, unsigned long addr, struct vm_area done: *first = -1; } -#else -static void walk_pmd_range_locked(pud_t *pud, unsigned long addr, struct vm_area_struct *vma, - struct mm_walk *args, unsigned long *bitmap, unsigned long *first) -{ -} -#endif static void walk_pmd_range(pud_t *pud, unsigned long start, unsigned long end, struct mm_walk *args) @@ -3513,7 +3504,6 @@ static void walk_pmd_range(pud_t *pud, unsigned long start, unsigned long end, continue; } -#ifdef CONFIG_TRANSPARENT_HUGEPAGE if (pmd_trans_huge(val)) { unsigned long pfn = pmd_pfn(val); struct pglist_data *pgdat = lruvec_pgdat(walk->lruvec); @@ -3532,7 +3522,7 @@ static void walk_pmd_range(pud_t *pud, unsigned long start, unsigned long end, walk_pmd_range_locked(pud, addr, vma, args, bitmap, &first); continue; } -#endif + walk->mm_stats[MM_NONLEAF_TOTAL]++; if (should_clear_pmd_young()) {