From patchwork Thu Apr 13 05:54:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Yang X-Patchwork-Id: 82812 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp816971vqo; Wed, 12 Apr 2023 23:10:11 -0700 (PDT) X-Google-Smtp-Source: AKy350bmiaD2fsr4Gk2kd6Vtyt/A29CKfqGoTGcrb4rw8OvHkW+2zGKRVlNlSJzDyj+/wXHhjlO5 X-Received: by 2002:aa7:dd47:0:b0:4fc:d331:515 with SMTP id o7-20020aa7dd47000000b004fcd3310515mr687127edw.5.1681366211425; Wed, 12 Apr 2023 23:10:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681366211; cv=none; d=google.com; s=arc-20160816; b=EsviLkNO1uw8bigefLZw6LittgPgoCHdr61DaVRm7Omqg/181zTxg+5axGvwM3XOwS gXPoAyrWQhABGoZDIR8g/FxgPRg2pMVTZcA9/mlnw+B6KcOfB7OSmCnIJ+3ahG2N9tFe hT+36JPRLJO0jPnXMLli54mE+qyetcix9PR7jXiPjHQZKgT+tKlcTOV8PjYRhjGsYECb DLCMa6HnoFJPrK2tBALsTWzMDBUVrONmAdiI6R9340Q8wk9IQG+JX9ygTaP8xT821Dho 6UIoh+mhHfRtMs92QOd3FThrKOTDWNRKQqniIjxUlfTu4DKUXcQbYjOPc9h2o8SA/Onk esLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=GSmcPwTaUQvWeJMPDmnm7a66QpkRWyC6WCGMUW/L4as=; b=xnEDpGPp5oEw5JWbr+qmxRqYds2f2ixDELvcYdIGz0oxIvbUPpzLHN2cjDslG9OWHq DG6sYuBs8/Lsaq/OlSqhK2I1C6VTJnzBZeIW5JHFjZ2K5SRLG2D/hpm9JPlBC1t9/TKo ovvnta0k4EzwheM2ubUUlVV+p3XcCVNDuDLHfCl1MQHZx1t5E6vCW51qLc3PWOX3dwJZ xRTl++9ajhTWc1s2f0QAusw4G3pJe6zqnzo34JwmsPkE6T1l4Q1MKrJ2CS2aLyv0wYBx MK6E2nDj1KwJ+WTEfp1gS8VzaEuASfY1EBL10oNM4mQH/W5BaSfqCK3/fJXgGMWz/viy 5MHQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=zte.com.cn Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id v25-20020a17090651d900b0094a80bd0a19si936355ejk.636.2023.04.12.23.09.47; Wed, 12 Apr 2023 23:10:11 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=zte.com.cn Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229703AbjDMFyq (ORCPT + 99 others); Thu, 13 Apr 2023 01:54:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229492AbjDMFyo (ORCPT ); Thu, 13 Apr 2023 01:54:44 -0400 Received: from ubuntu20 (unknown [193.203.214.57]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B85D810E5 for ; Wed, 12 Apr 2023 22:54:42 -0700 (PDT) Received: by ubuntu20 (Postfix, from userid 1003) id 66067E090F; Thu, 13 Apr 2023 05:54:41 +0000 (UTC) From: Yang Yang To: akpm@linux-foundation.org, david@redhat.com Cc: yang.yang29@zte.com.cn, imbrenda@linux.ibm.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, ran.xiaokai@zte.com.cn, xu.xin.sc@gmail.com, xu.xin16@zte.com.cn, Xuexin Jiang Subject: [PATCH v7 2/6] ksm: count all zero pages placed by KSM Date: Thu, 13 Apr 2023 13:54:39 +0800 Message-Id: <20230413055439.181039-1-yang.yang29@zte.com.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <202304131346489021903@zte.com.cn> References: <202304131346489021903@zte.com.cn> MIME-Version: 1.0 X-Spam-Status: No, score=3.4 required=5.0 tests=BAYES_00, FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM,FSL_HELO_NON_FQDN_1, HEADER_FROM_DIFFERENT_DOMAINS,HELO_NO_DOMAIN,NO_DNS_FOR_FROM, RCVD_IN_PBL,RDNS_NONE,SPF_SOFTFAIL,SPOOFED_FREEMAIL_NO_RDNS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Level: *** X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763040256440393774?= X-GMAIL-MSGID: =?utf-8?q?1763040256440393774?= From: xu xin As pages_sharing and pages_shared don't include the number of zero pages merged by KSM, we cannot know how many pages are zero pages placed by KSM when enabling use_zero_pages, which leads to KSM not being transparent with all actual merged pages by KSM. In the early days of use_zero_pages, zero-pages was unable to get unshared by the ways like MADV_UNMERGEABLE so it's hard to count how many times one of those zeropages was then unmerged. But now, unsharing KSM-placed zero page accurately has been achieved, so we can easily count both how many times a page full of zeroes was merged with zero-page and how many times one of those pages was then unmerged. and so, it helps to estimate memory demands when each and every shared page could get unshared. So we add ksm_zero_pages under /sys/kernel/mm/ksm/ to show the number of all zero pages placed by KSM. Signed-off-by: xu xin Suggested-by: David Hildenbrand Cc: Claudio Imbrenda Cc: Xuexin Jiang Reviewed-by: Xiaokai Ran Reviewed-by: Yang Yang --- include/linux/ksm.h | 16 ++++++++++++++++ mm/ksm.c | 18 ++++++++++++++++++ mm/memory.c | 7 ++++++- 3 files changed, 40 insertions(+), 1 deletion(-) diff --git a/include/linux/ksm.h b/include/linux/ksm.h index f0cc085be42a..ea628d2a9105 100644 --- a/include/linux/ksm.h +++ b/include/linux/ksm.h @@ -99,9 +99,25 @@ static inline void folio_migrate_ksm(struct folio *newfolio, struct folio *old) /* use pte_mkdirty to track a KSM-placed zero page */ #define set_pte_ksm_zero(pte) pte_mkdirty(pte_mkspecial(pte)) #define is_ksm_zero_pte(pte) (is_zero_pfn(pte_pfn(pte)) && pte_dirty(pte)) +extern unsigned long ksm_zero_pages; +static inline void inc_ksm_zero_pages(void) +{ + ksm_zero_pages++; +} + +static inline void dec_ksm_zero_pages(void) +{ + ksm_zero_pages--; +} #else /* !CONFIG_KSM_ZERO_PAGES_TRACK */ #define set_pte_ksm_zero(pte) pte_mkspecial(pte) #define is_ksm_zero_pte(pte) 0 +static inline void inc_ksm_zero_pages(void) +{ +} +static inline void dec_ksm_zero_pages(void) +{ +} #endif /* CONFIG_KSM_ZERO_PAGES_TRACK */ #endif /* __LINUX_KSM_H */ diff --git a/mm/ksm.c b/mm/ksm.c index 1d1771a6b3fe..232680393741 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -278,6 +278,11 @@ static unsigned int zero_checksum __read_mostly; /* Whether to merge empty (zeroed) pages with actual zero pages */ static bool ksm_use_zero_pages __read_mostly; +#ifdef CONFIG_KSM_ZERO_PAGES_TRACK +/* The number of zero pages which is placed by KSM */ +unsigned long ksm_zero_pages; +#endif + #ifdef CONFIG_NUMA /* Zeroed when merging across nodes is not allowed */ static unsigned int ksm_merge_across_nodes = 1; @@ -1243,6 +1248,7 @@ static int replace_page(struct vm_area_struct *vma, struct page *page, } else { newpte = set_pte_ksm_zero(pfn_pte(page_to_pfn(kpage), vma->vm_page_prot)); + inc_ksm_zero_pages(); /* * We're replacing an anonymous page with a zero page, which is * not anonymous. We need to do proper accounting otherwise we @@ -3216,6 +3222,15 @@ static ssize_t pages_volatile_show(struct kobject *kobj, } KSM_ATTR_RO(pages_volatile); +#ifdef CONFIG_KSM_ZERO_PAGES_TRACK +static ssize_t ksm_zero_pages_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + return sysfs_emit(buf, "%ld\n", ksm_zero_pages); +} +KSM_ATTR_RO(ksm_zero_pages); +#endif /* CONFIG_KSM_ZERO_PAGES_TRACK */ + static ssize_t general_profit_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) { @@ -3286,6 +3301,9 @@ static struct attribute *ksm_attrs[] = { &pages_sharing_attr.attr, &pages_unshared_attr.attr, &pages_volatile_attr.attr, +#ifdef CONFIG_KSM_ZERO_PAGES_TRACK + &ksm_zero_pages_attr.attr, +#endif &full_scans_attr.attr, #ifdef CONFIG_NUMA &merge_across_nodes_attr.attr, diff --git a/mm/memory.c b/mm/memory.c index 42dd1ab5e4e6..76598287280f 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1416,8 +1416,11 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, tlb_remove_tlb_entry(tlb, pte, addr); zap_install_uffd_wp_if_needed(vma, addr, pte, details, ptent); - if (unlikely(!page)) + if (unlikely(!page)) { + if (is_ksm_zero_pte(ptent)) + dec_ksm_zero_pages(); continue; + } delay_rmap = 0; if (!PageAnon(page)) { @@ -3118,6 +3121,8 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) inc_mm_counter(mm, MM_ANONPAGES); } } else { + if (is_ksm_zero_pte(vmf->orig_pte)) + dec_ksm_zero_pages(); inc_mm_counter(mm, MM_ANONPAGES); } flush_cache_page(vma, vmf->address, pte_pfn(vmf->orig_pte));