From patchwork Tue Sep 12 17:52:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 138356 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9ecd:0:b0:3f2:4152:657d with SMTP id t13csp590601vqx; Tue, 12 Sep 2023 11:10:26 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHgGFoO2TZSUd/2LcB7L/PRp31s+RrGu/0/FZiFqtUaogj0aNi6nnWsSO/khmUbD0MOynV1 X-Received: by 2002:a17:902:9007:b0:1b8:400a:48f2 with SMTP id a7-20020a170902900700b001b8400a48f2mr506979plp.62.1694542226211; Tue, 12 Sep 2023 11:10:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694542226; cv=none; d=google.com; s=arc-20160816; b=XqxiPBqGBpAaFGu0f724UPuOJIWbbA37rff6VshVkQ6L75iR7SosnlhHyzVe1o5wG6 GR69Ba3LPG0L2X7d5OMzXVC3wc2GhVokdWdDjnpW3Ci7wQz+PitVfOVgZ8z1DD059S/u X778JVd5Zj72eMUfQ7kXL2baVw778KhmzAcvTtxidwcdUMog8CowjSgvfn3+votnOUo/ lel/UjpJNjhVD/GdeTuAJtG7oh/n1hoqN8mSLsgOTZaXuIl/fi8K0O2BwHm4eh48u/PK HtfA7Hj/hyn+cfI5S5dfz8TLm5b3spNzwZlOs6DZduByJA+tjxJQoXGOYMxFGDaw8Imw ftfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=4cyzr/0K5Dw6Q79NtyCfj6eo018Xmxirx96tjFXharo=; fh=sS+J4OyOC0EcVLWJpS3mBHGeO+0+dYZJ+ImCUfzsrH4=; b=AQ15Hm/s4DNmMfkSyYI2a1YzUkPKvh2u2zSLrAIgNrKzb0KXUAJLDN25Nj2WZoU/HM gb5vuyP7sJVR7Q2Aho3eVONc6Kf7cuGAfKXtN3GNrO8xdE8YI4TdzVvxPJvjr8fh52tG 9Le+AfcysW0z40j/8J+orAGiIUYW4YWR99rEXAPy6mxmt0yg+PSCbdyyHnDWOg2DUJ72 G0sTJNQsuYWhkiJ8p2w96QULx4/eNZrUbrl0p73eu6jB0lYI8uekxAkCME8JrbTcbRuT HophXfU6nSZKFuJIZfn5mfbhVZp5C1hwfwjad/5vXspW9XoXAd5XSjjKqT/bGJSwb3Fy V//Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from fry.vger.email (fry.vger.email. [23.128.96.38]) by mx.google.com with ESMTPS id im23-20020a170902bb1700b001b3bd85f54bsi8095854plb.35.2023.09.12.11.10.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 11:10:26 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id 305348211222; Tue, 12 Sep 2023 10:54:01 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237436AbjILRxm (ORCPT + 36 others); Tue, 12 Sep 2023 13:53:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57116 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237285AbjILRxW (ORCPT ); Tue, 12 Sep 2023 13:53:22 -0400 Received: from 66-220-144-179.mail-mxout.facebook.com (66-220-144-179.mail-mxout.facebook.com [66.220.144.179]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9FDBF1FE3 for ; Tue, 12 Sep 2023 10:52:53 -0700 (PDT) Received: by devbig1114.prn1.facebook.com (Postfix, from userid 425415) id DF124BCD1629; Tue, 12 Sep 2023 10:52:38 -0700 (PDT) From: Stefan Roesch To: kernel-team@fb.com Cc: shr@devkernel.io, akpm@linux-foundation.org, david@redhat.com, hannes@cmpxchg.org, riel@surriel.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v1 1/4] mm/ksm: add "smart" page scanning mode Date: Tue, 12 Sep 2023 10:52:25 -0700 Message-Id: <20230912175228.952039-2-shr@devkernel.io> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230912175228.952039-1-shr@devkernel.io> References: <20230912175228.952039-1-shr@devkernel.io> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Tue, 12 Sep 2023 10:54:01 -0700 (PDT) X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776856309198706670 X-GMAIL-MSGID: 1776856309198706670 This change adds a "smart" page scanning mode for KSM. So far all the candidate pages are continuously scanned to find candidates for de-duplication. There are a considerably number of pages that cannot be de-duplicated. This is costly in terms of CPU. By using smart scanning considerable CPU savings can be achieved. This change takes the history of scanning pages into account and skips the page scanning of certain pages for a while if de-deduplication for this page has not been successful in the past. To do this it introduces two new fields in the ksm_rmap_item structure: age and skip_age. age, is the KSM age and skip_page is the age for how long page scanning of this page is skipped. The age field is incremented each time the page is scanned and the page cannot be de-duplicated. How often a page is skipped is dependent how often de-duplication has been tried so far and the number of skips is currently limited to 8. This value has shown to be effective with different workloads. The feature is currently disable by default and can be enabled with the new smart_scan knob. The feature has shown to be very effective: upt to 25% of the page scans can be eliminated; the pages_to_scan rate can be reduced by 40 - 50% and a similar de-duplication rate can be maintained. Signed-off-by: Stefan Roesch --- mm/ksm.c | 75 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+) diff --git a/mm/ksm.c b/mm/ksm.c index 981af9c72e7a..bfd5087c7d5a 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -56,6 +56,8 @@ #define DO_NUMA(x) do { } while (0) #endif +typedef u8 rmap_age_t; + /** * DOC: Overview * @@ -193,6 +195,8 @@ struct ksm_stable_node { * @node: rb node of this rmap_item in the unstable tree * @head: pointer to stable_node heading this list in the stable tree * @hlist: link into hlist of rmap_items hanging off that stable_node + * @age: number of scan iterations since creation + * @skip_age: skip rmap item until age reaches skip_age */ struct ksm_rmap_item { struct ksm_rmap_item *rmap_list; @@ -212,6 +216,8 @@ struct ksm_rmap_item { struct hlist_node hlist; }; }; + rmap_age_t age; + rmap_age_t skip_age; }; #define SEQNR_MASK 0x0ff /* low bits of unstable tree seqnr */ @@ -281,6 +287,9 @@ static unsigned int zero_checksum __read_mostly; /* Whether to merge empty (zeroed) pages with actual zero pages */ static bool ksm_use_zero_pages __read_mostly; +/* Skip pages that couldn't be de-duplicated previously */ +static bool ksm_smart_scan; + /* The number of zero pages which is placed by KSM */ unsigned long ksm_zero_pages; @@ -2305,6 +2314,45 @@ static struct ksm_rmap_item *get_next_rmap_item(struct ksm_mm_slot *mm_slot, return rmap_item; } +static unsigned int inc_skip_age(rmap_age_t age) +{ + if (age <= 3) + return 1; + if (age <= 5) + return 2; + if (age <= 8) + return 4; + + return 8; +} + +static bool skip_rmap_item(struct page *page, struct ksm_rmap_item *rmap_item) +{ + rmap_age_t age; + + if (!ksm_smart_scan) + return false; + + if (PageKsm(page)) + return false; + + age = rmap_item->age++; + if (age < 3) + return false; + + if (rmap_item->skip_age == age) { + rmap_item->skip_age = 0; + return false; + } + + if (rmap_item->skip_age == 0) { + rmap_item->skip_age = age + inc_skip_age(age); + remove_rmap_item_from_tree(rmap_item); + } + + return true; +} + static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) { struct mm_struct *mm; @@ -2409,6 +2457,10 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) if (rmap_item) { ksm_scan.rmap_list = &rmap_item->rmap_list; + + if (skip_rmap_item(*page, rmap_item)) + goto next_page; + ksm_scan.address += PAGE_SIZE; } else put_page(*page); @@ -3449,6 +3501,28 @@ static ssize_t full_scans_show(struct kobject *kobj, } KSM_ATTR_RO(full_scans); +static ssize_t smart_scan_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + return sysfs_emit(buf, "%u\n", ksm_smart_scan); +} + +static ssize_t smart_scan_store(struct kobject *kobj, + struct kobj_attribute *attr, + const char *buf, size_t count) +{ + int err; + bool value; + + err = kstrtobool(buf, &value); + if (err) + return -EINVAL; + + ksm_smart_scan = value; + return count; +} +KSM_ATTR(smart_scan); + static struct attribute *ksm_attrs[] = { &sleep_millisecs_attr.attr, &pages_to_scan_attr.attr, @@ -3469,6 +3543,7 @@ static struct attribute *ksm_attrs[] = { &stable_node_chains_prune_millisecs_attr.attr, &use_zero_pages_attr.attr, &general_profit_attr.attr, + &smart_scan_attr.attr, NULL, }; From patchwork Tue Sep 12 17:52:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 138345 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9ecd:0:b0:3f2:4152:657d with SMTP id t13csp583380vqx; Tue, 12 Sep 2023 11:00:28 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHj+p3pGpkmZQd+lN3TnvMDuJLxfRUtcbpmo71dYhj56hhWxi2LiFriDkbmcqhIpkOu1k9N X-Received: by 2002:a05:6a00:4293:b0:68f:c4a2:fb1f with SMTP id bx19-20020a056a00429300b0068fc4a2fb1fmr3915839pfb.8.1694541627713; Tue, 12 Sep 2023 11:00:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694541627; cv=none; d=google.com; s=arc-20160816; b=kVdCj2dNRWGvsJZO32y2fsIOX9QuCsGxVsAX53oGiY07WK5m4Yr4oWiEkbdffKViyj LrppD0Vil5SNQKSEMTI3en5IeYrzSA7sXoRpLgkkvYUQf+3c6TvXOacHoPGPO5tL33U6 zg7hoTFjd/rP5bD9lMGSmTWoV+tnEpFqtIoO7kX5hRxbAQGEnzf2gwJYsSROVmMfutU6 E2Gb50KQfSBOeXcp/F9/ndQUU7YtBpICI7/EGYcd1V9AEOoP/DjYe3odvIkbjjdxdg4l 61DSC4E30xYN9eqoocrnc3y9Ik0chhqKuX0GWTFJol3Kji0jQZ/+B8XaeLvr5fcY7NM9 f5qg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=CN2i2K1yeQi3CVF3AoCmDBQsHD9chBIcu/eGaGP4nGc=; fh=sS+J4OyOC0EcVLWJpS3mBHGeO+0+dYZJ+ImCUfzsrH4=; b=HVujV3Iazp2j/zD9FIeGPkdM4FAuDwrUQYbU0LT6biee3HSRDuywjycRDxJ2IHFng8 uK2s+ZGqDfXFTaE+bpi95dC62GukL8EzMCFPkPEuwqi2ilSkQUW1ifQY8Pi/PyUf50WA 3vHJHfvPoZCJ4j6nNrBA31FaGzEVWwzpU55T+RkBMRxETFSfKsCykM9q11Paj5M5gYfd hb1Q+jYus0UnZkltgjTGio2+RCJLO2rrJ6IEGPGTmReQssDdBYMjZMzLpVrpu4mSwqhA JWYbAAj5rd3e/qw8rbcDNWWDbqcUqfty+dGv0kIOCSZ5mY7oKuPZxLv0M4esV4tHJnzp LrqA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from morse.vger.email (morse.vger.email. [2620:137:e000::3:1]) by mx.google.com with ESMTPS id g18-20020a056a000b9200b0068e38752b4csi8551503pfj.205.2023.09.12.11.00.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 11:00:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) client-ip=2620:137:e000::3:1; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id 52312801FBB6; Tue, 12 Sep 2023 10:53:54 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237317AbjILRxo (ORCPT + 36 others); Tue, 12 Sep 2023 13:53:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57132 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237384AbjILRxW (ORCPT ); Tue, 12 Sep 2023 13:53:22 -0400 Received: from 66-220-144-179.mail-mxout.facebook.com (66-220-144-179.mail-mxout.facebook.com [66.220.144.179]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A0C801FE9 for ; Tue, 12 Sep 2023 10:52:54 -0700 (PDT) Received: by devbig1114.prn1.facebook.com (Postfix, from userid 425415) id 28DACBCD162C; Tue, 12 Sep 2023 10:52:38 -0700 (PDT) From: Stefan Roesch To: kernel-team@fb.com Cc: shr@devkernel.io, akpm@linux-foundation.org, david@redhat.com, hannes@cmpxchg.org, riel@surriel.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v1 2/4] mm/ksm: add pages_skipped metric Date: Tue, 12 Sep 2023 10:52:26 -0700 Message-Id: <20230912175228.952039-3-shr@devkernel.io> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230912175228.952039-1-shr@devkernel.io> References: <20230912175228.952039-1-shr@devkernel.io> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Tue, 12 Sep 2023 10:53:54 -0700 (PDT) X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776855681660729364 X-GMAIL-MSGID: 1776855681660729364 This change adds the "pages skipped" metric. To be able to evaluate how successful smart page scanning is, the pages skipped metric can be compared to the pages scanned metric. The pages skipped metric is a cumulative counter. The counter is stored under /sys/kernel/mm/ksm/pages_skipped. Signed-off-by: Stefan Roesch Reviewed-by: David Hildenbrand --- mm/ksm.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/mm/ksm.c b/mm/ksm.c index bfd5087c7d5a..728574a3033e 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -293,6 +293,9 @@ static bool ksm_smart_scan; /* The number of zero pages which is placed by KSM */ unsigned long ksm_zero_pages; +/* The number of pages that have been skipped due to "smart scanning" */ +static unsigned long ksm_pages_skipped; + #ifdef CONFIG_NUMA /* Zeroed when merging across nodes is not allowed */ static unsigned int ksm_merge_across_nodes = 1; @@ -2345,6 +2348,7 @@ static bool skip_rmap_item(struct page *page, struct ksm_rmap_item *rmap_item) return false; } + ksm_pages_skipped++; if (rmap_item->skip_age == 0) { rmap_item->skip_age = age + inc_skip_age(age); remove_rmap_item_from_tree(rmap_item); @@ -3435,6 +3439,13 @@ static ssize_t pages_volatile_show(struct kobject *kobj, } KSM_ATTR_RO(pages_volatile); +static ssize_t pages_skipped_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + return sysfs_emit(buf, "%lu\n", ksm_pages_skipped); +} +KSM_ATTR_RO(pages_skipped); + static ssize_t ksm_zero_pages_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) { @@ -3532,6 +3543,7 @@ static struct attribute *ksm_attrs[] = { &pages_sharing_attr.attr, &pages_unshared_attr.attr, &pages_volatile_attr.attr, + &pages_skipped_attr.attr, &ksm_zero_pages_attr.attr, &full_scans_attr.attr, #ifdef CONFIG_NUMA From patchwork Tue Sep 12 17:52:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 138343 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9ecd:0:b0:3f2:4152:657d with SMTP id t13csp582056vqx; Tue, 12 Sep 2023 10:57:56 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGa839gWTOiYJnjfc5EjRjgKycHZCxWjm1SVP8KqDguoMOAZzpeB4DzeMHR5wZ8RgcPStXb X-Received: by 2002:a17:902:cec4:b0:1bb:ce4a:5893 with SMTP id d4-20020a170902cec400b001bbce4a5893mr587841plg.30.1694541476297; Tue, 12 Sep 2023 10:57:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694541476; cv=none; d=google.com; s=arc-20160816; b=v5fo3WMOf1ETlXHDyknsaKwuVsZGcMOfNxXYZ8zEQ6NB/OTLcdSJez1iczPfK9fpIa T2zDPpUsyXMovfhJYlL3QcLmcqfI+akyc/2bskLe+THSX4z+fL7M9N8Bu8mTuDYzh7OV GbYZhpiBggqR5arB+FjwjjnWHZzEokVlpykA/GHimuxDO3laa93YyhYL3x8RvZMErUjs lNGikonNK9hyLIvZLlTbtTYG/TMKn++pS5uQc/N+HSwlLhLGA9tbuhi7TjgqKpGa6BEi M68YXjaVRRRSn3PKXbvtHzSgxef4dhOU6/If1OEd/cMUhsn+lm+I8ZtMfm8pJMrqdN91 QTqQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=7hiLlIvC4E6ZG09k10WszVkbuStDhbkQ3XWN2o6TQ4s=; fh=sS+J4OyOC0EcVLWJpS3mBHGeO+0+dYZJ+ImCUfzsrH4=; b=kJJgVUD3+6cspddN9J1ehXiCt0a+lqQLK3G+9jb5fnuoBwlyZVqEDCWCPg8dHC3gw2 PcsOfGjT6oJZH460xYBdKUKH3kiDvZU15lBWmP3p4cKkw9KfkU1jUFGj17Sh3Eca9khE UG4r/wniqKxtyP5VPOoef+VXacxVqZFjcbRZn9tQKEgRvIYBeI5R/AFtQqPelVLno6YN vEiqIfR9hBD+Siw8CgrfR35EgiY27ynQ6vwA5r1rL5pUwuzNdOk+RNEHmABcHz4pmp5Z isfHgHqjhbdmLUTjZvhTiD0jHbV7Q2DIOYuL9rtY0ZTvRRjUhf7ZbY7CyPoRCx1XLKTb EGdw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from agentk.vger.email (agentk.vger.email. [23.128.96.32]) by mx.google.com with ESMTPS id y6-20020a170902ed4600b001bc836180dcsi1951864plb.520.2023.09.12.10.57.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 10:57:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id DB5128144188; Tue, 12 Sep 2023 10:53:49 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237306AbjILRxk (ORCPT + 36 others); Tue, 12 Sep 2023 13:53:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55984 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237253AbjILRxW (ORCPT ); Tue, 12 Sep 2023 13:53:22 -0400 Received: from 66-220-144-178.mail-mxout.facebook.com (66-220-144-178.mail-mxout.facebook.com [66.220.144.178]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9FD111FE7 for ; Tue, 12 Sep 2023 10:52:53 -0700 (PDT) Received: by devbig1114.prn1.facebook.com (Postfix, from userid 425415) id 467C8BCD162E; Tue, 12 Sep 2023 10:52:39 -0700 (PDT) From: Stefan Roesch To: kernel-team@fb.com Cc: shr@devkernel.io, akpm@linux-foundation.org, david@redhat.com, hannes@cmpxchg.org, riel@surriel.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v1 3/4] mm/ksm: document smart scan mode Date: Tue, 12 Sep 2023 10:52:27 -0700 Message-Id: <20230912175228.952039-4-shr@devkernel.io> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230912175228.952039-1-shr@devkernel.io> References: <20230912175228.952039-1-shr@devkernel.io> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Tue, 12 Sep 2023 10:53:49 -0700 (PDT) X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776855522594284888 X-GMAIL-MSGID: 1776855522594284888 This adds documentation for the smart scan mode of KSM. Signed-off-by: Stefan Roesch Reviewed-by: David Hildenbrand --- Documentation/admin-guide/mm/ksm.rst | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/Documentation/admin-guide/mm/ksm.rst b/Documentation/admin-guide/mm/ksm.rst index 776f244bdae4..1762219baf51 100644 --- a/Documentation/admin-guide/mm/ksm.rst +++ b/Documentation/admin-guide/mm/ksm.rst @@ -155,6 +155,15 @@ stable_node_chains_prune_millisecs scan. It's a noop if not a single KSM page hit the ``max_page_sharing`` yet. +smart_scan + By default KSM checks every candidate page for each scan. It does + not take into account historic information. When smart scan is + enabled, pages that have previously not been de-duplicated get + skipped. How often these pages are skipped depends on how often + de-duplication has already been tried and failed. By default this + optimization is disabled. The ``pages_skipped`` metric shows how + effetive the setting is. + The effectiveness of KSM and MADV_MERGEABLE is shown in ``/sys/kernel/mm/ksm/``: general_profit From patchwork Tue Sep 12 17:52:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 138348 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9ecd:0:b0:3f2:4152:657d with SMTP id t13csp584799vqx; Tue, 12 Sep 2023 11:02:01 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF5GT0VPsxcXzpf4IgPPYCOb/TA/nERacjumA8sKWaThgZV2F3O3buVZpghgIzrVOAXgQOT X-Received: by 2002:a05:6830:1e04:b0:6b2:9bdb:a84a with SMTP id s4-20020a0568301e0400b006b29bdba84amr597212otr.32.1694541721387; Tue, 12 Sep 2023 11:02:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694541721; cv=none; d=google.com; s=arc-20160816; b=I1kwbd8kOTRgzDoDhPSVf/b58VwJnQGtuWlYlQVOF3F3mDXRxp3HrZxpoKdnQwT02p I07/zzmiSpOuhFFlwwvOr0dDpL1ZUjCVpgH7wj4wOnnUTdUOIfkGtEsYPTkLVmWmUU3e wSrYz5KMZL8kAI8xCqiEmU19kOs1Ub8ALYsufc27VyxfAam5mHNJuwkLSgM0J6QYSvaZ JzaUmv+9/8pJU73jlas8OI01epKuxleFG9kiv/HFZmZhUBxvoJS8drgLhw/Lbl/wPxu6 n/tNVnFQwuezjnyqoVzWXbl34wQ7d3LQziPIohPwPGUou/wwern604DcapTlzk6Ka/3H 3Yhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=9xcbn4tr09Qqc62Dv3C/v1C43Yd0eirFtwweDMiZotE=; fh=sS+J4OyOC0EcVLWJpS3mBHGeO+0+dYZJ+ImCUfzsrH4=; b=Ggcnx2jgm3nSuYnLlad7/MrMb/6lLax1B2daAfOD3xVE9uz+M0d4QyWrcmcLgzrk6w LUtZU/GfjhE3gmpQNRiOf7t6NoAs+/k6MjcyHpb+ZO5pfbrTEo+Y5QOvPHrmG0m8nsTE uqTzE7Pp/pqi43Q9gg9O1obb4CkRR7G3OtjFog1GKuwmzpDJwPNqjkYbLCyigqynNgUZ mD+59po2FJLVXARAmCsS5URaoMmHdhuMuIlIIZ/z12lRmt6Vb5AaHoSFa5C/uhllgEWl hbszXWu3XRS+i0QErAMUm6cZ/qGTD28i5+KjJBo+DjDf3DDL8hLc0ejbS7WI+nN5Nta4 r/mA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id e1-20020a630f01000000b00573f8a53378si8153063pgl.58.2023.09.12.11.01.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 11:02:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id F3EF580CD73E; Tue, 12 Sep 2023 10:53:44 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237415AbjILRxj (ORCPT + 36 others); Tue, 12 Sep 2023 13:53:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36810 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237374AbjILRxW (ORCPT ); Tue, 12 Sep 2023 13:53:22 -0400 Received: from 66-220-144-178.mail-mxout.facebook.com (66-220-144-178.mail-mxout.facebook.com [66.220.144.178]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9FB851FE5 for ; Tue, 12 Sep 2023 10:52:53 -0700 (PDT) Received: by devbig1114.prn1.facebook.com (Postfix, from userid 425415) id 9C4ADBCD1635; Tue, 12 Sep 2023 10:52:39 -0700 (PDT) From: Stefan Roesch To: kernel-team@fb.com Cc: shr@devkernel.io, akpm@linux-foundation.org, david@redhat.com, hannes@cmpxchg.org, riel@surriel.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v1 4/4] mm/ksm: document pages_skipped sysfs knob Date: Tue, 12 Sep 2023 10:52:28 -0700 Message-Id: <20230912175228.952039-5-shr@devkernel.io> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230912175228.952039-1-shr@devkernel.io> References: <20230912175228.952039-1-shr@devkernel.io> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Tue, 12 Sep 2023 10:53:45 -0700 (PDT) X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776855780145743347 X-GMAIL-MSGID: 1776855780145743347 This adds documentation for the new metric pages_skipped. Signed-off-by: Stefan Roesch Reviewed-by: David Hildenbrand --- Documentation/admin-guide/mm/ksm.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Documentation/admin-guide/mm/ksm.rst b/Documentation/admin-guide/mm/ksm.rst index 1762219baf51..27d949250b67 100644 --- a/Documentation/admin-guide/mm/ksm.rst +++ b/Documentation/admin-guide/mm/ksm.rst @@ -178,6 +178,8 @@ pages_unshared how many pages unique but repeatedly checked for merging pages_volatile how many pages changing too fast to be placed in a tree +pages_skipped + how many pages did the "smart" page scanning algorithm skip full_scans how many times all mergeable areas have been scanned stable_node_chains