From patchwork Fri Sep 15 02:52:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 140045 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp765677vqi; Thu, 14 Sep 2023 19:53:57 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEePT11n87SMzIFInSVpuemqWFz+wRmCj5J0VIRzQmueHNfpQTKZwggtSd/ncX+5XRluKis X-Received: by 2002:a17:90a:9417:b0:26f:87d1:e48 with SMTP id r23-20020a17090a941700b0026f87d10e48mr4980166pjo.20.1694746436956; Thu, 14 Sep 2023 19:53:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694746436; cv=none; d=google.com; s=arc-20160816; b=vMBdhD6ftfaMjzaTcXWJh+dj/t0EEEE4mARFCkdIoryOJ9ghOoNx+hoHOUVPBPj86R yCr0ORao0buLL8y6u+vOmbccVdXJ6gMtWkk7GQD5VlV9WL9+ZW9WCxJBzGiLTi7innYO VRkPmHTHS8Qwv2mGkI3VggyZbcxUguSyJTz3Df9tq1w6GVTSssm4eid5SxwMiEXmEECT XTyK0aQeIEHt90rfkl0psBJfbEPQtcbu/LAJXtBYnCGBv9Vo1InwHUa34mA8kXfpwdPe pckGDfSDL6FdEL84b/e9ZAazyT8rcgE5XtiWJPTg8rFuG0J/zg3WGMWdamSzdbtP+8WO ZOqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=yas1krmMytP5jdv3DOjnLBNiWh5GBRJ7GxfCNOjPUeE=; fh=BzdeVYqZhG5iuwuKJRNLP969rvCput73lx0iwx2zu7A=; b=YW9CSdIaQ4FBfpktZILnInj6XBwjJxDdhQf7bKqe+rJqAS+Qk5Lrs5E88EMigkRQeF mpbcktc4ndiN5C1uTKDUJu/sPKgiKzLkoVzwihhQ8j3Li2ApK2/Yg5T31H2XdH4SoC9A lzUXtFjymNjpHVNcx0efgDX/g/qLieKVXKYCFtxRtXWaIb1KXUVSxXUo9RuJ76XjUx9F Rl336ruUCh3OcFU970KCGPmzI6SNElLyrY+ztyouF/BWqlNy9q+f3BeY5rj0kQVrT2f7 YYXbwlDPzR4/SnJbDIffd2mStV7W6444e7xxENRi+YaeOXE01ugQKKS3N/PaRgWY6j+w IdYQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="gKRZh/eQ"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id o6-20020a6548c6000000b005780a1effa4si1369156pgs.375.2023.09.14.19.53.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Sep 2023 19:53:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="gKRZh/eQ"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 0AB3383CF4B5; Thu, 14 Sep 2023 19:53:45 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231877AbjIOCx2 (ORCPT + 33 others); Thu, 14 Sep 2023 22:53:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34296 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231808AbjIOCxU (ORCPT ); Thu, 14 Sep 2023 22:53:20 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E139D270C for ; Thu, 14 Sep 2023 19:53:09 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 22D18C433CA; Fri, 15 Sep 2023 02:53:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1694746389; bh=Nt+EQkpN6g4pK9W8aFPRK0//Zoaw9gTVad64o5lt3eA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gKRZh/eQUciGBq4zsaSk5NtoLQsupXcxMJ/a50/LGY+nOoGWuUjC6y2orIQp3obtI kXAhaTSvmkPcLcIBo83U43YcT5/rR5+67cCeNTXQlVjlTdinObZwE9rv8twhhAoFQ+ hfODNrii+3froco6CXwiwvl/To4vC87JSvSxXDv/eJWIQZELjCY1rxBMVpzzcvLU93 LKH7YRnIV2jtSEIA2qyPFoj0BJ7KiovZ49aLLrfHP/SjF8OE2hm2oY/XWCe7owdqgw /nEekEFdyI15efQdNOybauZ+1SdgCXF10FOBqA+Hs0IMEHvkEUrNqT1fUkNKZoBVdj COaVPLwQxu3zw== From: SeongJae Park To: Andrew Morton Cc: SeongJae Park , damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 6/8] mm/damon/core: use pseudo-moving sum for nr_accesses_bp Date: Fri, 15 Sep 2023 02:52:49 +0000 Message-Id: <20230915025251.72816-7-sj@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230915025251.72816-1-sj@kernel.org> References: <20230915025251.72816-1-sj@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Thu, 14 Sep 2023 19:53:45 -0700 (PDT) X-Spam-Status: No, score=-1.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777070439772545876 X-GMAIL-MSGID: 1777070439772545876 Let nr_accesses_bp be calculated as a pseudo-moving sum that updated for every sampling interval, using damon_moving_sum(). This is assumed to be useful for cases that the aggregation interval is set quite huge, but the monivoting results need to be collected earlier than next aggregation interval is passed. Signed-off-by: SeongJae Park --- include/linux/damon.h | 12 +++++++++--- mm/damon/core.c | 16 +++++++++++++++- mm/damon/paddr.c | 9 +++++---- mm/damon/vaddr.c | 12 +++++++----- 4 files changed, 36 insertions(+), 13 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index 15f24b23c9a0..0fe13482df63 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -40,7 +40,8 @@ struct damon_addr_range { * @ar: The address range of the region. * @sampling_addr: Address of the sample for the next access check. * @nr_accesses: Access frequency of this region. - * @nr_accesses_bp: @nr_accesses in basis point (0.01%). + * @nr_accesses_bp: @nr_accesses in basis point (0.01%) that updated for + * each sampling interval. * @list: List head for siblings. * @age: Age of this region. * @@ -51,7 +52,11 @@ struct damon_addr_range { * damon_update_region_access_rate(). * * @nr_accesses_bp is another representation of @nr_accesses in basis point - * (1 in 10,000) that updated every aggregation interval. + * (1 in 10,000) that updated for every &damon_attrs->sample_interval in a + * manner similar to moving sum. By the algorithm, this value becomes + * @nr_accesses * 10000 for every &struct damon_attrs->aggr_interval. This can + * be used when the aggregation interval is too huge and therefore cannot wait + * for it before getting the access monitoring results. * * @age is initially zero, increased for each aggregation interval, and reset * to zero again if the access frequency is significantly changed. If two @@ -629,7 +634,8 @@ int damon_set_regions(struct damon_target *t, struct damon_addr_range *ranges, unsigned int nr_ranges); unsigned int damon_moving_sum(unsigned int mvsum, unsigned int nomvsum, unsigned int len_window, unsigned int new_value); -void damon_update_region_access_rate(struct damon_region *r, bool accessed); +void damon_update_region_access_rate(struct damon_region *r, bool accessed, + struct damon_attrs *attrs); struct damos_filter *damos_new_filter(enum damos_filter_type type, bool matching); diff --git a/mm/damon/core.c b/mm/damon/core.c index ce85c00b0a4c..29ee1fc18393 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -1599,14 +1599,28 @@ unsigned int damon_moving_sum(unsigned int mvsum, unsigned int nomvsum, * damon_update_region_access_rate() - Update the access rate of a region. * @r: The DAMON region to update for its access check result. * @accessed: Whether the region has accessed during last sampling interval. + * @attrs: The damon_attrs of the DAMON context. * * Update the access rate of a region with the region's last sampling interval * access check result. * * Usually this will be called by &damon_operations->check_accesses callback. */ -void damon_update_region_access_rate(struct damon_region *r, bool accessed) +void damon_update_region_access_rate(struct damon_region *r, bool accessed, + struct damon_attrs *attrs) { + unsigned int len_window = 1; + + /* + * sample_interval can be zero, but cannot be larger than + * aggr_interval, owing to validation of damon_set_attrs(). + */ + if (attrs->sample_interval) + len_window = attrs->aggr_interval / attrs->sample_interval; + r->nr_accesses_bp = damon_moving_sum(r->nr_accesses_bp, + r->last_nr_accesses * 10000, len_window, + accessed ? 10000 : 0); + if (accessed) r->nr_accesses++; } diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 44f21860b555..081e2a325778 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -148,7 +148,8 @@ static bool damon_pa_young(unsigned long paddr, unsigned long *folio_sz) return accessed; } -static void __damon_pa_check_access(struct damon_region *r) +static void __damon_pa_check_access(struct damon_region *r, + struct damon_attrs *attrs) { static unsigned long last_addr; static unsigned long last_folio_sz = PAGE_SIZE; @@ -157,12 +158,12 @@ static void __damon_pa_check_access(struct damon_region *r) /* If the region is in the last checked page, reuse the result */ if (ALIGN_DOWN(last_addr, last_folio_sz) == ALIGN_DOWN(r->sampling_addr, last_folio_sz)) { - damon_update_region_access_rate(r, last_accessed); + damon_update_region_access_rate(r, last_accessed, attrs); return; } last_accessed = damon_pa_young(r->sampling_addr, &last_folio_sz); - damon_update_region_access_rate(r, last_accessed); + damon_update_region_access_rate(r, last_accessed, attrs); last_addr = r->sampling_addr; } @@ -175,7 +176,7 @@ static unsigned int damon_pa_check_accesses(struct damon_ctx *ctx) damon_for_each_target(t, ctx) { damon_for_each_region(r, t) { - __damon_pa_check_access(r); + __damon_pa_check_access(r, &ctx->attrs); max_nr_accesses = max(r->nr_accesses, max_nr_accesses); } } diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index e36303271f9d..af2cb82e1fad 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -557,26 +557,27 @@ static bool damon_va_young(struct mm_struct *mm, unsigned long addr, * r the region to be checked */ static void __damon_va_check_access(struct mm_struct *mm, - struct damon_region *r, bool same_target) + struct damon_region *r, bool same_target, + struct damon_attrs *attrs) { static unsigned long last_addr; static unsigned long last_folio_sz = PAGE_SIZE; static bool last_accessed; if (!mm) { - damon_update_region_access_rate(r, false); + damon_update_region_access_rate(r, false, attrs); return; } /* If the region is in the last checked page, reuse the result */ if (same_target && (ALIGN_DOWN(last_addr, last_folio_sz) == ALIGN_DOWN(r->sampling_addr, last_folio_sz))) { - damon_update_region_access_rate(r, last_accessed); + damon_update_region_access_rate(r, last_accessed, attrs); return; } last_accessed = damon_va_young(mm, r->sampling_addr, &last_folio_sz); - damon_update_region_access_rate(r, last_accessed); + damon_update_region_access_rate(r, last_accessed, attrs); last_addr = r->sampling_addr; } @@ -593,7 +594,8 @@ static unsigned int damon_va_check_accesses(struct damon_ctx *ctx) mm = damon_get_mm(t); same_target = false; damon_for_each_region(r, t) { - __damon_va_check_access(mm, r, same_target); + __damon_va_check_access(mm, r, same_target, + &ctx->attrs); max_nr_accesses = max(r->nr_accesses, max_nr_accesses); same_target = true; }