From patchwork Mon Feb 26 14:05:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 206661 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:a81b:b0:108:e6aa:91d0 with SMTP id bq27csp2099661dyb; Mon, 26 Feb 2024 06:09:57 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCX5xcXX9RzSbSKn7zbh0vDCBnhVvtQD+Wge95+YhiA++LuMy7pvhVO6UYBfe9qzbR4XhjzNbrnPVoraN9GUH0LWMlIhjQ== X-Google-Smtp-Source: AGHT+IEVmXVQfHYaIhLpRhQV3CcdprJ3KSBE9JFqJGiFVr84csONChe0Qf0T/50BFvkvDUwT9WpI X-Received: by 2002:a05:6830:908:b0:6e2:dc7f:f708 with SMTP id v8-20020a056830090800b006e2dc7ff708mr8637547ott.13.1708956597477; Mon, 26 Feb 2024 06:09:57 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708956597; cv=pass; d=google.com; s=arc-20160816; b=XpFqfE5nV4qEceOu9HvNebzehyy3OYpLjsB8Q4NzJuc+R+d9Ad37nrBlgjsJ0942X9 kLvI2hXAJRha8LaabVzM5kZPxvJqrG7GSlx6dAlamirstRJfkgFneAWROE/43fws3LFC yxDDL/X1m6w1ItZnDSKbA/4QX5mkcS1IoTj7u5NddUwx88CIcygZT39HUPnG7Mh24KGH opdpZe8cSLcUfHDofQPXErdbHvgVigZs59JedgDS2R24DVR99KKVltqkKxRS02Bqb7DP 0ntCBFAusqbBbMTEV2+qNDPiQu+0qZx18zxEov8/OJXqFsQuzfxZNvtc9UMAIOQgq3qM Z0Cw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=Y9OC4YXYFjc+2FMc0Wyr+afdOtmeZZzGlGXJ/Mb/OPk=; fh=egwJU2bb679B4Zi0WdxyKykgfACRkSkUGxyHY52POxM=; b=A6k+naiPQxvX4uXq2pUBjBqm5tbGGSyNdk8YZ9BlJegdxAHUPF1bCP7MFxWZveyoR3 PmPbF5RKGVtXdHt2CqEH1DvTYvZ0SEIoENqkelQ3LtJHHyxWbXsYB9QA/iZO/MxP3vre BUcFPfcp8X9Abb3qp+U/QsaQ+OxGNNek5eg9jt26c+IA48fX3PTATEv3p4KcH6J5rdwI kjga60Ow4kZHq2sNMKIkLbIPL4JI3ygHSwnGSXqmFxcL423+5tyYBhdJGgeEXnBgzu7/ vsGjtHaCP3nINg6P7BI5Hfp3yRPg7SUPYwHHrxNHOSL0oOYH7hK7tjsueRJ9o0PAhAES KxqA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=sk.com); spf=pass (google.com: domain of linux-kernel+bounces-81579-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-81579-ouuuleilei=gmail.com@vger.kernel.org" Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id hv3-20020a67e683000000b0046d198a1c80si599277vsb.445.2024.02.26.06.09.57 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Feb 2024 06:09:57 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-81579-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=sk.com); spf=pass (google.com: domain of linux-kernel+bounces-81579-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-81579-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 3C4AB1C23CBE for ; Mon, 26 Feb 2024 14:09:57 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4202012C545; Mon, 26 Feb 2024 14:06:21 +0000 (UTC) Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 583CA12A14A; Mon, 26 Feb 2024 14:06:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708956379; cv=none; b=IUGh9nU5qUeOmLHsYRjPdAE01R6O4/PP3nOehDM+/3gj7jLt2ItN+eOQKndGER6eFeHW77N/EMngRlTW84Wau9JX9fPzRErHErkUm4GHxb9x53pvwxBGdpb6Y8nyfXHXYQgI6f1QJih1vXRoDRx5qeBOWbMP0u0a/lExEybsD3U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708956379; c=relaxed/simple; bh=Ig1CU3NKL8/cdJN+rmc1e4GaDBcrnBkfyq6QFF/1g7U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=G8/0gnVdoWeg0h9p1ISwsgvd9dgak9cN0Xb2W9lgxjTbt+TcNWFktgr2tp1p4g6i1Kxj1DX13BD/VvODzQH+CkE4euCmUkniVa8zK0P6eYkyuYqmb5tTXYgSoSAbGATefGajivRjKeZKmvWPzH5D0DBIWFf+AwhIJkPnAR1mDIk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d6dff70000001748-5d-65dc9ad5f17d From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, lizhijian@cn.fujitsu.com, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com Subject: [PATCH v2 1/7] mm/damon: refactor DAMOS_PAGEOUT with migration_mode Date: Mon, 26 Feb 2024 23:05:47 +0900 Message-ID: <20240226140555.1615-2-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240226140555.1615-1-honggyu.kim@sk.com> References: <20240226140555.1615-1-honggyu.kim@sk.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrOIsWRmVeSWpSXmKPExsXC9ZZnoe61WXdSDRb+ZbSY2GNgMWf9GjaL XTdCLP7vPcZo8eT/b1aLEzcb2Sw6vy9lsbi8aw6bxb01/1ktjqw/y2Kx7haQtfnsGWaLxcvV LPZ1PGCyOPz1DZPF5EsL2CxeTDnDaHFy1mQWi9lH77E7CHv8PziJ2WPp6TdsHhuagMTOWXfZ PVr23WL3WLCp1KPlyFtWj8V7XjJ5bFrVyeax6dMkdo8TM36zeOx8aOnxYvNMRo/e5ndsHp83 yQXwR3HZpKTmZJalFunbJXBlzGnYwFzQwlsx78catgbGK1xdjJwcEgImEjfmPmXqYuQAsxdv 1QIJswmoSVx5OQksLCLgILHqq0IXIxcHs0ADi8SPz5dYQGqEBXwk1v67wQ5iswioSvzaNB8s zitgJvHoWC8jxHhNicfbf4LVcAqYS/w/3M4GMlMIqObobWaIckGJkzOfgLUyC8hLNG+dzQyy S0LgFLvE3yvrWSHmSEocXHGDZQIj/ywkPbOQ9CxgZFrFKJSZV5abmJljopdRmZdZoZecn7uJ ERh9y2r/RO9g/HQh+BCjAAejEg+vQ+2dVCHWxLLiytxDjBIczEoivOEyN1OFeFMSK6tSi/Lj i0pzUosPMUpzsCiJ8xp9K08REkhPLEnNTk0tSC2CyTJxcEo1MDK+N4yt27RA9+y/ac7bG3rd Dz+yUDrannj3QV6ted70+a3f/ZKifvgU12UsuB/YVTyxslSkL/lnqMwXP1sPWSGeB3JTFtmG 3z7qLOWYNm9R24NJbss36+yscjzc+K1eUO+lQNXsWZ2JgQd3xe8zfeywek10WvC1nbd1tSsZ rr5e/Ffj2KdFS5RYijMSDbWYi4oTAbUWVdi6AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprLIsWRmVeSWpSXmKPExsXCNUNLT/fqrDupBo0HdSwm9hhYzFm/hs1i 140Qi/97jzFaPPn/m9XixM1GNovPz14zW3Q++c5ocXjuSVaLzu9LWSwu75rDZnFvzX9WiyPr z7JYrLsFZG0+e4bZYvFyNYtD156zWuzreMBkcfjrGyaLyZcWsFm8mHKG0eLkrMksFrOP3mN3 EPf4f3ASs8fS02/YPDY0AYmds+6ye7Tsu8XusWBTqUfLkbesHov3vGTy2LSqk81j06dJ7B4n Zvxm8dj50NLjxeaZjB69ze/YPL7d9vBY/OIDU4BgFJdNSmpOZllqkb5dAlfGnIYNzAUtvBXz fqxha2C8wtXFyMEhIWAisXirVhcjJwebgJrElZeTmEDCIgIOEqu+KnQxcnEwCzSwSPz4fIkF pEZYwEdi7b8b7CA2i4CqxK9N88HivAJmEo+O9TKC2BICmhKPt/8Eq+EUMJf4f7idDWSmEFDN 0dvMEOWCEidnPgFrZRaQl2jeOpt5AiPPLCSpWUhSCxiZVjGKZOaV5SZm5pjqFWdnVOZlVugl 5+duYgRG2rLaPxN3MH657H6IUYCDUYmH16H2TqoQa2JZcWXuIUYJDmYlEd5wmZupQrwpiZVV qUX58UWlOanFhxilOViUxHm9wlMThATSE0tSs1NTC1KLYLJMHJxSDYxhHAuf6dzr6NtyNJy7 13qtTeBhUWmln4dEykuTjp9kPrPYvdXukZi588M8jvz/Okuuvs5e+U/k4+zt36xcRG4WSjyW j2Gbe+7GhJ35d3/qb7q6TXVr2Mlrj62Yb2bkL4n3/jzrYsN99qW/1PO3W39Rl/abUabYKMh2 /GZw+UrX89m3ys78F+1XYinOSDTUYi4qTgQATejUCbACAAA= X-CFilter-Loop: Reflected X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791970873231004283 X-GMAIL-MSGID: 1791970873231004283 This is a preparation patch that introduces migration modes. The damon_pa_pageout is renamed to damon_pa_migrate and it receives an extra argument for migration_mode. No functional changes applied. Signed-off-by: Honggyu Kim --- mm/damon/paddr.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 081e2a325778..277a1c4d833c 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -224,7 +224,12 @@ static bool damos_pa_filter_out(struct damos *scheme, struct folio *folio) return false; } -static unsigned long damon_pa_pageout(struct damon_region *r, struct damos *s) +enum migration_mode { + MIG_PAGEOUT, +}; + +static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, + enum migration_mode mm) { unsigned long addr, applied; LIST_HEAD(folio_list); @@ -249,7 +254,14 @@ static unsigned long damon_pa_pageout(struct damon_region *r, struct damos *s) put_folio: folio_put(folio); } - applied = reclaim_pages(&folio_list); + switch (mm) { + case MIG_PAGEOUT: + applied = reclaim_pages(&folio_list); + break; + default: + /* Unexpected migration mode. */ + return 0; + } cond_resched(); return applied * PAGE_SIZE; } @@ -297,7 +309,7 @@ static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, { switch (scheme->action) { case DAMOS_PAGEOUT: - return damon_pa_pageout(r, scheme); + return damon_pa_migrate(r, scheme, MIG_PAGEOUT); case DAMOS_LRU_PRIO: return damon_pa_mark_accessed(r, scheme); case DAMOS_LRU_DEPRIO: From patchwork Mon Feb 26 14:05:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 206686 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:a81b:b0:108:e6aa:91d0 with SMTP id bq27csp2115764dyb; Mon, 26 Feb 2024 06:34:43 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXohoUdknWrGZfA9NtVlXpQ30YcSNiRmc0oKtKKZsEuPtyjz7M5PhiGzGfLeBPbc1AyAgLPDtswLWX+bGa5+rs0xqIoYQ== X-Google-Smtp-Source: AGHT+IGk8AlaxpoMuoKKzxNqfoKW8T445KcYtOERrKXp26t5VXngeCLDOqq9KSILHZmtXsE/cM7F X-Received: by 2002:a05:6a20:c78d:b0:1a0:fde5:6bd9 with SMTP id hk13-20020a056a20c78d00b001a0fde56bd9mr4455776pzb.44.1708958083028; Mon, 26 Feb 2024 06:34:43 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708958083; cv=pass; d=google.com; s=arc-20160816; b=TeFtD0u0vDaqoAUx3WP2QUJFcW7F+4a7jb211URHTGSm2B1IaCpk/eMRf/ydF2UuDy o+vlDiawx0m7LpRrAIVCaye3VrvtuCNoxDNQtzX0sTDWhip3smPLoxj6/M768fZZegOx ziKRseU21P16WdvvP4oxR13YPOjIUkN6abDSp8B4Muk2GY34uT4ce7BMQDdWZORDu7sK gPBCrEX4F3sgsqXAlQf4JYvstCQ7dZ45CBXwRjKn3/263Zzlczc1z3xnQZa3KQs8u3Zy P06jXsH8ygBEnAp3xpBkUvvTugkO5Kgkh3zkk0ZdHJALw0xSHPA4SbblL6GqIztQEMmr 6yqg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=/O1IDxVh05t8M0nebcpVtrx+/Qgao8Jaq6x7QW3BvpY=; fh=egwJU2bb679B4Zi0WdxyKykgfACRkSkUGxyHY52POxM=; b=ZAj7JgTYw7BS988p77W72FPkmu9O9FtCDKvgBETTIrfaZhohGCqL0zYNKv0/L3AQFt NanEMrkdRzQcW2xRQIKtTp4gIZWBBM0NgMxrZIhNzTczvKyLRpTWyS816T3tB9+m1sJa qZP7pkYq+GwNWQQ8CkaWXT8xiU5EXT2j6UU2dQdB5nzXZfhKzfBEy60MVRYffzYX12S0 Fx6gQNRTmewSMok+ovfsBpli/8ZikS2Bp0qHPS73UCMNhhPN1Yge8tWSOqxrsHbfJOpa +GNf+jWg68DNDT8xgxAxmXwcAOHaHvUOy6cDbo2vUy+fAa+UkoTC4ApatwkskBjhbuHz KDXg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=sk.com); spf=pass (google.com: domain of linux-kernel+bounces-81582-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-81582-ouuuleilei=gmail.com@vger.kernel.org" Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id 27-20020a630f5b000000b005d8e12c85d9si3817352pgp.738.2024.02.26.06.34.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Feb 2024 06:34:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-81582-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=sk.com); spf=pass (google.com: domain of linux-kernel+bounces-81582-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-81582-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 93FB0B271CA for ; Mon, 26 Feb 2024 14:10:55 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6FD7212D75C; Mon, 26 Feb 2024 14:06:25 +0000 (UTC) Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D3D4412BF3C; Mon, 26 Feb 2024 14:06:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708956383; cv=none; b=HHXiI0j3QpZImVnrrAvN1e9OBNOgJsPWW8hz6N1WzbEPg5zwa1y6INO9yLb1iYxVznwzr2nCxKeWeq/2YQ23NYfhrlTMZRVdvRbWUTV41Ds8MRXIU52b7UlaHUF3LHGQJQ/s0sBE9nvTh4RP3DdK72nh7FkxyjPAMj8ZakAOhaM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708956383; c=relaxed/simple; bh=C468lHYjYAI9vBRck8tz19UeLn3N7jq8un3nPfdUOcw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=X9b+rOJlfvJtfKp0RIYUtmgyeS0HHWwiOcvltFCtIDlEdle9wcbHTXOm4TX8txbgL5/hFMiwhL7O6122PA5YetrC0TRAYN3+xnqWrsacwTNBPuU8jsaCD6xRVPbbWhgns4wo84/cyJMjW4LGYrqMibCq69JHazaYDcqmwOP/jh4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d6dff70000001748-65-65dc9ad7da0c From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, lizhijian@cn.fujitsu.com, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com Subject: [PATCH v2 2/7] mm: make alloc_demote_folio externally invokable for migration Date: Mon, 26 Feb 2024 23:05:48 +0900 Message-ID: <20240226140555.1615-3-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240226140555.1615-1-honggyu.kim@sk.com> References: <20240226140555.1615-1-honggyu.kim@sk.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrOIsWRmVeSWpSXmKPExsXC9ZZnoe71WXdSDU4sELaY2GNgMWf9GjaL XTdCLP7vPcZo8eT/b1aLEzcb2Sw6vy9lsbi8aw6bxb01/1ktjqw/y2Kx7haQtfnsGWaLxcvV LPZ1PGCyOPz1DZPF5EsL2CxeTDnDaHFy1mQWi9lH77E7CHv8PziJ2WPp6TdsHhuagMTOWXfZ PVr23WL3WLCp1KPlyFtWj8V7XjJ5bFrVyeax6dMkdo8TM36zeOx8aOnxYvNMRo/e5ndsHp83 yQXwR3HZpKTmZJalFunbJXBlbHjym73gkGDFtMmn2RsYN/N1MXJySAiYSByY2cnexcgBZl+f JwkSZhNQk7jychITSFhEwEFi1VeFLkYuDmaBBhaJH58vsYDUCAuESdxasBPMZhFQldix4ymY zStgJrF19jZWiPGaEo+3/2QHsTkFzCX+H25nA5kpBFRz9DYzRLmgxMmZT8BamQXkJZq3zmYG 2SUhcIpd4mPLUyaIOZISB1fcYJnAyD8LSc8sJD0LGJlWMQpl5pXlJmbmmOhlVOZlVugl5+du YgRG37LaP9E7GD9dCD7EKMDBqMTD61B7J1WINbGsuDL3EKMEB7OSCG+4zM1UId6UxMqq1KL8 +KLSnNTiQ4zSHCxK4rxG38pThATSE0tSs1NTC1KLYLJMHJxSDYwKC0Iq+iRk+wtOMx6fcP/m 4uKgw+oTP809kbxG4lzdzPaa+Z/lGrYcblm3/HBZ/dRHKVx/8r7tlPv5+yDXAkaGO+s2uXu7 7E1r3zVP6se5c6cVjD2inrp94nrs81HnV0+kW95hG8l3GVY6lhfdF4ao5NYo9RcaZB9vm7p9 RYu/UdcW1ZCPf94rsRRnJBpqMRcVJwIABBv4HboCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprLIsWRmVeSWpSXmKPExsXCNUNLT/f6rDupBgfPMltM7DGwmLN+DZvF rhshFv/3HmO0ePL/N6vFiZuNbBafn71mtuh88p3R4vDck6wWnd+Xslhc3jWHzeLemv+sFkfW n2WxWHcLyNp89gyzxeLlahaHrj1ntdjX8YDJ4vDXN0wWky8tYLN4MeUMo8XJWZNZLGYfvcfu IO7x/+AkZo+lp9+weWxoAhI7Z91l92jZd4vdY8GmUo+WI29ZPRbvecnksWlVJ5vHpk+T2D1O zPjN4rHzoaXHi80zGT16m9+xeXy77eGx+MUHpgDBKC6blNSczLLUIn27BK6MDU9+sxccEqyY Nvk0ewPjZr4uRg4OCQETievzJLsYOTnYBNQkrrycxAQSFhFwkFj1VaGLkYuDWaCBReLH50ss IDXCAmEStxbsBLNZBFQldux4CmbzCphJbJ29jRXElhDQlHi8/Sc7iM0pYC7x/3A7G8hMIaCa o7eZIcoFJU7OfALWyiwgL9G8dTbzBEaeWUhSs5CkFjAyrWIUycwry03MzDHVK87OqMzLrNBL zs/dxAiMtGW1fybuYPxy2f0QowAHoxIPr0PtnVQh1sSy4srcQ4wSHMxKIrzhMjdThXhTEiur Uovy44tKc1KLDzFKc7AoifN6hacmCAmkJ5akZqemFqQWwWSZODilGhjZNkyuCLp5K9RnIVvv BK/jzJd2fv5YuFJ0kqB47etlR0SNBHy9s7PU5Z+dnLlbueDFzfiM1A/Mc9tWv88t3V9o0+/j FhMptMBwh0blCYtZ65uz/z2Kaj2276LO18eTpxTeaWu8y78lz9Tn4b8TDNJ/9yxo+HR+S7zC pudtX1USPNaWlTpyTvuuxFKckWioxVxUnAgAPK1J3bACAAA= X-CFilter-Loop: Reflected X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791972430887255178 X-GMAIL-MSGID: 1791972430887255178 The alloc_demote_folio can be used out of vmscan.c so it'd be better to remove static keyword from it. This function can also be used for both demotion and promotion so it'd be better to rename it from alloc_demote_folio to alloc_migrate_folio. Signed-off-by: Honggyu Kim --- mm/internal.h | 1 + mm/vmscan.c | 10 +++++++--- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index b61034bd50f5..61af6641235d 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -866,6 +866,7 @@ extern unsigned long __must_check vm_mmap_pgoff(struct file *, unsigned long, unsigned long, unsigned long); extern void set_pageblock_order(void); +struct folio *alloc_migrate_folio(struct folio *src, unsigned long private); unsigned long reclaim_pages(struct list_head *folio_list); unsigned int reclaim_clean_pages_from_list(struct zone *zone, struct list_head *folio_list); diff --git a/mm/vmscan.c b/mm/vmscan.c index bba207f41b14..b8a1a1599e3c 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -910,8 +910,7 @@ static void folio_check_dirty_writeback(struct folio *folio, mapping->a_ops->is_dirty_writeback(folio, dirty, writeback); } -static struct folio *alloc_demote_folio(struct folio *src, - unsigned long private) +struct folio *alloc_migrate_folio(struct folio *src, unsigned long private) { struct folio *dst; nodemask_t *allowed_mask; @@ -935,6 +934,11 @@ static struct folio *alloc_demote_folio(struct folio *src, if (dst) return dst; + /* + * Allocation failed from the target node so try to allocate from + * fallback nodes based on allowed_mask. + * See fallback_alloc() at mm/slab.c. + */ mtc->gfp_mask &= ~__GFP_THISNODE; mtc->nmask = allowed_mask; @@ -973,7 +977,7 @@ static unsigned int demote_folio_list(struct list_head *demote_folios, node_get_allowed_targets(pgdat, &allowed_mask); /* Demotion ignores all cpuset and mempolicy settings */ - migrate_pages(demote_folios, alloc_demote_folio, NULL, + migrate_pages(demote_folios, alloc_migrate_folio, NULL, (unsigned long)&mtc, MIGRATE_ASYNC, MR_DEMOTION, &nr_succeeded); From patchwork Mon Feb 26 14:05:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 206670 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:a81b:b0:108:e6aa:91d0 with SMTP id bq27csp2104027dyb; Mon, 26 Feb 2024 06:16:02 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCX1yzONULk/3BxFBhahrZyigcZ/4G5yYs4/Zr58cyjSbi7ewUe2ppJFN8sxH562sgdwmfHwpdgmQ5K+Yaxmt7SDshF/DQ== X-Google-Smtp-Source: AGHT+IGC/Td+OV3KdUqLA/Kpib7WenWsICLN/40jzlDQv8GgIa3+ZJ/9hju5Y8cwy0XhTbboGXJj X-Received: by 2002:a05:6358:93a1:b0:17b:8afe:2d9c with SMTP id h33-20020a05635893a100b0017b8afe2d9cmr4393754rwb.18.1708956962310; Mon, 26 Feb 2024 06:16:02 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708956962; cv=pass; d=google.com; s=arc-20160816; b=VQP8cPnOYiYO3hX4aEz8Pl/ecfmyR97aXoZjUWya7NODKsnfq1qBi8z9uIwP34j/cX f60RtXfBm2nMSq158LZqSSXxVu9kumCQDiaY6NkFPSGVJvY3SiifbNBRb24Wgp0Q20F4 I0vuGreCaYFmdm/rgPEagf4Ymv64oj9pEfSlCkuM+G2yvQC/lixcxTnJbbIW2rnqnJ3n VpsIZaaHFqadDpP5KbNk8PP8us0j6mAVH/MTSi4uDys9qGlJhnb8LACnEKpAzA98xrht mQvvAqpOqu26MxvY4XeueBUXw3RLtJOcl26dfj/uijREEOTgwfNj025cqox8up+i2T4x 33yg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=IhyZjw+jpLJUwFy4dEx4+1Sb+mkQtPd245NYWtlHCwU=; fh=egwJU2bb679B4Zi0WdxyKykgfACRkSkUGxyHY52POxM=; b=nBQOO4EQuXFoEoWJlbuJdki3+OlkkHv2eaO2RSU9aEtiYjBs76+5XfInzLQUmbaTo0 T+zDDOTFRebpqeaFv8UPpCeBdGkwNAY9uqKhA3KxkXJG68UDs4iP/gndPsl1Y6a+BWSh 7YsXtfQnWAtbxN+EcekyN4E3UYvkazD4uRikFNeMWPEyKquq6QWE30379VHGqAcCgScF qlaHsyo0sdboxwyKxboVLZh5SA8XhRKOgOXHilqwFdPWQsME+vrftVYXnzTmGV1mtVNi Ff+oN77RuR4OWuPaQ8BkS4xVAXl6ayu7GY9fFACoRskSJ9VCjn9rnDYvUjE9TjUTPh4s sNWQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=sk.com); spf=pass (google.com: domain of linux-kernel+bounces-81583-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-81583-ouuuleilei=gmail.com@vger.kernel.org" Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id p37-20020a631e65000000b005dc7e779b7dsi3716095pgm.73.2024.02.26.06.16.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Feb 2024 06:16:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-81583-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=sk.com); spf=pass (google.com: domain of linux-kernel+bounces-81583-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-81583-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id A7D2C28932A for ; Mon, 26 Feb 2024 14:11:16 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1BAD712E1DC; Mon, 26 Feb 2024 14:06:27 +0000 (UTC) Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5EA7E12C525; Mon, 26 Feb 2024 14:06:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708956383; cv=none; b=QPmRRahs7k8CN6PM67SwTS40uPjWAA1QtDr1W+OKSS6BeFxSLd1K815tzY1UW+aphLrkxiT8QmPCHSl4f74aBxCP9dFK6hefwCYuCF8wo/JHJiIii26KZSubMjlFtH3IfXSx5Rol2CgS4amQlK2jqQsNIU2sBG+MrQJzF3QMKlw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708956383; c=relaxed/simple; bh=k6DbcpBVXMmCiXmnmNrP0yZrXB+ni5erIDbzS0ImpJ0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bpbDZHOdOWdoPIHUFOcQ4gEM9FNtn0Xsq+a9+sjqxMrwGPbjPi7EHtR0mQpMJzQYNFw/GyGYkVdERy10nLtDkQbziq933p4O1C7AC8igLVOjR8N8lMoyR5WlFL4j1naCA68aPKUkXxm9WkmH2fitrmyGSAQFnpe12Rq9hZb4vss= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d6dff70000001748-6b-65dc9ad87a7f From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, lizhijian@cn.fujitsu.com, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com Subject: [PATCH v2 3/7] mm/damon: introduce DAMOS_DEMOTE action for demotion Date: Mon, 26 Feb 2024 23:05:49 +0900 Message-ID: <20240226140555.1615-4-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240226140555.1615-1-honggyu.kim@sk.com> References: <20240226140555.1615-1-honggyu.kim@sk.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrBIsWRmVeSWpSXmKPExsXC9ZZnoe6NWXdSDebt0LaY2GNgMWf9GjaL XTdCLP7vPcZo8eT/b1aLEzcb2Sw6vy9lsbi8aw6bxb01/1ktjqw/y2Kx7haQtfnsGWaLxcvV LPZ1PGCyOPz1DZPF5EsL2CxeTDnDaHFy1mQWi9lH77E7CHv8PziJ2WPp6TdsHhuagMTOWXfZ PVr23WL3WLCp1KPlyFtWj8V7XjJ5bFrVyeax6dMkdo8TM36zeOx8aOnxYvNMRo/e5ndsHp83 yQXwR3HZpKTmZJalFunbJXBlvDqzlrlgtU/FtxdbmBsYb9l2MXJySAiYSDy53M4IY8+dcZYN xGYTUJO48nISUxcjB4eIgIPEqq8KXYxcHMwCDSwSPz5fYgGpERbwkeiceYYJxGYRUJW4N/sX M4jNK2Am8afrFjPETE2Jx9t/soPYnALmEv8Pt7OBzBQCqjl6G6pcUOLkzCdgI5kF5CWat85m BtklIXCKXeL/hLOsEHMkJQ6uuMEygZF/FpKeWUh6FjAyrWIUyswry03MzDHRy6jMy6zQS87P 3cQIjL9ltX+idzB+uhB8iFGAg1GJh9eh9k6qEGtiWXFl7iFGCQ5mJRHecJmbqUK8KYmVValF +fFFpTmpxYcYpTlYlMR5jb6VpwgJpCeWpGanphakFsFkmTg4pRoYF79myP4s1c6u8d//s72H 10GOnSfll05a0zc/NefqohmNm5yW8SvZx17mdy5d4DTzk9Mm1nf+tR3xBXFvqrOi3OYKmW9s DQ7f3iNuFle+aPHT9N9+XY8vW2WaL/0S91pgxwcGa/m47XcLrm1R5ew6+rVnd9v/oN37gnb7 mb53yFpdrqO0fw67EktxRqKhFnNRcSIAYaWl7rsCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprHIsWRmVeSWpSXmKPExsXCNUNLT/fGrDupBj2n5C0m9hhYzFm/hs1i 140Qi/97jzFaPPn/m9XixM1GNovPz14zW3Q++c5ocXjuSVaLzu9LWSwu75rDZnFvzX9WiyPr z7JYrLsFZG0+e4bZYvFyNYtD156zWuzreMBkcfjrGyaLyZcWsFm8mHKG0eLkrMksFrOP3mN3 EPf4f3ASs8fS02/YPDY0AYmds+6ye7Tsu8XusWBTqUfLkbesHov3vGTy2LSqk81j06dJ7B4n Zvxm8dj50NLjxeaZjB69ze/YPL7d9vBY/OIDU4BgFJdNSmpOZllqkb5dAlfGqzNrmQtW+1R8 e7GFuYHxlm0XIyeHhICJxNwZZ9lAbDYBNYkrLycxdTFycIgIOEis+qrQxcjFwSzQwCLx4/Ml FpAaYQEfic6ZZ5hAbBYBVYl7s38xg9i8AmYSf7puMUPM1JR4vP0nO4jNKWAu8f9wOxvITCGg mqO3ocoFJU7OfAI2kllAXqJ562zmCYw8s5CkZiFJLWBkWsUokplXlpuYmWOqV5ydUZmXWaGX nJ+7iREYa8tq/0zcwfjlsvshRgEORiUeXofaO6lCrIllxZW5hxglOJiVRHjDZW6mCvGmJFZW pRblxxeV5qQWH2KU5mBREuf1Ck9NEBJITyxJzU5NLUgtgskycXBKNTAaWRvZp3m2y1/ukm7Z eDFZ4tL0k58LjnHqsq/6ptO9xqZoWWKmmoSg0L7/OU7f7XUmMPPblv9dYWXUdH7eV6WXC9zO qX+98VfRv0tGwWyLcf3TTyeexJ1nvOPaMjtHOX++sH1woepsPbWN15uPvCx83vNHwCe6/t1K o9VOvCYWMoqJ2u6fVymxFGckGmoxFxUnAgBVpl34sQIAAA== X-CFilter-Loop: Reflected X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791971255275791681 X-GMAIL-MSGID: 1791971255275791681 This patch introduces DAMOS_DEMOTE action, which is similar to DAMOS_PAGEOUT, but demote folios instead of swapping them out. Since there are some common routines with pageout, many functions have similar logics between pageout and demote. damon_pa_migrate_folio_list() is a minimized version of shrink_folio_list(), but it's minified only for demotion. Signed-off-by: Honggyu Kim Signed-off-by: Hyeongtak Ji --- include/linux/damon.h | 2 + mm/damon/paddr.c | 222 ++++++++++++++++++++++++++++++++++++++- mm/damon/sysfs-schemes.c | 1 + 3 files changed, 224 insertions(+), 1 deletion(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index e00ddf1ed39c..86e66772766b 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -105,6 +105,7 @@ struct damon_target { * @DAMOS_NOHUGEPAGE: Call ``madvise()`` for the region with MADV_NOHUGEPAGE. * @DAMOS_LRU_PRIO: Prioritize the region on its LRU lists. * @DAMOS_LRU_DEPRIO: Deprioritize the region on its LRU lists. + * @DAMOS_DEMOTE: Do demotion for the given region. * @DAMOS_STAT: Do nothing but count the stat. * @NR_DAMOS_ACTIONS: Total number of DAMOS actions * @@ -122,6 +123,7 @@ enum damos_action { DAMOS_NOHUGEPAGE, DAMOS_LRU_PRIO, DAMOS_LRU_DEPRIO, + DAMOS_DEMOTE, DAMOS_STAT, /* Do nothing but only record the stat */ NR_DAMOS_ACTIONS, }; diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 277a1c4d833c..23e37ce57202 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -12,6 +12,9 @@ #include #include #include +#include +#include +#include #include "../internal.h" #include "ops-common.h" @@ -226,8 +229,214 @@ static bool damos_pa_filter_out(struct damos *scheme, struct folio *folio) enum migration_mode { MIG_PAGEOUT, + MIG_DEMOTE, }; +/* + * XXX: This is copied from demote_folio_list as renamed as migrate_folio_list. + * Take folios on @migrate_folios and attempt to migrate them to another node. + * Folios which are not migrated are left on @migrate_folios. + */ +static unsigned int migrate_folio_list(struct list_head *migrate_folios, + struct pglist_data *pgdat, + enum migration_mode mm) +{ + int target_nid = next_demotion_node(pgdat->node_id); + unsigned int nr_succeeded; + nodemask_t allowed_mask; + + struct migration_target_control mtc = { + /* + * Allocate from 'node', or fail quickly and quietly. + * When this happens, 'page' will likely just be discarded + * instead of migrated. + */ + .gfp_mask = (GFP_HIGHUSER_MOVABLE & ~__GFP_RECLAIM) | __GFP_NOWARN | + __GFP_NOMEMALLOC | GFP_NOWAIT, + .nid = target_nid, + .nmask = &allowed_mask + }; + + if (pgdat->node_id == target_nid || target_nid == NUMA_NO_NODE) + return 0; + + if (list_empty(migrate_folios)) + return 0; + + node_get_allowed_targets(pgdat, &allowed_mask); + + /* Migration ignores all cpuset and mempolicy settings */ + migrate_pages(migrate_folios, alloc_migrate_folio, NULL, + (unsigned long)&mtc, MIGRATE_ASYNC, MR_DEMOTION, + &nr_succeeded); + + __count_vm_events(PGDEMOTE_DIRECT, nr_succeeded); + + return nr_succeeded; +} + +enum folio_references { + FOLIOREF_RECLAIM, + FOLIOREF_KEEP, + FOLIOREF_ACTIVATE, +}; + +/* + * XXX: This is just copied and simplified from folio_check_references at + * mm/vmscan.c but without having scan_control. + */ +static enum folio_references folio_check_references(struct folio *folio) +{ + int referenced_ptes, referenced_folio; + unsigned long vm_flags; + + referenced_ptes = folio_referenced(folio, 1, NULL, &vm_flags); + referenced_folio = folio_test_clear_referenced(folio); + + /* rmap lock contention: rotate */ + if (referenced_ptes == -1) + return FOLIOREF_KEEP; + + if (referenced_ptes) { + /* + * All mapped folios start out with page table + * references from the instantiating fault, so we need + * to look twice if a mapped file/anon folio is used more + * than once. + * + * Mark it and spare it for another trip around the + * inactive list. Another page table reference will + * lead to its activation. + * + * Note: the mark is set for activated folios as well + * so that recently deactivated but used folios are + * quickly recovered. + */ + folio_set_referenced(folio); + + if (referenced_folio || referenced_ptes > 1) + return FOLIOREF_ACTIVATE; + + /* + * Activate file-backed executable folios after first usage. + */ + if ((vm_flags & VM_EXEC) && folio_is_file_lru(folio)) + return FOLIOREF_ACTIVATE; + + return FOLIOREF_KEEP; + } + + return FOLIOREF_RECLAIM; +} + +/* + * XXX: This is minimized implmentation based on shrink_folio_list only for + * the demotion calling demote_folio_list. + */ +static unsigned int damon_pa_migrate_folio_list(struct list_head *folio_list, + struct pglist_data *pgdat, + enum migration_mode mm) +{ + unsigned int nr_migrated = 0; + struct folio *folio; + LIST_HEAD(ret_folios); + LIST_HEAD(migrate_folios); + + cond_resched(); + + while (!list_empty(folio_list)) { + struct folio *folio; + enum folio_references references; + + cond_resched(); + + folio = lru_to_folio(folio_list); + list_del(&folio->lru); + + if (!folio_trylock(folio)) + goto keep; + + VM_BUG_ON_FOLIO(folio_test_active(folio), folio); + + references = folio_check_references(folio); + if (references == FOLIOREF_KEEP) + goto keep_locked; + + /* Relocate its contents to another node. */ + list_add(&folio->lru, &migrate_folios); + folio_unlock(folio); + continue; +keep_locked: + folio_unlock(folio); +keep: + list_add(&folio->lru, &ret_folios); + VM_BUG_ON_FOLIO(folio_test_lru(folio), folio); + } + /* 'folio_list' is always empty here */ + + /* Migrate folios selected for migration */ + nr_migrated += migrate_folio_list(&migrate_folios, pgdat, mm); + /* Folios that could not be migrated are still in @migrate_folios */ + if (!list_empty(&migrate_folios)) { + /* Folios which weren't migrated go back on @folio_list */ + list_splice_init(&migrate_folios, folio_list); + } + + try_to_unmap_flush(); + + list_splice(&ret_folios, folio_list); + + while (!list_empty(folio_list)) { + folio = lru_to_folio(folio_list); + list_del(&folio->lru); + folio_putback_lru(folio); + } + + return nr_migrated; +} + +/* + * XXX: This is almost identical to reclaim_pages() in mm/vmscan.c, but it + * internally calls damon_pa_migrate_folio_list() instead of + * reclaim_folio_list(). We might be better to think if we can have a + * common function for both cases. + */ +static unsigned long damon_pa_migrate_pages(struct list_head *folio_list, + enum migration_mode mm) +{ + int nid; + unsigned int nr_migrated = 0; + LIST_HEAD(node_folio_list); + unsigned int noreclaim_flag; + + if (list_empty(folio_list)) + return nr_migrated; + + noreclaim_flag = memalloc_noreclaim_save(); + + nid = folio_nid(lru_to_folio(folio_list)); + do { + struct folio *folio = lru_to_folio(folio_list); + + if (nid == folio_nid(folio)) { + folio_clear_active(folio); + list_move(&folio->lru, &node_folio_list); + continue; + } + + nr_migrated += damon_pa_migrate_folio_list(&node_folio_list, + NODE_DATA(nid), mm); + nid = folio_nid(lru_to_folio(folio_list)); + } while (!list_empty(folio_list)); + + nr_migrated += damon_pa_migrate_folio_list(&node_folio_list, + NODE_DATA(nid), mm); + + memalloc_noreclaim_restore(noreclaim_flag); + + return nr_migrated; +} + static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, enum migration_mode mm) { @@ -247,7 +456,11 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, folio_test_clear_young(folio); if (!folio_isolate_lru(folio)) goto put_folio; - if (folio_test_unevictable(folio)) + /* + * Since unevictable folios can be demoted or promoted, + * unevictable test is needed only for pageout. + */ + if (mm == MIG_PAGEOUT && folio_test_unevictable(folio)) folio_putback_lru(folio); else list_add(&folio->lru, &folio_list); @@ -258,6 +471,9 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, case MIG_PAGEOUT: applied = reclaim_pages(&folio_list); break; + case MIG_DEMOTE: + applied = damon_pa_migrate_pages(&folio_list, mm); + break; default: /* Unexpected migration mode. */ return 0; @@ -314,6 +530,8 @@ static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, return damon_pa_mark_accessed(r, scheme); case DAMOS_LRU_DEPRIO: return damon_pa_deactivate_pages(r, scheme); + case DAMOS_DEMOTE: + return damon_pa_migrate(r, scheme, MIG_DEMOTE); case DAMOS_STAT: break; default: @@ -334,6 +552,8 @@ static int damon_pa_scheme_score(struct damon_ctx *context, return damon_hot_score(context, r, scheme); case DAMOS_LRU_DEPRIO: return damon_cold_score(context, r, scheme); + case DAMOS_DEMOTE: + return damon_cold_score(context, r, scheme); default: break; } diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c index fe0fe2562000..53e47fad5021 100644 --- a/mm/damon/sysfs-schemes.c +++ b/mm/damon/sysfs-schemes.c @@ -1186,6 +1186,7 @@ static const char * const damon_sysfs_damos_action_strs[] = { "nohugepage", "lru_prio", "lru_deprio", + "demote", "stat", }; From patchwork Mon Feb 26 14:05:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 206664 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:a81b:b0:108:e6aa:91d0 with SMTP id bq27csp2100393dyb; Mon, 26 Feb 2024 06:11:02 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXEHZzK3emmJmlBSVqpS+e90XJVXiidAAjnhRJ5GNFCwn9PG92yLkKKZ8klV4rLWkBr5Ge/ucMowJmDw9kOUET4NdsE4Q== X-Google-Smtp-Source: AGHT+IFTa2U5IJ9QBbmTCH7oRLlhHCiHQLk6EIyDTgLGe3zeKRA1sMMf07MeM9HK6Rljts4Ut6xP X-Received: by 2002:a17:906:688a:b0:a43:4ac2:f16b with SMTP id n10-20020a170906688a00b00a434ac2f16bmr1949775ejr.13.1708956662684; Mon, 26 Feb 2024 06:11:02 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708956662; cv=pass; d=google.com; s=arc-20160816; b=plIytYnVB9M7JreV+aodoBeEtOLTVZJ7/gP8xRpP4dHUWgQ2+gq1azxWhbBteYZDPg mUhTl46IlhVN2cm8R9S+6Y/FRAN0gE3umQMzlSYkv+BwVplDv+CAeG8QTPWhzlslMG+W m8KFPPeey3UiQ+XmjrQ8bVN23U5P2Bfes2dnkrchpMKOF0jAxKxj4nUr/37O6NYprObU FJwkUIW898/VYmm3DLQ4JdJmtBvPUJ4NB2gz3ww6Ctr4XeiECGrye9sRfpxuJ3T/ambW bbnr9iqUOlZHVanwC0pYB3p069EQj0QVRuCI5KwtALTVUQSjNBjD2+hR8q4kwij99XE4 8jSw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=w9f3rfKd+jtQPwgacOekgJEcbI06gWAyOzMEzJgXU4U=; fh=egwJU2bb679B4Zi0WdxyKykgfACRkSkUGxyHY52POxM=; b=vcYn06t/iDYHvlT9NbQbdpJBvDEzzk58Qdu66nefes0z3SNXfTjffq6LbTaELfSI3X Z4T8rdnvJad7Q8Nqo1iMGq9Bh+PFP0/RXk7hztlqA4dNHiQ5HpArLpwJ6Idse/H8oH2u KYb+XgfOFm/049CBWWGkIgvOz2eY7YZ2uevNgl6HVrkchlnVdcgvlrHAWJGBUeQEgmpm NZtxCQ0bXRLlnrEMs6RNd3Dl3bR2aTEXKCX6JXeYEJY/aepmYi2f17d6ymiTEtDLaz3P 41GxdRUyAPXxxESoZHmQ3d8HQGL1WrgnhkKY+8I4RR6TyODrxkjruFK6016MApH63AwG IEig==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=sk.com); spf=pass (google.com: domain of linux-kernel+bounces-81584-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-81584-ouuuleilei=gmail.com@vger.kernel.org" Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id kh4-20020a170906f80400b00a3ebf421c69si2197779ejb.804.2024.02.26.06.11.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Feb 2024 06:11:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-81584-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=sk.com); spf=pass (google.com: domain of linux-kernel+bounces-81584-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-81584-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 2A57E1F2BD30 for ; Mon, 26 Feb 2024 14:11:02 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D63A112D77E; Mon, 26 Feb 2024 14:06:25 +0000 (UTC) Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 118CF12C53D; Mon, 26 Feb 2024 14:06:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708956383; cv=none; b=eJT2kB8XUbZHIB0mJqgshuzgHbPwNG2+gfFgJ9MHnlEpkL7i2NCf57I7rSKkpDPMUHvGe17uWs3Ju6+qmjeU8zFCAcbs8qHceZ8bbdtZ2TvtbdI0n7gSshY+WumTZxCXjZl8wZrUqLVGggsvjkAj5BXoToSZ6coY5ehiq3SfppA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708956383; c=relaxed/simple; bh=tP11PJE+B44zfRXa8blWSPevKcbSiOSl0SgRA/mAjGs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tvySXC2re05mUgHRLI9PI54eGCY53kaKFYpSMQZe6FWF1T1fOZ74z/mVunr7rp5SjUYWQZlMulK4GvPjPm3TOf9L8d4tvi5F/cKgm9esdFHNBkbW29PV4duMFmHlQ2x98uKGracACylobV6Hez+8cazLrQyxSANX4eihyfFNYbU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d6dff70000001748-7c-65dc9ad909a7 From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, lizhijian@cn.fujitsu.com, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com Subject: [PATCH v2 4/7] mm/memory-tiers: add next_promotion_node to find promotion target Date: Mon, 26 Feb 2024 23:05:50 +0900 Message-ID: <20240226140555.1615-5-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240226140555.1615-1-honggyu.kim@sk.com> References: <20240226140555.1615-1-honggyu.kim@sk.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrBIsWRmVeSWpSXmKPExsXC9ZZnoe6tWXdSDb7/YrOY2GNgMWf9GjaL XTdCLP7vPcZo8eT/b1aLEzcb2Sw6vy9lsbi8aw6bxb01/1ktjqw/y2Kx7haQtfnsGWaLxcvV LPZ1PGCyOPz1DZPF5EsL2CxeTDnDaHFy1mQWi9lH77E7CHv8PziJ2WPp6TdsHhuagMTOWXfZ PVr23WL3WLCp1KPlyFtWj8V7XjJ5bFrVyeax6dMkdo8TM36zeOx8aOnxYvNMRo/e5ndsHp83 yQXwR3HZpKTmZJalFunbJXBl7FvZwFrwSLziz+vJzA2M+4S7GDk5JARMJF6+3sMGY1/cspYJ xGYTUJO48nISkM3BISLgILHqq0IXIxcHs0ADi8SPz5dYQGqEBSIlZkxuYgWxWQRUJVpPXWIE sXkFzCSuP9nHCjFTU+Lx9p/sIDangLnE/8PtbCAzhYBqjt5mhigXlDg58wnYSGYBeYnmrbOZ QXZJCJxil9h0YC/UHEmJgytusExg5J+FpGcWkp4FjEyrGIUy88pyEzNzTPQyKvMyK/SS83M3 MQLjb1ntn+gdjJ8uBB9iFOBgVOLhdai9kyrEmlhWXJl7iFGCg1lJhDdc5maqEG9KYmVValF+ fFFpTmrxIUZpDhYlcV6jb+UpQgLpiSWp2ampBalFMFkmDk6pBsZ1bw5O7DnF3tp751DZCdvF SZc3nJBMjJE8kaukLfLu0KoGno1X57gtVPktNfkX87orou/rHqcrGKrOaozWVr6mMb+o7NPs 1cURJ/UNo17N4WxQLdD9ev+6ZJxgk+HNFgPzlx/3lE1mEQkx3nWkLcLCKbjw085lJwrmT+V7 /0L2VEalS/Y01UQlluKMREMt5qLiRAATmp6xuwIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprHIsWRmVeSWpSXmKPExsXCNUNLT/fmrDupBj39phYTewws5qxfw2ax 60aIxf+9xxgtnvz/zWpx4mYjm8XnZ6+ZLTqffGe0ODz3JKtF5/elLBaXd81hs7i35j+rxZH1 Z1ks1t0CsjafPcNssXi5msWha89ZLfZ1PGCyOPz1DZPF5EsL2CxeTDnDaHFy1mQWi9lH77E7 iHv8PziJ2WPp6TdsHhuagMTOWXfZPVr23WL3WLCp1KPlyFtWj8V7XjJ5bFrVyeax6dMkdo8T M36zeOx8aOnxYvNMRo/e5ndsHt9ue3gsfvGBKUAwissmJTUnsyy1SN8ugStj38oG1oJH4hV/ Xk9mbmDcJ9zFyMkhIWAicXHLWiYQm01ATeLKy0lANgeHiICDxKqvCl2MXBzMAg0sEj8+X2IB qREWiJSYMbmJFcRmEVCVaD11iRHE5hUwk7j+ZB8rxExNicfbf7KD2JwC5hL/D7ezgcwUAqo5 epsZolxQ4uTMJ2AjmQXkJZq3zmaewMgzC0lqFpLUAkamVYwimXlluYmZOaZ6xdkZlXmZFXrJ +bmbGIGxtqz2z8QdjF8uux9iFOBgVOLhdai9kyrEmlhWXJl7iFGCg1lJhDdc5maqEG9KYmVV alF+fFFpTmrxIUZpDhYlcV6v8NQEIYH0xJLU7NTUgtQimCwTB6dUA+PaINdoLfZf91+us46d M+uS9dVZPGz5fwVbW9xXNF27aR175kvOuZl+M6Tc2xjtV9hLXT3SNv32Qf/yvKWX7ZYkSpxm ur3k8ve/hVWzpmVtMmFR29aY9PC4cbLtp4U72J6/UF6+XuXSfvaQ7p0GDrz/q7uEtlbM5zYx mm2td+9xbItPYfdB704lluKMREMt5qLiRABxHBSHsQIAAA== X-CFilter-Loop: Reflected X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791970941406680624 X-GMAIL-MSGID: 1791970941406680624 From: Hyeongtak Ji This patch adds next_promotion_node that can be used to identify the appropriate promotion target based on memory tiers. When multiple promotion target nodes are available, the nearest node is selected based on numa distance. Signed-off-by: Hyeongtak Ji --- include/linux/memory-tiers.h | 11 +++++++++ mm/memory-tiers.c | 43 ++++++++++++++++++++++++++++++++++++ 2 files changed, 54 insertions(+) diff --git a/include/linux/memory-tiers.h b/include/linux/memory-tiers.h index 1e39d27bee41..0788e435fc50 100644 --- a/include/linux/memory-tiers.h +++ b/include/linux/memory-tiers.h @@ -50,6 +50,7 @@ int mt_set_default_dram_perf(int nid, struct node_hmem_attrs *perf, int mt_perf_to_adistance(struct node_hmem_attrs *perf, int *adist); #ifdef CONFIG_MIGRATION int next_demotion_node(int node); +int next_promotion_node(int node); void node_get_allowed_targets(pg_data_t *pgdat, nodemask_t *targets); bool node_is_toptier(int node); #else @@ -58,6 +59,11 @@ static inline int next_demotion_node(int node) return NUMA_NO_NODE; } +static inline int next_promotion_node(int node) +{ + return NUMA_NO_NODE; +} + static inline void node_get_allowed_targets(pg_data_t *pgdat, nodemask_t *targets) { *targets = NODE_MASK_NONE; @@ -101,6 +107,11 @@ static inline int next_demotion_node(int node) return NUMA_NO_NODE; } +static inline int next_promotion_node(int node) +{ + return NUMA_NO_NODE; +} + static inline void node_get_allowed_targets(pg_data_t *pgdat, nodemask_t *targets) { *targets = NODE_MASK_NONE; diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c index 8d5291add2bc..0060ee571cf4 100644 --- a/mm/memory-tiers.c +++ b/mm/memory-tiers.c @@ -335,6 +335,49 @@ int next_demotion_node(int node) return target; } +/* + * Select a promotion target that is close to the from node among the given + * two nodes. + * + * TODO: consider other decision policy as node_distance may not be precise. + */ +static int select_promotion_target(int a, int b, int from) +{ + if (node_distance(from, a) < node_distance(from, b)) + return a; + else + return b; +} + +/** + * next_promotion_node() - Get the next node in the promotion path + * @node: The starting node to lookup the next node + * + * Return: node id for next memory node in the promotion path hierarchy + * from @node; NUMA_NO_NODE if @node is the toptier. + */ +int next_promotion_node(int node) +{ + int target = NUMA_NO_NODE; + int nid; + + if (node_is_toptier(node)) + return NUMA_NO_NODE; + + rcu_read_lock(); + for_each_node_state(nid, N_MEMORY) { + if (node_isset(node, node_demotion[nid].preferred)) { + if (target == NUMA_NO_NODE) + target = nid; + else + target = select_promotion_target(nid, target, node); + } + } + rcu_read_unlock(); + + return target; +} + static void disable_all_demotion_targets(void) { struct memory_tier *memtier; From patchwork Mon Feb 26 14:05:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 206666 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:a81b:b0:108:e6aa:91d0 with SMTP id bq27csp2101727dyb; Mon, 26 Feb 2024 06:12:57 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVRBW1TZ9KkWK46egTdsxtPji65r0W/+95B/zJwOOFIPHSsXJN3LKMffp4RXaM6n7wVciwYRvUeiUtOOxI/O8v1yE76tg== X-Google-Smtp-Source: AGHT+IHXdoaFCusjvBoOQoNa5Sz5IjPhJpxL6TGUPDZPHFIaw9FVR5G3aGR6MKmHeSdOlXutdaBX X-Received: by 2002:a17:906:3487:b0:a3f:daf8:bd3b with SMTP id g7-20020a170906348700b00a3fdaf8bd3bmr4644358ejb.8.1708956777340; Mon, 26 Feb 2024 06:12:57 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708956777; cv=pass; d=google.com; s=arc-20160816; b=beVMKjAtjUmboDRhgj7/ERWbGMxuuQDHTuefm8B3+JDZy16JiVxqTRRjmLJePSERNS QnN/+Pmf+eftBf9uL4i7HkPpVElMuBpHTOu3vbaFJRS+vJIMxFRNqOC+G7V5d/1d/Ms5 QYYG6gWbVPLGyzZ4FmQ7jZuM/MWLWZ9ISpehat/fXzx2/3XkGHWn+in1QpX5DkeqgFIN h1u6vhUFq0//IkMQCqsTRNDToflZXn8UgU62XvpblHD9kJGnJPcLH6EupPOmT4WnF+8G g1x+tSLWU9v0I/7dVJdhYk2tYMk2kHR1ouskynE/dxK2v45S21Hyz7J6kLhhsBykM/kK /5gw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=WCJbCXrjn/PAdTWQ0hYIswG7MSzXLA6XzCdTSZGM83g=; fh=egwJU2bb679B4Zi0WdxyKykgfACRkSkUGxyHY52POxM=; b=Ax7/RXH3ukr7M++mHM/puYr7wBKKKjIcxH8lCApQWeGldMi+5D4symtLHocfRXH1kU KaVZ9Qo2gCgOPqY0AwauqW3RIpWKzHug0V4VUS6oaNlZqngtb5LjafYUbKkKk6IfBX9t gsCJlpJHjL21BMtXVXGLVuiLoRn7q3PTSsJrdAuEUTqIFHAajUA847IqEDDjRYmI/YQR +j8hjBD6XnPMchHblukzs5445ZBmhTl2L1VXTaWj4Aul4sxIu/SDnqk2wF3AIQBQ7bpi fBz1/bpOC6KDE6vdDXWtpEWZ4tPeoi8cA7l5wwPsLB6aMrUBkhMsQSFH2214DDlHrzie D0Nw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=sk.com); spf=pass (google.com: domain of linux-kernel+bounces-81585-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-81585-ouuuleilei=gmail.com@vger.kernel.org" Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id m21-20020a170906235500b00a3f7f0def2dsi2210251eja.512.2024.02.26.06.12.57 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Feb 2024 06:12:57 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-81585-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=sk.com); spf=pass (google.com: domain of linux-kernel+bounces-81585-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-81585-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 444271F24364 for ; Mon, 26 Feb 2024 14:11:40 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0710812EBF3; Mon, 26 Feb 2024 14:06:29 +0000 (UTC) Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6F18212C803; Mon, 26 Feb 2024 14:06:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708956386; cv=none; b=PGtoDyje7bJxWlnmQlrXyoFpHeXvxkn5fHe6ciVt31p/Q68KwGPG98JLCThsB92aL/7m7d+1ncacK69ym/QNE2ti0UTKkLsF/pzvMH+cBKJ9Gn5cuNZHA4VriqobW53xCv6oI3cEckfhmfxRu9qvrLUd0it1c/K2v2oSO8U/mm4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708956386; c=relaxed/simple; bh=DNlk804N1hnzt8DH0e5+lqyDTtt76Du4GvfOQwH+aBY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uiIeriUb9c2bUKBx73yqSetns4Ej8/clr264cqKaWhxsnfXdBBzpScHNOnhrCxJMbBJgc3jr3v7ysOq0Gki1Lh3BLLrr5AJ38ccjxRfgyYEF3i6gFn8MWRRIxIrZHrAYsrdyikuao+/SX+IcUa0jr8VS9IpwOPIzfzIZarWs/BQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d6dff70000001748-83-65dc9adb88dd From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, lizhijian@cn.fujitsu.com, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com Subject: [PATCH v2 5/7] mm/damon: introduce DAMOS_PROMOTE action for promotion Date: Mon, 26 Feb 2024 23:05:51 +0900 Message-ID: <20240226140555.1615-6-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240226140555.1615-1-honggyu.kim@sk.com> References: <20240226140555.1615-1-honggyu.kim@sk.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrKIsWRmVeSWpSXmKPExsXC9ZZnoe7tWXdSDbYckLeY2GNgMWf9GjaL XTdCLP7vPcZo8eT/b1aLEzcb2Sw6vy9lsbi8aw6bxb01/1ktjqw/y2Kx7haQtfnsGWaLxcvV LPZ1PGCyOPz1DZPF5EsL2CxeTDnDaHFy1mQWi9lH77E7CHv8PziJ2WPp6TdsHhuagMTOWXfZ PVr23WL3WLCp1KPlyFtWj8V7XjJ5bFrVyeax6dMkdo8TM36zeOx8aOnxYvNMRo/e5ndsHp83 yQXwR3HZpKTmZJalFunbJXBlTDz7hq1goUXFnL5tjA2Mv3S7GDk4JARMJL4uK+5i5AQzp545 yQxiswmoSVx5OYkJpEREwEFi1VeFLkYuDmaBBhaJH58vsYDEhQX8JDa/LAMxWQRUJW4tEQDp 5BUwk3h5aR4rxERNicfbf7KD2JwC5hL/D7ezgZQLAdUcvc0MUS4ocXLmExYQm1lAXqJ562xm kE0SAqfYJXYu2skEMUdS4uCKGywTGPlnIemZhaRnASPTKkahzLyy3MTMHBO9jMq8zAq95Pzc TYzAyFtW+yd6B+OnC8GHGAU4GJV4eB1q76QKsSaWFVfmHmKU4GBWEuENl7mZKsSbklhZlVqU H19UmpNafIhRmoNFSZzX6Ft5ipBAemJJanZqakFqEUyWiYNTqoExc5uYpGYdU+jLbvYbJc+c C/IN5vrMsZj+vszyRE2yOqvE4RuVi+afWLlBj+PY/aMJf68HfEtMi2UrnTJXfb9HyYybhU27 7QU1a/49rzsus76lnKnRUOTc1QvPN8yKdFG/qOXZelPnfnVxsv7t9TvvPMuTPH1iy2Mu/2vX lxxvjHe9cuvBhjc2SizFGYmGWsxFxYkA+pGG7LgCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprDIsWRmVeSWpSXmKPExsXCNUNLT/f2rDupBu9n8FlM7DGwmLN+DZvF rhshFv/3HmO0ePL/N6vFiZuNbBafn71mtuh88p3R4vDck6wWnd+Xslhc3jWHzeLemv+sFkfW n2WxWHcLyNp89gyzxeLlahaHrj1ntdjX8YDJ4vDXN0wWky8tYLN4MeUMo8XJWZNZLGYfvcfu IO7x/+AkZo+lp9+weWxoAhI7Z91l92jZd4vdY8GmUo+WI29ZPRbvecnksWlVJ5vHpk+T2D1O zPjN4rHzoaXHi80zGT16m9+xeXy77eGx+MUHpgDBKC6blNSczLLUIn27BK6MiWffsBUstKiY 07eNsYHxl24XIyeHhICJxNQzJ5lBbDYBNYkrLycxdTFycIgIOEis+qrQxcjFwSzQwCLx4/Ml FpC4sICfxOaXZSAmi4CqxK0lAiCdvAJmEi8vzWOFmKgp8Xj7T3YQm1PAXOL/4XY2kHIhoJqj t5khygUlTs58wgJiMwvISzRvnc08gZFnFpLULCSpBYxMqxhFMvPKchMzc0z1irMzKvMyK/SS 83M3MQLjbFntn4k7GL9cdj/EKMDBqMTD61B7J1WINbGsuDL3EKMEB7OSCG+4zM1UId6UxMqq 1KL8+KLSnNTiQ4zSHCxK4rxe4akJQgLpiSWp2ampBalFMFkmDk6pBsaZS5iZNt2Tdbt0unvm iovFUdcX3Wq791p78tTIUrU5c/hXW3OkcVzoSwmaaeu8+/eSDEVN0VZDyT3cux47NJxh37gi KrjL31clSkhh00MFPZb1r0QDVCdOMU741V437+TRA0+1+982M1+fkPPlt3nRZPu23BldO/rM ntjqx0yaLdCxjD/R9ZESS3FGoqEWc1FxIgBxEqWRrwIAAA== X-CFilter-Loop: Reflected X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791971061688901485 X-GMAIL-MSGID: 1791971061688901485 From: Hyeongtak Ji This patch introduces DAMOS_PROMOTE action for paddr mode. It includes renaming alloc_demote_folio to alloc_migrate_folio to use it for promotion as well. Signed-off-by: Hyeongtak Ji Signed-off-by: Honggyu Kim --- include/linux/damon.h | 2 ++ include/linux/migrate_mode.h | 1 + include/linux/vm_event_item.h | 1 + include/trace/events/migrate.h | 3 ++- mm/damon/paddr.c | 45 ++++++++++++++++++++++++++++------ mm/damon/sysfs-schemes.c | 1 + mm/vmstat.c | 1 + 7 files changed, 46 insertions(+), 8 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index 86e66772766b..d7e52d0228b4 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -105,6 +105,7 @@ struct damon_target { * @DAMOS_NOHUGEPAGE: Call ``madvise()`` for the region with MADV_NOHUGEPAGE. * @DAMOS_LRU_PRIO: Prioritize the region on its LRU lists. * @DAMOS_LRU_DEPRIO: Deprioritize the region on its LRU lists. + * @DAMOS_PROMOTE: Do promotion for the given region. * @DAMOS_DEMOTE: Do demotion for the given region. * @DAMOS_STAT: Do nothing but count the stat. * @NR_DAMOS_ACTIONS: Total number of DAMOS actions @@ -123,6 +124,7 @@ enum damos_action { DAMOS_NOHUGEPAGE, DAMOS_LRU_PRIO, DAMOS_LRU_DEPRIO, + DAMOS_PROMOTE, DAMOS_DEMOTE, DAMOS_STAT, /* Do nothing but only record the stat */ NR_DAMOS_ACTIONS, diff --git a/include/linux/migrate_mode.h b/include/linux/migrate_mode.h index f37cc03f9369..63f75eb9abf3 100644 --- a/include/linux/migrate_mode.h +++ b/include/linux/migrate_mode.h @@ -29,6 +29,7 @@ enum migrate_reason { MR_CONTIG_RANGE, MR_LONGTERM_PIN, MR_DEMOTION, + MR_PROMOTION, MR_TYPES }; diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h index 8abfa1240040..63cf920afeaa 100644 --- a/include/linux/vm_event_item.h +++ b/include/linux/vm_event_item.h @@ -44,6 +44,7 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT, PGDEMOTE_KSWAPD, PGDEMOTE_DIRECT, PGDEMOTE_KHUGEPAGED, + PGPROMOTE, PGSCAN_KSWAPD, PGSCAN_DIRECT, PGSCAN_KHUGEPAGED, diff --git a/include/trace/events/migrate.h b/include/trace/events/migrate.h index 0190ef725b43..f0dd569c1e62 100644 --- a/include/trace/events/migrate.h +++ b/include/trace/events/migrate.h @@ -22,7 +22,8 @@ EM( MR_NUMA_MISPLACED, "numa_misplaced") \ EM( MR_CONTIG_RANGE, "contig_range") \ EM( MR_LONGTERM_PIN, "longterm_pin") \ - EMe(MR_DEMOTION, "demotion") + EM( MR_DEMOTION, "demotion") \ + EMe(MR_PROMOTION, "promotion") /* * First define the enums in the above macros to be exported to userspace diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 23e37ce57202..37a7b34a36dd 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -229,6 +229,7 @@ static bool damos_pa_filter_out(struct damos *scheme, struct folio *folio) enum migration_mode { MIG_PAGEOUT, + MIG_PROMOTE, MIG_DEMOTE, }; @@ -241,9 +242,26 @@ static unsigned int migrate_folio_list(struct list_head *migrate_folios, struct pglist_data *pgdat, enum migration_mode mm) { - int target_nid = next_demotion_node(pgdat->node_id); + int target_nid; unsigned int nr_succeeded; nodemask_t allowed_mask; + int reason; + enum vm_event_item vm_event; + + switch (mm) { + case MIG_PROMOTE: + target_nid = next_promotion_node(pgdat->node_id); + reason = MR_PROMOTION; + vm_event = PGPROMOTE; + break; + case MIG_DEMOTE: + target_nid = next_demotion_node(pgdat->node_id); + reason = MR_DEMOTION; + vm_event = PGDEMOTE_DIRECT; + break; + default: + return 0; + } struct migration_target_control mtc = { /* @@ -263,14 +281,19 @@ static unsigned int migrate_folio_list(struct list_head *migrate_folios, if (list_empty(migrate_folios)) return 0; - node_get_allowed_targets(pgdat, &allowed_mask); + if (mm == MIG_DEMOTE) { + node_get_allowed_targets(pgdat, &allowed_mask); + } else if (mm == MIG_PROMOTE) { + /* TODO: Need to add upper_tier_mask at struct memory_tier. */ + allowed_mask = NODE_MASK_NONE; + } /* Migration ignores all cpuset and mempolicy settings */ migrate_pages(migrate_folios, alloc_migrate_folio, NULL, - (unsigned long)&mtc, MIGRATE_ASYNC, MR_DEMOTION, + (unsigned long)&mtc, MIGRATE_ASYNC, reason, &nr_succeeded); - __count_vm_events(PGDEMOTE_DIRECT, nr_succeeded); + __count_vm_events(vm_event, nr_succeeded); return nr_succeeded; } @@ -359,7 +382,8 @@ static unsigned int damon_pa_migrate_folio_list(struct list_head *folio_list, VM_BUG_ON_FOLIO(folio_test_active(folio), folio); references = folio_check_references(folio); - if (references == FOLIOREF_KEEP) + if (references == FOLIOREF_KEEP || + (references == FOLIOREF_RECLAIM && mm == MIG_PROMOTE)) goto keep_locked; /* Relocate its contents to another node. */ @@ -452,8 +476,10 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, if (damos_pa_filter_out(s, folio)) goto put_folio; - folio_clear_referenced(folio); - folio_test_clear_young(folio); + if (mm != MIG_PROMOTE) { + folio_clear_referenced(folio); + folio_test_clear_young(folio); + } if (!folio_isolate_lru(folio)) goto put_folio; /* @@ -471,6 +497,7 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, case MIG_PAGEOUT: applied = reclaim_pages(&folio_list); break; + case MIG_PROMOTE: case MIG_DEMOTE: applied = damon_pa_migrate_pages(&folio_list, mm); break; @@ -530,6 +557,8 @@ static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, return damon_pa_mark_accessed(r, scheme); case DAMOS_LRU_DEPRIO: return damon_pa_deactivate_pages(r, scheme); + case DAMOS_PROMOTE: + return damon_pa_migrate(r, scheme, MIG_PROMOTE); case DAMOS_DEMOTE: return damon_pa_migrate(r, scheme, MIG_DEMOTE); case DAMOS_STAT: @@ -552,6 +581,8 @@ static int damon_pa_scheme_score(struct damon_ctx *context, return damon_hot_score(context, r, scheme); case DAMOS_LRU_DEPRIO: return damon_cold_score(context, r, scheme); + case DAMOS_PROMOTE: + return damon_hot_score(context, r, scheme); case DAMOS_DEMOTE: return damon_cold_score(context, r, scheme); default: diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c index 53e47fad5021..9bc48932eb6c 100644 --- a/mm/damon/sysfs-schemes.c +++ b/mm/damon/sysfs-schemes.c @@ -1186,6 +1186,7 @@ static const char * const damon_sysfs_damos_action_strs[] = { "nohugepage", "lru_prio", "lru_deprio", + "promote", "demote", "stat", }; diff --git a/mm/vmstat.c b/mm/vmstat.c index 359460deb377..c703abdb8137 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1282,6 +1282,7 @@ const char * const vmstat_text[] = { "pgdemote_kswapd", "pgdemote_direct", "pgdemote_khugepaged", + "pgpromote", "pgscan_kswapd", "pgscan_direct", "pgscan_khugepaged", From patchwork Mon Feb 26 14:05:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 206667 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:a81b:b0:108:e6aa:91d0 with SMTP id bq27csp2101814dyb; Mon, 26 Feb 2024 06:13:04 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCW2iNy3178s+UgQ26SOyHX2Q97T2BzNnTZBKS6oJlTn7GSjcycXGGd0U+bGBUdG4pNXk8vybg6jaHxJxfQWnQ5MBoTxjA== X-Google-Smtp-Source: AGHT+IHg8vvdA/kpAtQ0ud/RVP4hk0qx3OgwIa6N/9DLv0OAUb6dCtKqPVPQHsImVydBTyz0K1Vf X-Received: by 2002:a17:906:34cb:b0:a40:2999:5404 with SMTP id h11-20020a17090634cb00b00a4029995404mr4460170ejb.3.1708956784278; Mon, 26 Feb 2024 06:13:04 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708956784; cv=pass; d=google.com; s=arc-20160816; b=CmS8jYINjqHeCEKoAfSPsdfT/thLAIjn/tyTovMdetVySqhI8W8zEEaz+ZkZl4mwGd fJMAb3zRWx26RPFHlMZMJHsk9XPKe/errCLP/S+URJJqG9eXdyXXEML1+u+iXJlTL1jC wkgME8u6UnM3upZZ4/FhugPOzidqpBq5XjL0t7cROrppbe/HsH2PSREx5sXAR6cI9b8l bOEpUt3D8X+9qm/KJ0MaOVAxDktB3rPTWMZ7qQTsGUU2lWpqvGKD5kFpLljo9nA7ou4o BFUzOvOVY3VLMYJ1D+2PDSyE9swUKOlIHumMuHIDVYxzGmPFersMgSqfOLan27FzYfkJ s/Wg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=4NxgZ3WLgt/p9y+eS2UytlMMie6cqo2iGxr0OVp+610=; fh=egwJU2bb679B4Zi0WdxyKykgfACRkSkUGxyHY52POxM=; b=rD18cPGuLat6CY92Mp8EakzeWrLjKBszPgbHIsN4HeVcTX9Xh34MFCfc2fSIJwjzKf gd+Pve5pfXR3FzYlGUzVYMdhfk2TKRlGa2iTYVN2LkiU3DFtEKTnT4GmGYgD7tbaGzY8 R7GzEx3pRE3N71zzKbS7qabtlB77aZn9XRq/8DlmLpgHPXopECU46TMkDo9fGp+IXtu2 a9TbWvqQayIQ+ZsBt3LAfRQTC35SrfsZjRcQUD4Ajci0UejKJhDXIV26lPIHoE5avZFP 3klPnrBC73nYlGmdpvAQ/7dr9QzXqfY7ufP7B+STrUSf1xJk+2ikcXvsDNyPn4ttBAzB 2pFw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=sk.com); spf=pass (google.com: domain of linux-kernel+bounces-81587-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-81587-ouuuleilei=gmail.com@vger.kernel.org" Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id p9-20020a17090628c900b00a3e788d3ba7si2011558ejd.384.2024.02.26.06.13.04 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Feb 2024 06:13:04 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-81587-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=sk.com); spf=pass (google.com: domain of linux-kernel+bounces-81587-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-81587-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 422681F24813 for ; Mon, 26 Feb 2024 14:11:53 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1BE3212F370; Mon, 26 Feb 2024 14:06:31 +0000 (UTC) Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6F45212C806; Mon, 26 Feb 2024 14:06:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708956388; cv=none; b=UxfT1S+XGWfbz333VuGVeNAns7fS9Sz30jHKyjWpQCPF+BBATMSL0A0x6f2wCy18BIP497MNWFUPTBgeyFf5g59ruZXQTP3xXTJ/Spl/tTdoXfp645QoG9r2/sMOy4HM+It+6RAIboOIGXMbknm2AAd4jR5O+wHzFTHfTyF6xw8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708956388; c=relaxed/simple; bh=NUk0r1jrb5I7n+/isN9uEsOAk3VhDiA1jU8BnjP6q5c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Q81FaF1r+8uHPUCTIHMXt8GTlaQWjNFyb+SJkpKD2yjd2MNF5/uTmKJlZ/gRy9nXOIav5iRRvYS/s2MICtxdnPIxNjSbu1XtPKLlAbQcU7WCo7gDgmMXkALXmO43o52ha5ofxscvpUR55BUMttmExXCPog9fC2QO3JvP3ZmcNDk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d6dff70000001748-8d-65dc9adceeac From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, lizhijian@cn.fujitsu.com, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com Subject: [PATCH v2 6/7] mm/damon/sysfs-schemes: add target_nid on sysfs-schemes Date: Mon, 26 Feb 2024 23:05:52 +0900 Message-ID: <20240226140555.1615-7-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240226140555.1615-1-honggyu.kim@sk.com> References: <20240226140555.1615-1-honggyu.kim@sk.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrOIsWRmVeSWpSXmKPExsXC9ZZnoe6dWXdSDabtMrSY2GNgMWf9GjaL XTdCLP7vPcZo8eT/b1aLEzcb2Sw6vy9lsbi8aw6bxb01/1ktjqw/y2Kx7haQtfnsGWaLxcvV LPZ1PGCyOPz1DZPF5EsL2CxeTDnDaHFy1mQWi9lH77E7CHv8PziJ2WPp6TdsHhuagMTOWXfZ PVr23WL3WLCp1KPlyFtWj8V7XjJ5bFrVyeax6dMkdo8TM36zeOx8aOnxYvNMRo/e5ndsHp83 yQXwR3HZpKTmZJalFunbJXBlXF0wnb3gulXFxwm32RsYlxp0MXJySAiYSBx/+4cFxr59dQcT iM0moCZx5eUkIJuDQ0TAQWLVV4UuRi4OZoEGFokfny+B1QsL+EscnrCcGaSGRUBVom2dEkiY V8BMouvfL0aIkZoSj7f/ZAexOQXMJf4fbmcDKRcCqjl6mxmiXFDi5MwnYBOZBeQlmrfOZgZZ JSFwjF3ie+dLqNMkJQ6uuMEygZF/FpKeWUh6FjAyrWIUyswry03MzDHRy6jMy6zQS87P3cQI jL5ltX+idzB+uhB8iFGAg1GJh9eh9k6qEGtiWXFl7iFGCQ5mJRHecJmbqUK8KYmVValF+fFF pTmpxYcYpTlYlMR5jb6VpwgJpCeWpGanphakFsFkmTg4pRoYQwSEvGbm/NZbP69KaPn8qaoV pU+Tf2QwKiztbJASzsrYlX5OUXf20QNbH7xc7zPpZcaeLdHGvB2nAqbpP+XPOchaqxLnq7bA 8PW+ythV5ncnrV5x8l/J6/0ORQfTP/mZTX3B8km+vNjqX810r3+Ovv9Dti0ve9K+v+CLynrn WO/sbrmp3Gu/KLEUZyQaajEXFScCANrcd+m6AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprLIsWRmVeSWpSXmKPExsXCNUNLT/fOrDupBr+/KFtM7DGwmLN+DZvF rhshFv/3HmO0ePL/N6vFiZuNbBafn71mtuh88p3R4vDck6wWnd+Xslhc3jWHzeLemv+sFkfW n2WxWHcLyNp89gyzxeLlahaHrj1ntdjX8YDJ4vDXN0wWky8tYLN4MeUMo8XJWZNZLGYfvcfu IO7x/+AkZo+lp9+weWxoAhI7Z91l92jZd4vdY8GmUo+WI29ZPRbvecnksWlVJ5vHpk+T2D1O zPjN4rHzoaXHi80zGT16m9+xeXy77eGx+MUHpgDBKC6blNSczLLUIn27BK6Mqwumsxdct6r4 OOE2ewPjUoMuRk4OCQETidtXdzCB2GwCahJXXk4Csjk4RAQcJFZ9Vehi5OJgFmhgkfjx+RIL SI2wgL/E4QnLmUFqWARUJdrWKYGEeQXMJLr+/WKEGKkp8Xj7T3YQm1PAXOL/4XY2kHIhoJqj t5khygUlTs58AjaRWUBeonnrbOYJjDyzkKRmIUktYGRaxSiSmVeWm5iZY6pXnJ1RmZdZoZec n7uJERhpy2r/TNzB+OWy+yFGAQ5GJR5eh9o7qUKsiWXFlbmHGCU4mJVEeMNlbqYK8aYkVlal FuXHF5XmpBYfYpTmYFES5/UKT00QEkhPLEnNTk0tSC2CyTJxcEo1MO7dPfsYU8OBR6rquvop 3mKHUifcvyzWssDszQmTtS7CwhFJe1pWvlKt8TB361ObJqC9aNf2ipjZ7hp1uTVrOPNMmzs1 //DzLbBZwFy39a+ClES9U2y8p5RK8X4p7utcq3bt3bygob3Mnq14SfQ+d/23H30nGm/bXXcu n6/TXrHmkl9XuO8lJZbijERDLeai4kQAbAwRP7ACAAA= X-CFilter-Loop: Reflected X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791971069085661154 X-GMAIL-MSGID: 1791971069085661154 From: Hyeongtak Ji This patch adds target_nid under /sys/kernel/mm/damon/admin/kdamonds//contexts//schemes/ target_nid can be used as the destination node for DAMOS actions such as DAMOS_DEMOTE or DAMOS_PROMOTE in the future. Signed-off-by: Hyeongtak Ji Signed-off-by: Honggyu Kim --- include/linux/damon.h | 11 ++++++++++- mm/damon/core.c | 5 ++++- mm/damon/dbgfs.c | 2 +- mm/damon/lru_sort.c | 3 ++- mm/damon/reclaim.c | 3 ++- mm/damon/sysfs-schemes.c | 37 ++++++++++++++++++++++++++++++++++++- 6 files changed, 55 insertions(+), 6 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index d7e52d0228b4..4d270956dbd0 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -321,6 +321,7 @@ struct damos_access_pattern { * @apply_interval_us: The time between applying the @action. * @quota: Control the aggressiveness of this scheme. * @wmarks: Watermarks for automated (in)activation of this scheme. + * @target_nid: Destination node if @action is "promote" or "demote". * @filters: Additional set of &struct damos_filter for &action. * @stat: Statistics of this scheme. * @list: List head for siblings. @@ -336,6 +337,10 @@ struct damos_access_pattern { * monitoring context are inactive, DAMON stops monitoring either, and just * repeatedly checks the watermarks. * + * @target_nid is used to set the destination node for promote or demote + * actions, which means it's only meaningful when @action is either "promote" or + * "demote". + * * Before applying the &action to a memory region, &struct damon_operations * implementation could check pages of the region and skip &action to respect * &filters @@ -357,6 +362,9 @@ struct damos { /* public: */ struct damos_quota quota; struct damos_watermarks wmarks; + union { + int target_nid; + }; struct list_head filters; struct damos_stat stat; struct list_head list; @@ -661,7 +669,8 @@ struct damos *damon_new_scheme(struct damos_access_pattern *pattern, enum damos_action action, unsigned long apply_interval_us, struct damos_quota *quota, - struct damos_watermarks *wmarks); + struct damos_watermarks *wmarks, + int target_nid); void damon_add_scheme(struct damon_ctx *ctx, struct damos *s); void damon_destroy_scheme(struct damos *s); diff --git a/mm/damon/core.c b/mm/damon/core.c index 3a05e71509b9..0c2472818fb9 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -316,7 +316,8 @@ struct damos *damon_new_scheme(struct damos_access_pattern *pattern, enum damos_action action, unsigned long apply_interval_us, struct damos_quota *quota, - struct damos_watermarks *wmarks) + struct damos_watermarks *wmarks, + int target_nid) { struct damos *scheme; @@ -341,6 +342,8 @@ struct damos *damon_new_scheme(struct damos_access_pattern *pattern, scheme->wmarks = *wmarks; scheme->wmarks.activated = true; + scheme->target_nid = target_nid; + return scheme; } diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c index dc0ea1fc30ca..29b427dd1186 100644 --- a/mm/damon/dbgfs.c +++ b/mm/damon/dbgfs.c @@ -279,7 +279,7 @@ static struct damos **str_to_schemes(const char *str, ssize_t len, pos += parsed; scheme = damon_new_scheme(&pattern, action, 0, "a, - &wmarks); + &wmarks, NUMA_NO_NODE); if (!scheme) goto fail; diff --git a/mm/damon/lru_sort.c b/mm/damon/lru_sort.c index f2e5f9431892..fd0492637fce 100644 --- a/mm/damon/lru_sort.c +++ b/mm/damon/lru_sort.c @@ -163,7 +163,8 @@ static struct damos *damon_lru_sort_new_scheme( /* under the quota. */ "a, /* (De)activate this according to the watermarks. */ - &damon_lru_sort_wmarks); + &damon_lru_sort_wmarks, + NUMA_NO_NODE); } /* Create a DAMON-based operation scheme for hot memory regions */ diff --git a/mm/damon/reclaim.c b/mm/damon/reclaim.c index ab974e477d2f..973ac5df84eb 100644 --- a/mm/damon/reclaim.c +++ b/mm/damon/reclaim.c @@ -147,7 +147,8 @@ static struct damos *damon_reclaim_new_scheme(void) /* under the quota. */ &damon_reclaim_quota, /* (De)activate this according to the watermarks. */ - &damon_reclaim_wmarks); + &damon_reclaim_wmarks, + NUMA_NO_NODE); } static int damon_reclaim_apply_parameters(void) diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c index 9bc48932eb6c..8bf5aa98d916 100644 --- a/mm/damon/sysfs-schemes.c +++ b/mm/damon/sysfs-schemes.c @@ -6,6 +6,7 @@ */ #include +#include #include "sysfs-common.h" @@ -1175,6 +1176,7 @@ struct damon_sysfs_scheme { struct damon_sysfs_scheme_filters *filters; struct damon_sysfs_stats *stats; struct damon_sysfs_scheme_regions *tried_regions; + int target_nid; }; /* This should match with enum damos_action */ @@ -1202,6 +1204,7 @@ static struct damon_sysfs_scheme *damon_sysfs_scheme_alloc( scheme->kobj = (struct kobject){}; scheme->action = action; scheme->apply_interval_us = apply_interval_us; + scheme->target_nid = NUMA_NO_NODE; return scheme; } @@ -1424,6 +1427,32 @@ static ssize_t apply_interval_us_store(struct kobject *kobj, return err ? err : count; } +static ssize_t target_nid_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + struct damon_sysfs_scheme *scheme = container_of(kobj, + struct damon_sysfs_scheme, kobj); + + return sysfs_emit(buf, "%d\n", scheme->target_nid); +} + +static ssize_t target_nid_store(struct kobject *kobj, + struct kobj_attribute *attr, const char *buf, size_t count) +{ + struct damon_sysfs_scheme *scheme = container_of(kobj, + struct damon_sysfs_scheme, kobj); + int err = 0; + + if (scheme->action != DAMOS_DEMOTE && + scheme->action != DAMOS_PROMOTE) + return -EINVAL; + + /* TODO: error handling for target_nid range. */ + err = kstrtoint(buf, 0, &scheme->target_nid); + + return err ? err : count; +} + static void damon_sysfs_scheme_release(struct kobject *kobj) { kfree(container_of(kobj, struct damon_sysfs_scheme, kobj)); @@ -1435,9 +1464,13 @@ static struct kobj_attribute damon_sysfs_scheme_action_attr = static struct kobj_attribute damon_sysfs_scheme_apply_interval_us_attr = __ATTR_RW_MODE(apply_interval_us, 0600); +static struct kobj_attribute damon_sysfs_scheme_target_nid_attr = + __ATTR_RW_MODE(target_nid, 0600); + static struct attribute *damon_sysfs_scheme_attrs[] = { &damon_sysfs_scheme_action_attr.attr, &damon_sysfs_scheme_apply_interval_us_attr.attr, + &damon_sysfs_scheme_target_nid_attr.attr, NULL, }; ATTRIBUTE_GROUPS(damon_sysfs_scheme); @@ -1690,7 +1723,8 @@ static struct damos *damon_sysfs_mk_scheme( }; scheme = damon_new_scheme(&pattern, sysfs_scheme->action, - sysfs_scheme->apply_interval_us, "a, &wmarks); + sysfs_scheme->apply_interval_us, "a, &wmarks, + sysfs_scheme->target_nid); if (!scheme) return NULL; @@ -1721,6 +1755,7 @@ static void damon_sysfs_update_scheme(struct damos *scheme, scheme->action = sysfs_scheme->action; scheme->apply_interval_us = sysfs_scheme->apply_interval_us; + scheme->target_nid = sysfs_scheme->target_nid; scheme->quota.ms = sysfs_quotas->ms; scheme->quota.sz = sysfs_quotas->sz; From patchwork Mon Feb 26 14:05:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 206665 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:a81b:b0:108:e6aa:91d0 with SMTP id bq27csp2101679dyb; Mon, 26 Feb 2024 06:12:54 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUREa3SjcI4qaM5FdUNmNvYdJpSke5VMwRHcuKGsC2/5TrOgiZosz1zAjdM4OQrm1Q+Y138nzf+aJHKtGfHlqxBasbJ9Q== X-Google-Smtp-Source: AGHT+IFCfYS1Ozg96IzDv4Gz0It/rV9EM8flH3G5UwACmr+mtYkHmlhbED8X868L0rHlBP65COMu X-Received: by 2002:a17:906:a1d5:b0:a3e:6d75:61df with SMTP id bx21-20020a170906a1d500b00a3e6d7561dfmr4869676ejb.45.1708956774236; Mon, 26 Feb 2024 06:12:54 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708956774; cv=pass; d=google.com; s=arc-20160816; b=uPKV8csA9SBQBlEVWwGuLXUplCNRrN6WiqzjR8GyMCF586hPsbkShNQkIwIZxHddN/ QZ36kzR82rid7N/i6tSn2r9yuBDtA+n6sZskPwskPD+VhPIH8mxmmT7g+ha7f6QsRLsF APc5M9p/ouZwjKxawF4dDssj2FAViiDDelQFYDnP8ydzALDvQ0BsMFdyy+jI8nlNKbwL VV8Tylxb5Fag+lPtSFB8SVmq6xFRUf+KHBXNJ/ADKiAp8Gtgc7PaVzsOWYDrE02KO6E9 v4OzyFApVemicXjLhlwKU8n6AAqgHlc74UCStbOlb6Lywq+zD68fTDWsLYVmE+C/joEg 1Lhg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=zXbLP2GSUHeXTzjP624Gn+oPXDlC4J4rQeiJ29BCBCE=; fh=egwJU2bb679B4Zi0WdxyKykgfACRkSkUGxyHY52POxM=; b=sjMyAbHotqe8j724JYUq2OVwEkT3yWrp4aGG1TPOypLHFA11Bysh5H7CKeuF7p5nEg E0GwlZ8JH9Brsblar7F8nD71l14IP/mL8vq37d8BLBiDXV2Ug4+0bPT1V0mlodcHL3RA TiZFYqReN/YJiMMcgnYNwPB+0wI2OKnyq0nbnwFEZPtgP3Gxb1tpAsH4P4QGYUCAlXmJ OmUh74Hzog9KQdw0yaBk+C05XaXgBmibkDDamLkNR0Cr30NTTh+IDG7bXaw5c3w2wNOm n2iarI11OyznrJGeiewGA6Anj4QTRMl/MGy8mBtyYhNqKkO65qJv/vr9ZBpPn9cX6dbh iseg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=sk.com); spf=pass (google.com: domain of linux-kernel+bounces-81586-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-81586-ouuuleilei=gmail.com@vger.kernel.org" Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id v9-20020a170906564900b00a3e52f5bee8si2114949ejr.1018.2024.02.26.06.12.54 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Feb 2024 06:12:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-81586-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=sk.com); spf=pass (google.com: domain of linux-kernel+bounces-81586-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-81586-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 1E7331F22A6F for ; Mon, 26 Feb 2024 14:11:34 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 93BD312EBE5; Mon, 26 Feb 2024 14:06:28 +0000 (UTC) Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A9B1712C819; Mon, 26 Feb 2024 14:06:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708956386; cv=none; b=jFOm5JhzzJkEpRw+VImrQ+nFNHGnsck0DYMwI1MkOCBLeGlw1O45XJQ2ut7JmASSruny3hznA0Ra9ns5+xsNqDlyzU+L3ZOjXpY2ZwJ2I1ptDeLBTNpFCZQbjp41w3enOztgfAFReaqv4xERmoXhPKpob0Mh4DTi2iPJX6u/HUg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708956386; c=relaxed/simple; bh=rwgghmTg7o4FR0Bv2KuCZkmTtPHv9JXSryRKMphOOMM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=X+dNMT/Vk+ENbb8+TCPs5edVVyZcZ9Cb+llQELWEwkuu9CjDs+0YziuTdZyJlMFtQZGB9mVLxHxFmC5MR2L20LwTEvgKIvBW4J1A4C3odNKDkHZu+qcJodgucBqmEXQPnfXG92Gz5bMHrpnYq8UXoMykdMF7o5G1kdZIS1jIztA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d6dff70000001748-93-65dc9ade7648 From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, lizhijian@cn.fujitsu.com, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com Subject: [PATCH v2 7/7] mm/damon/sysfs-schemes: apply target_nid for promote and demote actions Date: Mon, 26 Feb 2024 23:05:53 +0900 Message-ID: <20240226140555.1615-8-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240226140555.1615-1-honggyu.kim@sk.com> References: <20240226140555.1615-1-honggyu.kim@sk.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrBIsWRmVeSWpSXmKPExsXC9ZZnoe69WXdSDZa1cVtM7DGwmLN+DZvF rhshFv/3HmO0ePL/N6vFiZuNbBad35eyWFzeNYfN4t6a/6wWR9afZbFYdwvI2nz2DLPF4uVq Fvs6HjBZHP76hsli8qUFbBYvppxhtDg5azKLxeyj99gdhD3+H5zE7LH09Bs2jw1NQGLnrLvs Hi37brF7LNhU6tFy5C2rx+I9L5k8Nq3qZPPY9GkSu8eJGb9ZPHY+tPR4sXkmo0dv8zs2j8+b 5AL4o7hsUlJzMstSi/TtErgyPl+/wVTwQ76iqe8rWwNjr2QXIyeHhICJxLEtr1hh7L2rzzCC 2GwCahJXXk5i6mLk4BARcJBY9VWhi5GLg1mggUXix+dLLCA1wgLxEiu27wPrZRFQlTg77RqY zStgJrH9dhcjxExNicfbf7KD2JwC5hL/D7ezgcwUAqo5epsZolxQ4uTMJ2AjmQXkJZq3zmYG 2SUhcIpd4tSzU+wQcyQlDq64wTKBkX8Wkp5ZSHoWMDKtYhTKzCvLTczMMdHLqMzLrNBLzs/d xAiMv2W1f6J3MH66EHyIUYCDUYmH16H2TqoQa2JZcWXuIUYJDmYlEd5wmZupQrwpiZVVqUX5 8UWlOanFhxilOViUxHmNvpWnCAmkJ5akZqemFqQWwWSZODilGhgDSu/zu586uPSX6Afb/49c H9mJSL3S3Oza2bzPtvHlHn6HZ4efrk3U2mXIuNGh8s5Jmd4db1Y+nJi7XHX+Ufut0qt7jJb9 esMlvcxL6ajI/jlM4QdWZr8N3MNz/v+t4Cus5efixB2shdX+MbnsCYqb/UjEXDJ7ltbRPvNL joKHVk16Hv13+9JSJZbijERDLeai4kQAoSBEXbsCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprHIsWRmVeSWpSXmKPExsXCNUNLT/furDupBrPvWFlM7DGwmLN+DZvF rhshFv/3HmO0ePL/N6vFiZuNbBafn71mtuh88p3R4vDck6wWnd+Xslhc3jWHzeLemv+sFkfW n2WxWHcLyNp89gyzxeLlahaHrj1ntdjX8YDJ4vDXN0wWky8tYLN4MeUMo8XJWZNZLGYfvcfu IO7x/+AkZo+lp9+weWxoAhI7Z91l92jZd4vdY8GmUo+WI29ZPRbvecnksWlVJ5vHpk+T2D1O zPjN4rHzoaXHi80zGT16m9+xeXy77eGx+MUHpgDBKC6blNSczLLUIn27BK6Mz9dvMBX8kK9o 6vvK1sDYK9nFyMkhIWAisXf1GUYQm01ATeLKy0lMXYwcHCICDhKrvip0MXJxMAs0sEj8+HyJ BaRGWCBeYsX2fawgNouAqsTZadfAbF4BM4ntt7sYIWZqSjze/pMdxOYUMJf4f7idDWSmEFDN 0dvMEOWCEidnPgEbySwgL9G8dTbzBEaeWUhSs5CkFjAyrWIUycwry03MzDHVK87OqMzLrNBL zs/dxAiMtWW1fybuYPxy2f0QowAHoxIPr0PtnVQh1sSy4srcQ4wSHMxKIrzhMjdThXhTEiur Uovy44tKc1KLDzFKc7AoifN6hacmCAmkJ5akZqemFqQWwWSZODilGhjNfZR5/H8w5K5/oMaV Y7nh34Ppcn3CBydPUVNkPLBLYOv6U00bsl53xitqxr+L+F4TvfNi5YUdIfFxIqxS28oFOnQP JGv0b3lRYO4jc03guYyn96xV6R6z1vlvqTzA5jKD+9xvw00WryXCAn49CPO/LvWttOIS143F y+IzlzhrTombvf9R9SQlluKMREMt5qLiRACjXRGRsQIAAA== X-CFilter-Loop: Reflected X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791971058434992183 X-GMAIL-MSGID: 1791971058434992183 From: Hyeongtak Ji This patch changes DAMOS_PROMOTE and DAMOS_DEMOTE to use target_nid of sysfs as the destination NUMA node of migration. This has been tested on qemu as follows: $ cd /sys/kernel/mm/damon/admin/kdamonds/ $ cat contexts//schemes//action promote $ echo 1 > contexts//schemes//target_nid $ echo commit > state $ numactl -p 2 ./hot_cold 500M 600M & $ numastat -c -p hot_cold Per-node process memory usage (in MBs) PID Node 0 Node 1 Node 2 Total -------------- ------ ------ ------ ----- 701 (hot_cold) 0 501 601 1101 Signed-off-by: Hyeongtak Ji Signed-off-by: Honggyu Kim --- mm/damon/paddr.c | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 37a7b34a36dd..5e057a69464f 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -240,9 +240,9 @@ enum migration_mode { */ static unsigned int migrate_folio_list(struct list_head *migrate_folios, struct pglist_data *pgdat, - enum migration_mode mm) + enum migration_mode mm, + int target_nid) { - int target_nid; unsigned int nr_succeeded; nodemask_t allowed_mask; int reason; @@ -250,12 +250,14 @@ static unsigned int migrate_folio_list(struct list_head *migrate_folios, switch (mm) { case MIG_PROMOTE: - target_nid = next_promotion_node(pgdat->node_id); + if (target_nid == NUMA_NO_NODE) + target_nid = next_promotion_node(pgdat->node_id); reason = MR_PROMOTION; vm_event = PGPROMOTE; break; case MIG_DEMOTE: - target_nid = next_demotion_node(pgdat->node_id); + if (target_nid == NUMA_NO_NODE) + target_nid = next_demotion_node(pgdat->node_id); reason = MR_DEMOTION; vm_event = PGDEMOTE_DIRECT; break; @@ -358,7 +360,8 @@ static enum folio_references folio_check_references(struct folio *folio) */ static unsigned int damon_pa_migrate_folio_list(struct list_head *folio_list, struct pglist_data *pgdat, - enum migration_mode mm) + enum migration_mode mm, + int target_nid) { unsigned int nr_migrated = 0; struct folio *folio; @@ -399,7 +402,7 @@ static unsigned int damon_pa_migrate_folio_list(struct list_head *folio_list, /* 'folio_list' is always empty here */ /* Migrate folios selected for migration */ - nr_migrated += migrate_folio_list(&migrate_folios, pgdat, mm); + nr_migrated += migrate_folio_list(&migrate_folios, pgdat, mm, target_nid); /* Folios that could not be migrated are still in @migrate_folios */ if (!list_empty(&migrate_folios)) { /* Folios which weren't migrated go back on @folio_list */ @@ -426,7 +429,8 @@ static unsigned int damon_pa_migrate_folio_list(struct list_head *folio_list, * common function for both cases. */ static unsigned long damon_pa_migrate_pages(struct list_head *folio_list, - enum migration_mode mm) + enum migration_mode mm, + int target_nid) { int nid; unsigned int nr_migrated = 0; @@ -449,12 +453,14 @@ static unsigned long damon_pa_migrate_pages(struct list_head *folio_list, } nr_migrated += damon_pa_migrate_folio_list(&node_folio_list, - NODE_DATA(nid), mm); + NODE_DATA(nid), mm, + target_nid); nid = folio_nid(lru_to_folio(folio_list)); } while (!list_empty(folio_list)); nr_migrated += damon_pa_migrate_folio_list(&node_folio_list, - NODE_DATA(nid), mm); + NODE_DATA(nid), mm, + target_nid); memalloc_noreclaim_restore(noreclaim_flag); @@ -499,7 +505,8 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, break; case MIG_PROMOTE: case MIG_DEMOTE: - applied = damon_pa_migrate_pages(&folio_list, mm); + applied = damon_pa_migrate_pages(&folio_list, mm, + s->target_nid); break; default: /* Unexpected migration mode. */