From patchwork Tue Nov 22 20:38:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mina Almasry X-Patchwork-Id: 24553 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2428686wrr; Tue, 22 Nov 2022 12:40:49 -0800 (PST) X-Google-Smtp-Source: AA0mqf7DI+TRywupHVen93gfNhhMTFXleUWGDvEkzpcpqvi5C3oEOR4mUEB9CxBCmxp6QqWyAWRJ X-Received: by 2002:a17:902:efcf:b0:186:9cce:c59 with SMTP id ja15-20020a170902efcf00b001869cce0c59mr6282060plb.120.1669149649630; Tue, 22 Nov 2022 12:40:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669149649; cv=none; d=google.com; s=arc-20160816; b=ShDJUDhscJbzdFuvIzl/WZlTCs+Meyc6mi/Q5cBMoX8p/uYQ8BAZojZpTgH1l6d+WY j+4m1Wla+pYClbkuCArBuWCfv+b+5W0Tgj+BT28iLSHnwhsmg8gSIC0kz5LCVwmI/udb x+s7Vf7mlZfihpaS8TDF+Qi7udjp6hlSYvwPo+GDOlqvRN5mnnHKDWiytVsnDmOhWWA2 V+Gty57YFybCRGdf8kGCevbwMVy9taV/rm0HeMD96gTR8oRyamJAxzIzF9LAO/BcKWdt 5pJ7Mik2qKIprBNhBl96pIdU1DmFY0dVFBsnUs48el7fz/4pUqMTlFV/jomQUTtm+czy JMlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=mH2xKyLbtphuPdUJ732OjxOQc7w7f5rhMLqPKs8ZczA=; b=0Bxba6C7YGMG1rRwP4gPMj67NeO/HNVdAlhVWJ1hCmoruszyrsS7trixUogclUFf1C j2i3b7N+9IMJKGK97NYMFSqfLg4/Ss+TzLrqcJY6pmDYPRd3Zh/5r4aRSDG+j1VmZ8nM P45cnKjBwPJL78pP5EJ3OLJ1xqh0x+r5Of6qe10mMJCT77Vzb6PL8avGUSX4ZuvysGls sxL1iB8x0DxYee7cct46G8ds2LFG1MKP13VuaOPnk0U4wWsZoH5XbCFB6oTmQTY8vi91 DwttXSPzxaVJ5EnKU7gej3qJaERbRbXyp8f/eGiE1882qQxABVRfEaDVXRv1b/XpvKxJ Q/PA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=qdsmec8o; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id a10-20020a65640a000000b0044c3ec9ea71si870959pgv.630.2022.11.22.12.40.36; Tue, 22 Nov 2022 12:40:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=qdsmec8o; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234739AbiKVUjK (ORCPT + 99 others); Tue, 22 Nov 2022 15:39:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234512AbiKVUjF (ORCPT ); Tue, 22 Nov 2022 15:39:05 -0500 Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A601D697C5 for ; Tue, 22 Nov 2022 12:39:04 -0800 (PST) Received: by mail-pg1-x54a.google.com with SMTP id p7-20020a631e47000000b0047691854a86so9085055pgm.16 for ; Tue, 22 Nov 2022 12:39:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=mH2xKyLbtphuPdUJ732OjxOQc7w7f5rhMLqPKs8ZczA=; b=qdsmec8olNuJkHjim4tCNOzdWa/frQp8m5lufYNaqejkZMLoMUUyW+CWmAkGziybpf XVtKd/0LwOGTSKPzuW0laRVzBchDj/eYAtuTCSNB8IMPYcpApfCWC2zjH37lPwrTquSo e6VwPwg/ku5WTWy2ImXR0wgUFLZ1Bya8Re9NroqrrjtTrQdgeKHCixYvONLr+VBOSK7D y/z7y3/mili8PPvIi+kePEDEmzvJZh09v92ohyzbQ5RuROFwKw5KgQsz7VHG+PzHsW5E 1yg6L/6VSUWRD0wFpHrfizjK3o+Nif6zqdPMbBMhrgAWMlNgfhUP/5lOq0C7Wj8y1WM3 OO7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=mH2xKyLbtphuPdUJ732OjxOQc7w7f5rhMLqPKs8ZczA=; b=VNk9QvOUjkdQ3NPvDnKGiIxw1VSM8zh5+H5tySS9TVbIT98Eceplu2QuyEWnywb6on Ltb0DLHx7+SxoTDB1i1JPIU//jja+rf1OCGDopXWxEkTq+pq924o5QlmNMmeFETrTIMt GA0o3pYzJs4GdG5MmebvYYf6vRj0TAXS70EX/XsUeYXq3iZ2FzOKpdCdvFAIQhwn/agZ ZGpf8dikdoJkf2f6Toii/EqNI3Nx6nZOAydKTgaCixIiAoLM3fqogXz1sMduQTXFy74v aUSVfOFJcCYwAtvaGWsIYsbgu4xd0nPVNOIeWA6trhfkO1WFaLOmeJGElmhE7177cG0z o2Og== X-Gm-Message-State: ANoB5pn99rWThp9cfFqM9CZbaRfgv3Eh0xeqwJfRjYQKRPS4FV41SQYp X9RHZ0nkJ5uMDRbIA5dvwC7o3iTwc2453oIDbw== X-Received: from almasrymina.svl.corp.google.com ([2620:15c:2d4:203:b7d0:497d:abe4:3e6f]) (user=almasrymina job=sendgmr) by 2002:a63:7143:0:b0:46f:918e:7338 with SMTP id b3-20020a637143000000b0046f918e7338mr6371506pgn.8.1669149544152; Tue, 22 Nov 2022 12:39:04 -0800 (PST) Date: Tue, 22 Nov 2022 12:38:46 -0800 In-Reply-To: <20221122203850.2765015-1-almasrymina@google.com> Mime-Version: 1.0 References: <20221122203850.2765015-1-almasrymina@google.com> X-Mailer: git-send-email 2.38.1.584.g0f3c55d4c2-goog Message-ID: <20221122203850.2765015-2-almasrymina@google.com> Subject: [RFC PATCH v1] mm: Add memory.demote for proactive demotion only From: Mina Almasry To: Huang Ying , Yang Shi , Yosry Ahmed , Tim Chen , weixugc@google.com, shakeelb@google.com, gthelen@google.com, fvdl@google.com, Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Andrew Morton Cc: Mina Almasry , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750230262874346638?= X-GMAIL-MSGID: =?utf-8?q?1750230262874346638?= Add the proactive demotion interface memory.demote. This interface can be used as follows: echo "1m" > memory.demote At this command the kernel will attempt to demote 1m of memory from this cgroup. The kernel may not be able to demote the full amount requested by the userspace and in that case EAGAIN would be returned to the user (similar to memory.reclaim). The kernel will only attempt to demote pages with this interface. It will not attempt any other kind of reclaim (swap, writeback or reclaiming clean file pages). Signed-off-by: Mina Almasry --- mm/memcontrol.c | 38 ++++++++++++++++++++++++++++++++++++++ mm/vmscan.c | 18 ++++++++++++++---- 2 files changed, 52 insertions(+), 4 deletions(-) -- 2.38.1.584.g0f3c55d4c2-goog diff --git a/mm/memcontrol.c b/mm/memcontrol.c index fd4ff1c865a2..427c79e467eb 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -6623,6 +6623,39 @@ static ssize_t memory_reclaim(struct kernfs_open_file *of, char *buf, return nbytes; } +static ssize_t memory_demote(struct kernfs_open_file *of, char *buf, + size_t nbytes, loff_t off) +{ + struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of)); + unsigned int nr_retries = MAX_RECLAIM_RETRIES; + unsigned long nr_to_demote, nr_demoted = 0; + unsigned int reclaim_options = MEMCG_RECLAIM_ONLY_DEMOTE; + int err; + + buf = strstrip(buf); + err = page_counter_memparse(buf, "", &nr_to_demote); + if (err) + return err; + + while (nr_demoted < nr_to_demote) { + unsigned long demoted; + + if (signal_pending(current)) + return -EINTR; + + demoted = try_to_free_mem_cgroup_pages( + memcg, nr_to_demote - nr_demoted, GFP_KERNEL, + reclaim_options); + + if (!demoted && !nr_retries--) + return -EAGAIN; + + nr_demoted += demoted; + } + + return nbytes; +} + static struct cftype memory_files[] = { { .name = "current", @@ -6691,6 +6724,11 @@ static struct cftype memory_files[] = { .flags = CFTYPE_NS_DELEGATABLE, .write = memory_reclaim, }, + { + .name = "demote", + .flags = CFTYPE_NS_DELEGATABLE, + .write = memory_demote, + }, { } /* terminate */ }; diff --git a/mm/vmscan.c b/mm/vmscan.c index dea05ad8ece5..8c1f5416d789 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1657,12 +1657,13 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, LIST_HEAD(demote_folios); unsigned int nr_reclaimed = 0; unsigned int pgactivate = 0; - bool do_demote_pass; + bool do_demote_pass, only_demote_pass; struct swap_iocb *plug = NULL; memset(stat, 0, sizeof(*stat)); cond_resched(); do_demote_pass = can_demote(pgdat->node_id, sc); + only_demote_pass = sc->demotion == 2; retry: while (!list_empty(folio_list)) { @@ -2091,10 +2092,19 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, nr_reclaimed += demote_folio_list(&demote_folios, pgdat); /* Folios that could not be demoted are still in @demote_folios */ if (!list_empty(&demote_folios)) { - /* Folios which weren't demoted go back on @folio_list for retry: */ + /* + * Folios which weren't demoted go back on @folio_list. + */ list_splice_init(&demote_folios, folio_list); - do_demote_pass = false; - goto retry; + + /* + * goto retry to reclaim the undemoted folios in folio_list if + * desired. + */ + if (!only_demote_pass) { + do_demote_pass = false; + goto retry; + } } pgactivate = stat->nr_activate[0] + stat->nr_activate[1]; From patchwork Tue Nov 22 20:38:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mina Almasry X-Patchwork-Id: 24554 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2428706wrr; Tue, 22 Nov 2022 12:40:53 -0800 (PST) X-Google-Smtp-Source: AA0mqf6M04hBlpnxxbEp4vd6Hmns81OfWfcsXB3h/CJhW5TApVTUjIukauX3AuixdBjv6DMzf5Wl X-Received: by 2002:a63:b16:0:b0:46e:c052:a6b0 with SMTP id 22-20020a630b16000000b0046ec052a6b0mr4781243pgl.91.1669149653056; Tue, 22 Nov 2022 12:40:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669149653; cv=none; d=google.com; s=arc-20160816; b=zPp9sxq9Uklm1ao/IpjVGAq2gGBr1ZuEfoUI/e11isG+InVMvbXdwiu6HOwjErr8WA YNLDzqd6KT4IWB9RTwr8IfALdZwS5kUNrIx2BQmRxjFWJo4fX83gfBYFDAlWkBdPUPLw bdX5n9u3EQrqnN4LSLJ3B166EDk2S3/Bs8kkBGhTICrJ0zKTYHhzt4sDpCDpwXZzmZkZ kPyXWWy+vtLHcpBcfaM54qk8mTKgvZ5O+B17y2qz8ws94+c5b58zlyJZEC0d8bnqNMQO r8gWvWVlNRT3ErFGDKmV/XthsQCqnpk65Fb9Z1HgRU0DcjSmWh2+t9hJ/7HyWB8yN46l gr9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=FPSzCF4B4ZUUfFv+NywISnCJ1Cav2tvMYf25Hqqvvgg=; b=b8AOccZFkrNkqkvPCs6RoZkPSQLGDnxFeM2DpEaZXPCom2pdSvfRzOcMymjBfwMixQ RQNLw/bKcT5AtEwAzUezAUiAP9MSAkd+etBtAGnMyaTR08KAN80IxUZmHy0Nn2pi3Ulq 1VLwNu2pBYGpn41PFeN+3Bj2kiXdgbTHmwQaVehCUBMB/PQ7Qi8lCs11SOXL9abhjjCd 9QYTKGkEgeOs0Ex0R6Cqez4+uCfiKs+5bZCQvbi+xJHHscn31s9tloYmqivq3wKIFGlc FhNvdXmmRqhQ1Y1M7Zb23Usao9OiLXRtzTeerY5pi6wGaLUtJgfpbACGujxUoo9xSX3G ktdQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b="Ylw0t/TQ"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id v13-20020a63464d000000b004774c91418dsi9925480pgk.523.2022.11.22.12.40.38; Tue, 22 Nov 2022 12:40:53 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b="Ylw0t/TQ"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234899AbiKVUjW (ORCPT + 99 others); Tue, 22 Nov 2022 15:39:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45400 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234777AbiKVUjN (ORCPT ); Tue, 22 Nov 2022 15:39:13 -0500 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B5AF868C75 for ; Tue, 22 Nov 2022 12:39:11 -0800 (PST) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-393c38f7bcfso128045147b3.5 for ; Tue, 22 Nov 2022 12:39:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=FPSzCF4B4ZUUfFv+NywISnCJ1Cav2tvMYf25Hqqvvgg=; b=Ylw0t/TQAkkJ5ZZ1Pjx3hAcFXh/aS80QxJujdVf6jQSIZZqenDDpuUnVzgNX6ILpOT D+OjVdmtDW5/tAPK26Gb1zGy/ZoTCNYUJRF0pyT+nwSM+/hfu38zrqKsPnMqXGmEd+xx cuUwYrnWdEWlYYI+NWRholXeyBpeozHwyScvQAjTUOFxdVVobSIfg3zrrhjVGUP9l7Px HcnmajR3YLpKxiUY3qDInKNDQjuM2qZP0Bd8vfd5A7IwOfoP5RK2bZZ11acFXKOU83kT GwdUHROVxNzf/nQ9nlj5eX8S+GZeK8I5hMFTQ/pyIzMVhZn83FC27sX4jgh9Me5r10Ge mRkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=FPSzCF4B4ZUUfFv+NywISnCJ1Cav2tvMYf25Hqqvvgg=; b=hSg9oDOuTDIY7qYkqZIMmukoYNlf5GISiD+Eo0Ml7qK6I4frZ+gk3dB5UWFRXqFhS2 wLVT8pl+UZEaHlELig428vlV/h7/+I57VR5FMA5ZI8Gl0r4X2paNT/Thj9NCDWjTYfkz MFnRfxPMW5Ljzo3VsSZTd3M9qBd+rCXW1kvn0ZmveulmR+DrkoFyQkvt/MUrZbxULF/k V9TRny6uySrE9/yIDcppPpTTvxlgrLl92nwbbyDod163xOVrK75bPJih4nK6Su1T+8/d z2rcKfXmkwnz99TFpM8sr2CLX5FNPKI7pEA47Rkf9V/hSai3OW6+AnFXEqMxxUgBTQBH FUmA== X-Gm-Message-State: ANoB5pnMtnBKeBnTDw5t83zbimYs/3L+zf6iW5RHqONxqtylzbIExPoA TaamPS/HDvUtmBxelO2ajh91Z2O0epb1nIy52Q== X-Received: from almasrymina.svl.corp.google.com ([2620:15c:2d4:203:b7d0:497d:abe4:3e6f]) (user=almasrymina job=sendgmr) by 2002:a25:cccc:0:b0:6de:afe7:1c27 with SMTP id l195-20020a25cccc000000b006deafe71c27mr2ybf.642.1669149550378; Tue, 22 Nov 2022 12:39:10 -0800 (PST) Date: Tue, 22 Nov 2022 12:38:47 -0800 In-Reply-To: <20221122203850.2765015-1-almasrymina@google.com> Mime-Version: 1.0 References: <20221122203850.2765015-1-almasrymina@google.com> X-Mailer: git-send-email 2.38.1.584.g0f3c55d4c2-goog Message-ID: <20221122203850.2765015-3-almasrymina@google.com> Subject: [RFC PATCH v1 3/4] mm: Fix demotion-only scanning anon pages From: Mina Almasry To: Huang Ying , Yang Shi , Yosry Ahmed , Tim Chen , weixugc@google.com, shakeelb@google.com, gthelen@google.com, fvdl@google.com, Andrew Morton Cc: Mina Almasry , Muchun Song , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750230266855953067?= X-GMAIL-MSGID: =?utf-8?q?1750230266855953067?= This is likely a missed change from commit a2a36488a61c ("mm/vmscan: Consider anonymous pages without swap"). Current logic is if !may_swap _or_ !can_reclaim_anon_pages() then we don't scan anon memory. This should be an 'and'. We would like to scan anon memory if we may swap or if we can_reclaim_anon_pages(). Fixes: commit a2a36488a61c ("mm/vmscan: Consider anonymous pages without swap") Signed-off-by: Mina Almasry --- mm/vmscan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 2.38.1.584.g0f3c55d4c2-goog diff --git a/mm/vmscan.c b/mm/vmscan.c index 8c1f5416d789..d7e509b3f07f 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2931,7 +2931,7 @@ static void get_scan_count(struct lruvec *lruvec, struct scan_control *sc, enum lru_list lru; /* If we have no swap space, do not bother scanning anon folios. */ - if (!sc->may_swap || !can_reclaim_anon_pages(memcg, pgdat->node_id, sc)) { + if (!sc->may_swap && !can_reclaim_anon_pages(memcg, pgdat->node_id, sc)) { scan_balance = SCAN_FILE; goto out; } From patchwork Tue Nov 22 20:38:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mina Almasry X-Patchwork-Id: 24555 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp2428705wrr; Tue, 22 Nov 2022 12:40:53 -0800 (PST) X-Google-Smtp-Source: AA0mqf4uRrQrfwvW6m4ir7hDwKmdvDFq1Fc799FW4Jy8xkO14KRpfo4UoYr+iD9EnRUu91idzUN3 X-Received: by 2002:a62:ea18:0:b0:56c:2d:1e56 with SMTP id t24-20020a62ea18000000b0056c002d1e56mr5619430pfh.41.1669149653035; Tue, 22 Nov 2022 12:40:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669149653; cv=none; d=google.com; s=arc-20160816; b=rrMJxrCeDff77q9i6sO1HfBCg4jCCFBwrgiVnAqGmOxCyXT9qOmPFPD21PNtwnV3JA ++LyqyvzZJU+lMpEO9uWa3vhBZAJLuJmgOWZZkdFvrooz34v19YOQNSk6zFx9OVH3Izq 3qlyBuEJeQzb83CTU3m4dIHhiB3RVx7aOwpNUJ/2fH4Fa7CDGQAgT+KCx2cYQcEJpL5A sc+RGuwqueXJbtcEmt0Wz6TG4CLVYAc/AWkipAFSsVUuG/OuA4MnMPPcGMxAbpH6oxWp yum9J9L0Dr6bgvvaB2bHvGiguYAN4IBUgq5GK+OyDUxrvVSu8y21Sw9gKIgJ2jRimH9j s4SQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=BSER/B4DnSDEXzptnYVRvY0Bxd0pX06XJiUIMZfJs1c=; b=K7ayD1Xwu/RcxVbG5E0xbBkTvxucpwciirNmK6TPW8HNkeTmWrU0NuZpVBjptdgQSt xGPJaKfvsuNGsI19EaPHL0m/ig2nAB+tMBNrpxCJL0o7dcXLP6suxDsOi/PEEtkqoyij UZ8c4hBYnzZu9CoHNFUl988MZEYyCbkBdlB8nhcsn+l9azmM7hBPSSeGrKZ5ZgjoG+63 lKsjzsVPnD1Tnp3xHPpcYdLWc0SVz6P0wJYX5iOibQlnSr2zk92N4OYaiG3pVLJTkfwJ pThAa+FZphE8kJ9NxdmBLFSKW0/muzlfthAp3KEQf0vJRnbyX/u00F3WCMJF6t67MvsB E/fA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=Q8CGtJa+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j11-20020a63cf0b000000b0046249619a7fsi14573545pgg.553.2022.11.22.12.40.39; Tue, 22 Nov 2022 12:40:53 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=Q8CGtJa+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234818AbiKVUj0 (ORCPT + 99 others); Tue, 22 Nov 2022 15:39:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234809AbiKVUjS (ORCPT ); Tue, 22 Nov 2022 15:39:18 -0500 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 90AF1697EA for ; Tue, 22 Nov 2022 12:39:16 -0800 (PST) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-352e29ff8c2so151581217b3.21 for ; Tue, 22 Nov 2022 12:39:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=BSER/B4DnSDEXzptnYVRvY0Bxd0pX06XJiUIMZfJs1c=; b=Q8CGtJa+O00lg9Tqtd2xc0jsOMGOiJJRQKDl+HbQ4+RJuR4NE+I2vLParTPRJyK9dV HeSYxODpIYH636a5cGLJjdX3SPGMLDqJjaFcJ39VsuEELq4DgDIZX//2wGqsFd51QSJ+ AZM/FTrtKnbmUqDMcYElZ3IbHtiXrin84UOmwRdMpGISPBEI8woUDxAgbVU4ym/RyNK7 VWuGFbedYwWGh89zIthlnJvT8GU/8qlI9jP4fZERGstjvqu1lGm4VzeB6c1hICq2esGq a6HaxwII49NO4LMfOWNuAy3MP55sIxndD9RIDojCzutzGpi9VdKps5EppZgBJUct3an9 k1NA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=BSER/B4DnSDEXzptnYVRvY0Bxd0pX06XJiUIMZfJs1c=; b=aMS0hs/aCTOXXbbpCdmg9nn71/RBa5QRfveLYwMiEQnk5GijH0yvqHfw/rmaREvH7N JTge7WMlI1VprfmPDcIVaHiCGYgJJLCHhGEly5hzyhA40WgXufDgG75aJSjz3R1tj5kQ wTU4z2oBbkknUFmsLmdyCPWOrutbc6WozCL2vAXf+RsxzXxdo4gurcplQif425vA48Vr ouGm9vnzjQnNbjjqqSc7bzL31cC4riv4zZjItMdyGEGncdCq4vzZwpAQHU5K6hzuqWsO VMQSgumuIEuyiemv6ksdDvNOpi0pB3ENz21KkVxszsUBpr8ZRtLMv/o2znttBKtuLIOE pcNA== X-Gm-Message-State: ANoB5plmx14KtiGvtvLxCI94Ibs4LfZodTKqhjatB9shjkLuoyDMPIQK oPno04d9Vi/h+o7UlZRF7VTo8CWzNxZt1mv4KA== X-Received: from almasrymina.svl.corp.google.com ([2620:15c:2d4:203:b7d0:497d:abe4:3e6f]) (user=almasrymina job=sendgmr) by 2002:a25:7450:0:b0:6ee:8d5a:3bca with SMTP id p77-20020a257450000000b006ee8d5a3bcamr4305851ybc.300.1669149555882; Tue, 22 Nov 2022 12:39:15 -0800 (PST) Date: Tue, 22 Nov 2022 12:38:48 -0800 In-Reply-To: <20221122203850.2765015-1-almasrymina@google.com> Mime-Version: 1.0 References: <20221122203850.2765015-1-almasrymina@google.com> X-Mailer: git-send-email 2.38.1.584.g0f3c55d4c2-goog Message-ID: <20221122203850.2765015-4-almasrymina@google.com> Subject: [RFC PATCH v1 4/4] mm: Add nodes= arg to memory.demote From: Mina Almasry To: Huang Ying , Yang Shi , Yosry Ahmed , Tim Chen , weixugc@google.com, shakeelb@google.com, gthelen@google.com, fvdl@google.com, Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Andrew Morton Cc: Mina Almasry , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750230266654339068?= X-GMAIL-MSGID: =?utf-8?q?1750230266654339068?= The nodes= arg instructs the kernel to only scan the given nodes for demotion. For example use cases, consider a 3 tier memory system: nodes 0,1 -> top tier nodes 2,3 -> second tier nodes 4,5 -> third tier echo "1m nodes=2,3" > memory.demote This instructs the kernel to attempt to demote 1m memory in the second tier to the third, which can be desirable according to the userspace policy if the second tier is filling up and there is available memory on the third tier. echo "1m" > memory.demote Instructs the kernel to attempt to demote 1m of memory (regardless of which tier the memory is currently on). echo "1m nodes=0,1" Instructs the kernel to demote memory from the top tier nodes, which can be desirable according to the userspace policy if there is pressure on the top tiers. Signed-off-by: Mina Almasry --- include/linux/swap.h | 3 ++- mm/memcontrol.c | 64 ++++++++++++++++++++++++++++++++++++-------- mm/vmscan.c | 4 ++- 3 files changed, 58 insertions(+), 13 deletions(-) -- 2.38.1.584.g0f3c55d4c2-goog diff --git a/include/linux/swap.h b/include/linux/swap.h index f768171c2dc2..e195ee5f8efb 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -425,7 +425,8 @@ extern unsigned long try_to_free_pages(struct zonelist *zonelist, int order, extern unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg, unsigned long nr_pages, gfp_t gfp_mask, - unsigned int reclaim_options); + unsigned int reclaim_options, + nodemask_t nodemask); extern unsigned long mem_cgroup_shrink_node(struct mem_cgroup *mem, gfp_t gfp_mask, bool noswap, pg_data_t *pgdat, diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 427c79e467eb..cce446348358 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -63,6 +63,7 @@ #include #include #include +#include #include "internal.h" #include #include @@ -2392,7 +2393,8 @@ static unsigned long reclaim_high(struct mem_cgroup *memcg, psi_memstall_enter(&pflags); nr_reclaimed += try_to_free_mem_cgroup_pages(memcg, nr_pages, gfp_mask, - MEMCG_RECLAIM_DEFAULT); + MEMCG_RECLAIM_DEFAULT, + NODE_MASK_ALL); psi_memstall_leave(&pflags); } while ((memcg = parent_mem_cgroup(memcg)) && !mem_cgroup_is_root(memcg)); @@ -2683,7 +2685,8 @@ static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask, psi_memstall_enter(&pflags); nr_reclaimed = try_to_free_mem_cgroup_pages(mem_over_limit, nr_pages, - gfp_mask, reclaim_options); + gfp_mask, reclaim_options, + NODE_MASK_ALL); psi_memstall_leave(&pflags); if (mem_cgroup_margin(mem_over_limit) >= nr_pages) @@ -3504,7 +3507,8 @@ static int mem_cgroup_resize_max(struct mem_cgroup *memcg, if (!try_to_free_mem_cgroup_pages(memcg, 1, GFP_KERNEL, memsw ? MEMCG_RECLAIM_NO_SWAP : - MEMCG_RECLAIM_DEFAULT)) { + MEMCG_RECLAIM_DEFAULT, + NODE_MASK_ALL)) { ret = -EBUSY; break; } @@ -3615,7 +3619,8 @@ static int mem_cgroup_force_empty(struct mem_cgroup *memcg) return -EINTR; if (!try_to_free_mem_cgroup_pages(memcg, 1, GFP_KERNEL, - MEMCG_RECLAIM_DEFAULT)) + MEMCG_RECLAIM_DEFAULT, + NODE_MASK_ALL)) nr_retries--; } @@ -6408,7 +6413,8 @@ static ssize_t memory_high_write(struct kernfs_open_file *of, } reclaimed = try_to_free_mem_cgroup_pages(memcg, nr_pages - high, - GFP_KERNEL, MEMCG_RECLAIM_DEFAULT); + GFP_KERNEL, MEMCG_RECLAIM_DEFAULT, + NODE_MASK_ALL); if (!reclaimed && !nr_retries--) break; @@ -6457,7 +6463,8 @@ static ssize_t memory_max_write(struct kernfs_open_file *of, if (nr_reclaims) { if (!try_to_free_mem_cgroup_pages(memcg, nr_pages - max, - GFP_KERNEL, MEMCG_RECLAIM_DEFAULT)) + GFP_KERNEL, MEMCG_RECLAIM_DEFAULT, + NODE_MASK_ALL)) nr_reclaims--; continue; } @@ -6612,7 +6619,8 @@ static ssize_t memory_reclaim(struct kernfs_open_file *of, char *buf, reclaimed = try_to_free_mem_cgroup_pages(memcg, nr_to_reclaim - nr_reclaimed, - GFP_KERNEL, reclaim_options); + GFP_KERNEL, reclaim_options, + NODE_MASK_ALL); if (!reclaimed && !nr_retries--) return -EAGAIN; @@ -6623,6 +6631,16 @@ static ssize_t memory_reclaim(struct kernfs_open_file *of, char *buf, return nbytes; } +enum { + MEMORY_DEMOTE_NODES = 0, + MEMORY_DEMOTE_NULL, +}; + +static const match_table_t if_tokens = { + { MEMORY_DEMOTE_NODES, "nodes=%s" }, + { MEMORY_DEMOTE_NULL, NULL }, +}; + static ssize_t memory_demote(struct kernfs_open_file *of, char *buf, size_t nbytes, loff_t off) { @@ -6631,11 +6649,35 @@ static ssize_t memory_demote(struct kernfs_open_file *of, char *buf, unsigned long nr_to_demote, nr_demoted = 0; unsigned int reclaim_options = MEMCG_RECLAIM_ONLY_DEMOTE; int err; + char *old_buf, *start; + substring_t args[MAX_OPT_ARGS]; + int token; + char value[256]; + nodemask_t nodemask = NODE_MASK_ALL; buf = strstrip(buf); - err = page_counter_memparse(buf, "", &nr_to_demote); - if (err) - return err; + old_buf = buf; + nr_to_demote = memparse(buf, &buf) / PAGE_SIZE; + if (buf == old_buf) + return -EINVAL; + + buf = strstrip(buf); + + while ((start = strsep(&buf, " ")) != NULL) { + if (!strlen(start)) + continue; + token = match_token(start, if_tokens, args); + match_strlcpy(value, args, sizeof(value)); + switch (token) { + case MEMORY_DEMOTE_NODES: + err = nodelist_parse(value, nodemask); + if (err < 0) + return -EINVAL; + break; + default: + return -EINVAL; + } + } while (nr_demoted < nr_to_demote) { unsigned long demoted; @@ -6645,7 +6687,7 @@ static ssize_t memory_demote(struct kernfs_open_file *of, char *buf, demoted = try_to_free_mem_cgroup_pages( memcg, nr_to_demote - nr_demoted, GFP_KERNEL, - reclaim_options); + reclaim_options, nodemask); if (!demoted && !nr_retries--) return -EAGAIN; diff --git a/mm/vmscan.c b/mm/vmscan.c index d7e509b3f07f..df5ade259b3a 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -6719,7 +6719,8 @@ unsigned long mem_cgroup_shrink_node(struct mem_cgroup *memcg, unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg, unsigned long nr_pages, gfp_t gfp_mask, - unsigned int reclaim_options) + unsigned int reclaim_options, + nodemask_t nodemask) { unsigned long nr_reclaimed; unsigned int noreclaim_flag; @@ -6734,6 +6735,7 @@ unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg, .may_unmap = 1, .may_swap = !!(reclaim_options & MEMCG_RECLAIM_MAY_SWAP), .proactive = !!(reclaim_options & MEMCG_RECLAIM_PROACTIVE), + .nodemask = &nodemask, }; /* * Traverse the ZONELIST_FALLBACK zonelist of the current node to put