From patchwork Tue Nov 15 18:28:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 20501 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp2882794wru; Tue, 15 Nov 2022 10:34:41 -0800 (PST) X-Google-Smtp-Source: AA0mqf60MzeS7/Ny+GhsMoWp19kQa74zoaQV07z/FL+iplFfyPznrh1MKIkcWRWY6JLn/2FMnjvX X-Received: by 2002:a63:5c46:0:b0:43c:a5cb:5d1b with SMTP id n6-20020a635c46000000b0043ca5cb5d1bmr17606026pgm.134.1668537281699; Tue, 15 Nov 2022 10:34:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668537281; cv=none; d=google.com; s=arc-20160816; b=Vu1KMaukVSRfDDZnmuvsgLX2kewfE97H2S0zV029mZYjZJcD0HR7krQoyayESFXAwj WW/5hGBecMqC07uS7Iyo4JZAdH1cuw6O9Lr/1c+GGxgHYNfqIDxZiQtInCgFA0UKgEHV 9Cts2KQENb9bZt+I0s4gLITXN3q6A3eeWiXGPYpKofxHIrOESUcMvnq3YOb2i0/Z1lmU f95fAzqvxD7EZrF9sxMjhPA8TWeBnindNvVtVq9GP7tUuiPPNxMM/gcA7vAsmqCKrhU3 mn3vuDTFuNzurbj54OsEzFTN9kgelvUJSep3kTctxxUnSlSehS1wCoae2Bpq1jNVv9KT iBSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=npFjL6Dds3DKnQo07iUTlbqXMaucW9/yvERNl8ZNQRo=; b=FEPuNDVHHeAnD2NhUWxV4mytOjA2Ou+PfQwa2swjiSMqLW3bGMAHAy5ZIpJ++gJz6j ikjpEg3I/mGXkBhouh4hY1gGmnhi6Ed+XLiK6fruiM8p3oP7jFeQAZ9xlyCsLli0/u93 xJBFcwUOMSDprft6ywk+XlO61vuUH6vsU/pvzSaK9ZZf0S2DqOPnNyq8l9+hNMeGY6Xe S0tFaqTAFReNogUnodtjz7jvAHRw3Cx9r2QxgQ2I/O2hkSiUt3CZIO3dD4IWLdPXFq5c 3Qp1xZp0+SEsojZ7+nSEe0hqHqmuS/+oRGThgo0aUNtDMqd6D+0rwZwf5Qp49sKy6RQ6 vjtA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=NPDHr0Ql; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t18-20020a63dd12000000b004405047579csi12723471pgg.540.2022.11.15.10.34.27; Tue, 15 Nov 2022 10:34:41 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=NPDHr0Ql; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231540AbiKOS3b (ORCPT + 99 others); Tue, 15 Nov 2022 13:29:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39182 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231759AbiKOS3N (ORCPT ); Tue, 15 Nov 2022 13:29:13 -0500 Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F205F2FFD8 for ; Tue, 15 Nov 2022 10:29:04 -0800 (PST) Received: by mail-pg1-x532.google.com with SMTP id n17so6412319pgh.9 for ; Tue, 15 Nov 2022 10:29:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=npFjL6Dds3DKnQo07iUTlbqXMaucW9/yvERNl8ZNQRo=; b=NPDHr0QlHVIUIhIrDfHxvLDIuBOtKZfo7tv9Z1S0sFSK9TGsnpiguY2DVaj8KQqkUg 2SugxfLILvY2yZb9mSRU2q3++D6K9Vj+iOyidmF/ozO/p/7XOGWmFRfizSWm5iaLk5hj il4Q4TvN2HUyX2sk4tSfr2n2+tzXd77V5UybYcVVdBEnLxiMpx20HM6oe4QsCFyHtEs2 LJ3vSboCcqML9M7WDu6BnBSGDB+A5Hgh9HctMZlRn/EuhBDkfp2Exqkb6OqkY16lFgjL mqYbnVzmCbSOyK+6Jgh/buBRjxAFZ5UldLk6C7sULrRpi2KwbgfgjYgFwkYbz8uiAGV1 sx/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=npFjL6Dds3DKnQo07iUTlbqXMaucW9/yvERNl8ZNQRo=; b=S0sz/vWAjQCs/1P4XYe5XebJKXlS1wTddDknpvkdJTY80L4CblJNlZNf6XoKDRhuEN i1WcHhLkYl6QPXsvQRB2Y4duhd43JJUcZ97KxFJKHD/IMDdxsWdJeqcWaPqX9KGpJAJ2 LGRSJYtwcukxasvo4UThOYumC83uz7UNOKDKpBp7FZHQ6eWskcLIxieuQtRr8f+VqBKY +PJcSOWPQ/SCFqwKOy2Rs4ol48lCC8B7Sbix2uLQ4ef4E1nY4Q3TK2pnkg/WbUCBaLZH c+hYfCYd3fcFe/jIgmxTt/uRPIGkqRpD1uzk+v35gHufuYbOx05uR1DeFUtnaeVaw7hr vC3A== X-Gm-Message-State: ANoB5pmgPNfO81TdlooPUHx2zgGl9LQEFgMVxLqu/Z2zFcOY5Z0o/hvp Y4lIj8Rxwm0iqo9zsUG7d6Q= X-Received: by 2002:a63:560c:0:b0:476:9983:b4b5 with SMTP id k12-20020a63560c000000b004769983b4b5mr6888328pgb.516.1668536944373; Tue, 15 Nov 2022 10:29:04 -0800 (PST) Received: from localhost (fwdproxy-prn-116.fbsv.net. [2a03:2880:ff:74::face:b00c]) by smtp.gmail.com with ESMTPSA id k15-20020aa7972f000000b0056bbba4302dsm9034046pfg.119.2022.11.15.10.29.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Nov 2022 10:29:03 -0800 (PST) From: Nhat Pham To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: hannes@cmpxchg.org Subject: [PATCH 1/4] workingset: fix confusion around eviction vs refault container Date: Tue, 15 Nov 2022 10:28:58 -0800 Message-Id: <20221115182901.2755368-2-nphamcs@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221115182901.2755368-1-nphamcs@gmail.com> References: <20221115182901.2755368-1-nphamcs@gmail.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749588148922516787?= X-GMAIL-MSGID: =?utf-8?q?1749588148922516787?= From: Johannes Weiner Refault decisions are made based on the lruvec where the page was evicted, as that determined its LRU order while it was alive. Stats and workingset aging must then occur on the lruvec of the new page, as that's the node and cgroup that experience the refault and that's the lruvec whose nonresident info ages out by a new resident page. Those lruvecs could be different when a page is shared between cgroups, or the refaulting page is allocated on a different node. There are currently two mix-ups: 1. When swap is available, the resident anon set must be considered when comparing the refault distance. The comparison is made against the right anon set, but the check for swap is not. When pages get evicted from a cgroup with swap, and refault in one without, this can incorrectly consider a hot refault as cold - and vice versa. Fix that by using the eviction cgroup for the swap check. 2. The stats and workingset age are updated against the wrong lruvec altogether: the right cgroup but the wrong NUMA node. When a page refaults on a different NUMA node, this will have confusing stats and distort the workingset age on a different lruvec - again possibly resulting in hot/cold misclassifications down the line. Fix the swap check and the refault pgdat to address both concerns. This was found during code review. It hasn't caused notable issues in production, suggesting that those refault-migrations are relatively rare in practice. Signed-off-by: Johannes Weiner Co-developed-by: Nhat Pham Signed-off-by: Nhat Pham --- mm/workingset.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/workingset.c b/mm/workingset.c index ae7e984b23c6..79585d55c45d 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -457,6 +457,7 @@ void workingset_refault(struct folio *folio, void *shadow) */ nr = folio_nr_pages(folio); memcg = folio_memcg(folio); + pgdat = folio_pgdat(folio); lruvec = mem_cgroup_lruvec(memcg, pgdat); mod_lruvec_state(lruvec, WORKINGSET_REFAULT_BASE + file, nr); @@ -474,7 +475,7 @@ void workingset_refault(struct folio *folio, void *shadow) workingset_size += lruvec_page_state(eviction_lruvec, NR_INACTIVE_FILE); } - if (mem_cgroup_get_nr_swap_pages(memcg) > 0) { + if (mem_cgroup_get_nr_swap_pages(eviction_memcg) > 0) { workingset_size += lruvec_page_state(eviction_lruvec, NR_ACTIVE_ANON); if (file) {