From patchwork Wed Sep 20 19:02:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 142721 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp4599511vqi; Wed, 20 Sep 2023 21:21:18 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGOK7EXJDvccga0ayL9+fgeUxpldm9crP5GNrzaSNLcuX6Ftwxkmspy6/hPXt5dbl55FwLK X-Received: by 2002:a05:6808:4092:b0:3a7:44da:d5e4 with SMTP id db18-20020a056808409200b003a744dad5e4mr3719606oib.51.1695270078144; Wed, 20 Sep 2023 21:21:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695270078; cv=none; d=google.com; s=arc-20160816; b=BC/dKM/Ua24rBgf/0RY2tkzrPB9TzepBYONuKmyQ97XRhol9E+n1D8EBmdLeABqtm6 huijkNHoFY+32ueoOz4Nj3puUww9o9IA4LeeYW7ia+oAGXqLBFu3ZXAMXVrPRwXXMHIW 6xiZTBjW/ek+XjVaU8LPEDfT1mcMsL7AFAipye2t3vvf7/ZMp93y2aoEHX02FA8xG9+s ATfdsd98vv+sVTK0E/pkob+4uuZmfhrDn6vSNPqzmPb7NqBJkLW02DDWO3WWIc8ExBUP Ot6oiKd9jqfJbhWpDErCTRBf7x2VGzB3mrXuVw4Z03f2V1mMxjRXiqY60oYo5PO7THHj fWYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version:reply-to :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=zmtAWomE9UHhxu4UCSwc3QxDlD+DtTBiD3PMp93+3PE=; fh=E4Dx6FsRR0bUU1V9Gy1LgJPpghBs+NaR6FSgB1AfVlI=; b=ZkMJg0Mg9HTtnLzt8/WbF/hV90tbAO2uy96Byl9s5RRDdVHBqc023iC+jJ0DD4yvZo p2zobOlFEVFnhYZY68xzYoa3kDJXftsacq2w1Sd1GyPr4J8Pg0W5egcmWBi1iRW9SJ2r ZpNPCk2+4XOIIy9/Kc10MJSkPEvzm+JHID6U72sd7PmqzMbico25FKa4Dz+VTD7Je26R 61ETLa9kd2ABnQDOlKG5m7KAURUYnk0V+WqQd5b5Uyzsb8JZPost0k40+ZaVZ6NfiL5J Ur/Pjury8FFoS8Pm8SsmVyg/fMFOKswZYwlqF4bu9AdcVBZ90foV8VeLowwPBrefTQdw 4pGg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=RbMDCyef; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id p3-20020aa78603000000b0068bda9121ecsi586365pfn.72.2023.09.20.21.21.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Sep 2023 21:21:18 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=RbMDCyef; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id C719480EBCB7; Wed, 20 Sep 2023 12:03:47 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229902AbjITTDl (ORCPT + 27 others); Wed, 20 Sep 2023 15:03:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229906AbjITTD0 (ORCPT ); Wed, 20 Sep 2023 15:03:26 -0400 Received: from mail-pg1-x533.google.com (mail-pg1-x533.google.com [IPv6:2607:f8b0:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 56FB0E4 for ; Wed, 20 Sep 2023 12:03:20 -0700 (PDT) Received: by mail-pg1-x533.google.com with SMTP id 41be03b00d2f7-5789de5c677so64987a12.3 for ; Wed, 20 Sep 2023 12:03:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695236600; x=1695841400; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=zmtAWomE9UHhxu4UCSwc3QxDlD+DtTBiD3PMp93+3PE=; b=RbMDCyefZ18r4dVeYKpV5jTNhWp+lns/rjINo5dQExzLjthyWFHE7YkXjKlrQlIYUj r/YpDNrgoc7ZLUFts5AeuYEb2TKS80mLeZkdXuNrSOFMAc3sC1yGaYFoOElGSTF7uApc BScSTGlSU2fLOKK706SBDyviMRE7aNItX2WqCOea02XiJhXg6VmtCvGOuza2ExTNJlYe 9zsyEhzDmjacS17Ixf8br5LtYwxOsDAG08MFU6G4rIPFvkEEx4U/j7VEV4buxXdon0v+ 1i1n087r+tMa2c3RRvByZqkvGZ0MWkita+wFun/Kcjb6GPUyGOiydG9G5QkQ0z0jY5L+ +wrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695236600; x=1695841400; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=zmtAWomE9UHhxu4UCSwc3QxDlD+DtTBiD3PMp93+3PE=; b=FnkNIBYKKDw2j4Y+x+DzIsmkE7VLkrEImPw+bYgiTgRgenkVm3ud2cKomROySgxCP1 uA0zhNVYmwcep1KziJSF/dfxYu99HkZzef2lpVGpYzbzXsNO9jVR8qYIOhjUcXbSyLhC nHswrO79SnXr52YXESzq7dB8Z/5Dx7UWgiPjX3l6z0wY+lmlRDIfvgC4FjHsyYS4hDZh vJB7jo+VW+4rj92/rcZmFYvvtwQtHdAKdzkEnahV0YNU87VEU65EPZ9xwDQDu3/6LVK0 4poTZMeX0kW7CX/Dxpwi4vWG28aU/9LMZhWZAftePCUxROS0R+2akZKiF1vn0LHoeHCs FNZg== X-Gm-Message-State: AOJu0YzEWDOa5FyMcklkoset2XNJWz4nGyj+uiDXDazbEVAQ3gcfgqAl 75tyNwXOdB2WmVwHchaz6xo= X-Received: by 2002:a05:6a20:138c:b0:145:6857:457a with SMTP id hn12-20020a056a20138c00b001456857457amr3193200pzc.4.1695236599694; Wed, 20 Sep 2023 12:03:19 -0700 (PDT) Received: from KASONG-MB2.tencent.com ([124.127.145.18]) by smtp.gmail.com with ESMTPSA id m5-20020aa78a05000000b006871fdde2c7sm423935pfa.110.2023.09.20.12.03.15 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 20 Sep 2023 12:03:19 -0700 (PDT) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Yu Zhao , Roman Gushchin , Johannes Weiner , Michal Hocko , Hugh Dickins , Nhat Pham , Yuanchu Xie , Kalesh Singh , Suren Baghdasaryan , "T . J . Mercier" , linux-kernel@vger.kernel.org, Kairui Song Subject: [RFC PATCH v3 5/6] mm, lru_gen: convert avg_total and avg_refaulted to atomic Date: Thu, 21 Sep 2023 03:02:43 +0800 Message-ID: <20230920190244.16839-6-ryncsn@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230920190244.16839-1-ryncsn@gmail.com> References: <20230920190244.16839-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Wed, 20 Sep 2023 12:03:48 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777619517471476371 X-GMAIL-MSGID: 1777619517471476371 From: Kairui Song No feature change, prepare for later patch. Signed-off-by: Kairui Song --- include/linux/mmzone.h | 4 ++-- mm/vmscan.c | 16 ++++++++-------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 4106fbc5b4b3..d944987b67d3 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -425,9 +425,9 @@ struct lru_gen_folio { /* the multi-gen LRU sizes, eventually consistent */ long nr_pages[MAX_NR_GENS][ANON_AND_FILE][MAX_NR_ZONES]; /* the exponential moving average of refaulted */ - unsigned long avg_refaulted[ANON_AND_FILE][MAX_NR_TIERS]; + atomic_long_t avg_refaulted[ANON_AND_FILE][MAX_NR_TIERS]; /* the exponential moving average of evicted+protected */ - unsigned long avg_total[ANON_AND_FILE][MAX_NR_TIERS]; + atomic_long_t avg_total[ANON_AND_FILE][MAX_NR_TIERS]; /* the first tier doesn't need protection, hence the minus one */ unsigned long protected[NR_HIST_GENS][ANON_AND_FILE][MAX_NR_TIERS - 1]; /* can be modified without holding the LRU lock */ diff --git a/mm/vmscan.c b/mm/vmscan.c index 3f4de75e5186..82acc1934c86 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -3705,9 +3705,9 @@ static void read_ctrl_pos(struct lruvec *lruvec, int type, int tier, int gain, struct lru_gen_folio *lrugen = &lruvec->lrugen; int hist = lru_hist_from_seq(lrugen->min_seq[type]); - pos->refaulted = lrugen->avg_refaulted[type][tier] + + pos->refaulted = atomic_long_read(&lrugen->avg_refaulted[type][tier]) + atomic_long_read(&lrugen->refaulted[hist][type][tier]); - pos->total = lrugen->avg_total[type][tier] + + pos->total = atomic_long_read(&lrugen->avg_total[type][tier]) + atomic_long_read(&lrugen->evicted[hist][type][tier]); if (tier) pos->total += lrugen->protected[hist][type][tier - 1]; @@ -3732,15 +3732,15 @@ static void reset_ctrl_pos(struct lruvec *lruvec, int type, bool carryover) if (carryover) { unsigned long sum; - sum = lrugen->avg_refaulted[type][tier] + + sum = atomic_long_read(&lrugen->avg_refaulted[type][tier]) + atomic_long_read(&lrugen->refaulted[hist][type][tier]); - WRITE_ONCE(lrugen->avg_refaulted[type][tier], sum / 2); + atomic_long_set(&lrugen->avg_refaulted[type][tier], sum / 2); - sum = lrugen->avg_total[type][tier] + + sum = atomic_long_read(&lrugen->avg_total[type][tier]) + atomic_long_read(&lrugen->evicted[hist][type][tier]); if (tier) sum += lrugen->protected[hist][type][tier - 1]; - WRITE_ONCE(lrugen->avg_total[type][tier], sum / 2); + atomic_long_set(&lrugen->avg_total[type][tier], sum / 2); } if (clear) { @@ -5885,8 +5885,8 @@ static void lru_gen_seq_show_full(struct seq_file *m, struct lruvec *lruvec, if (seq == max_seq) { s = "RT "; - n[0] = READ_ONCE(lrugen->avg_refaulted[type][tier]); - n[1] = READ_ONCE(lrugen->avg_total[type][tier]); + n[0] = atomic_long_read(&lrugen->avg_refaulted[type][tier]); + n[1] = atomic_long_read(&lrugen->avg_total[type][tier]); } else if (seq == min_seq[type] || NR_HIST_GENS > 1) { s = "rep"; n[0] = atomic_long_read(&lrugen->refaulted[hist][type][tier]);