From patchwork Fri Apr 21 17:40:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 86449 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1239140vqo; Fri, 21 Apr 2023 10:43:54 -0700 (PDT) X-Google-Smtp-Source: AKy350ZNa8D41I3WUL/4owZQg2B8xM+kQ7OXbHkF/wy/Ro8Dsr+ssRwmXLmBdfzN2SJnl1ZeQ3TH X-Received: by 2002:a05:6a20:428e:b0:f2:bb0:4622 with SMTP id o14-20020a056a20428e00b000f20bb04622mr5946686pzj.12.1682099034062; Fri, 21 Apr 2023 10:43:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682099034; cv=none; d=google.com; s=arc-20160816; b=yb+H+epelgGTiOKsNGIjeBclC9ZXfwVF2Nq9XLht2+oUq07tSMxomnlU7A+2+bvF1k yX7phzohSg1w27CJOznSi0SpmUL1nv+R/PHuTAN4rW3Rr89LWcLmsFIkDhBRSgB2Cqvc /xoC8NE6RldtOgPqu6pUnl9sc8wr4DwXb+fxBetNRQ4yNu81m43F0hZscIobv9eJ380o QjFNpgVSq+GSGtAXBbG2TzmS/nJUL4PRFe57Wuya+xSOuFlqLRB2cDkCpOEfMGFwFlkr C2GvrkvT2HA06hc2EEait8O5tMRBn0Ee0kNOLs4p62yDPCKK8P5K7q54FYuGMcvuDIAP G8RQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:cc:to:from:subject :message-id:references:mime-version:in-reply-to:date:dkim-signature; bh=abvwWBWOn0vLHIokwPNC/tHyu1Fa1R/g8ZqTWIaeK3o=; b=sUBJwSsZPlXT7P8Gi33F+VAGfNCJ+/2TjwSD5ertvqco3pi+T/G4Rv+4pVsWggYKVe HphRHxB5yN21WB/EfATzMTsMcZ+D1J5yib/VrLl0omJKjCT28pkvPJKlZZ4hN4NllgUg pWT4UYnYttNPRRZ7Ny2wBd8y4906AT5qi8Pq6wU5Fph6lbGNMcLYK8h4qVr79LDTjYSq JR00+8//DzTkSGaXi6TEhP+s+Y2gKL2SqWZudfTBFMHU+vW6tpxx5iZWvckykrIxtNVW r9I7Z1fkRn19Qj4tpwZr6BDGbrSk/mHS8PMR0MWSdaWnjnmnJUH7KckB8BnQVdJqvL/Q ng8g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=XiTBjh0u; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s6-20020a656446000000b0050bede17e7bsi4311080pgv.347.2023.04.21.10.43.41; Fri, 21 Apr 2023 10:43:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=XiTBjh0u; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233286AbjDURkk (ORCPT + 99 others); Fri, 21 Apr 2023 13:40:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40450 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233178AbjDURk0 (ORCPT ); Fri, 21 Apr 2023 13:40:26 -0400 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C01112C96 for ; Fri, 21 Apr 2023 10:40:25 -0700 (PDT) Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-247c0ee87aaso1395325a91.0 for ; Fri, 21 Apr 2023 10:40:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1682098824; x=1684690824; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=abvwWBWOn0vLHIokwPNC/tHyu1Fa1R/g8ZqTWIaeK3o=; b=XiTBjh0uy9kBhESmjZrsmo27Fd2heaOUEeGy+wI7JgY+55BFi83KpJfUElFkBAdtm2 /9Y+dOWIDQQOwVTZr/9l2zpPw4NhLuSad5LwurXNF9rM5PxMETajnAk6IdSR9itjleO5 5RCZOU6uWoQn/XlBfG6L6Ut73QqfPkSzpxRKMZq9c5dNC03EQysfXGAmQsX7vQIaMlXI wrYrPEcgEz4/6R89vEXhItTLbmEid9U911aGVkkoSTQdm4KxH3ssyAravljY6M6dDyLy P40RfGznmXP1ykBLl7M1nQBK+thu7rJX9Fwoe+MatBRuu3PDwmPEoG+B+msi8iGqptyL Fwng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682098824; x=1684690824; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=abvwWBWOn0vLHIokwPNC/tHyu1Fa1R/g8ZqTWIaeK3o=; b=HE8BaaYa8bU9hIZTAP50ONOs5zzrOWwjw6gR0sx4aDhAtDZopqVVqU0Fsi6UzTgPKc xejn7iDZZsqIseDy/+re7ocegd3sL9wvO9DXB6WQTcuYwm09fduOU3PKAj0vEIp9NG5z VRGwZZSzS+WzxOCdwqpB2XJtaLWDGC9Ld2Qw8Os1aROYf1XAALYhlclqeNrU8GG2zAId uLIA02C5475/oYLxRCgOaoAOQqXlTrzJvv9Ve8m3OS1taOuMnhBnGucbzqQhtz5MFe4S 7zPy90QuGpyc/37w3QRFUMBE4cJxDCfmP3CjZ03+vpgHxGKLQvStvD+br12IYileotGB hIoQ== X-Gm-Message-State: AAQBX9dBK/WLxdtQpM0RWnHgxjz19dzyCAVZC1Wm2kev0raWE+t0Y5x0 K+gzzChQMTEDJGgtKuzWnrft7wxq1l64TlkM X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:90a:290:b0:247:2677:77b with SMTP id w16-20020a17090a029000b002472677077bmr1455398pja.8.1682098824669; Fri, 21 Apr 2023 10:40:24 -0700 (PDT) Date: Fri, 21 Apr 2023 17:40:16 +0000 In-Reply-To: <20230421174020.2994750-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230421174020.2994750-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog Message-ID: <20230421174020.2994750-2-yosryahmed@google.com> Subject: [PATCH v5 1/5] writeback: move wb_over_bg_thresh() call outside lock section From: Yosry Ahmed To: Alexander Viro , Christian Brauner , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , Tejun Heo Cc: Jan Kara , Jens Axboe , " =?utf-8?q?Micha?= =?utf-8?q?l_Koutn=C3=BD?= " , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763808676598658324?= X-GMAIL-MSGID: =?utf-8?q?1763808676598658324?= wb_over_bg_thresh() calls mem_cgroup_wb_stats() which invokes an rstat flush, which can be expensive on large systems. Currently, wb_writeback() calls wb_over_bg_thresh() within a lock section, so we have to do the rstat flush atomically. On systems with a lot of cpus and/or cgroups, this can cause us to disable irqs for a long time, potentially causing problems. Move the call to wb_over_bg_thresh() outside the lock section in preparation to make the rstat flush in mem_cgroup_wb_stats() non-atomic. The list_empty(&wb->work_list) check should be okay outside the lock section of wb->list_lock as it is protected by a separate lock (wb->work_lock), and wb_over_bg_thresh() doesn't seem like it is modifying any of wb->b_* lists the wb->list_lock is protecting. Also, the loop seems to be already releasing and reacquring the lock, so this refactoring looks safe. Signed-off-by: Yosry Ahmed Reviewed-by: Michal Koutný Reviewed-by: Jan Kara Acked-by: Shakeel Butt Acked-by: Tejun Heo --- fs/fs-writeback.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c index 1db3e3c24b43..11aa1652fb84 100644 --- a/fs/fs-writeback.c +++ b/fs/fs-writeback.c @@ -2024,7 +2024,6 @@ static long wb_writeback(struct bdi_writeback *wb, struct blk_plug plug; blk_start_plug(&plug); - spin_lock(&wb->list_lock); for (;;) { /* * Stop writeback when nr_pages has been consumed @@ -2049,6 +2048,9 @@ static long wb_writeback(struct bdi_writeback *wb, if (work->for_background && !wb_over_bg_thresh(wb)) break; + + spin_lock(&wb->list_lock); + /* * Kupdate and background works are special and we want to * include all inodes that need writing. Livelock avoidance is @@ -2078,13 +2080,19 @@ static long wb_writeback(struct bdi_writeback *wb, * mean the overall work is done. So we keep looping as long * as made some progress on cleaning pages or inodes. */ - if (progress) + if (progress) { + spin_unlock(&wb->list_lock); continue; + } + /* * No more inodes for IO, bail */ - if (list_empty(&wb->b_more_io)) + if (list_empty(&wb->b_more_io)) { + spin_unlock(&wb->list_lock); break; + } + /* * Nothing written. Wait for some inode to * become available for writeback. Otherwise @@ -2096,9 +2104,7 @@ static long wb_writeback(struct bdi_writeback *wb, spin_unlock(&wb->list_lock); /* This function drops i_lock... */ inode_sleep_on_writeback(inode); - spin_lock(&wb->list_lock); } - spin_unlock(&wb->list_lock); blk_finish_plug(&plug); return nr_pages - work->nr_pages; From patchwork Fri Apr 21 17:40:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 86450 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1239670vqo; Fri, 21 Apr 2023 10:45:02 -0700 (PDT) X-Google-Smtp-Source: AKy350bKH9+irJRCkQJFnHiTk10gnNGsLeD8HV9kIC0JNJc/mD1qqalcc19GKzXJ3XYahranWmAq X-Received: by 2002:a17:90a:1f46:b0:24b:3123:8dd8 with SMTP id y6-20020a17090a1f4600b0024b31238dd8mr5938117pjy.27.1682099102003; Fri, 21 Apr 2023 10:45:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682099101; cv=none; d=google.com; s=arc-20160816; b=kU1GLfDNxTmCVRr1b4u40WepRzkEmPLicdK0RsvPycw7YitPcIWSNZv1Q+q04i4bMQ CSNRW5Lmsom5V2nWOjqNUO0K08m8q73k5DgqwX3mUEp18bof2cX8eQAHyzSGmX7u67Oc iiSi1KMxn13rngeogk1vDbm5TiN0q9rAs+sWVgZ/88X87QEYPGfr2DS55y+qcekmo/Q6 r/9oiolO1LrD2G+58e7eLVTe/ARRXdNNFHVzqbLOsHhN12dTjLXMh7THMnsnz18+7+3o t/5KKLv5TMSlNQOwi/m2t7LLdBez7dABUkN83VGN1M5uKqu1CcoVNjMJNTDuedo39DjG K5jg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:cc:to:from:subject :message-id:references:mime-version:in-reply-to:date:dkim-signature; bh=ze/kPlw8gotaEo1sJnB2fYrZx8MA/zlu9x0xYHZ+g7E=; b=LAUZhqeLhW/dfe/jPg90yityxBSEL19ORqhmLHI/PcxPDK9zJl2hn68rrFiwe8LBnS 8KUQE/5lgKAjg9i/ijp9INu29K5eZVfPWpO6psFzr3Tf8H7tG3nuqIWb6eI2BXh9NDgP UN0J8dK+6oy7dN6/G35WQpyn45eNFYG4af5IdILSSxl72SPn0gary+iL4A8z6aoySJLx rtZS3pFTpMeyEuVEFl6rw7B678Ss476zOC/VD816dbWgGz+5wgL3dcBdfK17y1dOkREi Ozgeiz4pj9zdfWLVvJoyCbdAHqWubs8ozs3/Z9AAiMcxpkHztxvwwIv6ZGt2oY74/sC6 3l/g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=evkdgDpS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id je21-20020a170903265500b001a194bda2desi4420949plb.487.2023.04.21.10.44.48; Fri, 21 Apr 2023 10:45:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=evkdgDpS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233367AbjDURko (ORCPT + 99 others); Fri, 21 Apr 2023 13:40:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40476 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232300AbjDURk2 (ORCPT ); Fri, 21 Apr 2023 13:40:28 -0400 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [IPv6:2607:f8b0:4864:20::1049]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE9E612582 for ; Fri, 21 Apr 2023 10:40:26 -0700 (PDT) Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-24736ac595aso2200461a91.1 for ; Fri, 21 Apr 2023 10:40:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1682098826; x=1684690826; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=ze/kPlw8gotaEo1sJnB2fYrZx8MA/zlu9x0xYHZ+g7E=; b=evkdgDpSyqlLAXON2ZuU2lSGfLM1dQnhdPP+fmyZWeKwGNZTWFCJKPVGhM2bZZGKOC f7YfSGCo2Nf3oXwwYJmGjEDgBvFsw6i76b70QgcRaecrvTKD295qL9CqwnO4UztiMSAR wca4o1fBiZOb2DZIrqp+VDQfAwIcLci5K41MPeROy0/47HRyQgXuMC8OBwZyCxCgwh/S 8f4JUGNY1xeou8G6F0i+W3dKhEWQ3Zv+nckYooRRkUzwRcW77nDMDOwXtSb1IOOWcx0Y jIqwdtbmW/rhz4VKzZZ8k/jeg5TikDCeSv0Wmg2wr9UElKoOoUpGS9AakxlgHuAC0y31 KsJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682098826; x=1684690826; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=ze/kPlw8gotaEo1sJnB2fYrZx8MA/zlu9x0xYHZ+g7E=; b=EWXdK/obcCOZsOmX7JuJxP3nu6RPMu4Vopb6pSXd994RHzHGx0NGjylXq+M/2qrNXb ot0a1hFmJgC6aAkjufdXZ8zvcxzemUfHs7N0oSUPDTWmd7806BgNmQZpuMb12NV2W2iH GOxUA9STuQ/hBNbNIlD8MTQXKB2lql1ZhtdvCfda7s1E4MFBx5BhRJs3Gm+OgI8ix3FV 8hm43kIUFySARAFUrm064iPTWphMD0vrvoNbpFlVQUkQciu68kQ3FiYpOWSFn7DEsdgO arApE1Edzg2rwm+KMYwwE0s8qSMFI4eWZGjosfdmU0av39YA40PKVt5dIfc9z0wjlYF9 v06w== X-Gm-Message-State: AAQBX9d8mwtZrnCGr/haoRja+nQL8SnE9wGF5Nep54y/QLVM/BRuPjkh tfGryJMrKrufwJIHWvwlJt8KkuO0+PBQA8Kh X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:90a:3fcd:b0:24b:27db:3cfc with SMTP id u13-20020a17090a3fcd00b0024b27db3cfcmr1409596pjm.8.1682098826505; Fri, 21 Apr 2023 10:40:26 -0700 (PDT) Date: Fri, 21 Apr 2023 17:40:17 +0000 In-Reply-To: <20230421174020.2994750-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230421174020.2994750-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog Message-ID: <20230421174020.2994750-3-yosryahmed@google.com> Subject: [PATCH v5 2/5] memcg: flush stats non-atomically in mem_cgroup_wb_stats() From: Yosry Ahmed To: Alexander Viro , Christian Brauner , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , Tejun Heo Cc: Jan Kara , Jens Axboe , " =?utf-8?q?Micha?= =?utf-8?q?l_Koutn=C3=BD?= " , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763808748119797666?= X-GMAIL-MSGID: =?utf-8?q?1763808748119797666?= The previous patch moved the wb_over_bg_thresh()->mem_cgroup_wb_stats() code path in wb_writeback() outside the lock section. We no longer need to flush the stats atomically. Flush the stats non-atomically. Signed-off-by: Yosry Ahmed Reviewed-by: Michal Koutný Acked-by: Shakeel Butt Acked-by: Tejun Heo --- mm/memcontrol.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 4b27e245a055..5e79fdf8442b 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -4648,11 +4648,7 @@ void mem_cgroup_wb_stats(struct bdi_writeback *wb, unsigned long *pfilepages, struct mem_cgroup *memcg = mem_cgroup_from_css(wb->memcg_css); struct mem_cgroup *parent; - /* - * wb_writeback() takes a spinlock and calls - * wb_over_bg_thresh()->mem_cgroup_wb_stats(). Do not sleep. - */ - mem_cgroup_flush_stats_atomic(); + mem_cgroup_flush_stats(); *pdirty = memcg_page_state(memcg, NR_FILE_DIRTY); *pwriteback = memcg_page_state(memcg, NR_WRITEBACK); From patchwork Fri Apr 21 17:40:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 86456 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1241971vqo; Fri, 21 Apr 2023 10:49:31 -0700 (PDT) X-Google-Smtp-Source: AKy350ahlJtDn0rvVnDNfgOXKcw+igCh0TANusCIHh3tlzAb9JxyrSrl9SFZI8ERc1WRCqxTDq6A X-Received: by 2002:a17:903:41d1:b0:19a:7f4b:3ef6 with SMTP id u17-20020a17090341d100b0019a7f4b3ef6mr7867967ple.3.1682099370943; Fri, 21 Apr 2023 10:49:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682099370; cv=none; d=google.com; s=arc-20160816; b=JvGxFNnnG65s73DqIACohcE1pz4STw7IBCVej9l5scfGxyDp+Gm0f+SL2xbtSveXUz CIm8TP9HWzOgCr53BWalfT0wtOuHCW3TRW/0kYBk1vTlkiwsILEPd8p+UIgxKpg1NdSz MUpZbYOaaCkQbbl04KPzojgzt2rj4xTlWBXOD86ljrrlsdmjwbUNCvRinnsM7QPURLUy OJBgLqo4eNLftQ93OqJN38SJ9cVz3Ez33/+M7Q0iJHqkFGxtBp73m20eILMKEjAvklNe VQ2Pjw+jQNKUPGA6pZ3HnobMcxxtkdXsgAYqP+f51cL0AnZbEPcWytPSt6RJAjOPzohW zSaw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:cc:to:from:subject :message-id:references:mime-version:in-reply-to:date:dkim-signature; bh=ZwYNf0IQKL13Uvalja3XtUe1fz98Sj9XyCNuQ2xKGcs=; b=q0fhhUV4mx3UwyD6TtdnxPwzo7bsosARfS43khayPUWsSqq0kcIaqP8Ij5pTFNHNkf Ryowe+vqDRK2XVbzgLb5fyi5e/GhYgxtQUVsRKeupyI1rtuvkTNbLP7ewOuQ7ZdyozxX 9GkdvKzCgdDX4KdnbAABDa5CleiniOy7ATfxn6oLrNcZb4CixODyQoR/107ui26D/Erb FUXIVvo5npaM6xB3OoF5fi50fY+/PRwfEkoJR7DqKhHEmW7W3MlaIt2pza3OGYaYW3Ru 68Y+k3Uuux6cj7ATPJ1xWNXmow13Uc5uC2/hA+cyaTGOQNS6SlTAfzGwOqX0FTR5nO7o QfvQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=Vz7q7Ozm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 13-20020a170902c10d00b001a6cb6667f9si4726862pli.21.2023.04.21.10.49.17; Fri, 21 Apr 2023 10:49:30 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=Vz7q7Ozm; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233345AbjDURkt (ORCPT + 99 others); Fri, 21 Apr 2023 13:40:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40498 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233290AbjDURkb (ORCPT ); Fri, 21 Apr 2023 13:40:31 -0400 Received: from mail-pg1-x549.google.com (mail-pg1-x549.google.com [IPv6:2607:f8b0:4864:20::549]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C619212C96 for ; Fri, 21 Apr 2023 10:40:28 -0700 (PDT) Received: by mail-pg1-x549.google.com with SMTP id 41be03b00d2f7-520f3f18991so1458485a12.3 for ; Fri, 21 Apr 2023 10:40:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1682098828; x=1684690828; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=ZwYNf0IQKL13Uvalja3XtUe1fz98Sj9XyCNuQ2xKGcs=; b=Vz7q7OzmTKVhjKuL1Z10tx4sjzaZzQc1RfkV5JIKDa4QF+qc7PE1nRpKj4JbTW26CL 6HWg0Bk/Qn7IeGOg2v+aAU068B55qLODSnp8tJGuESSICbp+6Gjpfa3HASvT4P/INIIX FNJAyhgiBW8P7szSQRrC+9yXtrHR4LwEf5c+uGkIF6nWGq5t81yHqYblmUSNV6/Pj4a5 x4hPCDxoK7EryyUSzUifkrvbv6JClx5/LSyTe0bPdsXuYCvvh5cuu7sfplfgaUZyWnve cPw/375qmd31nmxG3Xi1PRi7CTxei8qHjgER9ienP6JVZlSSk1rMC1yFsifBD6+eDFuk /A6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682098828; x=1684690828; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=ZwYNf0IQKL13Uvalja3XtUe1fz98Sj9XyCNuQ2xKGcs=; b=gFPejYgdW72FEgimUZj+m6qd51Jia/Oum69pWEP8PCbH4CtsqFJpAPQex6WYPiAYfQ +wiDEJ7i265OUfXwmClDm7vbrHd5jv3Wr/2QqAv0YN+004KwXpjY2IkYp2V+602eQR9d HCRZIE2PCIhAp6z31OUPFqss+bfARyzJkiqGn+4LvFi3rCBDmLa4hUxO3IdKh4krPth7 M11EOGGzCVBh5nZLvxgPtONdghIrtilrc4McgoGiEt6DbKNgO0nLLrIFgG7HyUeIw8Qs VUvJI9U2OSNMtcaJLK4hunJE5IiTRPpkGA2NxQQqM4dlsvWYKkOfgUg0PjhdWguDQEb6 j1Eg== X-Gm-Message-State: AAQBX9cm2JSUKf9HwCK9qre6k7WwM0s1mY/dTrKaKk5xSaDHZaJPNNyi EFacF30X6co8/uBH6GYWviPwKA0iTZBPtoNJ X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:903:2447:b0:1a9:2c3e:b087 with SMTP id l7-20020a170903244700b001a92c3eb087mr1970101pls.0.1682098828387; Fri, 21 Apr 2023 10:40:28 -0700 (PDT) Date: Fri, 21 Apr 2023 17:40:18 +0000 In-Reply-To: <20230421174020.2994750-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230421174020.2994750-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog Message-ID: <20230421174020.2994750-4-yosryahmed@google.com> Subject: [PATCH v5 3/5] memcg: calculate root usage from global state From: Yosry Ahmed To: Alexander Viro , Christian Brauner , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , Tejun Heo Cc: Jan Kara , Jens Axboe , " =?utf-8?q?Micha?= =?utf-8?q?l_Koutn=C3=BD?= " , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763809029591840990?= X-GMAIL-MSGID: =?utf-8?q?1763809029591840990?= Currently, we approximate the root usage by adding the memcg stats for anon, file, and conditionally swap (for memsw). To read the memcg stats we need to invoke an rstat flush. rstat flushes can be expensive, they scale with the number of cpus and cgroups on the system. mem_cgroup_usage() is called by memcg_events()->mem_cgroup_threshold() with irqs disabled, so such an expensive operation with irqs disabled can cause problems. Instead, approximate the root usage from global state. This is not 100% accurate, but the root usage has always been ill-defined anyway. Signed-off-by: Yosry Ahmed Reviewed-by: Michal Koutný Acked-by: Shakeel Butt --- mm/memcontrol.c | 24 +++++------------------- 1 file changed, 5 insertions(+), 19 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 5e79fdf8442b..cb78bba5b4a4 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3699,27 +3699,13 @@ static unsigned long mem_cgroup_usage(struct mem_cgroup *memcg, bool swap) if (mem_cgroup_is_root(memcg)) { /* - * We can reach here from irq context through: - * uncharge_batch() - * |--memcg_check_events() - * |--mem_cgroup_threshold() - * |--__mem_cgroup_threshold() - * |--mem_cgroup_usage - * - * rstat flushing is an expensive operation that should not be - * done from irq context; use stale stats in this case. - * Arguably, usage threshold events are not reliable on the root - * memcg anyway since its usage is ill-defined. - * - * Additionally, other call paths through memcg_check_events() - * disable irqs, so make sure we are flushing stats atomically. + * Approximate root's usage from global state. This isn't + * perfect, but the root usage was always an approximation. */ - if (in_task()) - mem_cgroup_flush_stats_atomic(); - val = memcg_page_state(memcg, NR_FILE_PAGES) + - memcg_page_state(memcg, NR_ANON_MAPPED); + val = global_node_page_state(NR_FILE_PAGES) + + global_node_page_state(NR_ANON_MAPPED); if (swap) - val += memcg_page_state(memcg, MEMCG_SWAP); + val += total_swap_pages - get_nr_swap_pages(); } else { if (!swap) val = page_counter_read(&memcg->memory); From patchwork Fri Apr 21 17:40:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 86459 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1247363vqo; Fri, 21 Apr 2023 11:00:29 -0700 (PDT) X-Google-Smtp-Source: AKy350Z1iZbb4/UrLnOANf4MUwWlXAJ9O19ZJspW+zo13M3/YifujbHgXTsXg7j7ISMYGN4NiO7R X-Received: by 2002:a05:6a21:33a2:b0:f2:8007:a90 with SMTP id yy34-20020a056a2133a200b000f280070a90mr3044517pzb.40.1682100028914; Fri, 21 Apr 2023 11:00:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682100028; cv=none; d=google.com; s=arc-20160816; b=qBbWucvXAB3I6IcAOOtSecoyGOrt9Px/YNqZdyyQwXp9GhbMcQbBV7w3bg1sPR7pJq k0blNxNYecz0zfWIDKYbTp6TUy24wbSqLr7ufZgeYLHRW1V26pnauEiAwzkQnXGwReg5 yH9JERzAlxG6E1JLawju0/caQ1B7z/G2+tXVFMXgI00qUZHlGH/3us08O2Xb6HTSfYgj GoTHR5+3M0Ik3IISknPzXRoKm3f75YFFpKAZ8r64IGc6SmN9w4ZiEo/5daifFCdOiCEz gcrImfkafidEGE0Gxr5ybyoG+Lsiwt8vIqGIBMZLwf5QR0ItVZMhg5Enz8OZfVx4gCiF 1bmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=2xVbU1EXT3F/5uSzRhbvwRdAhlPEIhLbnGkOO/O2kc0=; b=hmdWE3+YOGjbImdg/3v20zOviHUbKXz58WJ5hihrELKpgJ8uQblgmjyhQ0OmP4YbDh com45ne7hPbj/OOPUTNLgIzYDTfqagk/LJZgw3uR2tdz4AWQAf/8lN4xd6+klu2EdVOK SCT/51xp8m5U+i+DNHk5MEHLjGkB64NYNAFfawZuZUcDYGhSGsSXosQN9YxSxnuqy9vC n6i6JZQH3LVD97hSDQp72fu41oxnQnla1rQ2kZi6KNM+WUPSTU6uc5XoZLa454ruPLun fHe5KtU1NWFF/Cjksq0ZhKQJWOYWKnBjoAOE4rgdIJEw4QZ7y0MZr2VKk1vk9vx39FVh 9Jgg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=VsD7Uvvl; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 78-20020a621751000000b0063b8b7d7a14si4949989pfx.70.2023.04.21.11.00.15; Fri, 21 Apr 2023 11:00:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=VsD7Uvvl; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233289AbjDURkx (ORCPT + 99 others); Fri, 21 Apr 2023 13:40:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233293AbjDURkc (ORCPT ); Fri, 21 Apr 2023 13:40:32 -0400 Received: from mail-pf1-x44a.google.com (mail-pf1-x44a.google.com [IPv6:2607:f8b0:4864:20::44a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8B6312588 for ; Fri, 21 Apr 2023 10:40:30 -0700 (PDT) Received: by mail-pf1-x44a.google.com with SMTP id d2e1a72fcca58-63b6bd2eebaso1799541b3a.1 for ; Fri, 21 Apr 2023 10:40:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1682098830; x=1684690830; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=2xVbU1EXT3F/5uSzRhbvwRdAhlPEIhLbnGkOO/O2kc0=; b=VsD7UvvlQQthDrDrmrxfA+6j9bxs+fBOtiWfjPuVqKRncTMVYwHyMK7lQKw4z7d6OO 8f3IhH6XmGqVDVmFY8Cyv6LrhOYmMlAEwO97OjVKp4EfQ7GvmXRUQBA2elyZi2HdAWT8 tC3JhABpjBI1uwry+YoMsz8jQ+5FjhTBlcxyLfQuh9MawFFcJmVXlkCxxE83KRAGfIRN izUhapvRy65QRBOUgMasmNbiSkr8HJo7bR2eGmnfXoyNgGu2rdwIe+txD2JLoEKv7Tja ItjbWxT/dXoPWnPJUuXxJqFt3IyOH6lsotFnhAXC0TTQySKSNCNtSjYyYHqymn6WRMKQ FgWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682098830; x=1684690830; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=2xVbU1EXT3F/5uSzRhbvwRdAhlPEIhLbnGkOO/O2kc0=; b=c+Wkc8tN3ZggZBxjCjd2SDX0WvLvZlsz/epZBGBlsijzeNBj+n9V4kvnIsRxxxVCFF /wxU+KxhazM4g8SKoPm3/kA9bnBA/zhcNvwM0ua96oDg400nTWgxKWBSZJISmtZ8cY46 bvamz0FoTi0LHehvVGI5FnZAzX8A9dpTXMd8he+CQMUUwvZ8YzUYgq3dXppEts2exv+s TgvtvJNSutZY44ZCuTgdMCUzPB7vC3H8iTRoYW83f0EomN8zKV+Dk0J1gwhlm9XTDobY Y5UlEpXsHrSe0hphQMh8fCZOo5ShZYuHG5dLZC1Bf345TMEIGCc1vP1nE0Fa5Vboo8Oe e5hw== X-Gm-Message-State: AAQBX9cedyP3EHPc7imBfgS1I98/7c/sqdePW4/mWeQVv4+Q27cCkqZu x35mhkEZIjEYKkXUc1mWn2mSiRod1w1/hf8Z X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a05:6a00:2e97:b0:62d:9bea:2a0c with SMTP id fd23-20020a056a002e9700b0062d9bea2a0cmr2546450pfb.4.1682098830169; Fri, 21 Apr 2023 10:40:30 -0700 (PDT) Date: Fri, 21 Apr 2023 17:40:19 +0000 In-Reply-To: <20230421174020.2994750-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230421174020.2994750-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog Message-ID: <20230421174020.2994750-5-yosryahmed@google.com> Subject: [PATCH v5 4/5] memcg: remove mem_cgroup_flush_stats_atomic() From: Yosry Ahmed To: Alexander Viro , Christian Brauner , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , Tejun Heo Cc: Jan Kara , Jens Axboe , " =?utf-8?q?Micha?= =?utf-8?q?l_Koutn=C3=BD?= " , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763809719872049972?= X-GMAIL-MSGID: =?utf-8?q?1763809719872049972?= Previous patches removed all callers of mem_cgroup_flush_stats_atomic(). Remove the function and simplify the code. Signed-off-by: Yosry Ahmed Acked-by: Shakeel Butt --- include/linux/memcontrol.h | 5 ----- mm/memcontrol.c | 24 +++++------------------- 2 files changed, 5 insertions(+), 24 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 222d7370134c..00a88cf947e1 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1038,7 +1038,6 @@ static inline unsigned long lruvec_page_state_local(struct lruvec *lruvec, } void mem_cgroup_flush_stats(void); -void mem_cgroup_flush_stats_atomic(void); void mem_cgroup_flush_stats_ratelimited(void); void __mod_memcg_lruvec_state(struct lruvec *lruvec, enum node_stat_item idx, @@ -1537,10 +1536,6 @@ static inline void mem_cgroup_flush_stats(void) { } -static inline void mem_cgroup_flush_stats_atomic(void) -{ -} - static inline void mem_cgroup_flush_stats_ratelimited(void) { } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index cb78bba5b4a4..eadfe371a8f2 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -639,7 +639,7 @@ static inline void memcg_rstat_updated(struct mem_cgroup *memcg, int val) } } -static void do_flush_stats(bool atomic) +static void do_flush_stats(void) { /* * We always flush the entire tree, so concurrent flushers can just @@ -652,30 +652,16 @@ static void do_flush_stats(bool atomic) WRITE_ONCE(flush_next_time, jiffies_64 + 2*FLUSH_TIME); - if (atomic) - cgroup_rstat_flush_atomic(root_mem_cgroup->css.cgroup); - else - cgroup_rstat_flush(root_mem_cgroup->css.cgroup); + cgroup_rstat_flush(root_mem_cgroup->css.cgroup); atomic_set(&stats_flush_threshold, 0); atomic_set(&stats_flush_ongoing, 0); } -static bool should_flush_stats(void) -{ - return atomic_read(&stats_flush_threshold) > num_online_cpus(); -} - void mem_cgroup_flush_stats(void) { - if (should_flush_stats()) - do_flush_stats(false); -} - -void mem_cgroup_flush_stats_atomic(void) -{ - if (should_flush_stats()) - do_flush_stats(true); + if (atomic_read(&stats_flush_threshold) > num_online_cpus()) + do_flush_stats(); } void mem_cgroup_flush_stats_ratelimited(void) @@ -690,7 +676,7 @@ static void flush_memcg_stats_dwork(struct work_struct *w) * Always flush here so that flushing in latency-sensitive paths is * as cheap as possible. */ - do_flush_stats(false); + do_flush_stats(); queue_delayed_work(system_unbound_wq, &stats_flush_dwork, FLUSH_TIME); } From patchwork Fri Apr 21 17:40:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 86451 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp1240028vqo; Fri, 21 Apr 2023 10:45:40 -0700 (PDT) X-Google-Smtp-Source: AKy350apkoOURpMvnUiTDRaon5XWbkKPjYkR1uqCy0oZu2MllARwzUCI0Gsk9lJzno4VBm3xguJn X-Received: by 2002:a05:6a00:2448:b0:63b:7a55:ae89 with SMTP id d8-20020a056a00244800b0063b7a55ae89mr7096272pfj.27.1682099139844; Fri, 21 Apr 2023 10:45:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682099139; cv=none; d=google.com; s=arc-20160816; b=ukEgfAna7eXrlnmfO/73THuANuwRB6gkSbFTZ2vjMw8iwILqs1dK91XZOeifQ/uA8q tdMMi58VwoleHrP9loxEWmFCIO/ijwPDzf94o6rJYot+AzMFfsIiQ02+EOtc1X80mYKI f/GEItfh1JRn0fZpR7WrroAi59PFiQUQr0UKTHzIWMeanhLVyqSoOsRtFei+lIeQF4Gi 4pz50OiLpgOy9GEH2ofQkIPj4m3lw24A9B6+v7oXI46sf/+Z/zvRzAONNFpcdDPrSVX6 AfOmWdpNZL7rT70fEJPVlTngnqQ0J2SZNusy6ych/aMCqV/N7t0/9WMFnVs/E2QJVmOs ITUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:dkim-signature; bh=xEOFJLcgICBTmNTLnyQD1ooBVf7w9cMBHK4zBwAxhps=; b=Klg4mZBjV+VXK6rjFc7+8ifqCyruclho2eWmA4ZJcTwP8iksnjOG9xG1gF+V52atkb DfEtFUX5vd1ncUXfd+JQT7K90L/0/955aR3j9F1lPVwfN1bvkVB1WZumNjo1Rpi2oivo JvlkzUs0bpGvV20vcUWbwOZhg5QHRT28RGJDw/bOVUpunZcuWfmSiHl86yp0RVRTqO3z 5tmqrnlNErMb28cX6ogZVL94uBwJ6/4AsUimXIsUTLtySHsa/qM8VDfGHLS9tNtiLQ7d iuGiS7lQu6ScyTzhZc09FWhldk4BPl4QhlRv9J2o3JDZqpjFXnIFd/yr9B7yLGoNUCCd UI1g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=2LLI0uS0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k16-20020aa79d10000000b0063b731e3938si4656135pfp.2.2023.04.21.10.45.25; Fri, 21 Apr 2023 10:45:39 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20221208 header.b=2LLI0uS0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233386AbjDURk5 (ORCPT + 99 others); Fri, 21 Apr 2023 13:40:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40552 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233318AbjDURke (ORCPT ); Fri, 21 Apr 2023 13:40:34 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B9CD512C96 for ; Fri, 21 Apr 2023 10:40:32 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-b922a14e089so3322273276.2 for ; Fri, 21 Apr 2023 10:40:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1682098832; x=1684690832; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=xEOFJLcgICBTmNTLnyQD1ooBVf7w9cMBHK4zBwAxhps=; b=2LLI0uS0HCGgUG+To5FG73zbxk+tuzk5tTcOBZNGKwnVbYk9yq2r0zkyTl0MD4wfQ3 kKiggVfa7NNclULg8NuxAqHV3zYnq7LZjfMrcVD62yChQCNHWbk54zVsBhxomTb0t0SN +XgJr7wul3fQD2JopF/MZfktxPtuUWKER8IL9ZU+RSEl7sW66jFXxgj6K5ErZztO7Nce l4ZWAyWNanyDDUgKfIgOcWOkpHNwJKJvYyr6/84gH8i9GqoBVPKeVaSusHPk+GdeSMBt A9hidHBqS6YCIVpadKtHiQ8uYxCY8mC3l7iI3bjfr9izY/KQoToKFIDt3lP79eIF321D fMOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682098832; x=1684690832; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xEOFJLcgICBTmNTLnyQD1ooBVf7w9cMBHK4zBwAxhps=; b=HzRxe58rlppsG17pmttBxLwlASnYQQ4eEbDVOPjLvLy+3xSc5RDvNUspzG7x/UjPQ1 zNs1VsxEcCXCf1ISFzJhKdkvT4+fjsBQxGIYEF1NHf+MZ32ZObQPQvi1mBGHadP/xgkS fHz+SIA3pTBu7lpFxiO+PfHh3BqCCo6JFKjYyGDj4OkH5yr6nhpf6GI+WrrMmLDJFz0D w07Fx5XuqjE0lR/Ioqv8/M7y4h94tQ4y+2i+nnVEB5aObnOURcLnhDC+pjZMbGB9R4+R jukJphhxDF6W6I9fxZdyYiq7nolXZWbTpyuWvRFTFNyyE2COgWybkoOuAqXr7lrQ4Pno 9C/A== X-Gm-Message-State: AAQBX9fnlpdw2MYEFHdE6JSpDibH9ONKPUFUmsmnD+WYFyH4XedzRgoo cgAPtRQt+zFsnXuuo0CJNqm1lHdcKJza5oJ8 X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a25:bc4a:0:b0:b33:531b:3dd4 with SMTP id d10-20020a25bc4a000000b00b33531b3dd4mr1583021ybk.1.1682098831943; Fri, 21 Apr 2023 10:40:31 -0700 (PDT) Date: Fri, 21 Apr 2023 17:40:20 +0000 In-Reply-To: <20230421174020.2994750-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230421174020.2994750-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog Message-ID: <20230421174020.2994750-6-yosryahmed@google.com> Subject: [PATCH v5 5/5] cgroup: remove cgroup_rstat_flush_atomic() From: Yosry Ahmed To: Alexander Viro , Christian Brauner , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , Tejun Heo Cc: Jan Kara , Jens Axboe , " =?utf-8?q?Micha?= =?utf-8?q?l_Koutn=C3=BD?= " , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763808787582619108?= X-GMAIL-MSGID: =?utf-8?q?1763808787582619108?= Previous patches removed the only caller of cgroup_rstat_flush_atomic(). Remove the function and simplify the code. Signed-off-by: Yosry Ahmed Acked-by: Shakeel Butt Acked-by: Tejun Heo --- include/linux/cgroup.h | 1 - kernel/cgroup/rstat.c | 26 +++++--------------------- 2 files changed, 5 insertions(+), 22 deletions(-) diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h index 885f5395fcd0..567c547cf371 100644 --- a/include/linux/cgroup.h +++ b/include/linux/cgroup.h @@ -692,7 +692,6 @@ static inline void cgroup_path_from_kernfs_id(u64 id, char *buf, size_t buflen) */ void cgroup_rstat_updated(struct cgroup *cgrp, int cpu); void cgroup_rstat_flush(struct cgroup *cgrp); -void cgroup_rstat_flush_atomic(struct cgroup *cgrp); void cgroup_rstat_flush_hold(struct cgroup *cgrp); void cgroup_rstat_flush_release(void); diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index d3252b0416b6..f9ad33f117c8 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -171,7 +171,7 @@ __weak noinline void bpf_rstat_flush(struct cgroup *cgrp, __diag_pop(); /* see cgroup_rstat_flush() */ -static void cgroup_rstat_flush_locked(struct cgroup *cgrp, bool may_sleep) +static void cgroup_rstat_flush_locked(struct cgroup *cgrp) __releases(&cgroup_rstat_lock) __acquires(&cgroup_rstat_lock) { int cpu; @@ -207,9 +207,8 @@ static void cgroup_rstat_flush_locked(struct cgroup *cgrp, bool may_sleep) } raw_spin_unlock_irqrestore(cpu_lock, flags); - /* if @may_sleep, play nice and yield if necessary */ - if (may_sleep && (need_resched() || - spin_needbreak(&cgroup_rstat_lock))) { + /* play nice and yield if necessary */ + if (need_resched() || spin_needbreak(&cgroup_rstat_lock)) { spin_unlock_irq(&cgroup_rstat_lock); if (!cond_resched()) cpu_relax(); @@ -236,25 +235,10 @@ __bpf_kfunc void cgroup_rstat_flush(struct cgroup *cgrp) might_sleep(); spin_lock_irq(&cgroup_rstat_lock); - cgroup_rstat_flush_locked(cgrp, true); + cgroup_rstat_flush_locked(cgrp); spin_unlock_irq(&cgroup_rstat_lock); } -/** - * cgroup_rstat_flush_atomic- atomic version of cgroup_rstat_flush() - * @cgrp: target cgroup - * - * This function can be called from any context. - */ -void cgroup_rstat_flush_atomic(struct cgroup *cgrp) -{ - unsigned long flags; - - spin_lock_irqsave(&cgroup_rstat_lock, flags); - cgroup_rstat_flush_locked(cgrp, false); - spin_unlock_irqrestore(&cgroup_rstat_lock, flags); -} - /** * cgroup_rstat_flush_hold - flush stats in @cgrp's subtree and hold * @cgrp: target cgroup @@ -269,7 +253,7 @@ void cgroup_rstat_flush_hold(struct cgroup *cgrp) { might_sleep(); spin_lock_irq(&cgroup_rstat_lock); - cgroup_rstat_flush_locked(cgrp, true); + cgroup_rstat_flush_locked(cgrp); } /**