From patchwork Thu Oct 13 22:41:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Derrick X-Patchwork-Id: 2435 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4ac7:0:0:0:0:0 with SMTP id y7csp512661wrs; Thu, 13 Oct 2022 15:46:57 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4MH8kHD95aTeAFOQkzKvPCs/rbgx2DtLXUD0JwhMXfef5EEk2Vx9YDGkzWIJa/1zxEWKsl X-Received: by 2002:a17:906:5dda:b0:78d:e7d2:7499 with SMTP id p26-20020a1709065dda00b0078de7d27499mr1438460ejv.588.1665701217098; Thu, 13 Oct 2022 15:46:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1665701217; cv=none; d=google.com; s=arc-20160816; b=Hsn/7dbKlR0OzZrlkUWeHauFJL9JS1p/tUdp/JTpqwWoCbOtCcseIrKtp2Rbk+ExYF CT+oh00ISu/Qk/Nn6lrR2Jy9Qrev1Ndy09XOaaEG9S1Lg0KEiXMO3wIG3fPr9cBaCZrr /5WksDPPXC8dnNW+9MzC1C0tEa4nK/CyeVsIXPaBiKKSdGMkJZmKfCo4R/joTgij3cnC wof22kq94ud2VA+JOuCZa931ItRfP0F0QIgwCqkDpVY1QvUbebrPCYbAKBFrO1qoAhIb wKuqlFbZZts3wdgWxnOANcZxzeMBhZc76Xo4Niycq8hEqS2+WmGghX/k378bctkCW7vj 8v9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Vod9bfBwBD3lRXXsfZrFXG7EXQYLJ2cHVyFKENUAAzU=; b=lHwuu0u4NDJHYGP2pho4B4V6/VhONrLenlNtUBUBUHlzb/CYsBH96vHziyizQ3WcX3 fS7ssSF/HCzSlds1ZThgpLD78/AxFJxIXoUYN9gOjQ7a6OZhpx8WNv9PgXSqkLkmywbc Dg0smxI9deovuacDtgScjSm9p0mSzYSbIpSSnWvxmNivXphuMzX9GDN+3fMyflVYf017 33ZI6bv0v37t78whpN8p9wABuTPx/ekhOqvLsYnGYhEy/uvuZf4FGJbvzPUaNYsgAt9D FoQQbGvFi6A3JJy1yb8iuTR042TAOeLJzClFDXehCCukt9eAvwHmjS3EYgk3kSY2mC4s fSRw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@comcastmailservice.net header.s=20211018a header.b=J690zIjB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id et3-20020a056402378300b004534c7d4ebfsi773407edb.434.2022.10.13.15.46.30; Thu, 13 Oct 2022 15:46:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@comcastmailservice.net header.s=20211018a header.b=J690zIjB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229833AbiJMWpL (ORCPT + 99 others); Thu, 13 Oct 2022 18:45:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33300 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229773AbiJMWpE (ORCPT ); Thu, 13 Oct 2022 18:45:04 -0400 Received: from resqmta-c1p-023462.sys.comcast.net (resqmta-c1p-023462.sys.comcast.net [IPv6:2001:558:fd00:56::2]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B454A5B05E for ; Thu, 13 Oct 2022 15:44:56 -0700 (PDT) Received: from resomta-c1p-023267.sys.comcast.net ([96.102.18.232]) by resqmta-c1p-023462.sys.comcast.net with ESMTP id j6RpomdrT03bPj6uGoD3ZD; Thu, 13 Oct 2022 22:42:24 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcastmailservice.net; s=20211018a; t=1665700944; bh=Vod9bfBwBD3lRXXsfZrFXG7EXQYLJ2cHVyFKENUAAzU=; h=Received:Received:From:To:Subject:Date:Message-Id:MIME-Version; b=J690zIjBgP0kPbDXXI+Neca+cVOheOOsmD6EbSP2jjoIDXf24bi9A40PBQMNAfYAH f3ELOCc4QbG3JzZTRCawGr1f4VqxcbPv8bWvI7LdOQ7NA4pW4kr+cYBR1qRd7e8N7h SWxTvgvbnZGxVGIIY+xzhNAgIglF793NBeWjsZ18UNXvMfes1vR6aBOYJNKA3Iaujc PpS/wBZTN3TJYdw6xQCk4McWlLRyqQRnvLh05sSWN9eWE7D7SDFhx4OtFX8QfAJxEf +jjzrTZfjKbqKUe2JuW3OwHxsMZSxj6KNnSpKdqkVLQTuDCtotW3xdvj9qzwVL8NQF Kg7TmB2D1jFqA== Received: from jderrick-mobl4.amr.corp.intel.com ([71.205.181.50]) by resomta-c1p-023267.sys.comcast.net with ESMTPA id j6toofOVmA6uYj6tvozg6y; Thu, 13 Oct 2022 22:42:04 +0000 X-Xfinity-VAAS: gggruggvucftvghtrhhoucdtuddrgedvfedrfeekuddgudefucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuvehomhgtrghsthdqtfgvshhipdfqfgfvpdfpqffurfetoffkrfenuceurghilhhouhhtmecufedtudenucesvcftvggtihhpihgvnhhtshculddquddttddmnecujfgurhephffvvefufffkofgjfhgggfestdekredtredttdenucfhrhhomheplfhonhgrthhhrghnucffvghrrhhitghkuceojhhonhgrthhhrghnrdguvghrrhhitghksehlihhnuhigrdguvghvqeenucggtffrrghtthgvrhhnpedtteeljeffgfffveehhfetveefuedvheevffffhedtjeeuvdevgfeftddtheeftdenucfkphepjedurddvtdehrddukedurdehtdenucevlhhushhtvghrufhiiigvpedvnecurfgrrhgrmhephhgvlhhopehjuggvrhhrihgtkhdqmhhosghlgedrrghmrhdrtghorhhprdhinhhtvghlrdgtohhmpdhinhgvthepjedurddvtdehrddukedurdehtddpmhgrihhlfhhrohhmpehjohhnrghthhgrnhdruggvrhhrihgtkheslhhinhhugidruggvvhdpnhgspghrtghpthhtohepjedprhgtphhtthhopehsohhngheskhgvrhhnvghlrdhorhhgpdhrtghpthhtoheplhhinhhugidqrhgrihgusehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheplhhinhhugidqkhgvrhhnvghlsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepjhhonhgrthhhrghnrdguvghrrhhitghkse hsohhlihguihhgmhdrtghomhdprhgtphhtthhopehjohhnrghthhgrnhigrdhskhdruggvrhhrihgtkhesihhnthgvlhdrtghomhdprhgtphhtthhopehmrghrihhushiirdhtkhgrtgiihihksehlihhnuhigrdhinhhtvghlrdgtohhmpdhrtghpthhtohepjhhonhgrthhhrghnrdguvghrrhhitghksehlihhnuhigrdguvghv X-Xfinity-VMeta: sc=-100.00;st=legit From: Jonathan Derrick To: Song Liu Cc: , , jonathan.derrick@solidigm.com, jonathanx.sk.derrick@intel.com, Mariusz Tkaczyk , Jonathan Derrick Subject: [PATCH v2 3/3] md/bitmap: Convert daemon_work to proper timer Date: Thu, 13 Oct 2022 16:41:51 -0600 Message-Id: <20221013224151.300-4-jonathan.derrick@linux.dev> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221013224151.300-1-jonathan.derrick@linux.dev> References: <20221013224151.300-1-jonathan.derrick@linux.dev> MIME-Version: 1.0 X-Spam-Status: No, score=-1.2 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,SPF_HELO_PASS,SPF_SOFTFAIL autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1746614319580109673?= X-GMAIL-MSGID: =?utf-8?q?1746614319580109673?= It was observed that with certain high I/O workloads that the daemon work may never run, preventing the bitmap from fully flushing. Ensure the bitmap fully flushes by converting the daemon worker to a proper timer. Signed-off-by: Jonathan Derrick --- drivers/md/md-bitmap.c | 30 +++++++++++++----------------- drivers/md/md-bitmap.h | 3 +-- drivers/md/md.c | 9 +++++++-- drivers/md/md.h | 1 + 4 files changed, 22 insertions(+), 21 deletions(-) diff --git a/drivers/md/md-bitmap.c b/drivers/md/md-bitmap.c index cd8250368860..34feb906243f 100644 --- a/drivers/md/md-bitmap.c +++ b/drivers/md/md-bitmap.c @@ -1250,8 +1250,9 @@ static bitmap_counter_t *md_bitmap_get_counter(struct bitmap_counts *bitmap, * out to disk */ -void md_bitmap_daemon_work(struct mddev *mddev) +void md_bitmap_daemon_work(struct timer_list *t) { + struct mddev *mddev = from_timer(mddev, t, daemon_timer); struct bitmap *bitmap; unsigned long j; unsigned long nextpage; @@ -1267,11 +1268,7 @@ void md_bitmap_daemon_work(struct mddev *mddev) mutex_unlock(&mddev->bitmap_info.mutex); return; } - if (time_before(jiffies, bitmap->daemon_lastrun - + mddev->bitmap_info.daemon_sleep)) - goto done; - bitmap->daemon_lastrun = jiffies; if (bitmap->allclean) { mddev->thread->timeout = MAX_SCHEDULE_TIMEOUT; goto done; @@ -1372,6 +1369,7 @@ void md_bitmap_daemon_work(struct mddev *mddev) if (bitmap->allclean == 0) mddev->thread->timeout = mddev->bitmap_info.daemon_sleep; + mod_timer(&mddev->daemon_timer, jiffies + mddev->bitmap_info.daemon_sleep); mutex_unlock(&mddev->bitmap_info.mutex); } @@ -1735,21 +1733,16 @@ void md_bitmap_dirty_bits(struct bitmap *bitmap, unsigned long s, unsigned long void md_bitmap_flush(struct mddev *mddev) { struct bitmap *bitmap = mddev->bitmap; - long sleep; if (!bitmap) /* there was no bitmap */ return; /* run the daemon_work three time to ensure everything is flushed - * that can be - */ - sleep = mddev->bitmap_info.daemon_sleep * 2; - bitmap->daemon_lastrun -= sleep; - md_bitmap_daemon_work(mddev); - bitmap->daemon_lastrun -= sleep; - md_bitmap_daemon_work(mddev); - bitmap->daemon_lastrun -= sleep; - md_bitmap_daemon_work(mddev); + * that can be + */ + md_bitmap_daemon_work(&mddev->daemon_timer); + md_bitmap_daemon_work(&mddev->daemon_timer); + md_bitmap_daemon_work(&mddev->daemon_timer); if (mddev->bitmap_info.external) md_super_wait(mddev); md_bitmap_update_sb(bitmap); @@ -1826,7 +1819,7 @@ void md_bitmap_destroy(struct mddev *mddev) mutex_unlock(&mddev->bitmap_info.mutex); if (mddev->thread) mddev->thread->timeout = MAX_SCHEDULE_TIMEOUT; - + del_timer_sync(&mddev->daemon_timer); md_bitmap_free(bitmap); } @@ -1904,7 +1897,10 @@ struct bitmap *md_bitmap_create(struct mddev *mddev, int slot) if (err) goto error; - bitmap->daemon_lastrun = jiffies; + timer_setup(&mddev->daemon_timer, md_bitmap_daemon_work, 0); + mddev->daemon_timer.expires = jiffies + mddev->bitmap_info.daemon_sleep; + add_timer(&mddev->daemon_timer); + err = md_bitmap_resize(bitmap, blocks, mddev->bitmap_info.chunksize, 1); if (err) goto error; diff --git a/drivers/md/md-bitmap.h b/drivers/md/md-bitmap.h index 49a93d8ff307..b7e8f2266bf2 100644 --- a/drivers/md/md-bitmap.h +++ b/drivers/md/md-bitmap.h @@ -220,7 +220,6 @@ struct bitmap { * the bitmap daemon - periodically wakes up and sweeps the bitmap * file, cleaning up bits and flushing out pages to disk as necessary */ - unsigned long daemon_lastrun; /* jiffies of last run */ unsigned long last_end_sync; /* when we lasted called end_sync to * update bitmap with resync progress */ unsigned long unplugged_count; /* last dirty count from md_bitmap_unplug */ @@ -265,7 +264,7 @@ void md_bitmap_sync_with_cluster(struct mddev *mddev, sector_t new_lo, sector_t new_hi); void md_bitmap_unplug(struct bitmap *bitmap); -void md_bitmap_daemon_work(struct mddev *mddev); +void md_bitmap_daemon_work(struct timer_list *t); int md_bitmap_resize(struct bitmap *bitmap, sector_t blocks, int chunksize, int init); diff --git a/drivers/md/md.c b/drivers/md/md.c index afaf36b2f6ab..9f8a9f62b3db 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -9265,8 +9265,13 @@ void md_check_recovery(struct mddev *mddev) if (mddev->suspended) return; - if (mddev->bitmap) - md_bitmap_daemon_work(mddev); + if (mddev->bitmap) { + spin_lock(&pers_lock); + if (mddev->bitmap->allclean == 0) + mddev->thread->timeout = + mddev->bitmap_info.daemon_sleep; + spin_unlock(&pers_lock); + } if (signal_pending(current)) { if (mddev->pers->sync_request && !mddev->external) { diff --git a/drivers/md/md.h b/drivers/md/md.h index 1a558cb18bd4..578cc496c325 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -503,6 +503,7 @@ struct mddev { char cluster_name[64]; /* Name of the cluster */ unsigned int flush_threshold; /* how many dirty chunks between updates */ } bitmap_info; + struct timer_list daemon_timer; atomic_t max_corr_read_errors; /* max read retries */ struct list_head all_mddevs;