From patchwork Thu Apr 20 11:29:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 85878 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp265822vqo; Thu, 20 Apr 2023 04:40:58 -0700 (PDT) X-Google-Smtp-Source: AKy350asr1NqKU47u3nwyWSTQaID8/DzFDEwI6ZENuDHLrjtGHhQOaaM2uvgQ6T52QqsDFk2c0sH X-Received: by 2002:a17:903:1250:b0:1a6:cd08:5594 with SMTP id u16-20020a170903125000b001a6cd085594mr1240642plh.69.1681990858161; Thu, 20 Apr 2023 04:40:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681990858; cv=none; d=google.com; s=arc-20160816; b=S7JLbg3/QkiVuuFneMO/RmWA97PBTpQauCYpXj9lCh8APIAZ0RoezgJZRwU1PYEX2W bg6sGRaqTFjoBPzs28/LR/2ue5v9jKSxT4KY6mhEgJpi2DMhrALA/v5+JKunuOmcfClS PVpkMnl/qZTzIE39ECGgKveUntAG3KsqOvbv63uubvuDNRjSPDy1wodc+MeFy0PHm5BG GIFpDcIbxS5jb5J8+V3ZLoqIJw73gW/B2s/2xtB5WIU2o77dEcME6jyqhgqgN9+RlpHK lb0FpqhEx5+NPH8hYaKjovNLNidGzXzsPP2DyARAZc21InRM72BtMWcwKliCttBPUk2z W9dw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=fcaONA0Ons4m36pBA9WoLnnqyvns7eV5VTqteBNA9tQ=; b=qtlXZKXT5c4S/W1C3l92Dc29htsRMpgoL0NhTlWTGrs0QUHbbFOtDpemHBlmxfd9Ah iWSh1qMvxHxSAFg1IBPGhwjnQUceXIAC2i0oORnsRlFxm/W/68i7vrXRwHDcI7Db/c+v +z4TY4PHWWOgdf3iZmKHx29eWKAoYSyKyS6w/0oNwSA63gyW7vWCgfwguX6w9W+pIFIx 0hEmGSk+nO5E2+51GbHfw2HAREd0mwADc2FXGD2l6FZg5AHfQQHoEjIXY+AjzVh6mU4P KH0e7XVdaKgVoW9Xj3dgY0J2SqIYFpmlcrD7Ir2MoTaisZPSwz+ktaIClZSEXc4CqKjn DQkQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id v6-20020a655c46000000b0051b10dc3a47si1519763pgr.158.2023.04.20.04.40.45; Thu, 20 Apr 2023 04:40:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234584AbjDTLcO (ORCPT + 99 others); Thu, 20 Apr 2023 07:32:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34020 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234139AbjDTLbp (ORCPT ); Thu, 20 Apr 2023 07:31:45 -0400 Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7472C1FC4; Thu, 20 Apr 2023 04:31:22 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.153]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4Q2Fpq6XV8z4f3yNq; Thu, 20 Apr 2023 19:31:15 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCnD7OAIkFkVY8hHw--.17021S12; Thu, 20 Apr 2023 19:31:17 +0800 (CST) From: Yu Kuai To: song@kernel.org, neilb@suse.de, akpm@osdl.org Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai3@huawei.com, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH -next 8/8] md/raid1-10: limit the number of plugged bio Date: Thu, 20 Apr 2023 19:29:46 +0800 Message-Id: <20230420112946.2869956-9-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230420112946.2869956-1-yukuai1@huaweicloud.com> References: <20230420112946.2869956-1-yukuai1@huaweicloud.com> MIME-Version: 1.0 X-CM-TRANSID: gCh0CgCnD7OAIkFkVY8hHw--.17021S12 X-Coremail-Antispam: 1UD129KBjvJXoWxWw1DZF4xXr18CFyxXw47Jwb_yoW5uryfpa 1Uta4avrWUZrWxX3yDJa1UCFyFqw4DXFWqkFZ5C395tFy7XFWjga1rJFWrur1DZFW3Gry3 JFn0krWxGF15tF7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUU9K14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_Ar0_tr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwCF04k20xvY0x0EwIxGrw CFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE 14v26r106r1rMI8E67AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2 IY67AKxVWUCVW8JwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr1UMIIF0xvE42xK8VAv wI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14 v26r4j6r4UJbIYCTnIWIevJa73UjIFyTuYvjfUOBTYUUUUU X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,KHOP_HELO_FCRDNS, MAY_BE_FORGED,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763695245845230305?= X-GMAIL-MSGID: =?utf-8?q?1763695245845230305?= From: Yu Kuai bio can be added to plug infinitely, and following writeback test can trigger huge amount of pluged bio: Test script: modprobe brd rd_nr=4 rd_size=10485760 mdadm -CR /dev/md0 -l10 -n4 /dev/ram[0123] --assume-clean echo 0 > /proc/sys/vm/dirty_background_ratio echo 60 > /proc/sys/vm/dirty_ratio fio -filename=/dev/md0 -ioengine=libaio -rw=write -thread -bs=1k-8k -numjobs=1 -iodepth=128 -name=xxx Test result: Monitor /sys/block/md0/inflight will found that inflight keep increasing until fio finish writing, after running for about 2 minutes: [root@fedora ~]# cat /sys/block/md0/inflight 0 4474191 Fix the problem by limiting the number of pluged bio based on the number of copies for orininal bio. Signed-off-by: Yu Kuai --- drivers/md/raid1-10.h | 9 ++++++++- drivers/md/raid1.c | 2 +- drivers/md/raid10.c | 2 +- 3 files changed, 10 insertions(+), 3 deletions(-) diff --git a/drivers/md/raid1-10.h b/drivers/md/raid1-10.h index 95b2fb4dd9aa..2785ae805953 100644 --- a/drivers/md/raid1-10.h +++ b/drivers/md/raid1-10.h @@ -33,9 +33,12 @@ struct resync_pages { struct page *pages[RESYNC_PAGES]; }; +#define MAX_PLUG_BIO 32 + struct raid1_plug_cb { struct blk_plug_cb cb; struct bio_list pending; + unsigned int count; }; static inline void rbio_pool_free(void *rbio, void *data) @@ -132,7 +135,7 @@ static inline void md_submit_write(struct bio *bio) } static inline bool md_add_bio_to_plug(struct mddev *mddev, struct bio *bio, - blk_plug_cb_fn unplug) + blk_plug_cb_fn unplug, int copies) { struct raid1_plug_cb *plug = NULL; struct blk_plug_cb *cb; @@ -152,6 +155,10 @@ static inline bool md_add_bio_to_plug(struct mddev *mddev, struct bio *bio, plug = container_of(cb, struct raid1_plug_cb, cb); bio_list_add(&plug->pending, bio); + if (++plug->count / MAX_PLUG_BIO >= copies) { + list_del(&cb->list); + cb->callback(cb, false); + } return true; } diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index 91e1dbc48228..6a38104a7b89 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -1561,7 +1561,7 @@ static void raid1_write_request(struct mddev *mddev, struct bio *bio, r1_bio->sector); /* flush_pending_writes() needs access to the rdev so...*/ mbio->bi_bdev = (void *)rdev; - if (!md_add_bio_to_plug(mddev, mbio, raid1_unplug)) { + if (!md_add_bio_to_plug(mddev, mbio, raid1_unplug, disks)) { spin_lock_irqsave(&conf->device_lock, flags); bio_list_add(&conf->pending_bio_list, mbio); spin_unlock_irqrestore(&conf->device_lock, flags); diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index d92b1efe9eee..721d50646043 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -1300,7 +1300,7 @@ static void raid10_write_one_disk(struct mddev *mddev, struct r10bio *r10_bio, atomic_inc(&r10_bio->remaining); - if (!md_add_bio_to_plug(mddev, mbio, raid10_unplug)) { + if (!md_add_bio_to_plug(mddev, mbio, raid10_unplug, conf->copies)) { spin_lock_irqsave(&conf->device_lock, flags); bio_list_add(&conf->pending_bio_list, mbio); spin_unlock_irqrestore(&conf->device_lock, flags);