Message ID | 20230619204826.755559-5-yukuai1@huaweicloud.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp2985882vqr; Mon, 19 Jun 2023 05:54:18 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5xwxDV44arWbY6ifpQL8LiM+FAZziGHmGiRS+nM6WF6uGOe6ega0bbIHgrUxXf/oMza3HW X-Received: by 2002:a05:6a20:6a11:b0:120:e40a:a10d with SMTP id p17-20020a056a206a1100b00120e40aa10dmr6782211pzk.38.1687179257945; Mon, 19 Jun 2023 05:54:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687179257; cv=none; d=google.com; s=arc-20160816; b=xp7yv8h8rrtgv4xzjm8yJo1bomDZzAc5XwCWpQhLGOP3u424bCjzp7a/HuaL3Gl2/I z2SVE8+9VmkDUAkG/HQdra8aXL0CgR1Vw/TNS7KW+Wf19PchBm9Ra2UIkO3nFdF5bnvC 1CJhzFBiVyjQCdwvG0Dwc0VGSA0xfVqC5ZD8P3CQvhuyClNL1AxSA3A+RNaCvVjtq+lP y+wLwGKARm+leOr2SXoWYucfOBob+AmhMGcBQbSoWt9YAAmkVOu22CCVE2YA9i1vQ7Tn XHx26LrSD4GvCsPWWq4/v4RiNwxcaeKBIhBZGhnphWMm3jmirm3Y85AVmi0ODtdCCE14 YsUQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=aPjuBRuGJ9MnDqD418eSB9PMX7WlKTu8AEshCRcs1T4=; b=CvWA4OeXQ1qNaTgrjdHItY8fpJPtg0u3hyqQf789juk+9iG/dG9wWm270hZ8Z8mwaL 42hRV6bKkFz71mY7MF9DzWqeJTmgJg+w8EYl2Fx/IdtHtI4EUci7pFagCH4ZMRSqGWDz 3P89IgAg+Tsqxllj7nDKpvfht8qOF0+XKP2CHV6M3nTuqWshxrdUBqF7ZqvxRlQfedMi Ysd3V7P65cOD4k5SkmuppshkeksefhBFzjlOhHnLqMVY73J3xbQxgPtTlgnO/IXaHFk6 rbwIyOJDv5onfca691qqEMDg7OF+jEN+NIr28Mp89324EOQItTnqkAgC8mPutxSRWzj7 Uxvg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l189-20020a6388c6000000b0055382f1e985si6463696pgd.333.2023.06.19.05.54.03; Mon, 19 Jun 2023 05:54:17 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231239AbjFSMuI (ORCPT <rfc822;duw91626@gmail.com> + 99 others); Mon, 19 Jun 2023 08:50:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34536 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231318AbjFSMt4 (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Mon, 19 Jun 2023 08:49:56 -0400 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E042F4; Mon, 19 Jun 2023 05:49:54 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.153]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4Ql8jn1F3Fz4f4HGQ; Mon, 19 Jun 2023 20:49:49 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgBH_rHrTpBkoNw8MA--.51801S8; Mon, 19 Jun 2023 20:49:51 +0800 (CST) From: Yu Kuai <yukuai1@huaweicloud.com> To: xni@redhat.com, song@kernel.org Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai3@huawei.com, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH -next 4/8] md/raid1: switch to use md_account_bio() for io accounting Date: Tue, 20 Jun 2023 04:48:22 +0800 Message-Id: <20230619204826.755559-5-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230619204826.755559-1-yukuai1@huaweicloud.com> References: <20230619204826.755559-1-yukuai1@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-CM-TRANSID: gCh0CgBH_rHrTpBkoNw8MA--.51801S8 X-Coremail-Antispam: 1UD129KBjvJXoW7tw17ur4rZrW8Wr1UtrWrXwb_yoW8uF4fpa 1UWFyru3yrX3yq9w1DJFWUuF1fAan0ga42krWxX3sxZF1avF90qa18XFWfKrn8JF95WFy7 t3WvkF4DCF47tFDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPY14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2jI8I6cxK62vIxIIY0VWUZVW8XwA2048vs2IY02 0E87I2jVAFwI0_JF0E3s1l82xGYIkIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0 rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6x IIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xv wVC2z280aVCY1x0267AKxVW0oVCq3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFc xC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_ Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2 IErcIFxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E 14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_Jw0_GFylIx kGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAF wI0_Gr0_Cr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r 4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x0pRvJPtU UUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-CFilter-Loop: Reflected X-Spam-Status: No, score=0.0 required=5.0 tests=BAYES_00,DATE_IN_FUTURE_06_12, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1769135677641719722?= X-GMAIL-MSGID: =?utf-8?q?1769135677641719722?= |
Series |
md: fix and refactor io accounting and 'active_io'
|
|
Commit Message
Yu Kuai
June 19, 2023, 8:48 p.m. UTC
From: Yu Kuai <yukuai3@huawei.com> Two problems can be fixed this way: 1) 'active_io' will represent inflight io instead of io that is dispatching. 2) If io accounting is enabled or disabled while io is still inflight, bio_start_io_acct() and bio_end_io_acct() is not balanced and io inflight counter will be leaked. Signed-off-by: Yu Kuai <yukuai3@huawei.com> --- drivers/md/raid1.c | 14 ++++++-------- drivers/md/raid1.h | 1 - 2 files changed, 6 insertions(+), 9 deletions(-)
Comments
On Mon, Jun 19, 2023 at 8:49 PM Yu Kuai <yukuai1@huaweicloud.com> wrote: > > From: Yu Kuai <yukuai3@huawei.com> > > Two problems can be fixed this way: > > 1) 'active_io' will represent inflight io instead of io that is > dispatching. > > 2) If io accounting is enabled or disabled while io is still inflight, > bio_start_io_acct() and bio_end_io_acct() is not balanced and io > inflight counter will be leaked. > > Signed-off-by: Yu Kuai <yukuai3@huawei.com> > --- > drivers/md/raid1.c | 14 ++++++-------- > drivers/md/raid1.h | 1 - > 2 files changed, 6 insertions(+), 9 deletions(-) > > diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c > index dd25832eb045..06fa1580501f 100644 > --- a/drivers/md/raid1.c > +++ b/drivers/md/raid1.c > @@ -304,8 +304,6 @@ static void call_bio_endio(struct r1bio *r1_bio) > if (!test_bit(R1BIO_Uptodate, &r1_bio->state)) > bio->bi_status = BLK_STS_IOERR; > > - if (blk_queue_io_stat(bio->bi_bdev->bd_disk->queue)) > - bio_end_io_acct(bio, r1_bio->start_time); > bio_endio(bio); > } > > @@ -1303,10 +1301,10 @@ static void raid1_read_request(struct mddev *mddev, struct bio *bio, > } > > r1_bio->read_disk = rdisk; > - > - if (!r1bio_existed && blk_queue_io_stat(bio->bi_bdev->bd_disk->queue)) > - r1_bio->start_time = bio_start_io_acct(bio); > - > + if (!r1bio_existed) { > + md_account_bio(mddev, &bio); > + r1_bio->master_bio = bio; > + } > read_bio = bio_alloc_clone(mirror->rdev->bdev, bio, gfp, > &mddev->bio_set); > > @@ -1500,8 +1498,8 @@ static void raid1_write_request(struct mddev *mddev, struct bio *bio, > r1_bio->sectors = max_sectors; > } > > - if (blk_queue_io_stat(bio->bi_bdev->bd_disk->queue)) > - r1_bio->start_time = bio_start_io_acct(bio); > + md_account_bio(mddev, &bio); > + r1_bio->master_bio = bio; > atomic_set(&r1_bio->remaining, 1); > atomic_set(&r1_bio->behind_remaining, 0); > > diff --git a/drivers/md/raid1.h b/drivers/md/raid1.h > index 468f189da7a0..14d4211a123a 100644 > --- a/drivers/md/raid1.h > +++ b/drivers/md/raid1.h > @@ -157,7 +157,6 @@ struct r1bio { > sector_t sector; > int sectors; > unsigned long state; > - unsigned long start_time; > struct mddev *mddev; > /* > * original bio going to /dev/mdx > -- > 2.39.2 > Hi Kuai After this patch, raid1 will have one more memory allocation in the I/O path. Not sure if it can affect performance. Beside this, the patch is good for me. Reviewed-by: Xiao Ni <xni@redhat.com>
Hi, 在 2023/06/20 17:07, Xiao Ni 写道: > On Mon, Jun 19, 2023 at 8:49 PM Yu Kuai <yukuai1@huaweicloud.com> wrote: >> >> From: Yu Kuai <yukuai3@huawei.com> >> >> Two problems can be fixed this way: >> >> 1) 'active_io' will represent inflight io instead of io that is >> dispatching. >> >> 2) If io accounting is enabled or disabled while io is still inflight, >> bio_start_io_acct() and bio_end_io_acct() is not balanced and io >> inflight counter will be leaked. >> >> Signed-off-by: Yu Kuai <yukuai3@huawei.com> >> --- >> drivers/md/raid1.c | 14 ++++++-------- >> drivers/md/raid1.h | 1 - >> 2 files changed, 6 insertions(+), 9 deletions(-) >> >> diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c >> index dd25832eb045..06fa1580501f 100644 >> --- a/drivers/md/raid1.c >> +++ b/drivers/md/raid1.c >> @@ -304,8 +304,6 @@ static void call_bio_endio(struct r1bio *r1_bio) >> if (!test_bit(R1BIO_Uptodate, &r1_bio->state)) >> bio->bi_status = BLK_STS_IOERR; >> >> - if (blk_queue_io_stat(bio->bi_bdev->bd_disk->queue)) >> - bio_end_io_acct(bio, r1_bio->start_time); >> bio_endio(bio); >> } >> >> @@ -1303,10 +1301,10 @@ static void raid1_read_request(struct mddev *mddev, struct bio *bio, >> } >> >> r1_bio->read_disk = rdisk; >> - >> - if (!r1bio_existed && blk_queue_io_stat(bio->bi_bdev->bd_disk->queue)) >> - r1_bio->start_time = bio_start_io_acct(bio); >> - >> + if (!r1bio_existed) { >> + md_account_bio(mddev, &bio); >> + r1_bio->master_bio = bio; >> + } >> read_bio = bio_alloc_clone(mirror->rdev->bdev, bio, gfp, >> &mddev->bio_set); >> >> @@ -1500,8 +1498,8 @@ static void raid1_write_request(struct mddev *mddev, struct bio *bio, >> r1_bio->sectors = max_sectors; >> } >> >> - if (blk_queue_io_stat(bio->bi_bdev->bd_disk->queue)) >> - r1_bio->start_time = bio_start_io_acct(bio); >> + md_account_bio(mddev, &bio); >> + r1_bio->master_bio = bio; >> atomic_set(&r1_bio->remaining, 1); >> atomic_set(&r1_bio->behind_remaining, 0); >> >> diff --git a/drivers/md/raid1.h b/drivers/md/raid1.h >> index 468f189da7a0..14d4211a123a 100644 >> --- a/drivers/md/raid1.h >> +++ b/drivers/md/raid1.h >> @@ -157,7 +157,6 @@ struct r1bio { >> sector_t sector; >> int sectors; >> unsigned long state; >> - unsigned long start_time; >> struct mddev *mddev; >> /* >> * original bio going to /dev/mdx >> -- >> 2.39.2 >> > > Hi Kuai > > After this patch, raid1 will have one more memory allocation in the > I/O path. Not sure if it can affect performance. Beside this, the > patch is good for me. Yes, I'm aware of this additional memory allocation, however, raid1(and similar to other levels) already need to allocate r1bio and some bios(1 for read, and copies for write), so this is not a none -> new case, it's just a allocate 2 -> allocate 3 case. I think performance under memory pressure are both bad with or without this patch, and one one bio clone latency without memory reclaim should be fine. Thanks, Kuai > > Reviewed-by: Xiao Ni <xni@redhat.com> > > . >
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index dd25832eb045..06fa1580501f 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -304,8 +304,6 @@ static void call_bio_endio(struct r1bio *r1_bio) if (!test_bit(R1BIO_Uptodate, &r1_bio->state)) bio->bi_status = BLK_STS_IOERR; - if (blk_queue_io_stat(bio->bi_bdev->bd_disk->queue)) - bio_end_io_acct(bio, r1_bio->start_time); bio_endio(bio); } @@ -1303,10 +1301,10 @@ static void raid1_read_request(struct mddev *mddev, struct bio *bio, } r1_bio->read_disk = rdisk; - - if (!r1bio_existed && blk_queue_io_stat(bio->bi_bdev->bd_disk->queue)) - r1_bio->start_time = bio_start_io_acct(bio); - + if (!r1bio_existed) { + md_account_bio(mddev, &bio); + r1_bio->master_bio = bio; + } read_bio = bio_alloc_clone(mirror->rdev->bdev, bio, gfp, &mddev->bio_set); @@ -1500,8 +1498,8 @@ static void raid1_write_request(struct mddev *mddev, struct bio *bio, r1_bio->sectors = max_sectors; } - if (blk_queue_io_stat(bio->bi_bdev->bd_disk->queue)) - r1_bio->start_time = bio_start_io_acct(bio); + md_account_bio(mddev, &bio); + r1_bio->master_bio = bio; atomic_set(&r1_bio->remaining, 1); atomic_set(&r1_bio->behind_remaining, 0); diff --git a/drivers/md/raid1.h b/drivers/md/raid1.h index 468f189da7a0..14d4211a123a 100644 --- a/drivers/md/raid1.h +++ b/drivers/md/raid1.h @@ -157,7 +157,6 @@ struct r1bio { sector_t sector; int sectors; unsigned long state; - unsigned long start_time; struct mddev *mddev; /* * original bio going to /dev/mdx