From patchwork Wed Aug 16 01:27:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 135798 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b82d:0:b0:3f2:4152:657d with SMTP id z13csp149473vqi; Wed, 16 Aug 2023 09:54:22 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEm87/O8oKy15DIU71HJxAQuMxyBFUERnxM/1YtKPe+fwAHjSUtVUAdjQom9ykmpIoAlM3e X-Received: by 2002:a2e:8541:0:b0:2b6:df5d:8e05 with SMTP id u1-20020a2e8541000000b002b6df5d8e05mr2108392ljj.33.1692204862081; Wed, 16 Aug 2023 09:54:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692204862; cv=none; d=google.com; s=arc-20160816; b=v0KaeSK6GS7oumuenxPb/RFpQzrcgYC5RU8Iavzr6sKLjX+fIgJW5hF0MaXJOsfuS4 fVBKKO0txVb28VMgx9c1+/vmSZbo3FDdC37a8VQL4RobEKg0m2lWfY9FayAFCOwjN/Xj Rri2wYPzwqcitGDjEQYki747l2a65Qw2VOgUjODTzJFxfOgzHEIrT8ECqDZ+klYUjtyK uUMQ9BE1linbTuFCrTikRCQParG+ifjjDwh9vnnoewrb1ru+L5Jx8oA7DoGZVdT2CM10 KISt7/fdhpidDGpulHapNLTL4WV33JzH4i3tfFxQHpIUGz0ewTb51zvZXHMWVa7YKK/M wN6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=fGcHa4wxK8KTILJlomxUt8qBCrCKcfbG3FWGNsJqGr0=; fh=wTguGu9iHUye/+4XPiOZzVDD/C/LKyDnbGOrb/C9kEo=; b=o/uJ9EcI9410RE9mvlUt3Uch11QKAj8X0nleSVSqWIaEMz46+533gheI0p91w+TN6c xzn2GdH1EW3vXo7BzkQsvGaIknr1m4I0sxQ5FieaotOK9i6zL+RsuROmFk+AP7Mr+g5b Bhw9lWbrByqgTlBNWzqVvhyTv3PrB/vsu/ma5G+jekXZkOC4hS9lj7ACdSYiY+X2h5wX yAF5kkj6wKTm8LY1O+SKBeFQ5O4W2RThyDHiohHxCZ8K5DV1M3gJcNzXDJST/xEBQMoD 0JorjiomkQherfSsjzVwsB3D2E03blBNuYZQ+w4ukUQbfo61Z8bTWh7d3/Qn1Z4Khtjm tmnw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id rk8-20020a170907214800b00992e26642ddsi11245442ejb.251.2023.08.16.09.53.58; Wed, 16 Aug 2023 09:54:22 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241078AbjHPBbA (ORCPT + 99 others); Tue, 15 Aug 2023 21:31:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46750 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241114AbjHPBax (ORCPT ); Tue, 15 Aug 2023 21:30:53 -0400 Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 64888211E; Tue, 15 Aug 2023 18:30:52 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.169]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4RQVvX5Ktsz4f3nV9; Wed, 16 Aug 2023 09:30:48 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgAH5KbHJtxkQuVAAw--.55976S5; Wed, 16 Aug 2023 09:30:49 +0800 (CST) From: Yu Kuai To: tj@kernel.org, josef@toxicpanda.com, axboe@kernel.dk, yukuai3@huawei.com, mkoutny@suse.com Cc: cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH -next v2 1/4] blk-throttle: print signed value 'carryover_bytes/ios' for user Date: Wed, 16 Aug 2023 09:27:05 +0800 Message-Id: <20230816012708.1193747-2-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230816012708.1193747-1-yukuai1@huaweicloud.com> References: <20230816012708.1193747-1-yukuai1@huaweicloud.com> MIME-Version: 1.0 X-CM-TRANSID: gCh0CgAH5KbHJtxkQuVAAw--.55976S5 X-Coremail-Antispam: 1UD129KBjvJXoW7ZFWrAF1UJr48WF1fArWxCrg_yoW8XF4kpF W3KrW8GF12qFnxCa13G3W5t3yUZan7Jry8A390kF13AF12k34qgr95ur1Skay0yFn3CF4v v34qqryxJF1Uu37anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBK14x267AKxVW5JVWrJwAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_Jr4l82xGYIkIc2 x26xkF7I0E14v26r4j6ryUM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_Xr0_Ar1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr1UM2 8EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AI xVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20x vE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xv r2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4IIrI8v6xkF7I0E8cxan2IY04 v7MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAFwI0_ Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVWUtVW8ZwCIc40Y0x 0EwIxGrwCI42IY6xIIjxv20xvE14v26r1j6r1xMIIF0xvE2Ix0cI8IcVCY1x0267AKxVW8 JVWxJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Jr0_Gr1lIx AIcVC2z280aVCY1x0267AKxVW8JVW8JrUvcSsGvfC2KfnxnUUI43ZEXa7VUjeWlDUUUUU= = X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1774405405534036865 X-GMAIL-MSGID: 1774405405534036865 From: Yu Kuai 'carryover_bytes/ios' can be negative, indicate that some bio is dispatched in advance within slice while configuration is updated. Print a huge value is not user-friendly. Signed-off-by: Yu Kuai --- block/blk-throttle.c | 2 +- block/blk-throttle.h | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/block/blk-throttle.c b/block/blk-throttle.c index 7397ff199d66..5184f17f5129 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -816,7 +816,7 @@ static void tg_update_carryover(struct throtl_grp *tg) __tg_update_carryover(tg, WRITE); /* see comments in struct throtl_grp for meaning of these fields. */ - throtl_log(&tg->service_queue, "%s: %llu %llu %u %u\n", __func__, + throtl_log(&tg->service_queue, "%s: %lld %lld %d %d\n", __func__, tg->carryover_bytes[READ], tg->carryover_bytes[WRITE], tg->carryover_ios[READ], tg->carryover_ios[WRITE]); } diff --git a/block/blk-throttle.h b/block/blk-throttle.h index d1ccbfe9f797..bffbc9cfc8ab 100644 --- a/block/blk-throttle.h +++ b/block/blk-throttle.h @@ -127,8 +127,8 @@ struct throtl_grp { * bytes/ios are waited already in previous configuration, and they will * be used to calculate wait time under new configuration. */ - uint64_t carryover_bytes[2]; - unsigned int carryover_ios[2]; + long long carryover_bytes[2]; + int carryover_ios[2]; unsigned long last_check_time; From patchwork Wed Aug 16 01:27:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 135789 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b82d:0:b0:3f2:4152:657d with SMTP id z13csp114646vqi; Wed, 16 Aug 2023 09:02:27 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFqTqPJiGuXxkemdyGDuqzi/y3BrGca1YL7flahL1zGXZXZiqX6EAWpBfvBpjvluiOF5Loy X-Received: by 2002:a17:906:1da1:b0:98e:2097:f23e with SMTP id u1-20020a1709061da100b0098e2097f23emr1630809ejh.77.1692201747426; Wed, 16 Aug 2023 09:02:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692201747; cv=none; d=google.com; s=arc-20160816; b=tVwxjZzg7J3oRbnxq5gfaNah94qKWVIRRD7mln488qvOdbqdRnKICN0wQBef2kSk9G 6oNwYq0AX99CKeUrTLd57MZmkLCFfDjOg/RzKhd6eGSpEc//Vx3iYPvP6oLyFdrNiWDb JW+193VKHh4yMRlZz5iJw+eawLgrpNsGvbp6R8iaysjmiJqQ3jKpIK0xBeESUtuv/WzP KEZJXnqwTQn5R4O/3V8USpe/NBsadPON1bqmwnQfuzehM/MzMR6dlv/ZIcEVallgAqv2 Ip4H9kNfI/qXa+N/HenfrEOhyVbtAnRpdU4h/8C1dnwtaPpwbt/FhpDBS1lI9o4potyK EITg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=/E9z+uoKlmyEKT45UXKTuolh2Bjl+fypQoDKOTYn4/k=; fh=wTguGu9iHUye/+4XPiOZzVDD/C/LKyDnbGOrb/C9kEo=; b=VjZh8LKc67cvBArwCGAnKhYwBxkdhHGI3J4UoWE3qvfPU7AUESwJsfJO4WvzZikBhG lBPt11cxL/6jEZqrjLoYXfWB3HrFHDv8J0tlOzZHg6Uyk+4aVxyDh/C9nOTZs7CH+g0N sxqiYvCj7YugFmO9h3eRty6LtN0MFRP1Xj+hgqAf0S0XQ0enFNrh7THk8a3fK5Aoo4yO TdPkxDkPaRVfCQrmKj58IDN+nb/CU4kV8mIv0/WTAYe2DyCcWP0Qj7tIaY+7Fib732Ee fTw6ts2IYl8w52Ly3HcIp1xYLYiJtd6pkO/ANIbbCCrAEKBY0WyThNZmZi2cOAappmSW WLkQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id us11-20020a170906bfcb00b0098e266c9592si11059130ejb.262.2023.08.16.09.01.42; Wed, 16 Aug 2023 09:02:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241055AbjHPBa4 (ORCPT + 99 others); Tue, 15 Aug 2023 21:30:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46752 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241115AbjHPBay (ORCPT ); Tue, 15 Aug 2023 21:30:54 -0400 Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 286C42121; Tue, 15 Aug 2023 18:30:53 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.169]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4RQVvY6Zh1z4f3q3b; Wed, 16 Aug 2023 09:30:49 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgAH5KbHJtxkQuVAAw--.55976S6; Wed, 16 Aug 2023 09:30:50 +0800 (CST) From: Yu Kuai To: tj@kernel.org, josef@toxicpanda.com, axboe@kernel.dk, yukuai3@huawei.com, mkoutny@suse.com Cc: cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH -next v2 2/4] blk-throttle: fix wrong comparation while 'carryover_ios/bytes' is negative Date: Wed, 16 Aug 2023 09:27:06 +0800 Message-Id: <20230816012708.1193747-3-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230816012708.1193747-1-yukuai1@huaweicloud.com> References: <20230816012708.1193747-1-yukuai1@huaweicloud.com> MIME-Version: 1.0 X-CM-TRANSID: gCh0CgAH5KbHJtxkQuVAAw--.55976S6 X-Coremail-Antispam: 1UD129KBjvJXoWxAFWkAF4xAry8WF17KF17ZFb_yoW5ArWrpr WfGF1IgF4rX3Z3tFnxJan8AFyrt39rAr98GrW3WayrCFn8GFyktrn5uFWFyayUZFs3uF4S kw1FqFn7AF4qyaDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBK14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_Jryl82xGYIkIc2 x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_Xr0_Ar1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr1UM2 8EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AI xVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20x vE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xv r2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4IIrI8v6xkF7I0E8cxan2IY04 v7MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAFwI0_ Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVWUtVW8ZwCIc40Y0x 0EwIxGrwCI42IY6xIIjxv20xvE14v26r1j6r1xMIIF0xvE2Ix0cI8IcVCY1x0267AKxVW8 JVWxJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Jr0_Gr1lIx AIcVC2z280aVCY1x0267AKxVW8JVW8JrUvcSsGvfC2KfnxnUUI43ZEXa7VUbdOz7UUUUU= = X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1774402139445476603 X-GMAIL-MSGID: 1774402139445476603 From: Yu Kuai carryover_ios/bytes[] can be negative in the case that ios are dispatched in the slice in advance, and then configuration is updated. For example: 1) set iops limit to 1000, and slice start is 0, slice end is 100ms; 2) current time is 0, and 100 ios are dispatched, those ios will not be throttled, hence io_disp is 100; 3) still at current time 0, update iops limit to 100, then carryover_ios is (0 - 100) = -100; 4) then, dispatch a new io at time 0, the expected result is that this io will wait for 1s. The calculation in tg_within_iops_limit: io_disp = 0; io_allowed = calculate_io_allowed + carryover_ios = 10 + (-100) = -90; io won't be throttled if (io_disp + 1 < io_allowed) passed. Before this patch, in step 4) (io_disp + 1 < io_allowed) is passed, because -90 for unsigned value is very huge, and such io won't be throttled. Fix this problem by checking if 'io/bytes_allowed' is negative first. Signed-off-by: Yu Kuai --- block/blk-throttle.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/block/blk-throttle.c b/block/blk-throttle.c index 5184f17f5129..7c93144d03da 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -825,7 +825,7 @@ static unsigned long tg_within_iops_limit(struct throtl_grp *tg, struct bio *bio u32 iops_limit) { bool rw = bio_data_dir(bio); - unsigned int io_allowed; + int io_allowed; unsigned long jiffy_elapsed, jiffy_wait, jiffy_elapsed_rnd; if (iops_limit == UINT_MAX) { @@ -838,9 +838,8 @@ static unsigned long tg_within_iops_limit(struct throtl_grp *tg, struct bio *bio jiffy_elapsed_rnd = roundup(jiffy_elapsed + 1, tg->td->throtl_slice); io_allowed = calculate_io_allowed(iops_limit, jiffy_elapsed_rnd) + tg->carryover_ios[rw]; - if (tg->io_disp[rw] + 1 <= io_allowed) { + if (io_allowed > 0 && tg->io_disp[rw] + 1 <= io_allowed) return 0; - } /* Calc approx time to dispatch */ jiffy_wait = jiffy_elapsed_rnd - jiffy_elapsed; @@ -851,7 +850,8 @@ static unsigned long tg_within_bps_limit(struct throtl_grp *tg, struct bio *bio, u64 bps_limit) { bool rw = bio_data_dir(bio); - u64 bytes_allowed, extra_bytes; + long long bytes_allowed; + u64 extra_bytes; unsigned long jiffy_elapsed, jiffy_wait, jiffy_elapsed_rnd; unsigned int bio_size = throtl_bio_data_size(bio); @@ -869,9 +869,8 @@ static unsigned long tg_within_bps_limit(struct throtl_grp *tg, struct bio *bio, jiffy_elapsed_rnd = roundup(jiffy_elapsed_rnd, tg->td->throtl_slice); bytes_allowed = calculate_bytes_allowed(bps_limit, jiffy_elapsed_rnd) + tg->carryover_bytes[rw]; - if (tg->bytes_disp[rw] + bio_size <= bytes_allowed) { + if (bytes_allowed > 0 && tg->bytes_disp[rw] + bio_size <= bytes_allowed) return 0; - } /* Calc approx time to dispatch */ extra_bytes = tg->bytes_disp[rw] + bio_size - bytes_allowed; From patchwork Wed Aug 16 01:27:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 135746 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a888:0:b0:3f2:4152:657d with SMTP id x8csp1041433vqo; Wed, 16 Aug 2023 04:33:44 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGT+eFGqVu78ZTOawF2eOVh0GjuVEIHK19JgXyHhvAWs9sX2HWYqAEQUD6cm2sTjWGfkz2E X-Received: by 2002:a05:6402:1854:b0:525:69ec:e1c8 with SMTP id v20-20020a056402185400b0052569ece1c8mr1066176edy.40.1692185624360; Wed, 16 Aug 2023 04:33:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692185624; cv=none; d=google.com; s=arc-20160816; b=XPG+McHxd3JXiddQqBQou9IGvRUxaa0NqJ1hQfxfkOePaukTON7ntNB/bapIxHGyqS Uhi129K3iURlLjk8DswK+yQhKrj4/6ug/LookczYWWAjwWRm4JprPSsk/Y4YOH2Iek7n 65XpfhtdCnZmiaUG85exBNkVEqwve+kpIRsYz0zvs0xub/NPnTk1m2RDLOZgng2bLPI7 bhCxvMJSz/ZExkXOUn8I6diZDGxzLXS79/5vNqGrhhMb+0hHTi0GmSsr6nW9srqPiBRZ Vjy4ur8ZGGvrvw9ZVAfeWZhxT00Q9Dqhq9x8NFRGxmskQryOQv4g03tZU98i/GgJlYIA ChJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=NXJWFlTQsNz81JtbMRoQU8KaZQk/ll081+0972Wr1hU=; fh=wTguGu9iHUye/+4XPiOZzVDD/C/LKyDnbGOrb/C9kEo=; b=HAvjHoHUN1vXuD8UJa85okjd1VoqDh+AHgjRUt8EJG+HV4+BuD9lY22gGCVkFuF5BY nl0R+vRzQw3GDPhIyHc7g3Ew032ejhlPagbCRQSfD0fsQcJ8cc7cgw5axEDU2RmKxZp0 WnnJwc4gzkUgAIHEmsP3e2Q+egYrB2c8KEeM765uBiHgdgS5Bc4LJktQdWO2nBgHeGL5 7dmaQMgzrd/MWFNjMwC4dlvGfjfDNDTrXswIGv7VWKgE3Ix9tlLib7KzMi0ogc45arjm sn5xl21t1bmXCyXpuvwtBE9z+MJ+mlr6j3WIAsXuqGuWkbm2l825RV6h7yPpNwBsS1xZ 8cBQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l13-20020aa7d94d000000b005233f769158si11205349eds.348.2023.08.16.04.33.18; Wed, 16 Aug 2023 04:33:44 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241072AbjHPBa7 (ORCPT + 99 others); Tue, 15 Aug 2023 21:30:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46762 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241116AbjHPBay (ORCPT ); Tue, 15 Aug 2023 21:30:54 -0400 Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 52E062123; Tue, 15 Aug 2023 18:30:53 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.169]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4RQVvY4wDNz4f3pG4; Wed, 16 Aug 2023 09:30:49 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgAH5KbHJtxkQuVAAw--.55976S7; Wed, 16 Aug 2023 09:30:50 +0800 (CST) From: Yu Kuai To: tj@kernel.org, josef@toxicpanda.com, axboe@kernel.dk, yukuai3@huawei.com, mkoutny@suse.com Cc: cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH -next v2 3/4] blk-throttle: use calculate_io/bytes_allowed() for throtl_trim_slice() Date: Wed, 16 Aug 2023 09:27:07 +0800 Message-Id: <20230816012708.1193747-4-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230816012708.1193747-1-yukuai1@huaweicloud.com> References: <20230816012708.1193747-1-yukuai1@huaweicloud.com> MIME-Version: 1.0 X-CM-TRANSID: gCh0CgAH5KbHJtxkQuVAAw--.55976S7 X-Coremail-Antispam: 1UD129KBjvJXoWxur1UJF1DXryxXF4rAr43KFg_yoWrGrykpr WfAF42grs5Xas7tFW3Awn0yayrCwnxJFyUJ3y3Xr9YyFWUAFy8GF1kAryUta4xAas7ua1f Aw1DZrn8Cr42yFJanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBE14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JrWl82xGYIkIc2 x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_Ar0_tr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJw A2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq3wAS 0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2 IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0 Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2kIc2 xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v2 6r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_Jw0_GFylIxkGc2 Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_ Gr0_Cr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMI IF0xvEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x0JUd8n5UUUUU = X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1774385233408538945 X-GMAIL-MSGID: 1774385233408538945 From: Yu Kuai There are no functional changes, just make the code cleaner. Signed-off-by: Yu Kuai --- block/blk-throttle.c | 86 +++++++++++++++++++++----------------------- 1 file changed, 41 insertions(+), 45 deletions(-) diff --git a/block/blk-throttle.c b/block/blk-throttle.c index 7c93144d03da..69a994156772 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -697,11 +697,40 @@ static bool throtl_slice_used(struct throtl_grp *tg, bool rw) return true; } +static unsigned int calculate_io_allowed(u32 iops_limit, + unsigned long jiffy_elapsed) +{ + unsigned int io_allowed; + u64 tmp; + + /* + * jiffy_elapsed should not be a big value as minimum iops can be + * 1 then at max jiffy elapsed should be equivalent of 1 second as we + * will allow dispatch after 1 second and after that slice should + * have been trimmed. + */ + + tmp = (u64)iops_limit * jiffy_elapsed; + do_div(tmp, HZ); + + if (tmp > UINT_MAX) + io_allowed = UINT_MAX; + else + io_allowed = tmp; + + return io_allowed; +} + +static u64 calculate_bytes_allowed(u64 bps_limit, unsigned long jiffy_elapsed) +{ + return mul_u64_u64_div_u64(bps_limit, (u64)jiffy_elapsed, (u64)HZ); +} + /* Trim the used slices and adjust slice start accordingly */ static inline void throtl_trim_slice(struct throtl_grp *tg, bool rw) { - unsigned long nr_slices, time_elapsed, io_trim; - u64 bytes_trim, tmp; + unsigned long time_elapsed, io_trim; + u64 bytes_trim; BUG_ON(time_before(tg->slice_end[rw], tg->slice_start[rw])); @@ -723,19 +752,14 @@ static inline void throtl_trim_slice(struct throtl_grp *tg, bool rw) throtl_set_slice_end(tg, rw, jiffies + tg->td->throtl_slice); - time_elapsed = jiffies - tg->slice_start[rw]; - - nr_slices = time_elapsed / tg->td->throtl_slice; - - if (!nr_slices) + time_elapsed = rounddown(jiffies - tg->slice_start[rw], + tg->td->throtl_slice); + if (!time_elapsed) return; - tmp = tg_bps_limit(tg, rw) * tg->td->throtl_slice * nr_slices; - do_div(tmp, HZ); - bytes_trim = tmp; - - io_trim = (tg_iops_limit(tg, rw) * tg->td->throtl_slice * nr_slices) / - HZ; + bytes_trim = calculate_bytes_allowed(tg_bps_limit(tg, rw), + time_elapsed); + io_trim = calculate_io_allowed(tg_iops_limit(tg, rw), time_elapsed); if (!bytes_trim && !io_trim) return; @@ -749,41 +773,13 @@ static inline void throtl_trim_slice(struct throtl_grp *tg, bool rw) else tg->io_disp[rw] = 0; - tg->slice_start[rw] += nr_slices * tg->td->throtl_slice; + tg->slice_start[rw] += time_elapsed; throtl_log(&tg->service_queue, "[%c] trim slice nr=%lu bytes=%llu io=%lu start=%lu end=%lu jiffies=%lu", - rw == READ ? 'R' : 'W', nr_slices, bytes_trim, io_trim, - tg->slice_start[rw], tg->slice_end[rw], jiffies); -} - -static unsigned int calculate_io_allowed(u32 iops_limit, - unsigned long jiffy_elapsed) -{ - unsigned int io_allowed; - u64 tmp; - - /* - * jiffy_elapsed should not be a big value as minimum iops can be - * 1 then at max jiffy elapsed should be equivalent of 1 second as we - * will allow dispatch after 1 second and after that slice should - * have been trimmed. - */ - - tmp = (u64)iops_limit * jiffy_elapsed; - do_div(tmp, HZ); - - if (tmp > UINT_MAX) - io_allowed = UINT_MAX; - else - io_allowed = tmp; - - return io_allowed; -} - -static u64 calculate_bytes_allowed(u64 bps_limit, unsigned long jiffy_elapsed) -{ - return mul_u64_u64_div_u64(bps_limit, (u64)jiffy_elapsed, (u64)HZ); + rw == READ ? 'R' : 'W', time_elapsed / tg->td->throtl_slice, + bytes_trim, io_trim, tg->slice_start[rw], tg->slice_end[rw], + jiffies); } static void __tg_update_carryover(struct throtl_grp *tg, bool rw) From patchwork Wed Aug 16 01:27:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 135728 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a888:0:b0:3f2:4152:657d with SMTP id x8csp917798vqo; Tue, 15 Aug 2023 23:50:51 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHBUVXFuXXA9mWgB0Ym8IGhBN0dQMgjHzWZIlemAIObOJU1kKDFMkUGqytgqFkmtm5Fbuas X-Received: by 2002:a17:906:113:b0:99d:de25:89bb with SMTP id 19-20020a170906011300b0099dde2589bbmr808249eje.0.1692168651089; Tue, 15 Aug 2023 23:50:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692168651; cv=none; d=google.com; s=arc-20160816; b=sX7A1STWkRQYrXlhzgbq6UdU+12vabRWlk6nVVEIBXU+MSwR0oJEmgPKFfAgDMRxwV uDrmFffOw23U/y+JeVhK2KuNJnPLlxJlWtWvnfN1vsK/OalCt/6fLrb9aEiredVzhbbS sgHyI2g9ShKVXbi1P82IvK1OdfWifzggJRSQCfWpUQFdA269JTxdxpp8alVk9WOC7Z75 NBqILslbzTSa/L2eE+LufQns++0XKgZf7AZvQj3Kf7xEwRXJI+9cEuAgFcJmpz6qlP6O qDVUB2cPjlXrIoz/qGxVgRu2Q+isBJoSM85e7IbJZuNIQ2OL+USD1iwDQGsDU+xQcwsY cAnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=4oSoM44qqIQ2DA/B09W8RIMWqlRKaC1OLQXfgJtieaI=; fh=wTguGu9iHUye/+4XPiOZzVDD/C/LKyDnbGOrb/C9kEo=; b=SeUyj+6JWUVYCH/J65e2OGhyu8rTZvgnWHb14ebwqgE3QThRKICXuSgnVMr51iXGq6 50YSQ/rn/f06WTe/FTnAp+PTc3gjO3sNoaTrfBC30WBKrAEGshqc/Tjp9VVurqwysWf9 ucu5WJ2o8OOI7jy/cKbkHXwLGkavetfw2fziDw0c3+vd5MSwGLsRwTPhjsqR29MiKcL0 aFZh43wzoAHVFCuYuYZhEpTS0aQbAc4dbjDHW2ipB2ZpP+fLC22uHfTtl83gXjIhD9l1 PZeifZFG824JEXkwiRKsQo2zkrl6NRejMl8fQkqEAeAzRAEapQxO09ojOWnLNGuFgxcw C1VQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 17-20020a170906059100b0099ca6e1b1f1si10475511ejn.1055.2023.08.15.23.50.25; Tue, 15 Aug 2023 23:50:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241085AbjHPBba (ORCPT + 99 others); Tue, 15 Aug 2023 21:31:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46776 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241065AbjHPBa5 (ORCPT ); Tue, 15 Aug 2023 21:30:57 -0400 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 898D81BF8; Tue, 15 Aug 2023 18:30:56 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.169]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4RQVvZ5DMnz4f3nwk; Wed, 16 Aug 2023 09:30:50 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgAH5KbHJtxkQuVAAw--.55976S8; Wed, 16 Aug 2023 09:30:51 +0800 (CST) From: Yu Kuai To: tj@kernel.org, josef@toxicpanda.com, axboe@kernel.dk, yukuai3@huawei.com, mkoutny@suse.com Cc: cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH -next v2 4/4] blk-throttle: consider 'carryover_ios/bytes' in throtl_trim_slice() Date: Wed, 16 Aug 2023 09:27:08 +0800 Message-Id: <20230816012708.1193747-5-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230816012708.1193747-1-yukuai1@huaweicloud.com> References: <20230816012708.1193747-1-yukuai1@huaweicloud.com> MIME-Version: 1.0 X-CM-TRANSID: gCh0CgAH5KbHJtxkQuVAAw--.55976S8 X-Coremail-Antispam: 1UD129KBjvJXoWxWrW3ZF47Zw43Ww13trW8WFg_yoW5Zw1fpF WfJF47twsIqFn3K3ZxZ3Wvv3Wjk3yDJryDG3y5trWfAF90kry8Kr1vkrW2qay2yas7ua1f J3409r9rAr4qkrJanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPF14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_Ar0_tr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E 14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_Jw0_GFylIx kGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAF wI0_Cr0_Gr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVWUJV W8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWIevJa73UjIFyTuYvjfUOBTY UUUUU X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1774367435583656149 X-GMAIL-MSGID: 1774367435583656149 From: Yu Kuai Currently, 'carryover_ios/bytes' is not handled in throtl_trim_slice(), for consequence, 'carryover_ios/bytes' will be used to throttle bio multiple times, for example: 1) set iops limit to 100, and slice start is 0, slice end is 100ms; 2) current time is 0, and 10 ios are dispatched, those io won't be throttled and io_disp is 10; 3) still at current time 0, update iops limit to 1000, carryover_ios is updated to (0 - 10) = -10; 4) in this slice(0 - 100ms), io_allowed = 100 + (-10) = 90, which means only 90 ios can be dispatched without waiting; 5) assume that io is throttled in slice(0 - 100ms), and throtl_trim_slice() update silce to (100ms - 200ms). In this case, 'carryover_ios/bytes' is not cleared and still only 90 ios can be dispatched between 100ms - 200ms. Fix this problem by updating 'carryover_ios/bytes' in throtl_trim_slice(). Fixes: a880ae93e5b5 ("blk-throttle: fix io hung due to configuration updates") Reported-by: zhuxiaohui Link: https://lore.kernel.org/all/20230812072116.42321-1-zhuxiaohui.400@bytedance.com/ Signed-off-by: Yu Kuai --- block/blk-throttle.c | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/block/blk-throttle.c b/block/blk-throttle.c index 69a994156772..38a881cf97d0 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -729,8 +729,9 @@ static u64 calculate_bytes_allowed(u64 bps_limit, unsigned long jiffy_elapsed) /* Trim the used slices and adjust slice start accordingly */ static inline void throtl_trim_slice(struct throtl_grp *tg, bool rw) { - unsigned long time_elapsed, io_trim; - u64 bytes_trim; + unsigned long time_elapsed; + long long bytes_trim; + int io_trim; BUG_ON(time_before(tg->slice_end[rw], tg->slice_start[rw])); @@ -758,17 +759,21 @@ static inline void throtl_trim_slice(struct throtl_grp *tg, bool rw) return; bytes_trim = calculate_bytes_allowed(tg_bps_limit(tg, rw), - time_elapsed); - io_trim = calculate_io_allowed(tg_iops_limit(tg, rw), time_elapsed); - if (!bytes_trim && !io_trim) + time_elapsed) + + tg->carryover_bytes[rw]; + io_trim = calculate_io_allowed(tg_iops_limit(tg, rw), time_elapsed) + + tg->carryover_ios[rw]; + if (bytes_trim <= 0 && io_trim <= 0) return; - if (tg->bytes_disp[rw] >= bytes_trim) + tg->carryover_bytes[rw] = 0; + if ((long long)tg->bytes_disp[rw] >= bytes_trim) tg->bytes_disp[rw] -= bytes_trim; else tg->bytes_disp[rw] = 0; - if (tg->io_disp[rw] >= io_trim) + tg->carryover_ios[rw] = 0; + if ((int)tg->io_disp[rw] >= io_trim) tg->io_disp[rw] -= io_trim; else tg->io_disp[rw] = 0; @@ -776,7 +781,7 @@ static inline void throtl_trim_slice(struct throtl_grp *tg, bool rw) tg->slice_start[rw] += time_elapsed; throtl_log(&tg->service_queue, - "[%c] trim slice nr=%lu bytes=%llu io=%lu start=%lu end=%lu jiffies=%lu", + "[%c] trim slice nr=%lu bytes=%lld io=%d start=%lu end=%lu jiffies=%lu", rw == READ ? 'R' : 'W', time_elapsed / tg->td->throtl_slice, bytes_trim, io_trim, tg->slice_start[rw], tg->slice_end[rw], jiffies);