From patchwork Thu Jun 1 05:39:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chengming Zhou X-Patchwork-Id: 101740 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp76010vqr; Wed, 31 May 2023 23:00:57 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6hPgRI1aAb+HHY5md/V7UKnoX/hpSqCPBtJiR26eeQvJKmvH+M1uYhbo2PszyjLOLZvFlP X-Received: by 2002:aa7:88d6:0:b0:64f:74d9:eb4b with SMTP id k22-20020aa788d6000000b0064f74d9eb4bmr10710305pff.15.1685599257374; Wed, 31 May 2023 23:00:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685599257; cv=none; d=google.com; s=arc-20160816; b=kZtn0c/MyiMBwYYeySJlnT7HneIm1PuN8sDHJj66++EnPkhaFhVJMC9JPZfqWSA6K3 Mo67abMqrPKDz9MxtZt/tVV1CYP5Y1eZ+XwDkwSkjox/kJo7NkRqSbbrinC2cGcOXzno 7eHAxjCe7VJo4HUK3meskjndEt92k4cJTi1Lnl65OwxXQDUcNWsRTcaAWTDq1HnwCmUd 2tERhnpZRx6OCQKKj0tc9eKAxaT6BrLu5ahYzmZXL8O/OrHSK9ir3YPVufBPXwCcJy97 JkONF0EnpwnM/Uf/FcTC18hT4nRRTFWXtliT+iAFtYGR7LqJSMMRyHK3H6yKV5RrVbn7 4+Sw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=wMraD2clGsbsww7FgpOj2nLzIuUm/oP4AQqhudlO+CU=; b=maN6mcgKHbcXtS37VPCjZXw5v+KTw2ZxinR5TCxRT8x9h8Sgf2by2lXJfsQP/ANmtb 1jZ9YEoUtpKszqE2rZ2DFBkxhfGkAqOdvoyey20IedTEMZEussWyCjVGy6499k//gqbT XpUWBY7t7lMOlNrCdL4QrT0zTmmMDatFVa6hAbq7Mq5QTXHf4FvZU5T5XyaK2VzzBNnv 2OB/a/siYPFZIePF9kv8KaPo83zlpEZp1Cf23MM/lkzQu+X3cI9IxQdhbp0tkHb9SP7i 3rJqD6vONdsiqlNlvxghLJGbesaMJgJC71Pdi3uKEvokCh+evNbx//RSCZCkWjvSHjtL K0UQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=FkyWsbBZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z3-20020aa79903000000b0064d10dee6f1si4908570pff.33.2023.05.31.23.00.42; Wed, 31 May 2023 23:00:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=FkyWsbBZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231775AbjFAFya (ORCPT + 99 others); Thu, 1 Jun 2023 01:54:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58060 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231628AbjFAFyF (ORCPT ); Thu, 1 Jun 2023 01:54:05 -0400 X-Greylist: delayed 590 seconds by postgrey-1.37 at lindbergh.monkeyblade.net; Wed, 31 May 2023 22:52:38 PDT Received: from out-6.mta0.migadu.com (out-6.mta0.migadu.com [91.218.175.6]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6BAFE198E for ; Wed, 31 May 2023 22:52:38 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1685598097; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=wMraD2clGsbsww7FgpOj2nLzIuUm/oP4AQqhudlO+CU=; b=FkyWsbBZ2oZLCAcBImfx6W1IHJc3i/7Zx7aiKeb4ibNEDe4ItNvXQ055eYXhHUKeeeNI80 QsTlFsysUgtFRAULTHNVAFK4WDrfCURYAnLgKGeK5GJUpW1ybrWuwAm/fo5xXor0NoIPCo pKfYZcuMo1xdrXJyz/zYeU/YONzok2g= From: chengming.zhou@linux.dev To: tj@kernel.org, axboe@kernel.dk Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, Chengming Zhou Subject: [PATCH] blk-mq: fix incorrect rq start_time_ns and alloc_time_ns after throttled Date: Thu, 1 Jun 2023 13:39:19 +0800 Message-Id: <20230601053919.3639954-1-chengming.zhou@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1767478926659060549?= X-GMAIL-MSGID: =?utf-8?q?1767478926659060549?= From: Chengming Zhou iocost rely on rq start_time_ns and alloc_time_ns to tell the saturation state of the block device. If any qos ->throttle() end up blocking, the cached rq start_time_ns and alloc_time_ns will include its throtted time, which can confuse its user. This patch add nr_flush counter in blk_plug, so we can tell if the task has throttled in any qos ->throttle(), in which case we need to correct the rq start_time_ns and alloc_time_ns. Another solution may be make rq_qos_throttle() return bool to indicate if it has throttled in any qos ->throttle(). But this need more changes. Signed-off-by: Chengming Zhou --- block/blk-core.c | 3 +++ block/blk-mq.c | 18 ++++++++++++++++++ include/linux/blkdev.h | 8 +++++--- 3 files changed, 26 insertions(+), 3 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 00c74330fa92..5109f7f5606c 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1053,6 +1053,7 @@ void blk_start_plug_nr_ios(struct blk_plug *plug, unsigned short nr_ios) plug->cached_rq = NULL; plug->nr_ios = min_t(unsigned short, nr_ios, BLK_MAX_REQUEST_COUNT); plug->rq_count = 0; + plug->nr_flush = 0; plug->multiple_queues = false; plug->has_elevator = false; plug->nowait = false; @@ -1150,6 +1151,8 @@ void __blk_flush_plug(struct blk_plug *plug, bool from_schedule) */ if (unlikely(!rq_list_empty(plug->cached_rq))) blk_mq_free_plug_rqs(plug); + + plug->nr_flush++; } /** diff --git a/block/blk-mq.c b/block/blk-mq.c index f6dad0886a2f..8731f2815790 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -2871,6 +2871,7 @@ static inline struct request *blk_mq_get_cached_request(struct request_queue *q, { struct request *rq; enum hctx_type type, hctx_type; + unsigned short nr_flush; if (!plug) return NULL; @@ -2897,8 +2898,25 @@ static inline struct request *blk_mq_get_cached_request(struct request_queue *q, * before we throttle. */ plug->cached_rq = rq_list_next(rq); + nr_flush = plug->nr_flush; rq_qos_throttle(q, *bio); + /* + * If any qos ->throttle() end up blocking, we will have flushed the + * plug and we need to correct the rq start_time_ns and alloc_time_ns. + */ + if (nr_flush != plug->nr_flush) { + if (blk_mq_need_time_stamp(rq)) { + u64 now = ktime_get_ns(); + +#ifdef CONFIG_BLK_RQ_ALLOC_TIME + if (rq->alloc_time_ns) + rq->alloc_time_ns += now - rq->start_time_ns; +#endif + rq->start_time_ns = now; + } + } + rq->cmd_flags = (*bio)->bi_opf; INIT_LIST_HEAD(&rq->queuelist); return rq; diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index e3242e67a8e3..cf66871a1844 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -978,9 +978,11 @@ struct blk_plug { unsigned short rq_count; - bool multiple_queues; - bool has_elevator; - bool nowait; + unsigned short nr_flush; + + bool multiple_queues:1; + bool has_elevator:1; + bool nowait:1; struct list_head cb_list; /* md requires an unplug callback */ };