From patchwork Wed Sep 13 15:16:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chengming Zhou X-Patchwork-Id: 138889 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9ecd:0:b0:3f2:4152:657d with SMTP id t13csp1161672vqx; Wed, 13 Sep 2023 08:21:46 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHhd73a4Z9eC/Xmch0nKGrj7/xa0O+Zfq3DCiPNDFZiPNrGkC6aTyHyrsKh8GdrlPDhkn1L X-Received: by 2002:a17:90b:38c1:b0:26b:455b:8d61 with SMTP id nn1-20020a17090b38c100b0026b455b8d61mr8067138pjb.22.1694618505951; Wed, 13 Sep 2023 08:21:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694618505; cv=none; d=google.com; s=arc-20160816; b=PUte85sfn/U0tfjk98uIHqC5k8VbiwMRlPcda4cD4czhFw8aaRxQDm+MncmsJ2wjf7 lxHlqWcdnYXBbSW1m8IO2+riCKRZCf2jvVqwlP5bC7c4XvsiqcUYwkkNY3FWriuljQTu Sr4sVzKufdmYqaoEU9XRoWaznSX1xu5mjeGCMxaBnwnutwd9pU0R5hUgxYcVTnnGN5kc ZNyk4Nxt/u22Mn28H6K586T772YBY5tbSrgB+95pqN8lWirTODsRotssGc5iYmvXXIzV 72bpg2NzC6Uy6u5sj2ZYthwS+NzGcOIZJtXxaasv514Ks6KJkTwlWZs39HgKo35OgI0U 7Vgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=lESR3NSM7ZpkD2NbKZdz4U/iR2sY5wKzp5qn1n2DwS4=; fh=szS6nKOHqWqdmo+pDmdMHkpBM8bpzBlMJyKfLi9jRHY=; b=QZbqEgzupT5Y/ZAcrDGaV26bO12Zmn7sFdCiZJ0j6ReZyUSdbaiIY8ZfmQ19Zl5iJ2 5zgldUmRJckMjoIGjkLW6qmm2frwDZFKuu82flhCagewuqLXZJxyxJJq6ic2myH52P/M iEeah0IBxpujxC9+Q5oCuq0IAD89rTlxlsQGFzUIqcPWY9O9/aHymo0S85p8tDKtbqO1 qcFLl9XZv/J8kmcWQWg/qrDH4yMTehyHwadp3H/21NJBTeBAqJLNdR1CvceIRQNevPD2 svE73050SxQg7JXI7BRGD4iJLy+fM/6eOO1EDoDdwJWF/GxlZ2vA6RLduXgSF03Ecsd6 GPMA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=eQoP2NQu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id y64-20020a638a43000000b0057761dd9c56si7047360pgd.322.2023.09.13.08.21.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Sep 2023 08:21:45 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=eQoP2NQu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 5555B802388E; Wed, 13 Sep 2023 08:17:13 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241123AbjIMPQ7 (ORCPT + 36 others); Wed, 13 Sep 2023 11:16:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39338 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241648AbjIMPQg (ORCPT ); Wed, 13 Sep 2023 11:16:36 -0400 Received: from out-227.mta1.migadu.com (out-227.mta1.migadu.com [95.215.58.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 42B65115 for ; Wed, 13 Sep 2023 08:16:32 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1694618190; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lESR3NSM7ZpkD2NbKZdz4U/iR2sY5wKzp5qn1n2DwS4=; b=eQoP2NQujqkGpNZBwuYx1EBwjuTeEsBLyGQQI5TWRpwwNydZdk7R82YhJuwO94gwBwnVOd vnnh7ecQBss6Ttv5dALvKmPHCaWTl+uHSZPyQmak1XfyvC2nE82luTN2Vylq6ez6OJG585 RXIHH4XuhTIE7huOZ14+FKvs6O/GcOc= From: chengming.zhou@linux.dev To: axboe@kernel.dk, hch@lst.de, ming.lei@redhat.com, bvanassche@acm.org Cc: kbusch@kernel.org, mst@redhat.com, damien.lemoal@opensource.wdc.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, chengming.zhou@linux.dev, Chengming Zhou Subject: [PATCH v2 1/5] blk-mq: account active requests when get driver tag Date: Wed, 13 Sep 2023 15:16:12 +0000 Message-Id: <20230913151616.3164338-2-chengming.zhou@linux.dev> In-Reply-To: <20230913151616.3164338-1-chengming.zhou@linux.dev> References: <20230913151616.3164338-1-chengming.zhou@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Wed, 13 Sep 2023 08:17:13 -0700 (PDT) X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776936294705049224 X-GMAIL-MSGID: 1776936294705049224 From: Chengming Zhou There is a limit that batched queue_rqs() can't work on shared tags queue, since the account of active requests can't be done there. Now we account the active requests only in blk_mq_get_driver_tag(), which is not the time we get driver tag actually (with none elevator). To support batched queue_rqs() on shared tags queue, we move the account of active requests to where we get the driver tag: 1. none elevator: blk_mq_get_tags() and blk_mq_get_tag() 2. other elevator: __blk_mq_alloc_driver_tag() This is clearer and match with the unaccount side, which just happen when we put the driver tag. The other good point is that we don't need RQF_MQ_INFLIGHT trick anymore, which used to avoid double account of flush request. Now we only account when actually get the driver tag, so all is good. We will remove RQF_MQ_INFLIGHT in the next patch. Signed-off-by: Chengming Zhou Reviewed-by: Ming Lei --- block/blk-mq.c | 33 +++++++++++------------------ block/blk-mq.h | 56 ++++++++++++++++++++++++++++++++++++-------------- 2 files changed, 53 insertions(+), 36 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 1fafd54dce3c..e776388decc3 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -426,6 +426,8 @@ __blk_mq_alloc_requests_batch(struct blk_mq_alloc_data *data) rq_list_add(data->cached_rq, rq); nr++; } + if (!(data->rq_flags & RQF_SCHED_TAGS)) + blk_mq_add_active_requests(data->hctx, nr); /* caller already holds a reference, add for remainder */ percpu_ref_get_many(&data->q->q_usage_counter, nr - 1); data->nr_tags -= nr; @@ -510,6 +512,8 @@ static struct request *__blk_mq_alloc_requests(struct blk_mq_alloc_data *data) goto retry; } + if (!(data->rq_flags & RQF_SCHED_TAGS)) + blk_mq_inc_active_requests(data->hctx); rq = blk_mq_rq_ctx_init(data, blk_mq_tags_from_data(data), tag); blk_mq_rq_time_init(rq, alloc_time_ns); return rq; @@ -669,6 +673,8 @@ struct request *blk_mq_alloc_request_hctx(struct request_queue *q, tag = blk_mq_get_tag(&data); if (tag == BLK_MQ_NO_TAG) goto out_queue_exit; + if (!(data.rq_flags & RQF_SCHED_TAGS)) + blk_mq_inc_active_requests(data.hctx); rq = blk_mq_rq_ctx_init(&data, blk_mq_tags_from_data(&data), tag); blk_mq_rq_time_init(rq, alloc_time_ns); rq->__data_len = 0; @@ -708,11 +714,10 @@ static void __blk_mq_free_request(struct request *rq) blk_pm_mark_last_busy(rq); rq->mq_hctx = NULL; - if (rq->rq_flags & RQF_MQ_INFLIGHT) - __blk_mq_dec_active_requests(hctx); - - if (rq->tag != BLK_MQ_NO_TAG) + if (rq->tag != BLK_MQ_NO_TAG) { + blk_mq_dec_active_requests(hctx); blk_mq_put_tag(hctx->tags, ctx, rq->tag); + } if (sched_tag != BLK_MQ_NO_TAG) blk_mq_put_tag(hctx->sched_tags, ctx, sched_tag); blk_mq_sched_restart(hctx); @@ -1065,8 +1070,7 @@ static inline void blk_mq_flush_tag_batch(struct blk_mq_hw_ctx *hctx, * All requests should have been marked as RQF_MQ_INFLIGHT, so * update hctx->nr_active in batch */ - if (hctx->flags & BLK_MQ_F_TAG_QUEUE_SHARED) - __blk_mq_sub_active_requests(hctx, nr_tags); + blk_mq_sub_active_requests(hctx, nr_tags); blk_mq_put_tags(hctx->tags, tag_array, nr_tags); percpu_ref_put_many(&q->q_usage_counter, nr_tags); @@ -1748,7 +1752,7 @@ struct request *blk_mq_dequeue_from_ctx(struct blk_mq_hw_ctx *hctx, return data.rq; } -static bool __blk_mq_alloc_driver_tag(struct request *rq) +bool __blk_mq_alloc_driver_tag(struct request *rq) { struct sbitmap_queue *bt = &rq->mq_hctx->tags->bitmap_tags; unsigned int tag_offset = rq->mq_hctx->tags->nr_reserved_tags; @@ -1769,20 +1773,7 @@ static bool __blk_mq_alloc_driver_tag(struct request *rq) return false; rq->tag = tag + tag_offset; - return true; -} - -bool __blk_mq_get_driver_tag(struct blk_mq_hw_ctx *hctx, struct request *rq) -{ - if (rq->tag == BLK_MQ_NO_TAG && !__blk_mq_alloc_driver_tag(rq)) - return false; - - if ((hctx->flags & BLK_MQ_F_TAG_QUEUE_SHARED) && - !(rq->rq_flags & RQF_MQ_INFLIGHT)) { - rq->rq_flags |= RQF_MQ_INFLIGHT; - __blk_mq_inc_active_requests(hctx); - } - hctx->tags->rqs[rq->tag] = rq; + blk_mq_inc_active_requests(rq->mq_hctx); return true; } diff --git a/block/blk-mq.h b/block/blk-mq.h index 1743857e0b01..560a76df290a 100644 --- a/block/blk-mq.h +++ b/block/blk-mq.h @@ -271,12 +271,18 @@ static inline int blk_mq_get_rq_budget_token(struct request *rq) return -1; } -static inline void __blk_mq_inc_active_requests(struct blk_mq_hw_ctx *hctx) +static inline void __blk_mq_add_active_requests(struct blk_mq_hw_ctx *hctx, + int val) { if (blk_mq_is_shared_tags(hctx->flags)) - atomic_inc(&hctx->queue->nr_active_requests_shared_tags); + atomic_add(val, &hctx->queue->nr_active_requests_shared_tags); else - atomic_inc(&hctx->nr_active); + atomic_add(val, &hctx->nr_active); +} + +static inline void __blk_mq_inc_active_requests(struct blk_mq_hw_ctx *hctx) +{ + __blk_mq_add_active_requests(hctx, 1); } static inline void __blk_mq_sub_active_requests(struct blk_mq_hw_ctx *hctx, @@ -293,6 +299,32 @@ static inline void __blk_mq_dec_active_requests(struct blk_mq_hw_ctx *hctx) __blk_mq_sub_active_requests(hctx, 1); } +static inline void blk_mq_add_active_requests(struct blk_mq_hw_ctx *hctx, + int val) +{ + if (hctx->flags & BLK_MQ_F_TAG_QUEUE_SHARED) + __blk_mq_add_active_requests(hctx, val); +} + +static inline void blk_mq_inc_active_requests(struct blk_mq_hw_ctx *hctx) +{ + if (hctx->flags & BLK_MQ_F_TAG_QUEUE_SHARED) + __blk_mq_inc_active_requests(hctx); +} + +static inline void blk_mq_sub_active_requests(struct blk_mq_hw_ctx *hctx, + int val) +{ + if (hctx->flags & BLK_MQ_F_TAG_QUEUE_SHARED) + __blk_mq_sub_active_requests(hctx, val); +} + +static inline void blk_mq_dec_active_requests(struct blk_mq_hw_ctx *hctx) +{ + if (hctx->flags & BLK_MQ_F_TAG_QUEUE_SHARED) + __blk_mq_dec_active_requests(hctx); +} + static inline int __blk_mq_active_requests(struct blk_mq_hw_ctx *hctx) { if (blk_mq_is_shared_tags(hctx->flags)) @@ -302,13 +334,9 @@ static inline int __blk_mq_active_requests(struct blk_mq_hw_ctx *hctx) static inline void __blk_mq_put_driver_tag(struct blk_mq_hw_ctx *hctx, struct request *rq) { + blk_mq_dec_active_requests(hctx); blk_mq_put_tag(hctx->tags, rq->mq_ctx, rq->tag); rq->tag = BLK_MQ_NO_TAG; - - if (rq->rq_flags & RQF_MQ_INFLIGHT) { - rq->rq_flags &= ~RQF_MQ_INFLIGHT; - __blk_mq_dec_active_requests(hctx); - } } static inline void blk_mq_put_driver_tag(struct request *rq) @@ -319,19 +347,17 @@ static inline void blk_mq_put_driver_tag(struct request *rq) __blk_mq_put_driver_tag(rq->mq_hctx, rq); } -bool __blk_mq_get_driver_tag(struct blk_mq_hw_ctx *hctx, struct request *rq); +bool __blk_mq_alloc_driver_tag(struct request *rq); static inline bool blk_mq_get_driver_tag(struct request *rq) { struct blk_mq_hw_ctx *hctx = rq->mq_hctx; - if (rq->tag != BLK_MQ_NO_TAG && - !(hctx->flags & BLK_MQ_F_TAG_QUEUE_SHARED)) { - hctx->tags->rqs[rq->tag] = rq; - return true; - } + if (rq->tag == BLK_MQ_NO_TAG && !__blk_mq_alloc_driver_tag(rq)) + return false; - return __blk_mq_get_driver_tag(hctx, rq); + hctx->tags->rqs[rq->tag] = rq; + return true; } static inline void blk_mq_clear_mq_map(struct blk_mq_queue_map *qmap) From patchwork Wed Sep 13 15:16:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chengming Zhou X-Patchwork-Id: 139106 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:a8d:b0:3f2:4152:657d with SMTP id gr13csp100428vqb; Wed, 13 Sep 2023 14:37:52 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEWI3GuNNBAmbu3I2U9JOpITRjCeszr8WukK8gn6aMufp3ld2bu4MfmsNkXfF6VDG34hJws X-Received: by 2002:a17:90a:8c95:b0:274:4b04:392f with SMTP id b21-20020a17090a8c9500b002744b04392fmr2435217pjo.24.1694641071644; Wed, 13 Sep 2023 14:37:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694641071; cv=none; d=google.com; s=arc-20160816; b=mfZlx0I+RpDNaYc7bUgDUB2mvhkTDmpVf+Of+wVjqCJRAdA1jXCpCXRNfIKiW+GpD2 igrOHMVkF2JWShIKCysisDXDrayoyL6UPuUOWjUkLBnjEaRyIPESiT7i15vht+z7e4f4 CC63UBeBGVTHaCRK4zN14MqYtdvdPjF4hZvNoWaKe+vbm7vyQPCinVrNs9EUsJpzRdHV 7AAfohfKOaG/hCwtezX6Eytwure6Sgqbjdsbz9ihztQbZypWtq+k5LI0ZlaSJYgRLVuk cAz10TWnM6DAS21SlfndHNPJJMD5Zvv6sdsCOKzII45JTp6CnDmh3L9kY6S6v2YI2o3k MFxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=54/TH6NwZmycBuSVYU7gHf61WfubLeQthagty1Zb+Ho=; fh=szS6nKOHqWqdmo+pDmdMHkpBM8bpzBlMJyKfLi9jRHY=; b=A2rAPFTBuR6mVFBxWLCPbFEqwBUIk4atgY9EN3qmhBIZrTLvM33FOBc1BIC66jkTo2 WZYI/SBA9kClTLT61aZq9SB8RBcKxJ19KqtQhFCkxDBIgUi5NASv2CDb7fZbHw7AXod7 SDZZbFx8cwML4l3fYCrmgvIVueqeDUgg7SV+HzKcubRHJG3y2PjCtHztSI6Xj1CY0y44 DzBm33OUPRMtTePGO6WQD3mNPJQuTfarNvln3j4nijNBqvjaCbmoEnj7+nLvrzzAu+m5 X3NFOZ23Ltr7sYXWSd5ctZklEIEYzCkGMKuQeKUokIbB50S2XE+vxsnEbBxNjCoJgyIQ DCMg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=NTumMwyN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id z8-20020a17090a468800b00265e48c115csi2391221pjf.114.2023.09.13.14.37.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Sep 2023 14:37:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=NTumMwyN; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id A03C384A6C42; Wed, 13 Sep 2023 08:17:24 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241637AbjIMPRJ (ORCPT + 36 others); Wed, 13 Sep 2023 11:17:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38784 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241665AbjIMPQi (ORCPT ); Wed, 13 Sep 2023 11:16:38 -0400 Received: from out-222.mta1.migadu.com (out-222.mta1.migadu.com [95.215.58.222]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C7363DC for ; Wed, 13 Sep 2023 08:16:34 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1694618193; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=54/TH6NwZmycBuSVYU7gHf61WfubLeQthagty1Zb+Ho=; b=NTumMwyNHBdjcIricrwm9zKU91F6+4vINw2RVn5cVHIyaTqrJSZE1Mg/sY1E8kCeupdeyr Mk+w9MGQRmxoZpZWD0F5Hqi3P6GnuNxVBtZuHXwpRoOTwqymnTnr6HMNHSW+Jrm7T5Bts+ GVowM6zQPg+BYhyrI4+b2Z2GeH1x8Pw= From: chengming.zhou@linux.dev To: axboe@kernel.dk, hch@lst.de, ming.lei@redhat.com, bvanassche@acm.org Cc: kbusch@kernel.org, mst@redhat.com, damien.lemoal@opensource.wdc.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, chengming.zhou@linux.dev, Chengming Zhou Subject: [PATCH v2 2/5] blk-mq: remove RQF_MQ_INFLIGHT Date: Wed, 13 Sep 2023 15:16:13 +0000 Message-Id: <20230913151616.3164338-3-chengming.zhou@linux.dev> In-Reply-To: <20230913151616.3164338-1-chengming.zhou@linux.dev> References: <20230913151616.3164338-1-chengming.zhou@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Wed, 13 Sep 2023 08:17:24 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776959956564282515 X-GMAIL-MSGID: 1776959956564282515 From: Chengming Zhou Since the previous patch change to only account active requests when we really allocate the driver tag, the RQF_MQ_INFLIGHT can be removed and no double account problem. 1. none elevator: flush request will use the first pending request's driver tag, won't double account. 2. other elevator: flush request will be accounted when allocate driver tag when issue, and will be unaccounted when it put the driver tag. Signed-off-by: Chengming Zhou Reviewed-by: Ming Lei --- block/blk-flush.c | 11 ++--------- block/blk-mq-debugfs.c | 1 - block/blk-mq.c | 4 ---- include/linux/blk-mq.h | 2 -- 4 files changed, 2 insertions(+), 16 deletions(-) diff --git a/block/blk-flush.c b/block/blk-flush.c index e73dc22d05c1..3f4d41952ef2 100644 --- a/block/blk-flush.c +++ b/block/blk-flush.c @@ -323,16 +323,9 @@ static void blk_kick_flush(struct request_queue *q, struct blk_flush_queue *fq, flush_rq->mq_ctx = first_rq->mq_ctx; flush_rq->mq_hctx = first_rq->mq_hctx; - if (!q->elevator) { + if (!q->elevator) flush_rq->tag = first_rq->tag; - - /* - * We borrow data request's driver tag, so have to mark - * this flush request as INFLIGHT for avoiding double - * account of this driver tag - */ - flush_rq->rq_flags |= RQF_MQ_INFLIGHT; - } else + else flush_rq->internal_tag = first_rq->internal_tag; flush_rq->cmd_flags = REQ_OP_FLUSH | REQ_PREFLUSH; diff --git a/block/blk-mq-debugfs.c b/block/blk-mq-debugfs.c index c3b5930106b2..5cbeb9344f2f 100644 --- a/block/blk-mq-debugfs.c +++ b/block/blk-mq-debugfs.c @@ -246,7 +246,6 @@ static const char *const rqf_name[] = { RQF_NAME(STARTED), RQF_NAME(FLUSH_SEQ), RQF_NAME(MIXED_MERGE), - RQF_NAME(MQ_INFLIGHT), RQF_NAME(DONTPREP), RQF_NAME(SCHED_TAGS), RQF_NAME(USE_SCHED), diff --git a/block/blk-mq.c b/block/blk-mq.c index e776388decc3..c209a7dddee3 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1066,10 +1066,6 @@ static inline void blk_mq_flush_tag_batch(struct blk_mq_hw_ctx *hctx, { struct request_queue *q = hctx->queue; - /* - * All requests should have been marked as RQF_MQ_INFLIGHT, so - * update hctx->nr_active in batch - */ blk_mq_sub_active_requests(hctx, nr_tags); blk_mq_put_tags(hctx->tags, tag_array, nr_tags); diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 958ed7e89b30..1ab3081c82ed 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -32,8 +32,6 @@ typedef __u32 __bitwise req_flags_t; #define RQF_FLUSH_SEQ ((__force req_flags_t)(1 << 4)) /* merge of different types, fail separately */ #define RQF_MIXED_MERGE ((__force req_flags_t)(1 << 5)) -/* track inflight for MQ */ -#define RQF_MQ_INFLIGHT ((__force req_flags_t)(1 << 6)) /* don't call prep for this one */ #define RQF_DONTPREP ((__force req_flags_t)(1 << 7)) /* use hctx->sched_tags */ From patchwork Wed Sep 13 15:16:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chengming Zhou X-Patchwork-Id: 138890 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9ecd:0:b0:3f2:4152:657d with SMTP id t13csp1161909vqx; Wed, 13 Sep 2023 08:22:05 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHTgQ//ZuRfJ1wTg5K1S5+fpvgpBlWAphJEW5O3aXIHRXfeSbjp8JNVjreCUVmDdg7V1fGD X-Received: by 2002:a17:903:11d1:b0:1c0:ce0f:ab57 with SMTP id q17-20020a17090311d100b001c0ce0fab57mr4735142plh.3.1694618525234; Wed, 13 Sep 2023 08:22:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694618525; cv=none; d=google.com; s=arc-20160816; b=fdesP+us4TlThrcNxwsjlctQNL6J3OCt/I5S8F8yKpzXQ7E5woKNQXgHdKg7EvPQ8H xRluFfQFopQpedpnflweVsxmEm79t/yIUtLmYtJ01ajORFcPxwXhVSNM0PbLx1lPiOZe V5IqAspD40Vq6WEBOA/+ox61hE044srmbeCNpKhLrOmQcGuUSQANe0aDy/FoB25CUwlE LFbpYy65+exx80GXJIYsUXEwD+u6733OSjmJqKwJJj7zl1V4JoFrgL1hBIqyHkESiNYO yEpdlJ8w282S9fyXen4cvR998duhYX8rt3kglVyJAI7KVa0v6Mbp/b9VBFhYLa7RPh0n ehWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=0/BBRvz91/qS3KlzgnvKJU+Ez3qMEnH1nXMNaUxqVE0=; fh=szS6nKOHqWqdmo+pDmdMHkpBM8bpzBlMJyKfLi9jRHY=; b=ywfxyndzGfkkMp7AeU1BQ5XM28JSwVT3woYREoNUJNzTOnf8SYMzCiGtdw9VyWzRtn G8xSzta9tPyx7N+h7u01tcJ9kMQ0guLznM8li0lObtUXgt1Tw4XWnQXF6KBtgm4waEHu vxkcCCSQYqHaJufnD1gSib+rXijeZRQIQo/hKZYhhK0RdkoKfRAOxRHI0p91t92FHnVf hOEkfCC5+FMxZIXehl5YjGlF5s5b6N2DPgB8e+ClyOiEP/jJ7GF2fDOr2i7JnpGgTZAw CigGp+KEfnS6hkW7jxoUuRszTbQBtpl2atV0CfuTPoXkpS1RGKDXw5SLRceGm6fqjKj6 PVoA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=F4q3Ygj6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id d6-20020a170902cec600b001c3868f594bsi9948403plg.634.2023.09.13.08.22.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Sep 2023 08:22:05 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=F4q3Ygj6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 373FF802389B; Wed, 13 Sep 2023 08:17:40 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241631AbjIMPRE (ORCPT + 36 others); Wed, 13 Sep 2023 11:17:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39250 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241673AbjIMPQl (ORCPT ); Wed, 13 Sep 2023 11:16:41 -0400 Received: from out-227.mta1.migadu.com (out-227.mta1.migadu.com [95.215.58.227]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 16861B9 for ; Wed, 13 Sep 2023 08:16:37 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1694618195; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0/BBRvz91/qS3KlzgnvKJU+Ez3qMEnH1nXMNaUxqVE0=; b=F4q3Ygj6OhewGfFQk8HXu//5xlTYgIC1sqR6KVRQA0SVq6L0RiUD/ZRfRu1NkH95snA+s5 woXrvUBK7WVCG2aACvaVJKPnZ7UHGDwOPfJcKH9deghCemJVWWxDZ1PVKJ/Jg5WEa9Ni0W MpDPeTrjt51ApEQrfPSEPDq7qam3WVY= From: chengming.zhou@linux.dev To: axboe@kernel.dk, hch@lst.de, ming.lei@redhat.com, bvanassche@acm.org Cc: kbusch@kernel.org, mst@redhat.com, damien.lemoal@opensource.wdc.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, chengming.zhou@linux.dev, Chengming Zhou Subject: [PATCH v2 3/5] blk-mq: support batched queue_rqs() on shared tags queue Date: Wed, 13 Sep 2023 15:16:14 +0000 Message-Id: <20230913151616.3164338-4-chengming.zhou@linux.dev> In-Reply-To: <20230913151616.3164338-1-chengming.zhou@linux.dev> References: <20230913151616.3164338-1-chengming.zhou@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Wed, 13 Sep 2023 08:17:40 -0700 (PDT) X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776936315153521531 X-GMAIL-MSGID: 1776936315153521531 From: Chengming Zhou Since active requests have been accounted when allocate driver tags, we can remove this limit now. Signed-off-by: Chengming Zhou Reviewed-by: Ming Lei --- block/blk-mq.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index c209a7dddee3..68ce9357463b 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -2781,13 +2781,8 @@ void blk_mq_flush_plug_list(struct blk_plug *plug, bool from_schedule) * If we do, we can dispatch the whole plug list in one go. We * already know at this point that all requests belong to the * same queue, caller must ensure that's the case. - * - * Since we pass off the full list to the driver at this point, - * we do not increment the active request count for the queue. - * Bypass shared tags for now because of that. */ - if (q->mq_ops->queue_rqs && - !(rq->mq_hctx->flags & BLK_MQ_F_TAG_QUEUE_SHARED)) { + if (q->mq_ops->queue_rqs) { blk_mq_run_dispatch_ops(q, __blk_mq_flush_plug_list(q, plug)); if (rq_list_empty(plug->mq_list)) From patchwork Wed Sep 13 15:16:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chengming Zhou X-Patchwork-Id: 139003 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:ce21:0:b0:3f4:cb6f:45e2 with SMTP id m1csp1248545vqx; Wed, 13 Sep 2023 11:20:46 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE1gF47adVVsZBHX1Y8Rn+7z9oF8BscNyJ/sJUcqylXdn3hBZFUn68BMEXNWt5gwm+zfstO X-Received: by 2002:a05:6358:71c2:b0:132:ce36:ee31 with SMTP id u2-20020a05635871c200b00132ce36ee31mr3880846rwu.6.1694629246179; Wed, 13 Sep 2023 11:20:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694629246; cv=none; d=google.com; s=arc-20160816; b=izahiB0z1sTfcsczusSGsy7jCWayvr95zBQu8wtpTE3hqOmiwu2FwoMg4Ph5rMiEbu x2WJXZK6HpB3bkj6GSbxbVfBdTyQRxqL5Gp6VlfQ8TsLw2v3JOjCNvxIqY97Qg3/VsJQ nXBIlOJrgg5Apr2siyBOrnGtIDyWwxDBHveLr2yBlFmQzbdk1a8D0Q33BKujr8+JIQ28 HjNx9WxBPqOKvbJlg4JrO7GyvSGdMUCK73ZyWrjs2Z5oDNKlmmjkejm7Xi2qnOjgMgAU wWe9BlJFpkF+JxrzQsyF06ysr7xbkFioj6wSwg1+p5RA1nE2LMDELPJcVqSmDM5H4lT9 7bvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Dj7vqocDNNbQLsHxPquEzULPZP2DWtKivkF6Ijm0nQA=; fh=szS6nKOHqWqdmo+pDmdMHkpBM8bpzBlMJyKfLi9jRHY=; b=JwInMMfS1tqzXeUPGIq6Vq9FfX7KXMD+Rb0O6TYNnPBvnmJH/fjmpU9Hnxyr1dqc9M rFL2YiMVUM+Vi/BAu9R5FxQ0FiqE/yU02WfWDPmy2yluG7XcKXQJdsT3xVrHUceBj8pX coLZcH3vTIGcrG6yuvm0lFy3lPIeF3Cwd+wQOthLcRzEQdw7xARFMMFxH4NzvDoysnuU BlMh9Gq5k02VKgzcx7hf113aV4Pg1LMcaNwAaji0RhBJgSFBjBiIURpsUhr4o60zV4PC FFPmygBnmVCfMoNGhVTrnnY+Bm2V8+iEOUzngCdJDYzlrHGKgs+9RtPtK9FGf7RNaK/F faAg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=wTrJglrX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id v202-20020a6361d3000000b0057751b4abe2si7643438pgb.111.2023.09.13.11.20.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Sep 2023 11:20:46 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=wTrJglrX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 27ECE84A6C75; Wed, 13 Sep 2023 08:17:30 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241649AbjIMPRK (ORCPT + 36 others); Wed, 13 Sep 2023 11:17:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241684AbjIMPQn (ORCPT ); Wed, 13 Sep 2023 11:16:43 -0400 Received: from out-215.mta1.migadu.com (out-215.mta1.migadu.com [IPv6:2001:41d0:203:375::d7]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A15F5C1 for ; Wed, 13 Sep 2023 08:16:39 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1694618198; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Dj7vqocDNNbQLsHxPquEzULPZP2DWtKivkF6Ijm0nQA=; b=wTrJglrXKddvN2j1EuQ7ruRuUNx10JYg6pOcbN5+v6fpthbUIZB3gjo7wBPSvcHVeAJvFl XX5FhjgJa2RNnEO0bK9JIXJ1McJ27k3vYWhAGCVrmkIVdlBRUF6nJ4rIukP+ProFpe/Sj1 kqdvjCi6k4cdtXNXQLCqCfGBJ5k8Tbg= From: chengming.zhou@linux.dev To: axboe@kernel.dk, hch@lst.de, ming.lei@redhat.com, bvanassche@acm.org Cc: kbusch@kernel.org, mst@redhat.com, damien.lemoal@opensource.wdc.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, chengming.zhou@linux.dev, Chengming Zhou Subject: [PATCH v2 4/5] blk-mq: update driver tags request table when start request Date: Wed, 13 Sep 2023 15:16:15 +0000 Message-Id: <20230913151616.3164338-5-chengming.zhou@linux.dev> In-Reply-To: <20230913151616.3164338-1-chengming.zhou@linux.dev> References: <20230913151616.3164338-1-chengming.zhou@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Wed, 13 Sep 2023 08:17:30 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776947556675551539 X-GMAIL-MSGID: 1776947556675551539 From: Chengming Zhou Now we update driver tags request table in blk_mq_get_driver_tag(), so the driver that support queue_rqs() have to update that inflight table by itself. Move it to blk_mq_start_request(), which is a better place where we setup the deadline for request timeout check. And it's just where the request becomes inflight. Signed-off-by: Chengming Zhou Reviewed-by: Ming Lei --- block/blk-mq.c | 1 + block/blk-mq.h | 3 --- drivers/block/virtio_blk.c | 2 -- drivers/nvme/host/pci.c | 1 - 4 files changed, 1 insertion(+), 6 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 68ce9357463b..e2d11183f62e 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1259,6 +1259,7 @@ void blk_mq_start_request(struct request *rq) blk_add_timer(rq); WRITE_ONCE(rq->state, MQ_RQ_IN_FLIGHT); + rq->mq_hctx->tags->rqs[rq->tag] = rq; #ifdef CONFIG_BLK_DEV_INTEGRITY if (blk_integrity_rq(rq) && req_op(rq) == REQ_OP_WRITE) diff --git a/block/blk-mq.h b/block/blk-mq.h index 560a76df290a..f75a9ecfebde 100644 --- a/block/blk-mq.h +++ b/block/blk-mq.h @@ -351,12 +351,9 @@ bool __blk_mq_alloc_driver_tag(struct request *rq); static inline bool blk_mq_get_driver_tag(struct request *rq) { - struct blk_mq_hw_ctx *hctx = rq->mq_hctx; - if (rq->tag == BLK_MQ_NO_TAG && !__blk_mq_alloc_driver_tag(rq)) return false; - hctx->tags->rqs[rq->tag] = rq; return true; } diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c index 1fe011676d07..4689ac2e0c0e 100644 --- a/drivers/block/virtio_blk.c +++ b/drivers/block/virtio_blk.c @@ -470,8 +470,6 @@ static bool virtblk_prep_rq_batch(struct request *req) struct virtio_blk *vblk = req->mq_hctx->queue->queuedata; struct virtblk_req *vbr = blk_mq_rq_to_pdu(req); - req->mq_hctx->tags->rqs[req->tag] = req; - return virtblk_prep_rq(req->mq_hctx, vblk, req, vbr) == BLK_STS_OK; } diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 2f57da12d983..c2e942808eff 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -924,7 +924,6 @@ static bool nvme_prep_rq_batch(struct nvme_queue *nvmeq, struct request *req) if (unlikely(!nvme_check_ready(&nvmeq->dev->ctrl, req, true))) return false; - req->mq_hctx->tags->rqs[req->tag] = req; return nvme_prep_rq(nvmeq->dev, req) == BLK_STS_OK; } From patchwork Wed Sep 13 15:16:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chengming Zhou X-Patchwork-Id: 139221 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp48348vqi; Wed, 13 Sep 2023 18:39:25 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE7XD7m48RTrJZovaryQ9flZUpIa6GH38fhvKFJFLKvhC9aUQyWxDut5S680u2dLZtzkEW3 X-Received: by 2002:a05:6a00:21cf:b0:68b:c562:da65 with SMTP id t15-20020a056a0021cf00b0068bc562da65mr4402300pfj.26.1694655565575; Wed, 13 Sep 2023 18:39:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694655565; cv=none; d=google.com; s=arc-20160816; b=JFV3sfstJ3rju/fOV+daIP582PoAYNbe5+dpbHxLXf/4Y6kvfFkLopTgVUQgl2IN3m TmPTGTm07BagVJx/87je8F+aK3dWzPkSnYOK2ylm9sawOJjT9G2cT7pBgHWpGFKuM5a9 oOCmB2tNzIo9qYacP5JhoK8D08g6ulKWL1e4hbdTF7OqFuJWzRRDSmCNX4pBUSn7oOFp blJxkZYf8c/er1U9TMSmuOZ92gyrVZLkm51RGgGOFzxXuXfye/0fLEPxufZ1aqzmB5zo oZN4HwfENn61mCmffHS77TenlwIRFW+sdUA9Bt1HdjgFVyWLHg5FvTUE/j7mtaXoLfv+ SygQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=hCOMDgQx+bgpwKLnJ2wBA8f+pdNiIK3o3ggHoT0hM7g=; fh=szS6nKOHqWqdmo+pDmdMHkpBM8bpzBlMJyKfLi9jRHY=; b=jfSxCrZR2GJt0SNEZ+vTbZmofW1oEVMcbjgaJI5KUHk7Pnp0vXrc/90zgToGHCc9bZ V5Q3OJVsaDsR6ezUMhebusGOH6I3XtStG4m1T1eivcSH74nZl+ZanDUEjS7C9MAbtwdU AaseyY9aRo7gd/s1mOcRzdaDp8C6SDRKFp7LwWO34kNoi8Wm9tlGHzGGmM47GShcUD0t DMIUDfT8XB867dYHG7ohAmOw0CxQo/fYSuXe6lV8K5DvUUn365QVt3eBgMy1NdkNN15n uV/da5wWipCVDtbMSIG0vz8bfFEESJWoKCkZ6EVvom07bhFdZ8xFoq4TyfZk9pj6DOfN G42g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=tTN41pCZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id p10-20020a056a0026ca00b0068e2f34e100si480463pfw.67.2023.09.13.18.39.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Sep 2023 18:39:25 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=tTN41pCZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id D7C7A84A6C7B; Wed, 13 Sep 2023 08:17:33 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241589AbjIMPRM (ORCPT + 36 others); Wed, 13 Sep 2023 11:17:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55816 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241701AbjIMPQq (ORCPT ); Wed, 13 Sep 2023 11:16:46 -0400 Received: from out-220.mta1.migadu.com (out-220.mta1.migadu.com [IPv6:2001:41d0:203:375::dc]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3633EDF for ; Wed, 13 Sep 2023 08:16:42 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1694618200; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hCOMDgQx+bgpwKLnJ2wBA8f+pdNiIK3o3ggHoT0hM7g=; b=tTN41pCZ//w5UNw1GVbdKGdqCSWpAtDlyNGtHOLmA/YyKL9qApU6+LurZMfZ+k7Lyhit9u ULGyDYi6YxwhAFwIanaBI2Eeg/Og6Ph7SECgHyu9Uc2MVF9JKexnFtaoBaVl9EOFG9WDv/ 0BvG7Xgr04o7ABH9n1TVozi+Y8JsEYQ= From: chengming.zhou@linux.dev To: axboe@kernel.dk, hch@lst.de, ming.lei@redhat.com, bvanassche@acm.org Cc: kbusch@kernel.org, mst@redhat.com, damien.lemoal@opensource.wdc.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, chengming.zhou@linux.dev, Chengming Zhou Subject: [PATCH v2 5/5] block/null_blk: add queue_rqs() support Date: Wed, 13 Sep 2023 15:16:16 +0000 Message-Id: <20230913151616.3164338-6-chengming.zhou@linux.dev> In-Reply-To: <20230913151616.3164338-1-chengming.zhou@linux.dev> References: <20230913151616.3164338-1-chengming.zhou@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Wed, 13 Sep 2023 08:17:33 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1776975154525741547 X-GMAIL-MSGID: 1776975154525741547 From: Chengming Zhou Add batched mq_ops.queue_rqs() support in null_blk for testing. The implementation is much easy since null_blk doesn't have commit_rqs(). We simply handle each request one by one, if errors are encountered, leave them in the passed in list and return back. There is about 3.6% improvement in IOPS of fio/t/io_uring on null_blk with hw_queue_depth=256 on my test VM, from 1.09M to 1.13M. Signed-off-by: Chengming Zhou Reviewed-by: Ming Lei --- drivers/block/null_blk/main.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c index 968090935eb2..79d6cd3c3d41 100644 --- a/drivers/block/null_blk/main.c +++ b/drivers/block/null_blk/main.c @@ -1750,6 +1750,25 @@ static blk_status_t null_queue_rq(struct blk_mq_hw_ctx *hctx, return null_handle_cmd(cmd, sector, nr_sectors, req_op(rq)); } +static void null_queue_rqs(struct request **rqlist) +{ + struct request *requeue_list = NULL; + struct request **requeue_lastp = &requeue_list; + struct blk_mq_queue_data bd = { }; + blk_status_t ret; + + do { + struct request *rq = rq_list_pop(rqlist); + + bd.rq = rq; + ret = null_queue_rq(rq->mq_hctx, &bd); + if (ret != BLK_STS_OK) + rq_list_add_tail(&requeue_lastp, rq); + } while (!rq_list_empty(*rqlist)); + + *rqlist = requeue_list; +} + static void cleanup_queue(struct nullb_queue *nq) { bitmap_free(nq->tag_map); @@ -1802,6 +1821,7 @@ static int null_init_hctx(struct blk_mq_hw_ctx *hctx, void *driver_data, static const struct blk_mq_ops null_mq_ops = { .queue_rq = null_queue_rq, + .queue_rqs = null_queue_rqs, .complete = null_complete_rq, .timeout = null_timeout_rq, .poll = null_poll,