[RFC,0/7] blk-mq: improve tag fair sharing

  From: Yu Kuai <yukuai3@huawei.com>

This is not a formal version and not fully tested, I send this RFC
because I want to make sure if people think doing this is meaningful,
before I spend too much time on this.

This patchset tries to refactor how tag is shared:
 - patch 2 delay tag sharing from issue io to when get driver tag faild;
 - patch 3 support to access which queues/hctxs is sharing tags through
 blk_mq_tags;
 - patch 4 move the caculation that how many tags is available from
 hctx_may_queue() to when queue/hctx start or stop to sharing tags.
 - patch 5 record new information that how many times fail to get driver
 tag recently; And patch 7 use this to adjust available tags for each
 shared queue.

Yu Kuai (7):
  blk-mq: factor out a structure from blk_mq_tags to control tag sharing
  blk-mq: delay tag fair sharing until fail to get driver tag
  blk-mq: support to track active queues from blk_mq_tags
  blk-mq: precalculate available tags for hctx_may_queue()
  blk-mq: record the number of times fail to get driver tag while
    sharing tags
  blk-mq: move active request counter to struct tag_sharing
  blk-mq: allow shared queue to get more driver tags

 block/blk-core.c       |   2 -
 block/blk-mq-debugfs.c |   6 +-
 block/blk-mq-tag.c     | 154 ++++++++++++++++++++++++++++++++++++++---
 block/blk-mq.c         |  10 ++-
 block/blk-mq.h         |  39 ++++++-----
 include/linux/blk-mq.h |  24 ++++---
 include/linux/blkdev.h |  13 +++-
 7 files changed, 201 insertions(+), 47 deletions(-)
  

On 6/18/23 09:07, Yu Kuai wrote:
> This is not a formal version and not fully tested, I send this RFC
> because I want to make sure if people think doing this is meaningful,
> before I spend too much time on this.
The approach looks good to me but I'd like to hear from Jens and 
Christoph what their opinion is about the approach of this patch series 
before doing an in-depth review.

Thanks,

Bart.
  

Hi, Christoph and Jens and anyone who's interested

在 2023/06/20 23:20, Bart Van Assche 写道:
> On 6/18/23 09:07, Yu Kuai wrote:
>> This is not a formal version and not fully tested, I send this RFC
>> because I want to make sure if people think doing this is meaningful,
>> before I spend too much time on this.
> The approach looks good to me but I'd like to hear from Jens and 
> Christoph what their opinion is about the approach of this patch series 
> before doing an in-depth review.
> 
Any suggestions on this topic? It'll be great to hear that if anyone
thinks it's meaningful to refactor tag fair sharing.

Thanks,
Kuai
> Thanks,
> 
> Bart.
> 
> .
>
  

On 7/3/23 06:29, Yu Kuai wrote:
> 在 2023/06/20 23:20, Bart Van Assche 写道:
>> On 6/18/23 09:07, Yu Kuai wrote:
>>> This is not a formal version and not fully tested, I send this RFC
>>> because I want to make sure if people think doing this is meaningful,
>>> before I spend too much time on this.
>> The approach looks good to me but I'd like to hear from Jens and 
>> Christoph what their opinion is about the approach of this patch 
>> series before doing an in-depth review.
>>
> Any suggestions on this topic? It'll be great to hear that if anyone
> thinks it's meaningful to refactor tag fair sharing.

The cover letter of this patch series says "This is not a formal version 
and not fully tested". If a fully tested version will be posted, I will 
help with an in-depth review.

Thanks,

Bart.
  

Hi,

在 2023/07/04 2:08, Bart Van Assche 写道:
> On 7/3/23 06:29, Yu Kuai wrote:
>> 在 2023/06/20 23:20, Bart Van Assche 写道:
>>> On 6/18/23 09:07, Yu Kuai wrote:
>>>> This is not a formal version and not fully tested, I send this RFC
>>>> because I want to make sure if people think doing this is meaningful,
>>>> before I spend too much time on this.
>>> The approach looks good to me but I'd like to hear from Jens and 
>>> Christoph what their opinion is about the approach of this patch 
>>> series before doing an in-depth review.
>>>
>> Any suggestions on this topic? It'll be great to hear that if anyone
>> thinks it's meaningful to refactor tag fair sharing.
> 
> The cover letter of this patch series says "This is not a formal version 
> and not fully tested". If a fully tested version will be posted, I will 
> help with an in-depth review.

Thanks for taking time on this patchset, do you think I need to do
following for the formal version, or these improvements can be done
later?

- current algorithm to borrow tags in patch 7 is very easy and
straightforward, it should work fine on simple caces like the case you
reported for ufs, but this algorithm should be improved for more
complicated cases.
- currently borrowed tags will never be returned untill queue is idle,
I should figure out a way to return borrowed tags if this queue is not
busy, so that other queues can borrow tag from this queue.

Thanks,
Kuai

> 
> Thanks,
> 
> Bart.
> 
> 
> .
>
  

On 7/4/23 20:17, Yu Kuai wrote:
> - currently borrowed tags will never be returned untill queue is idle,
> I should figure out a way to return borrowed tags if this queue is not
> busy, so that other queues can borrow tag from this queue.

At least for UFS this is a significant disadvantage. If e.g. one SCSI
command is sent to the UFS WLUN every 20 seconds and the request queue
timeout is 30 seconds then borrowed tags will never be returned.

The complexity of this patch series is a concern to me. The complexity
of this patch series may be a barrier towards merging this patch series
in the upstream kernel.

Thanks,

Bart.
  

Hi,

在 2023/07/07 2:43, Bart Van Assche 写道:
> On 7/4/23 20:17, Yu Kuai wrote:
>> - currently borrowed tags will never be returned untill queue is idle,
>> I should figure out a way to return borrowed tags if this queue is not
>> busy, so that other queues can borrow tag from this queue.
> 
> At least for UFS this is a significant disadvantage. If e.g. one SCSI
> command is sent to the UFS WLUN every 20 seconds and the request queue
> timeout is 30 seconds then borrowed tags will never be returned.

Ok, thanks for the notice.
> 
> The complexity of this patch series is a concern to me. The complexity
> of this patch series may be a barrier towards merging this patch series
> in the upstream kernel.

Yeah, I see. Thanks for the review! However, all I have in mind will end
up make this patch series more complicated, I'll try my best to make the
code more readable in the next version.

Thanks,
Kuai
> 
> Thanks,
> 
> Bart.
> 
> 
> .
>