[V4,0/6] genirq/affinity: Abstract APIs from managed irq affinity spread

  Hello,

irq_build_affinity_masks() actually grouping CPUs evenly into each managed
irq vector according to NUMA and CPU locality, and it is reasonable to abstract
one generic API for grouping CPUs evenly, the idea is suggested by Thomas
Gleixner.

group_cpus_evenly() is abstracted and put into lib/, so blk-mq can re-use
it to build default queue mapping.

blk-mq IO perf data is observed as more stable, meantime with big
improvement, see detailed data in the last patch.

Please consider it for v6.3!

V4:
	- address comments from John, not export the API, given so far no
	  module uses this symbol
	- add maintainer entry for new added lib/group_cpus.c
	- rebase on 6.2

V3:
	- fix build failure in case of !CONFIG_SMP, only 6/7 is changed

V2:
	- fix build failure in case of !CONFIG_SMP
	- fix commit log typo
	- fix memory leak in last patch
	- add reviewed-by

Since RFC:
	- remove RFC
	- rebase on -next tree


Ming Lei (6):
  genirq/affinity: Remove the 'firstvec' parameter from
    irq_build_affinity_masks
  genirq/affinity: Pass affinity managed mask array to
    irq_build_affinity_masks
  genirq/affinity: Don't pass irq_affinity_desc array to
    irq_build_affinity_masks
  genirq/affinity: Rename irq_build_affinity_masks as group_cpus_evenly
  genirq/affinity: Move group_cpus_evenly() into lib/
  blk-mq: Build default queue map via group_cpus_evenly()

 MAINTAINERS                |   2 +
 block/blk-mq-cpumap.c      |  63 ++----
 include/linux/group_cpus.h |  14 ++
 kernel/irq/affinity.c      | 405 +----------------------------------
 lib/Makefile               |   2 +
 lib/group_cpus.c           | 427 +++++++++++++++++++++++++++++++++++++
 6 files changed, 467 insertions(+), 446 deletions(-)
 create mode 100644 include/linux/group_cpus.h
 create mode 100644 lib/group_cpus.c
  

Hello Thomas, Jens and guys,

On Tue, Dec 27, 2022 at 10:28:59AM +0800, Ming Lei wrote:
> Hello,
> 
> irq_build_affinity_masks() actually grouping CPUs evenly into each managed
> irq vector according to NUMA and CPU locality, and it is reasonable to abstract
> one generic API for grouping CPUs evenly, the idea is suggested by Thomas
> Gleixner.
> 
> group_cpus_evenly() is abstracted and put into lib/, so blk-mq can re-use
> it to build default queue mapping.
> 
> blk-mq IO perf data is observed as more stable, meantime with big
> improvement, see detailed data in the last patch.
> 
> Please consider it for v6.3!
> 
> V4:
> 	- address comments from John, not export the API, given so far no
> 	  module uses this symbol
> 	- add maintainer entry for new added lib/group_cpus.c
> 	- rebase on 6.2

Any chance to take a look at this patchset?


thanks,
Ming
  

On 27/12/2022 02:28, Ming Lei wrote:
> irq_build_affinity_masks() actually grouping CPUs evenly into each managed
> irq vector according to NUMA and CPU locality, and it is reasonable to abstract
> one generic API for grouping CPUs evenly, the idea is suggested by Thomas
> Gleixner.
> 
> group_cpus_evenly() is abstracted and put into lib/, so blk-mq can re-use
> it to build default queue mapping.
> 
> blk-mq IO perf data is observed as more stable, meantime with big
> improvement, see detailed data in the last patch.
> 
> Please consider it for v6.3!

Just wondering could this be a better alternative for some drivers than 
using cpumask_local_spread()?

Thanks,
John
  

On Wed, Jan 11 2023 at 10:18, Ming Lei wrote:
> On Tue, Dec 27, 2022 at 10:28:59AM +0800, Ming Lei wrote:
>> Hello,
>> 
>> irq_build_affinity_masks() actually grouping CPUs evenly into each managed
>> irq vector according to NUMA and CPU locality, and it is reasonable to abstract
>> one generic API for grouping CPUs evenly, the idea is suggested by Thomas
>> Gleixner.
>> 
>> group_cpus_evenly() is abstracted and put into lib/, so blk-mq can re-use
>> it to build default queue mapping.
>> 
>> blk-mq IO perf data is observed as more stable, meantime with big
>> improvement, see detailed data in the last patch.
>> 
>> Please consider it for v6.3!
>> 
>> V4:
>> 	- address comments from John, not export the API, given so far no
>> 	  module uses this symbol
>> 	- add maintainer entry for new added lib/group_cpus.c
>> 	- rebase on 6.2
>
> Any chance to take a look at this patchset?

I'm at it. My sickness+vacation induced backlog is horrible ....
  

On 1/10/23 7:18 PM, Ming Lei wrote:
> Hello Thomas, Jens and guys,

I took a look and it looks good to me, no immediate issues spotted.
  

On Wed, Jan 11, 2023 at 07:58:50PM +0100, Thomas Gleixner wrote:
> On Wed, Jan 11 2023 at 10:18, Ming Lei wrote:
> > On Tue, Dec 27, 2022 at 10:28:59AM +0800, Ming Lei wrote:
> >> Hello,
> >> 
> >> irq_build_affinity_masks() actually grouping CPUs evenly into each managed
> >> irq vector according to NUMA and CPU locality, and it is reasonable to abstract
> >> one generic API for grouping CPUs evenly, the idea is suggested by Thomas
> >> Gleixner.
> >> 
> >> group_cpus_evenly() is abstracted and put into lib/, so blk-mq can re-use
> >> it to build default queue mapping.
> >> 
> >> blk-mq IO perf data is observed as more stable, meantime with big
> >> improvement, see detailed data in the last patch.
> >> 
> >> Please consider it for v6.3!
> >> 
> >> V4:
> >> 	- address comments from John, not export the API, given so far no
> >> 	  module uses this symbol
> >> 	- add maintainer entry for new added lib/group_cpus.c
> >> 	- rebase on 6.2
> >
> > Any chance to take a look at this patchset?
> 
> I'm at it. My sickness+vacation induced backlog is horrible ....

Thanks for the response, and wish you good health.


thanks, 
Ming
  

Jens!

On Wed, Jan 11 2023 at 12:04, Jens Axboe wrote:
> On 1/10/23 7:18 PM, Ming Lei wrote:
>> Hello Thomas, Jens and guys,
>
> I took a look and it looks good to me, no immediate issues spotted.

Want me to take the blk-mq change through tip too or do you prefer to
merge that into your tree?

If this goes through tip, I'd appreciate an Ack. If you want it through
block, then I give you a tag with the irq parts for you to pull.

Thanks,

        tglx
  

On 1/16/23 12:13 PM, Thomas Gleixner wrote:
> Jens!
> 
> On Wed, Jan 11 2023 at 12:04, Jens Axboe wrote:
>> On 1/10/23 7:18 PM, Ming Lei wrote:
>>> Hello Thomas, Jens and guys,
>>
>> I took a look and it looks good to me, no immediate issues spotted.
> 
> Want me to take the blk-mq change through tip too or do you prefer to
> merge that into your tree?
> 
> If this goes through tip, I'd appreciate an Ack. If you want it through
> block, then I give you a tag with the irq parts for you to pull.

I think the risk of conflicts there is going to be very small, so
please just take it through the tip tree. You can add my:

Reviewed-by: Jens Axboe <axboe@kernel.dk>

to the series. Thanks!