mbox series

[wq/for-6.9,v5,0/4] workqueue: Enable unbound cpumask update on ordered workqueues

Message ID 20240208161014.1084943-1-longman@redhat.com
Headers
Series workqueue: Enable unbound cpumask update on ordered workqueues |

Message

Waiman Long Feb. 8, 2024, 4:10 p.m. UTC 
  v5:
  - [v4] https://lore.kernel.org/lkml/20240207011911.975608-1-longman@redhat.com/
  - Streamline patch 2 by simplifying unplug_oldest_pwq() and calling it
    only in pwq_release_workfn().

 v4:
  - [v3] https://lore.kernel.org/lkml/20240205194602.871505-1-longman@redhat.com/
  - Rebase on the latest for-6.9 branch again & discard the use of
    __WQ_ORDERED_EXPLICIT and resetting of __WQ_ORDERED.
  - Add a new patch 1 to change the ordering of pwq's in wq->pwqs from
    oldest to newest.
  - Change the terminalogy from freeze/thaw to plug/unplug.
  - Allow more than 2 pwq's in wq->pwqs of ordered workqueue but only the
    oldest one is unplugged. This eliminates the need to wait for
    the draining of extra pwq in workqueue_apply_unbound_cpumask().

 v3:
  - [v2] https://lore.kernel.org/lkml/20240203154334.791910-1-longman@redhat.com/
  - Drop patch 1 as it has been merged into the for-6.9 branch.
  - Use rcu_access_pointer() to access wq->dfl_pwq.
  - Use RCU protection instead of acquiring wq->mutex in
    apply_wqattrs_cleanup().

Ordered workqueues does not currently follow changes made to the
global unbound cpumask because per-pool workqueue changes may break
the ordering guarantee. IOW, a work function in an ordered workqueue
may run on a cpuset isolated CPU.

This series enables ordered workqueues to follow changes made to the
global unbound cpumask by temporaily suspending (plugging) the execution
of work items in the newly allocated pool_workqueue until the old pwq
has been properly drained.

The cpumask of the rescuer task of each workqueue is also made to follow
changes in workqueue unbound cpumask as well as its sysfs cpumask,
if available.

Juri Lelli (1):
  kernel/workqueue: Let rescuers follow unbound wq cpumask changes

Waiman Long (3):
  workqueue: Link pwq's into wq->pwqs from oldest to newest
  workqueue: Enable unbound cpumask update on ordered workqueues
  workqueue: Bind unbound workqueue rescuer to wq_unbound_cpumask

 kernel/workqueue.c | 81 +++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 69 insertions(+), 12 deletions(-)

Comments

Tejun Heo Feb. 8, 2024, 7:26 p.m. UTC | #1 
On Thu, Feb 08, 2024 at 11:10:10AM -0500, Waiman Long wrote:
>  v5:
>   - [v4] https://lore.kernel.org/lkml/20240207011911.975608-1-longman@redhat.com/
>   - Streamline patch 2 by simplifying unplug_oldest_pwq() and calling it
>     only in pwq_release_workfn().
> 
>  v4:
>   - [v3] https://lore.kernel.org/lkml/20240205194602.871505-1-longman@redhat.com/
>   - Rebase on the latest for-6.9 branch again & discard the use of
>     __WQ_ORDERED_EXPLICIT and resetting of __WQ_ORDERED.
>   - Add a new patch 1 to change the ordering of pwq's in wq->pwqs from
>     oldest to newest.
>   - Change the terminalogy from freeze/thaw to plug/unplug.
>   - Allow more than 2 pwq's in wq->pwqs of ordered workqueue but only the
>     oldest one is unplugged. This eliminates the need to wait for
>     the draining of extra pwq in workqueue_apply_unbound_cpumask().
> 
>  v3:
>   - [v2] https://lore.kernel.org/lkml/20240203154334.791910-1-longman@redhat.com/
>   - Drop patch 1 as it has been merged into the for-6.9 branch.
>   - Use rcu_access_pointer() to access wq->dfl_pwq.
>   - Use RCU protection instead of acquiring wq->mutex in
>     apply_wqattrs_cleanup().

Applied the series w/ (w/ the updated second patch) to wq/for-6.9.

Thanks for working on this. It's really great that this got solved finally.
Waiman Long Feb. 8, 2024, 7:33 p.m. UTC | #2 
On 2/8/24 14:26, Tejun Heo wrote:
> On Thu, Feb 08, 2024 at 11:10:10AM -0500, Waiman Long wrote:
>>   v5:
>>    - [v4] https://lore.kernel.org/lkml/20240207011911.975608-1-longman@redhat.com/
>>    - Streamline patch 2 by simplifying unplug_oldest_pwq() and calling it
>>      only in pwq_release_workfn().
>>
>>   v4:
>>    - [v3] https://lore.kernel.org/lkml/20240205194602.871505-1-longman@redhat.com/
>>    - Rebase on the latest for-6.9 branch again & discard the use of
>>      __WQ_ORDERED_EXPLICIT and resetting of __WQ_ORDERED.
>>    - Add a new patch 1 to change the ordering of pwq's in wq->pwqs from
>>      oldest to newest.
>>    - Change the terminalogy from freeze/thaw to plug/unplug.
>>    - Allow more than 2 pwq's in wq->pwqs of ordered workqueue but only the
>>      oldest one is unplugged. This eliminates the need to wait for
>>      the draining of extra pwq in workqueue_apply_unbound_cpumask().
>>
>>   v3:
>>    - [v2] https://lore.kernel.org/lkml/20240203154334.791910-1-longman@redhat.com/
>>    - Drop patch 1 as it has been merged into the for-6.9 branch.
>>    - Use rcu_access_pointer() to access wq->dfl_pwq.
>>    - Use RCU protection instead of acquiring wq->mutex in
>>      apply_wqattrs_cleanup().
> Applied the series w/ (w/ the updated second patch) to wq/for-6.9.
>
> Thanks for working on this. It's really great that this got solved finally.

Me too. Thanks for all the good suggestions that you had given me as I 
don't have a full understanding of all the different pieces of the 
workqueue code.

Cheers,
Longman