mbox series

[PATCH-wq,v3,0/4] workqueue: Enable unbound cpumask update on ordered workqueues

Message ID 20240205194602.871505-1-longman@redhat.com
Headers
Series workqueue: Enable unbound cpumask update on ordered workqueues |

Message

Waiman Long Feb. 5, 2024, 7:45 p.m. UTC 
  v3:
  - [v2] https://lore.kernel.org/lkml/20240203154334.791910-1-longman@redhat.com/
  - Drop patch 1 as it has been merged into the for-6.9 branch.
  - Use rcu_access_pointer() to access wq->dfl_pwq.
  - Use RCU protection instead of acquiring wq->mutex in
    apply_wqattrs_cleanup().

 v2:
  - [v1] https://lore.kernel.org/all/20240130183336.511948-1-longman@redhat.com/
  - Rebased on top of wq's for-v6.9 branch.
  - Use the new pwq_tryinc_nr_active() mechanism to freeze the new
    pwq of an ordered workqueue until the old pwq has been properly
    drained to maintain ordering.
  - Make rescuer follow changes in workqueue unbound cpumask as well
    as its sysfs cpumask, if available.

Ordered workqueues does not currently follow changes made to the
global unbound cpumask because per-pool workqueue changes may break
the ordering guarantee. IOW, a work function in an ordered workqueue
may run on a cpuset isolated CPU.

This series enables ordered workqueues to follow changes made to
the global unbound cpumask by temporaily freeze the newly allocated
pool_workqueue by using the new frozen flag to freeze execution of
newly queued work items until the old pwq has been properly flushed.

The cpumask of the rescuer task of each workqueue is also made to follow
changes in workqueue unbound cpumask as well as its sysfs cpumask,
if available.

Juri Lelli (1):
  kernel/workqueue: Let rescuers follow unbound wq cpumask changes

Waiman Long (3):
  workqueue: Enable unbound cpumask update on ordered workqueues
  workqueue: Thaw frozen pwq in workqueue_apply_unbound_cpumask()
  workqueue: Bind unbound workqueue rescuer to wq_unbound_cpumask

 kernel/workqueue.c | 127 ++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 113 insertions(+), 14 deletions(-)

Comments

Tejun Heo Feb. 6, 2024, 12:04 a.m. UTC | #1 
On Mon, Feb 05, 2024 at 09:53:09AM -1000, Tejun Heo wrote:
> On Mon, Feb 05, 2024 at 02:45:58PM -0500, Waiman Long wrote:
> >  v3:
> >   - [v2] https://lore.kernel.org/lkml/20240203154334.791910-1-longman@redhat.com/
> >   - Drop patch 1 as it has been merged into the for-6.9 branch.
> >   - Use rcu_access_pointer() to access wq->dfl_pwq.
> >   - Use RCU protection instead of acquiring wq->mutex in
> >     apply_wqattrs_cleanup().
> 
> Looks like we raced each other. I'll wait for v4.

BTW, please don't bother to handle __WQ_ORDERED being cleared. We are very
close to removing the implicit ORDERED promotion, so we should be able to
apply the patch to remove the distinction between explicitly and implicitly
ordered workqueues.

Thanks.
Waiman Long Feb. 6, 2024, 1:07 a.m. UTC | #2 
On 2/5/24 19:04, Tejun Heo wrote:
> On Mon, Feb 05, 2024 at 09:53:09AM -1000, Tejun Heo wrote:
>> On Mon, Feb 05, 2024 at 02:45:58PM -0500, Waiman Long wrote:
>>>   v3:
>>>    - [v2] https://lore.kernel.org/lkml/20240203154334.791910-1-longman@redhat.com/
>>>    - Drop patch 1 as it has been merged into the for-6.9 branch.
>>>    - Use rcu_access_pointer() to access wq->dfl_pwq.
>>>    - Use RCU protection instead of acquiring wq->mutex in
>>>      apply_wqattrs_cleanup().
>> Looks like we raced each other. I'll wait for v4.
> BTW, please don't bother to handle __WQ_ORDERED being cleared. We are very
> close to removing the implicit ORDERED promotion, so we should be able to
> apply the patch to remove the distinction between explicitly and implicitly
> ordered workqueues.

OK, I saw your new commit 3bc1e711c26b ("workqueue: Don't implicitly 
make UNBOUND workqueues w/ @max_active==1 ordered") in the for-6.9 
branch. Will rebase my patch series on top of that and make the 
necessary modification.

Thanks,
Longman
Waiman Long Feb. 6, 2024, 1:24 a.m. UTC | #3 
On 2/5/24 19:04, Tejun Heo wrote:
> On Mon, Feb 05, 2024 at 09:53:09AM -1000, Tejun Heo wrote:
>> On Mon, Feb 05, 2024 at 02:45:58PM -0500, Waiman Long wrote:
>>>   v3:
>>>    - [v2] https://lore.kernel.org/lkml/20240203154334.791910-1-longman@redhat.com/
>>>    - Drop patch 1 as it has been merged into the for-6.9 branch.
>>>    - Use rcu_access_pointer() to access wq->dfl_pwq.
>>>    - Use RCU protection instead of acquiring wq->mutex in
>>>      apply_wqattrs_cleanup().
>> Looks like we raced each other. I'll wait for v4.
> BTW, please don't bother to handle __WQ_ORDERED being cleared. We are very
> close to removing the implicit ORDERED promotion, so we should be able to
> apply the patch to remove the distinction between explicitly and implicitly
> ordered workqueues.

BTW, the workqueue.c file in your latest for-6.9 branch still has a 
reference to __WQ_ORDERED_EXPLICIT in workqueue_apply_unbound_cpumask(). 
Will that break compilation?

Regards,
Longman
Tejun Heo Feb. 6, 2024, 1:40 a.m. UTC | #4 
On Mon, Feb 05, 2024 at 08:24:06PM -0500, Waiman Long wrote:
> 
> On 2/5/24 19:04, Tejun Heo wrote:
> > On Mon, Feb 05, 2024 at 09:53:09AM -1000, Tejun Heo wrote:
> > > On Mon, Feb 05, 2024 at 02:45:58PM -0500, Waiman Long wrote:
> > > >   v3:
> > > >    - [v2] https://lore.kernel.org/lkml/20240203154334.791910-1-longman@redhat.com/
> > > >    - Drop patch 1 as it has been merged into the for-6.9 branch.
> > > >    - Use rcu_access_pointer() to access wq->dfl_pwq.
> > > >    - Use RCU protection instead of acquiring wq->mutex in
> > > >      apply_wqattrs_cleanup().
> > > Looks like we raced each other. I'll wait for v4.
> > BTW, please don't bother to handle __WQ_ORDERED being cleared. We are very
> > close to removing the implicit ORDERED promotion, so we should be able to
> > apply the patch to remove the distinction between explicitly and implicitly
> > ordered workqueues.
> 
> BTW, the workqueue.c file in your latest for-6.9 branch still has a
> reference to __WQ_ORDERED_EXPLICIT in workqueue_apply_unbound_cpumask().
> Will that break compilation?

Right you are. Will post a followup patch.

Thanks.