From patchwork Thu Jan 12 16:14:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Schneider X-Patchwork-Id: 42528 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp3972966wrt; Thu, 12 Jan 2023 08:22:16 -0800 (PST) X-Google-Smtp-Source: AMrXdXtDvny4gNd2gBdWFqNwEF74StiM0pLaAKXq/HZ2LrwF66HIifHBpSdK9OnGXo8vQ0cYMqkV X-Received: by 2002:a17:907:3f9f:b0:840:4901:901f with SMTP id hr31-20020a1709073f9f00b008404901901fmr87758786ejc.3.1673540536738; Thu, 12 Jan 2023 08:22:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673540536; cv=none; d=google.com; s=arc-20160816; b=PQGSfLE4W+Xbl3RJfqvGobxS9RAmiQuzc/4btSs9IJ8qadaO/mLItOA5IvR+mFbuWA D/IuACG/CvAAFDEu4q1UcRkqJyV1BKatBD6ppoLwS7tPTynOplaCFe1l8QSPVDNtSVQY 3YlBYfrdEe1j5SRcjoNbQxxFpxsJQkkRMeYHzTjbu+w36TSic1u8MGCS1lI4QB6zBkZn bCxNJuxq82cvRJS9JAIxqxFmYk9CgE23RQ905d0BOODz8R2NVDWlSQjpf992vgtK0lwS i3gz4H4e4OIT/LWcSTybwmKNCf372C1YK6BkpP1Sp7Oa3ChozNbVPcn7T8ZMt96cRIS/ 7dPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=PMLRW5gvg+irO4xJtjlod4Jc3xlBzVZYp1kicQUxcG4=; b=D6MDxWWv93s9dC02Pe+pdOp0o5tPW3DEr073Z+BYf2BKfv6LO+jM+qV6lyIULUtgzC kacq7agPdnulgtH9W5GToLf5vgYfCKQnSNq21p+Z6iPF65zJUsBylNDVvy8w942r+gnR HVzmrK/3ycBe/bp7luaLfnLZiCAcj0wu4+ZxzgsV7To24Ov0/mV/BHm5OKrNVMxlrAJz NWnz1yID2wqYkZKLpkSB13YfxHflls5F/SfixSn3QqDKHx+TCUS99H0dk3y3LaYDazVm UGX0N1twnV6I2VQY3SPqI5Rc6ytihLPcLfOHUEpTqc8eajC+R5Q9V4fgYmeKgoe3ERiy uumg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=Kagaz6iE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id wz4-20020a170906fe4400b0084d43e4541csi11905065ejb.720.2023.01.12.08.21.52; Thu, 12 Jan 2023 08:22:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=Kagaz6iE; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232568AbjALQUE (ORCPT + 99 others); Thu, 12 Jan 2023 11:20:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56464 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236311AbjALQSg (ORCPT ); Thu, 12 Jan 2023 11:18:36 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 920A81DF30 for ; Thu, 12 Jan 2023 08:14:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1673540087; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PMLRW5gvg+irO4xJtjlod4Jc3xlBzVZYp1kicQUxcG4=; b=Kagaz6iEzmO8wZDkXbvODyunl6SesJiIiNefVBMyIA3cVujWIhCKblrEJ28OtL8mUAOgSH +DuY61S6KT14swGJKGglNlr83JsqLAeAjCbj9KVWpMjXHwZNOL12OvaPmAz5uwtz4YCijB EOEhxKHNQ+eXE6a6ikuyCEROX8DV1Tk= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-385-5MdotZiLPrelFSWPXgfy8w-1; Thu, 12 Jan 2023 11:14:44 -0500 X-MC-Unique: 5MdotZiLPrelFSWPXgfy8w-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CDF96299E74C; Thu, 12 Jan 2023 16:14:42 +0000 (UTC) Received: from vschneid.remote.csb (unknown [10.33.36.188]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D26462166B26; Thu, 12 Jan 2023 16:14:41 +0000 (UTC) From: Valentin Schneider To: linux-kernel@vger.kernel.org Cc: Tejun Heo , Lai Jiangshan , Peter Zijlstra , Frederic Weisbecker , Juri Lelli , Phil Auld , Marcelo Tosatti Subject: [PATCH v8 4/5] workqueue: Don't hold any lock while rcuwait'ing for !POOL_MANAGER_ACTIVE Date: Thu, 12 Jan 2023 16:14:30 +0000 Message-Id: <20230112161431.860196-5-vschneid@redhat.com> In-Reply-To: <20230112161431.860196-1-vschneid@redhat.com> References: <20230112161431.860196-1-vschneid@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1754834441549587651?= X-GMAIL-MSGID: =?utf-8?q?1754834441549587651?= put_unbound_pool() currently passes wq_manager_inactive() as exit condition to rcuwait_wait_event(), which grabs pool->lock to check for pool->flags & POOL_MANAGER_ACTIVE A later patch will require destroy_worker() to be invoked with wq_pool_attach_mutex held, which needs to be acquired before pool->lock. A mutex cannot be acquired within rcuwait_wait_event(), as it could clobber the task state set by rcuwait_wait_event() Instead, restructure the waiting logic to acquire any necessary lock outside of rcuwait_wait_event(). Since further work cannot be inserted into unbound pwqs that have reached ->refcnt==0, this is bound to make forward progress as eventually the worklist will be drained and need_more_worker(pool) will remain false, preventing any worker from stealing the manager position from us. Suggested-by: Tejun Heo Signed-off-by: Valentin Schneider --- kernel/workqueue.c | 36 +++++++++++++++++++----------------- 1 file changed, 19 insertions(+), 17 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 11bb657059bcd..8ece9bfa04a09 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -3604,18 +3604,6 @@ static void rcu_free_pool(struct rcu_head *rcu) kfree(pool); } -/* This returns with the lock held on success (pool manager is inactive). */ -static bool wq_manager_inactive(struct worker_pool *pool) -{ - raw_spin_lock_irq(&pool->lock); - - if (pool->flags & POOL_MANAGER_ACTIVE) { - raw_spin_unlock_irq(&pool->lock); - return false; - } - return true; -} - /** * put_unbound_pool - put a worker_pool * @pool: worker_pool to put @@ -3651,12 +3639,26 @@ static void put_unbound_pool(struct worker_pool *pool) * Become the manager and destroy all workers. This prevents * @pool's workers from blocking on attach_mutex. We're the last * manager and @pool gets freed with the flag set. - * Because of how wq_manager_inactive() works, we will hold the - * spinlock after a successful wait. + * + * Having a concurrent manager is quite unlikely to happen as we can + * only get here with + * pwq->refcnt == pool->refcnt == 0 + * which implies no work queued to the pool, which implies no worker can + * become the manager. However a worker could have taken the role of + * manager before the refcnts dropped to 0, since maybe_create_worker() + * drops pool->lock */ - rcuwait_wait_event(&manager_wait, wq_manager_inactive(pool), - TASK_UNINTERRUPTIBLE); - pool->flags |= POOL_MANAGER_ACTIVE; + while (true) { + rcuwait_wait_event(&manager_wait, + !(pool->flags & POOL_MANAGER_ACTIVE), + TASK_UNINTERRUPTIBLE); + raw_spin_lock_irq(&pool->lock); + if (!(pool->flags & POOL_MANAGER_ACTIVE)) { + pool->flags |= POOL_MANAGER_ACTIVE; + break; + } + raw_spin_unlock_irq(&pool->lock); + } while ((worker = first_idle_worker(pool))) destroy_worker(worker);