From patchwork Tue Jan 30 18:33:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 194267 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp1421796dyb; Tue, 30 Jan 2024 10:38:28 -0800 (PST) X-Google-Smtp-Source: AGHT+IFMb66IjheJ1c9OwPkHIDZRzXTuYycQgi2oQG+JKI7XmgeNSab527RljdeJpd8E7afVboZQ X-Received: by 2002:a17:902:bd8b:b0:1d8:d3bf:b367 with SMTP id q11-20020a170902bd8b00b001d8d3bfb367mr1769601pls.29.1706639908619; Tue, 30 Jan 2024 10:38:28 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706639908; cv=pass; d=google.com; s=arc-20160816; b=ZOJWfz0iaVfL+f7+T6zNlC3okQxqTUt+CWHc6Xvxso9zkyJAOKBRbnd11zLm/iTEx+ YCXTxsjzSCjmGIb85PB/qAuNY/Z8PH6PzUxy3KnXYv1Duo8fcDFIJMMeMIQsFct684I0 vd0kv5GTgZLUv4ZebxI8owrJlGGhqnEIwpEKaRTnXJnzrwZjRNxzm4wCRmQqJ90/mDjU wGJmvMZtn4J5OkQ3pF0bEsky4ntlR+G2202hEG4p9fWr43vYRA7ghQYz9hIuT6KujOn7 wnxXK+bU6VMyDkKwbl2U4NnmpP9Oh6mCcybY57sCoTyIvuuVuvQW5Hl3PsGK8/1Xp7Cw 8dKg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=pPefEzI7IPB8Pl2sd/W5lhgO3NwL39dr5Md44erL9pY=; fh=lt2js4+oSVfpbDjw8gm9/JJoU0YKcT6Mty9go3Ytfy0=; b=s+W0BCDe2ZQrmJNriLeCpE7uZWtfTz9cSaOuFugse3Mll5LRPXTg8d8DIzsv33CKXw LM7VrQh0NZo0RSqjYtBU/6VVvejpZg6mlzrr4Ljs069wCcJoNB2a0YKQEZkMnsl/dkO/ Pvb9Nwsd/caJymSDMROrhH0o4FHp6UHppBeG+77zB/JJRl+Kbc91aSDl2L96B30M4CWe rSsvBAxBP0W+ubqP2Pya6JAHPbu1lFUdUGqVlp7fwRoG42whSdsXJSf4M46lzVNoI/Fh nYXGETnsgPjARhu8X3J0xiyUFs5m2abn5brHZpSiVUhKNACQ6CHQSOu1OmN1WZx1QDAk YY7w== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b="N1P/12Gm"; arc=pass (i=1 spf=pass spfdomain=redhat.com dkim=pass dkdomain=redhat.com dmarc=pass fromdomain=redhat.com); spf=pass (google.com: domain of linux-kernel+bounces-45112-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-45112-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id d3-20020a170902cec300b001d8fae321f6si2509415plg.88.2024.01.30.10.38.28 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jan 2024 10:38:28 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-45112-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b="N1P/12Gm"; arc=pass (i=1 spf=pass spfdomain=redhat.com dkim=pass dkdomain=redhat.com dmarc=pass fromdomain=redhat.com); spf=pass (google.com: domain of linux-kernel+bounces-45112-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-45112-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 14F73B22E4B for ; Tue, 30 Jan 2024 18:34:33 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id AEFBC69E05; Tue, 30 Jan 2024 18:34:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="N1P/12Gm" Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 51DB1762CE for ; Tue, 30 Jan 2024 18:33:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706639641; cv=none; b=ptk0n2KCozO+eYd7r8IL3WqWvzTkx48XLywK/qlNQdlfpHu17MkIc3SCtj0K+ITxiuJqXgt1OUQBnQUqK5HZBwF/piZPGLH71btb2XwrqU80c2Wj7b5/6HyJiHXM1n2WyI9wHEw8UvS9eERgyOM13M++nFpIi2Pg+38R7rJzazA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706639641; c=relaxed/simple; bh=lL5qdWL2PB4mhrehemlKORG0nbLblNVPf8HOOqtCxN4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Pvp5F4qSWIcMqZYjSfYx3HEB8fvt21FcQrb+Kn/c0YlevCPCJFDEdFWaH7FVocs8+BT1Pl6DcOw9YklohtSGbDmkWMaAJCSvDh+iYEnlVzoxya0XWrLfu/jBwPZrf9R35/rbjyng7alyOJrgaRHdJTNOQ4g/BWc2CGAMc5U3ZrM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=N1P/12Gm; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706639639; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pPefEzI7IPB8Pl2sd/W5lhgO3NwL39dr5Md44erL9pY=; b=N1P/12GmAJRt5gbJ9ND3JoCYoGaDo0gzkMhMI04MZRgeJ5r87TOq+ExnBxNAmy8jidFCUQ QzCwjB54+lSilqtkEfnLKXERijSvkRoEVFhFppzOSLQ9DBX2SOPXo3d/pT8i3mqoGh/dMj n4qe7jq2S1o0lTSmTz2PuNTkdIiV00s= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-661-ndeUQeKQNOqMBNfJxAS7pw-1; Tue, 30 Jan 2024 13:33:57 -0500 X-MC-Unique: ndeUQeKQNOqMBNfJxAS7pw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 091EA38562CA; Tue, 30 Jan 2024 18:33:57 +0000 (UTC) Received: from llong.com (unknown [10.22.8.207]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9BC2640CD14B; Tue, 30 Jan 2024 18:33:56 +0000 (UTC) From: Waiman Long To: Tejun Heo , Lai Jiangshan Cc: linux-kernel@vger.kernel.org, Juri Lelli , Cestmir Kalina , Alex Gladkov , Waiman Long Subject: [RFC PATCH 2/3] workqueue: Break out __queue_work_rcu_locked() from __queue_work() Date: Tue, 30 Jan 2024 13:33:35 -0500 Message-Id: <20240130183336.511948-3-longman@redhat.com> In-Reply-To: <20240130183336.511948-1-longman@redhat.com> References: <20240130183336.511948-1-longman@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789541648680330202 X-GMAIL-MSGID: 1789541648680330202 Break out the core __queue_work() code into a __queue_work_rcu_locked helper to be used by later patches. There is no functional change. Signed-off-by: Waiman Long --- kernel/workqueue.c | 46 ++++++++++++++++++++++++++-------------------- 1 file changed, 26 insertions(+), 20 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 350179382667..98c741eb43af 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -1702,32 +1702,14 @@ static int wq_select_unbound_cpu(int cpu) return new_cpu; } -static void __queue_work(int cpu, struct workqueue_struct *wq, - struct work_struct *work) +static void __queue_work_rcu_locked(int cpu, struct workqueue_struct *wq, + struct work_struct *work) { struct pool_workqueue *pwq; struct worker_pool *last_pool, *pool; unsigned int work_flags; unsigned int req_cpu = cpu; - /* - * While a work item is PENDING && off queue, a task trying to - * steal the PENDING will busy-loop waiting for it to either get - * queued or lose PENDING. Grabbing PENDING and queueing should - * happen with IRQ disabled. - */ - lockdep_assert_irqs_disabled(); - - - /* - * For a draining wq, only works from the same workqueue are - * allowed. The __WQ_DESTROYING helps to spot the issue that - * queues a new work item to a wq after destroy_workqueue(wq). - */ - if (unlikely(wq->flags & (__WQ_DESTROYING | __WQ_DRAINING) && - WARN_ON_ONCE(!is_chained_work(wq)))) - return; - rcu_read_lock(); retry: /* pwq which will be used unless @work is executing elsewhere */ if (req_cpu == WORK_CPU_UNBOUND) { @@ -1808,6 +1790,30 @@ static void __queue_work(int cpu, struct workqueue_struct *wq, out: raw_spin_unlock(&pool->lock); +} + +static void __queue_work(int cpu, struct workqueue_struct *wq, + struct work_struct *work) +{ + /* + * While a work item is PENDING && off queue, a task trying to + * steal the PENDING will busy-loop waiting for it to either get + * queued or lose PENDING. Grabbing PENDING and queueing should + * happen with IRQ disabled. + */ + lockdep_assert_irqs_disabled(); + + /* + * For a draining wq, only works from the same workqueue are + * allowed. The __WQ_DESTROYING helps to spot the issue that + * queues a new work item to a wq after destroy_workqueue(wq). + */ + if (unlikely(wq->flags & (__WQ_DESTROYING | __WQ_DRAINING) && + WARN_ON_ONCE(!is_chained_work(wq)))) + return; + + rcu_read_lock(); + __queue_work_rcu_locked(cpu, wq, work); rcu_read_unlock(); }