From patchwork Fri Feb 16 18:04:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 202294 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:c619:b0:108:e6aa:91d0 with SMTP id hn25csp691618dyb; Fri, 16 Feb 2024 10:09:50 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWnRI4mMrPq0adoGNyTYYze3HYmN+D+LbD1bS1HJ08DcI8i07Ttb8TzTAyBPuPtlFHvUOI94bfPHWMoChvr/9BiV7eHZA== X-Google-Smtp-Source: AGHT+IGUZ65s66kXDYqcT+jMa6nha0eeWX0xkFFDJMNXGIUdHBf2+CPQJhNer7UJ+uY5dpD+5sVT X-Received: by 2002:a67:e3cd:0:b0:46e:d77f:65cc with SMTP id k13-20020a67e3cd000000b0046ed77f65ccmr6134625vsm.28.1708106990009; Fri, 16 Feb 2024 10:09:50 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708106989; cv=pass; d=google.com; s=arc-20160816; b=k1TEv43l+ad5OzyscQlkG6gqod2A8qaPoM+jgc6nQCPOKviTQOkFIQGYmR+0QSniPX 6G2QvEx4HmCoYGcQA2iXIm4/WprrZHi2Fv5YLAxux0nivP6pHQ+f3IoQK8huNFD8daRv ZJb/9+3CA2QN5pbBIExLyj22lHu+6XUSYiAhTFCs40z9ubK60CNSb+b3PbdJTQ5/KTYT wn30tshC5qsZ42RFiNmAalJzbWTRgz4VTmG/YQl++8/8xP0L6GqhHjQ9XaB2YQ9X5aSH hyxhTcJQZHKXm4YmpN1S2G9+jckuGrk8nri0WhrYUMYeQJUUd5DXTSpqbMx3oocfzckh tG0A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:sender:dkim-signature; bh=4xh5X4jP++uuYjnXCzqnuN7YFRsni8pg3IhpUUiVDrs=; fh=3xcE0Yd7si6KXTUg1D2wjIX/6P5lVhhHypU0OOCNbsM=; b=B1MvzIa2L+XWe1VQnzmc3itStdyI/HzNoYx4lAyh0uHOEwTEcAARO2QCsMPdVxrseH 0eQsqJqDOff41ObhI54Jj6V6nLojFWEHwE0QDpp5nOugtUPTBRNSBInKRqtFZjj0sZWT YVc7hZZcSx3PYFEp5VAAkLDtKnU80QuMKKlSPCWqpP8DYM4p7OUIdN7ozos0iJKbb8/+ /OuTuKrt7LIDLkZ2XbDH/6JqCce+Xbyj3t131nYxn03b1pVcs+glhAnp2bCe+rcJ9CJq nAmv3aOLK3T3W31ZCMiglx8u9fZ+ouxUf/z8Nf58sIr7RPZQsDyxPqFB33MoqC1C3l9h S8JA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b="HxQg/DY1"; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69133-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69133-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id b7-20020a0561023a0700b0046d3399b832si121254vsu.305.2024.02.16.10.09.49 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:09:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-69133-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b="HxQg/DY1"; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69133-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69133-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 7F4091C21E86 for ; Fri, 16 Feb 2024 18:09:49 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 91B871468E9; Fri, 16 Feb 2024 18:06:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="HxQg/DY1" Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D01CC145FE7 for ; Fri, 16 Feb 2024 18:06:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106803; cv=none; b=gRtk5Hw4EiGKFBGje4Wb6SD5ipWy0LuKyAQ3bybpWUXy+I3NKiBapk8EW+w5MWLnJufN0uu8pi2dUavvdurrfi3PssmMF/EtHbSjdXK/082F0sSVoxxKgJoH1JdPX6TvbmlAS4UrH0P1oVfHfiyy+bhUC+3io8T++pSH8BjGoaw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106803; c=relaxed/simple; bh=GeKVjnbdns5bK22n91xVMXpiQjcDzrlmZrUwQuwmJNg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XLfaZ14KT9vainHJ73McKL8UTRy68bxtYdUMM13/+rnu7gjdxO8bsSUDPM7VaSpaIDPOV3VZc2ByPrNjM/Xw+VAM5nhh7ChFfjxRlsiW8+CJP5vIQlden4kSYWXV9Vb6JNf2Upk5FO6ze4daG+Vl8oiSXqqSMjH1wmJdGmYLLYk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=HxQg/DY1; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-1d934c8f8f7so24377895ad.2 for ; Fri, 16 Feb 2024 10:06:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708106801; x=1708711601; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=4xh5X4jP++uuYjnXCzqnuN7YFRsni8pg3IhpUUiVDrs=; b=HxQg/DY19kgVTQYYQeSe163DCTvpVZfhgbSGL/0kk9ctmF1v6fkVYnDdLfsaU3+b5o +k3vsAg7RiTb6QX1rHLwxajUgMVCW5B09mgV7e5y+7qPOKZJXeTLQsDaBzGLb+deHZdu 9s+LZayjhU6Roy+Q1/ZdN8jEKQzHHB4B/mncFzMTu9w5N5gMsO01AMm7bdCyUU2Qj+o5 rEkUzYbH8KSI3hlE8DCl6k9ZkklGAU3KbuNMs7Vxzj8IDbaLCH0QFVO1ZSuhcuzCdmCa 81TuMerbDPv9tVo1iRdkf/EToAzoVOw2LuzmYTQO0nOCmTJ8iY3qmS33d7CDEWp8IzOE un6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708106801; x=1708711601; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=4xh5X4jP++uuYjnXCzqnuN7YFRsni8pg3IhpUUiVDrs=; b=WSrd3vj7wsPPiF2fS/Vqy9dKBXR97W4l/awTOZQgN4rzu5O6DU6a4gG/OikpA3VoN+ myNXovh0oR4+s8rmLbQFhk+aWYNy9AOrWaitLU5625OvK5NvdjNwNLvylu/RuBcD9TjU 6BbZVbJiGoPudOppkrJw27j4NFqdZ0aRneEc5ClOFGwwJtmDsMJIxKPTPQL5tJ0c+gK9 dGxYWRnUrsqffdnGQLvkHYgDttJlyDoYqGOYmyKv6xwxbpDTE+ngF0FbMSxzmdutxIR6 1TvGzsNtboGKDXtNu13ivXp7cwb1w63WKH8QpAresbYAhOwSCQ1J/Jgc8PVRw3772tVM ynTw== X-Forwarded-Encrypted: i=1; AJvYcCUZp4PESHbAmnCBIp5/gCiK+4dzWbnuwxSxNadOlaexb+lN8dzzTfOJNZlg4XMQpaI9/yNzdfNRLJ10cWCu+CCVojGDi/W0vw8wXw6v X-Gm-Message-State: AOJu0Yzt5a6iK1FddYS/pPZttM1Ay3hBT8j3nCraCOgmKTU0G1qtsujz nb5/qUOC5GP3p/Px2JdGLTijJR/R9n73VjphnFinjcxPy8PhlcwH X-Received: by 2002:a17:903:1c2:b0:1db:a2bc:22bb with SMTP id e2-20020a17090301c200b001dba2bc22bbmr3223151plh.41.1708106800941; Fri, 16 Feb 2024 10:06:40 -0800 (PST) Received: from localhost (dhcp-141-239-158-86.hawaiiantel.net. [141.239.158.86]) by smtp.gmail.com with ESMTPSA id e17-20020a17090301d100b001d720a7a616sm146700plh.165.2024.02.16.10.06.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:06:40 -0800 (PST) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, allen.lkml@gmail.com, kernel-team@meta.com, Tejun Heo Subject: [PATCH 15/17] workqueue: Update how start_flush_work() is called Date: Fri, 16 Feb 2024 08:04:56 -1000 Message-ID: <20240216180559.208276-16-tj@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240216180559.208276-1-tj@kernel.org> References: <20240216180559.208276-1-tj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791079995125190634 X-GMAIL-MSGID: 1791079995125190634 In prepartion of in-BH canceling of BH work items, update start_flush_work() so that: - rcu_read_lock()'ing is moved to the caller. - Instead of true or false, it now returns the worker_pool associated with the work item if the work item needs to be waited for. NULL if waiting is not needed. - Add a WARN if it encounters a queued work item when @from_cancel. This shouldn't happen. No behavior changes are intended. Signed-off-by: Tejun Heo --- kernel/workqueue.c | 39 ++++++++++++++++++++++----------------- 1 file changed, 22 insertions(+), 17 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 6a2abc81ae2b..f6ea25628701 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -3995,8 +3995,9 @@ void drain_workqueue(struct workqueue_struct *wq) } EXPORT_SYMBOL_GPL(drain_workqueue); -static bool start_flush_work(struct work_struct *work, struct wq_barrier *barr, - bool from_cancel) +static struct worker_pool *start_flush_work(struct work_struct *work, + struct wq_barrier *barr, + bool from_cancel) { struct worker *worker = NULL; struct worker_pool *pool; @@ -4005,12 +4006,9 @@ static bool start_flush_work(struct work_struct *work, struct wq_barrier *barr, might_sleep(); - rcu_read_lock(); pool = get_work_pool(work); - if (!pool) { - rcu_read_unlock(); - return false; - } + if (!pool) + return NULL; raw_spin_lock_irq(&pool->lock); /* see the comment in try_to_grab_pending() with the same code */ @@ -4018,6 +4016,12 @@ static bool start_flush_work(struct work_struct *work, struct wq_barrier *barr, if (pwq) { if (unlikely(pwq->pool != pool)) goto already_gone; + /* + * Cancel path should already have removed @work from worklist + * in try_to_grab_pending(). Control should get here iff we need + * to wait for the current execution to finish. + */ + WARN_ON_ONCE(from_cancel); } else { worker = find_worker_executing_work(pool, work); if (!worker) @@ -4045,17 +4049,16 @@ static bool start_flush_work(struct work_struct *work, struct wq_barrier *barr, if (!from_cancel && (wq->saved_max_active == 1 || wq->rescuer)) touch_wq_lockdep_map(wq); - rcu_read_unlock(); - return true; + return pool; already_gone: raw_spin_unlock_irq(&pool->lock); - rcu_read_unlock(); - return false; + return NULL; } static bool __flush_work(struct work_struct *work, bool from_cancel) { struct wq_barrier barr; + struct worker_pool *pool; if (WARN_ON(!wq_online)) return false; @@ -4063,13 +4066,15 @@ static bool __flush_work(struct work_struct *work, bool from_cancel) if (WARN_ON(!work->func)) return false; - if (start_flush_work(work, &barr, from_cancel)) { - wait_for_completion(&barr.done); - destroy_work_on_stack(&barr.work); - return true; - } else { + rcu_read_lock(); + pool = start_flush_work(work, &barr, from_cancel); + rcu_read_unlock(); + if (!pool) return false; - } + + wait_for_completion(&barr.done); + destroy_work_on_stack(&barr.work); + return true; } /**