From patchwork Fri Feb 16 18:04:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 202281 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:c619:b0:108:e6aa:91d0 with SMTP id hn25csp689079dyb; Fri, 16 Feb 2024 10:06:46 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXRp1dzqSzSHfcO64XzOIbo+Np+KXuUMc2oFa8fFmapSw4oobc8JVu9SNwsuDA9QTcbAqf4L0sPCYcLB/89wp8y95M+Ag== X-Google-Smtp-Source: AGHT+IFXcEyYYmoEboimgqLcwyDtjxh3Ox3pb+yH5FBTd6r6Mth/vU97W5X2+x0DNoabHehqQ39V X-Received: by 2002:a05:6214:5403:b0:68f:1ac5:9ccb with SMTP id ql3-20020a056214540300b0068f1ac59ccbmr9414136qvb.10.1708106806679; Fri, 16 Feb 2024 10:06:46 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708106806; cv=pass; d=google.com; s=arc-20160816; b=vJdXWNtrFsx99JnPvTILN4Ob2UnGvzBUqCYWAUqGwDKUJbV0gGx+13kNycRHMwqhsZ ZtwD1CJOIKyDj/kx1odhPHRqdU9VRagydSMaUJowDDObSh0SQY7TBmGYfhx0Jq31uPi4 adA/4/4b2qyk6oUfwzoIpDHuET0piSO/H2CGM/FXB9noVxErwIQbZxRAaM1EHukOjO03 /ta2Zy3exNy978mOPJpQ0TZ6DXR3rZO0tWqXFztGSQXER+jUDiKYxxRc4GIY4Iuw16QA TeCLNG7clxB5s8MBiRGJkJVGYcv1HfwNuD3oyD+A9vbafbIwMLvMNnJmQ1+38CWnIvPe I/Hg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:sender:dkim-signature; bh=XQL03BXEohlBiV8WF4+w4OjRLuExQ3dvTWReO6nS+hc=; fh=Yyvv7A1G+IA3IZKItKyxQAUxUKItGpLb/phXYyiMx6I=; b=eAYMkzxrCaDjIPWi0UwdQ/vdb5AY+nrVM+AwlIvNCWcvZAqkjCL58TpNiFiASfX4w0 08F5VyKPvjjW/5bw5HugsPrDdYTtof2vXNDU1buSObWfz8XfGWN6AC4LMf0ZovMNEIws 0jlTzuX0mxHYyOqnDov9HpbsO9e3zx/YyT76va2EG0y5sW2i/XGTaCg2R1sqmMPzcCQk FfPFQO/VcFnl5E6a8sy/VM+DzZu9eNt96WI8M79Lmc+u/JUTHQXiICFFvy81ljM/zKDc nP54/xAKpgK3DDF3aFxePhsFvxv7mPR5yEmA3RaSXauUH0+yA7vpq6GhZMxZwpSIvqrC TWsA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b="km/nJS2u"; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69119-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69119-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id t26-20020ac5c97a000000b004c0293f50d1si197959vkm.249.2024.02.16.10.06.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:06:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-69119-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b="km/nJS2u"; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69119-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69119-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 3FE821C20B6C for ; Fri, 16 Feb 2024 18:06:46 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7CE231353E5; Fri, 16 Feb 2024 18:06:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="km/nJS2u" Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1B040131E22 for ; Fri, 16 Feb 2024 18:06:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106777; cv=none; b=uXT2cDuIouKtJKNJAUowZYr8jDv1G7Lr8XMwG+yHo15e+vwQrimaCER6dg7d9nef9ZGz47e4s1igEEQerjjfwcLdCCtJu8ieTNTfsniMo+85Z3VHxt/s6Ni8rxcHBUQ7CRUQE8lgPKziSA3HVtBAhE8r2F2dHVDGscvLo5f4ZO0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106777; c=relaxed/simple; bh=rA+Bhv3/TvEC8fa8VDcKiAHnAwvTr3i2sZPLuuN/ifQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PrrO767H0CFZQAG/Z1SnzKSa9JxfVMMMQep0cJH2prpzqEYobQBSWDb3tZx1J3LarZ+Jjkk9w1UOfZYy/lb599RKVf/eXh12KyyN8AVfPX0V7NXTRhS1lqJt4j7gOMqPOEdDBc2yvn/13b5fk06LwnQaAjb51I10zb/SYsgAbHM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=km/nJS2u; arc=none smtp.client-ip=209.85.215.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pg1-f172.google.com with SMTP id 41be03b00d2f7-5c66b093b86so2527049a12.0 for ; Fri, 16 Feb 2024 10:06:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708106775; x=1708711575; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=XQL03BXEohlBiV8WF4+w4OjRLuExQ3dvTWReO6nS+hc=; b=km/nJS2uKTVSPSmB3uSgBRZoILEZOGWc64KaHRJMn8xbSbTtpWqhwzrxQUwFLqK+O1 H3Yh3EUNyi7CndeCb3tRgcPLM88dim21MqdAyKb4/qpaBOsYYOe6LWfNsoYl3I0VKZnC lxYx9oAhRbv8v4Nut7C8fmOWN8+wtQVGctZxgKl+e7J7HJ8Ta1XheFET4U+g+q5gdTOn gjHJRSe7/tO76eKTpdppk5+Cmiegqw/v86BUoD4lCfhkMfV/ywIRpEGqJ6/0dFhizDGx cCsG9+13LXm6IDH4u6z+1Z3O6jEY3zaqftzbR2WphTSwAKmT0+8yiNKAIzRMC3KvXOGU ODDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708106775; x=1708711575; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=XQL03BXEohlBiV8WF4+w4OjRLuExQ3dvTWReO6nS+hc=; b=v27WeMic0e/UwgYKUSiGL14486F0kHYXeJSgbHRidPoEhoVPtEOOVAcPaWa9ktwDt1 Pd/16+9MunQxBFcaFZAIRw3IJS4cJPUBwFIM5aym+hWdchHZnQTxoKxPM8BXu55nzz2e Enh0I7olhDd7gJM1TV9bi4l9RFlygu2UHwMwdEsozekQek1YSwzu3F6p3O2mpXv9Afhd CRQBD8iXIJF9C85ctJEk5doc9yZYZYR3REe0LadJoN2UXODiDy6NdswLvqAhY7gV2d68 GB/M9CijrF6t94cQZuIxItAGdXMcx76GAozDBypBwqqzx94yYNZJUVD88+cFtX83dJ8u qAFw== X-Forwarded-Encrypted: i=1; AJvYcCVjdyd4zlJdWQOeTROeHYbqz7yvd5ATQ+DMaSY3hH7hJOvhgHADwbTVYKDfQnpts10uc7DngU4llYP18Bh7ETZnvT5Nnqj4QjVI0+5j X-Gm-Message-State: AOJu0YxZpe1wsIE8ExwmDZZgJd/OeJp4YlQiSd4JvLxal2qHzv7VQXhH s8uTPztqXWlXGGaItX8m0oCs9xEoNJMhy5Trc6ofaF+hroM3nklV X-Received: by 2002:a17:90b:381:b0:299:336e:512d with SMTP id ga1-20020a17090b038100b00299336e512dmr3741056pjb.11.1708106775230; Fri, 16 Feb 2024 10:06:15 -0800 (PST) Received: from localhost (dhcp-141-239-158-86.hawaiiantel.net. [141.239.158.86]) by smtp.gmail.com with ESMTPSA id cm18-20020a17090afa1200b00299268defb9sm297679pjb.41.2024.02.16.10.06.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:06:14 -0800 (PST) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, allen.lkml@gmail.com, kernel-team@meta.com, Tejun Heo Subject: [PATCH 01/17] workqueue: Cosmetic changes Date: Fri, 16 Feb 2024 08:04:42 -1000 Message-ID: <20240216180559.208276-2-tj@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240216180559.208276-1-tj@kernel.org> References: <20240216180559.208276-1-tj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791079802896911276 X-GMAIL-MSGID: 1791079802896911276 Reorder some global declarations and adjust comments and whitespaces for clarity and consistency. No functional changes. Signed-off-by: Tejun Heo --- kernel/workqueue.c | 30 ++++++++++++++---------------- 1 file changed, 14 insertions(+), 16 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 6ae441e13804..b280caf81fb2 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -376,8 +376,6 @@ struct workqueue_struct { struct wq_node_nr_active *node_nr_active[]; /* I: per-node nr_active */ }; -static struct kmem_cache *pwq_cache; - /* * Each pod type describes how CPUs should be grouped for unbound workqueues. * See the comment above workqueue_attrs->affn_scope. @@ -389,20 +387,15 @@ struct wq_pod_type { int *cpu_pod; /* cpu -> pod */ }; -static struct wq_pod_type wq_pod_types[WQ_AFFN_NR_TYPES]; -static enum wq_affn_scope wq_affn_dfl = WQ_AFFN_CACHE; - static const char *wq_affn_names[WQ_AFFN_NR_TYPES] = { - [WQ_AFFN_DFL] = "default", - [WQ_AFFN_CPU] = "cpu", - [WQ_AFFN_SMT] = "smt", - [WQ_AFFN_CACHE] = "cache", - [WQ_AFFN_NUMA] = "numa", - [WQ_AFFN_SYSTEM] = "system", + [WQ_AFFN_DFL] = "default", + [WQ_AFFN_CPU] = "cpu", + [WQ_AFFN_SMT] = "smt", + [WQ_AFFN_CACHE] = "cache", + [WQ_AFFN_NUMA] = "numa", + [WQ_AFFN_SYSTEM] = "system", }; -static bool wq_topo_initialized __read_mostly = false; - /* * Per-cpu work items which run for longer than the following threshold are * automatically considered CPU intensive and excluded from concurrency @@ -418,6 +411,12 @@ static bool wq_power_efficient = IS_ENABLED(CONFIG_WQ_POWER_EFFICIENT_DEFAULT); module_param_named(power_efficient, wq_power_efficient, bool, 0444); static bool wq_online; /* can kworkers be created yet? */ +static bool wq_topo_initialized __read_mostly = false; + +static struct kmem_cache *pwq_cache; + +static struct wq_pod_type wq_pod_types[WQ_AFFN_NR_TYPES]; +static enum wq_affn_scope wq_affn_dfl = WQ_AFFN_CACHE; /* buf for wq_update_unbound_pod_attrs(), protected by CPU hotplug exclusion */ static struct workqueue_attrs *wq_update_pod_attrs_buf; @@ -2231,7 +2230,6 @@ static void __queue_work(int cpu, struct workqueue_struct *wq, */ lockdep_assert_irqs_disabled(); - /* * For a draining wq, only works from the same workqueue are * allowed. The __WQ_DESTROYING helps to spot the issue that @@ -4121,8 +4119,8 @@ static bool __cancel_work_timer(struct work_struct *work, bool is_dwork) local_irq_restore(flags); /* - * This allows canceling during early boot. We know that @work - * isn't executing. + * Skip __flush_work() during early boot when we know that @work isn't + * executing. This allows canceling during early boot. */ if (wq_online) __flush_work(work, true); From patchwork Fri Feb 16 18:04:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 202302 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:c619:b0:108:e6aa:91d0 with SMTP id hn25csp702701dyb; Fri, 16 Feb 2024 10:25:49 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXkxYdoZQccZaOT/83qhAedakDsDUlkiEI1hVr2+kZx+yPp9aNj4vVGMHCKo3peboGoOODcXYTqP4j/wyQDMqg/DJ47lw== X-Google-Smtp-Source: AGHT+IGjpPqluJKbqpV+zGoOspQomt1iDfj+YVtbuDSh5MTW3QXR2NP4qYQiXDHq8wnhJRaU+kIS X-Received: by 2002:a05:6a21:150b:b0:1a0:6856:d19b with SMTP id nq11-20020a056a21150b00b001a06856d19bmr7234386pzb.6.1708107948879; Fri, 16 Feb 2024 10:25:48 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708107948; cv=pass; d=google.com; s=arc-20160816; b=hQlaNMHbhotUWMsQlBlh/YpgB8e8TpZ5XenN0qDP/SM34e8eYx6ZH0OdWr/6gEMJqo qb99u9d4LWrwWpq5jnJ5BeKiHzc8g0mDtaOtrYAGeFJPJvzl4ysxajAHT+z9IE9dvZTh h0/aTyexntNJi5G5+Dk+adkPS/NRLgbcwbAluawYsea2WyuHa1ljvL1Ti24bnsru3mJ+ pTPqUXYMcjIKJxwU2dutPAtLJLQabKNgZ9Y0mf+ynzBdEptniB3dPNskeNRJec7Li2Fe 2xPrWzcBxL8SsMkdIYqC3fn39I6dNQ0HAgV9QZDDCr6UoM33/SCpAa+5nqrOQOl/23fL HffA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:sender:dkim-signature; bh=pIltjVUaiFwUeua93wdSXtPQLjPPFRChCUpwRSlDMH8=; fh=YfyOijUT4aWTsMaJNRcspdtRsIP3TsjAX2n/ORofxNA=; b=YYYSgYpd/fCalFR1AK1/Srfydobnl6tp17qvGgc4/h56Ei8pbKxDZ2lm9KEfxszD4j 7DY2Icxn/fazlRlZmt5kdKioNeYRoBOaoPIqdcz3Mt9WYLJvt0hj6D6/+LD389y8N3uP zO2eFpI8OhuwprzQEEo+/nA4HktQz1CfReAHyfCl7+nhUdcJmHuEVQUYFbwZKlI1pmBa FMujteiJ5WSYO11+WzALnqCLZ2LWmby2Xws02q2jcOjpIAYJgq70cRrPDP8Lj9UKaYZ1 pBTIUMgVO80mW1JurK/40Bo2w+Syd4dY5xYdcYZfHA95yFzqRgkubxe5FIHwjzb93PkW ae4g==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=HawJlmN2; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69120-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69120-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id c133-20020a63358b000000b005d8c19ca98dsi258485pga.137.2024.02.16.10.25.48 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:25:48 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-69120-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=HawJlmN2; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69120-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69120-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 07640B25639 for ; Fri, 16 Feb 2024 18:06:58 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D28ED135405; Fri, 16 Feb 2024 18:06:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="HawJlmN2" Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE99C1350CD for ; Fri, 16 Feb 2024 18:06:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106779; cv=none; b=i9Wc8zoKEYYDSVK2xZPf1hBqaCF582QO8dyGufKzG84pb6lu//R6NFSqsDbhNr8gWk6JLlGqMqWJKXaU/xz/JKQX9DploDPnpUpcewcCEF4whsZsQqiFcc98kK25BdwrIbCOoR7+V5BAR2sNraoiLENvHEShgn4+SX/eTApbIDY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106779; c=relaxed/simple; bh=qzo6BYrcleW4zRxM5K55RwFrekOKOLB8qjWI1ALQZYk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=m30wm7NmeA/q9CYLIzQvlp0O66IBB2WVc69hvO9dohP1xhhJBlJr2QxILT4rfUgJCOMHyoMKIonv00HBQyPNytM6pBzqpnBTzYdNbUWjiWqnloXwV+XEdl4fJq9a6yIa4jVsH61Fys0a/w2r7PYWgwBxdSTDct1h3Ne867ts0Xg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=HawJlmN2; arc=none smtp.client-ip=209.85.210.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-6e0f5934813so1886683b3a.2 for ; Fri, 16 Feb 2024 10:06:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708106777; x=1708711577; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=pIltjVUaiFwUeua93wdSXtPQLjPPFRChCUpwRSlDMH8=; b=HawJlmN2IwfBdNnChc+iiHUToYKabT/gsQ2w0BC1igkWyNAPNpsINkqEWpSSKiwYJZ pETtzMpZCe3XxkhJstCHMKW/K+4Yu0bmopax9sAPhQ3BuH+sFhY/6sKA6omH4voWVOQE K75UKnNRMuU2OKX33dpIQEt37nakUOPFySpEVPHYvzN5FuHoS47fKpicjuGcHY+r9MBk F9Ajk9Nplhc4/6MgWYl/v31lB9usbRRrojVpBZAx3f/38MlbKEmTtp/jSVOt2XqnrVev kE0lE5UNXhQ+0ijX+rUQnYau5Bz6cuhWepfUEr7xzZadEH2D1FqdIkmrcxUBhwrk1gZn o8Gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708106777; x=1708711577; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=pIltjVUaiFwUeua93wdSXtPQLjPPFRChCUpwRSlDMH8=; b=fhNvYg1XrGkLTf9MAjuuQfjRdNdaA/gFGYkBhG8HqvAYDo1d/xAqatdL8DSQJyW+x9 fuUJiIqhWFnVTpjvuVWrHCwA1sr6qsRnYALBya9pPHwprjh6Xtiju4eYlULC+OC5RB8G ZkisxOSYqI+TPwd3xgLRAXDK1pbkdyi+jErjByzrMZ/VKj8DbFyVKsoL8xIOJlQ+zlRS 6Qp30EQH+f8Tmgq+ubidZLFb0kb/sp1YHPs4/z8SgefiOl2Xx7P4wE7FevVS9HZRFo2C DqaJZ0b1azkGDeMX6HUzkvqhLfsjLa13VOFQuno7hOgYAvPB4jfLB23x60vOBQuUCz3d TIFQ== X-Forwarded-Encrypted: i=1; AJvYcCUHL7FT/hygCv97UQf/ShnnmS0SpNUsXLBbTOFTj0ZaeiSCkWc857MrVSBR+LYTGz1gRmgOFWva4xMLQFEZFdmRWDl4tlJ9+B8QuVeT X-Gm-Message-State: AOJu0Yw4O6ALJDG5IxksXJAWQX2oePa/HDXFAdrNAXEKaqUcsU2mdLzk z5Kz/cmm+PY+I2GcU/6NlWMkk18I9RyYwHNaxGupfrRDxT2CyBz/ X-Received: by 2002:a05:6a20:d046:b0:19c:ac7f:c3e4 with SMTP id hv6-20020a056a20d04600b0019cac7fc3e4mr5432849pzb.42.1708106776937; Fri, 16 Feb 2024 10:06:16 -0800 (PST) Received: from localhost (dhcp-141-239-158-86.hawaiiantel.net. [141.239.158.86]) by smtp.gmail.com with ESMTPSA id x40-20020a056a000be800b006ddcf5d5b0bsm235158pfu.153.2024.02.16.10.06.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:06:16 -0800 (PST) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, allen.lkml@gmail.com, kernel-team@meta.com, Tejun Heo Subject: [PATCH 02/17] workqueue: Use rcu_read_lock_any_held() instead of rcu_read_lock_held() Date: Fri, 16 Feb 2024 08:04:43 -1000 Message-ID: <20240216180559.208276-3-tj@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240216180559.208276-1-tj@kernel.org> References: <20240216180559.208276-1-tj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791081000264178560 X-GMAIL-MSGID: 1791081000264178560 The different flavors of RCU read critical sections have been unified. Let's update the locking assertion macros accordingly to avoid requiring unnecessary explicit rcu_read_[un]lock() calls. Signed-off-by: Tejun Heo --- kernel/workqueue.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index b280caf81fb2..87750e70b638 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -515,12 +515,12 @@ static void show_one_worker_pool(struct worker_pool *pool); #include #define assert_rcu_or_pool_mutex() \ - RCU_LOCKDEP_WARN(!rcu_read_lock_held() && \ + RCU_LOCKDEP_WARN(!rcu_read_lock_any_held() && \ !lockdep_is_held(&wq_pool_mutex), \ "RCU or wq_pool_mutex should be held") #define assert_rcu_or_wq_mutex_or_pool_mutex(wq) \ - RCU_LOCKDEP_WARN(!rcu_read_lock_held() && \ + RCU_LOCKDEP_WARN(!rcu_read_lock_any_held() && \ !lockdep_is_held(&wq->mutex) && \ !lockdep_is_held(&wq_pool_mutex), \ "RCU, wq->mutex or wq_pool_mutex should be held") From patchwork Fri Feb 16 18:04:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 202282 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:c619:b0:108:e6aa:91d0 with SMTP id hn25csp689351dyb; Fri, 16 Feb 2024 10:07:07 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUQ57LgJfMI2Fi1Yj9gCeiBl96Gad3fdmga0u1HIOV1zJLmnkRCNAI1VKSupTz8JUile5RkfvDXblwOGmvV9rejFlLRjQ== X-Google-Smtp-Source: AGHT+IGG9SUyBCUCqJiROL34kPPt5LQ6D+weIS//5XpIqY0rujJ2We3uThPMyWovKJJRFA5+Uiuh X-Received: by 2002:a05:6e02:cc4:b0:363:b695:5d5a with SMTP id c4-20020a056e020cc400b00363b6955d5amr4921191ilj.18.1708106827724; Fri, 16 Feb 2024 10:07:07 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708106827; cv=pass; d=google.com; s=arc-20160816; b=b1GcbGTYf3O/Cj64X/dvrniNIQP7wMPXff4DHMAbFuvqKArrYInykzxhhFDNwwR2N4 6uds0JtQ6BefhMVPQgCoyz9y18OTxGbctkfc/lVAdMiu0pjQyujPAiMhEZFDTWraUox4 aXDkzwOMpMfbg6wpXBW2lWXX68iFpN8d7+j5tExA+uKbp33ZgUHnSKKeVEVn6jNG833G makw96ISroU8LQuJ6QlZHcYC0pACXmP64VULwGNstjl+yjIlHl7OeGkCu+3a4AVmVm79 FID9HPsU4ywhHZwbCas76O9iPlW4ty3HlRV3iBhFyEo4dIlKjBDzoeNfbAeLLvoDlSQu osnQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:sender:dkim-signature; bh=RtvoN2Te8lsuMrpULW6AGCbHjbIm4X87ZQHxVBkQdBg=; fh=e0Zcnf0zLSn0Tv9ivwir6OEkJPfljHn69SOQu70nsl8=; b=wt2FFFINwohWO0o4n/+rTGDKS0AGdykflMduurGkUK0pNVkfvx2z1XDSsKQOeij5XV iVwGzheBvmYv0UBA90tQ4p0Qb4d4S1fv0hjTCtzB4MhjsLANvyGfwXd0Fmr9vKXfUSMU FojcMYXBMGAe7F1hKF5qJ29L003o2pGzIE+kRv6pmCW882cV7AznD50RCcU9jmQ0dc1J ZkljrQQt7mOUzxbWe28Wyx+7VCDeP+wHQXfqQBGXM7gigX04XXfX+5iDS3hHc/utud0I pkwo2byyX+Yc+DSFopg6/N0PxHBb1zgwMGpvgWqm6ircVBpOnYN7q34QuwcRLVZR4fOR 2WiQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=EhM6jn3u; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69121-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69121-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id n29-20020a63721d000000b005dc927ee2fasi217476pgc.710.2024.02.16.10.07.07 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:07:07 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-69121-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=EhM6jn3u; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69121-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69121-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 74890281E24 for ; Fri, 16 Feb 2024 18:07:07 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BB783135A4B; Fri, 16 Feb 2024 18:06:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="EhM6jn3u" Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 98A341350F6 for ; Fri, 16 Feb 2024 18:06:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106781; cv=none; b=Wn0qTxawFULeQrq7xlzj/HogsEgemBgv6ZqyXb/BioAJStfA4uuqs4g/GKmpDWtkRadI7zwXWm+bsbaiiTX9BtH6Jpwd3Bl4fPaV4luW/Y164dbq/SKf+HMTrl5/1FoXpTJzyB3H0Q/A0dZqC5WvX7wG2zl4gEp3hNBO4qOo4Ds= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106781; c=relaxed/simple; bh=dwcDbWOPxRgiZhLXY8mPYkaEIPRTLSYlPdZIIJdc+Lw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZVUZBYKKGSBlspMrK8TdqLbhkXNqr1hfOrG4tCxpvzRDOE5qfUS/3ApjYWAQIbHjXs7O5sfz9A7/oGN+TGOR9ICIx/+9DuhPSSAylQd3XN2Y/yY0OzEObFvUaW6HcarxN1/5mzQ1QrBv5XnFggH9tYRRlufJq8mSmO5fGakqM7A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=EhM6jn3u; arc=none smtp.client-ip=209.85.214.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-1d7881b1843so20778315ad.3 for ; Fri, 16 Feb 2024 10:06:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708106779; x=1708711579; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=RtvoN2Te8lsuMrpULW6AGCbHjbIm4X87ZQHxVBkQdBg=; b=EhM6jn3uGgGZb73vI9RkmtFYIcS+R/vLQe/gdAGCWzbu4a/iP76puwbvIGS2PitS6g Ukh1HuXnRl1O1JdYnKwXA6OA91SvJyK+t1YiykCvf6IwGSOyYbtiBHH/gWQm6CzxTOHL LAgexkt0+SN/hi5+b8fnW4ldJqDhKwbeCcq9zLx9Z1vO2M59S57R5LnUWjNFw/HtkSZi iuPin6jmDuNUoE7Cx4s5HdmBedPngMB1vQH1xenws1dDZbeqxcigOP+SiKL2h/1pJjow blwP4iY6AoWQKofqUdjZU6X4AEjmwFfaVuFltaLas2POIFxmhc0R6bMra47bNW+d6fws tQxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708106779; x=1708711579; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=RtvoN2Te8lsuMrpULW6AGCbHjbIm4X87ZQHxVBkQdBg=; b=frdW6zAniv5rNA6mnblt5O8tUHPPiJxSu8TyK7s/scYg8rg3r2rqJ6c6adKhtijgyn Vdb4/rm5jeHC1p06VItY0kFAH67orD7yKyTVzQeTb1rRMrIq4RysfJnZ4UtY+yzdmn0s vSahvRAW2/9HP0il2wYBVBN5CKg+AzepZQwkcmSiTEwhFRm03iGYYFHaSmwlgnzUTCnH b7sUadnIQDir+xVOdjJR4qi9g2zdqs3vmpYddNIkq1zICMz/TEFB5meKj1aJMi65zyDn /xrhLiNdKGHedgddVSrbkR7vPCrXjOe6PlMES8mLIRUKtgexQhKVe7hP7fBaNZHkFjh2 ZwAA== X-Forwarded-Encrypted: i=1; AJvYcCUlAvF6w3/+LE56/DLjzeJVWAuc8yb8Ekr/+nzU+f01HEb6XGEMvbAxEwLrew1yIS84Swjcta3yt4HcA4JUGt1RE8T41FGaW8cK+ciE X-Gm-Message-State: AOJu0YzhjvRP6i46olZeU9zLNs0PZgPmfcrdnM3o7f7KLq27rvqSg0zk 6hPQkwdsR+oJjGFD7tbDG1RX275vmS18GEUjVSzOZP7QlpfbUVcp X-Received: by 2002:a17:903:1106:b0:1d9:e18b:d916 with SMTP id n6-20020a170903110600b001d9e18bd916mr7103005plh.28.1708106778800; Fri, 16 Feb 2024 10:06:18 -0800 (PST) Received: from localhost (dhcp-141-239-158-86.hawaiiantel.net. [141.239.158.86]) by smtp.gmail.com with ESMTPSA id mn8-20020a1709030a4800b001d8d0487edbsm145269plb.223.2024.02.16.10.06.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:06:18 -0800 (PST) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, allen.lkml@gmail.com, kernel-team@meta.com, Tejun Heo Subject: [PATCH 03/17] workqueue: Rename __cancel_work_timer() to __cancel_timer_sync() Date: Fri, 16 Feb 2024 08:04:44 -1000 Message-ID: <20240216180559.208276-4-tj@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240216180559.208276-1-tj@kernel.org> References: <20240216180559.208276-1-tj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791079824620468514 X-GMAIL-MSGID: 1791079824620468514 __cancel_work_timer() is used to implement cancel_work_sync() and cancel_delayed_work_sync(), similarly to how __cancel_work() is used to implement cancel_work() and cancel_delayed_work(). ie. The _timer part of the name is a complete misnomer. The difference from __cancel_work() is the fact that it syncs against work item execution not whether it handles timers or not. Let's rename it to less confusing __cancel_work_sync(). No functional change. Signed-off-by: Tejun Heo --- kernel/workqueue.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 87750e70b638..7e2af79bfa62 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -4075,7 +4075,7 @@ static int cwt_wakefn(wait_queue_entry_t *wait, unsigned mode, int sync, void *k return autoremove_wake_function(wait, mode, sync, key); } -static bool __cancel_work_timer(struct work_struct *work, bool is_dwork) +static bool __cancel_work_sync(struct work_struct *work, bool is_dwork) { static DECLARE_WAIT_QUEUE_HEAD(cancel_waitq); unsigned long flags; @@ -4159,7 +4159,7 @@ static bool __cancel_work_timer(struct work_struct *work, bool is_dwork) */ bool cancel_work_sync(struct work_struct *work) { - return __cancel_work_timer(work, false); + return __cancel_work_sync(work, false); } EXPORT_SYMBOL_GPL(cancel_work_sync); @@ -4264,7 +4264,7 @@ EXPORT_SYMBOL(cancel_delayed_work); */ bool cancel_delayed_work_sync(struct delayed_work *dwork) { - return __cancel_work_timer(&dwork->work, true); + return __cancel_work_sync(&dwork->work, true); } EXPORT_SYMBOL(cancel_delayed_work_sync); From patchwork Fri Feb 16 18:04:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 202284 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:c619:b0:108:e6aa:91d0 with SMTP id hn25csp689562dyb; Fri, 16 Feb 2024 10:07:22 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVItjz4U/T+fka8kicnycVIXC7a2JgAgLEK8TA8NZtoLb5270fjD0/mh8dCoIp0jB0ZeR/afI9We6CCPcteH24qTHp1fA== X-Google-Smtp-Source: AGHT+IHHNG0Dv5sN6qe3WSGe/h5Xq1ZSKjxX38VRTLkSq5b7WZvEy5wEQ5kE7ALkghdDpvGVR7cp X-Received: by 2002:a05:6870:2b04:b0:21e:8603:cf20 with SMTP id ld4-20020a0568702b0400b0021e8603cf20mr78488oab.52.1708106842351; Fri, 16 Feb 2024 10:07:22 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708106842; cv=pass; d=google.com; s=arc-20160816; b=muVhawCaIsNJz2cM62BSWOBZ3nePjihsqdx1hAqDqG+4cDWq/GnwuGiA5wB870xJEA li7gGD7kVGEtng52fpZnkOfxxgXra6HTNHS0JPCDf6ZZ1xqyMP+dZt208kMG0dv2OpGC 5BbRS6wEiXM1A7KgKmmkL4x7mMOxtjUxbMVdQ9bBltVcfVSnLDP6Q0emzkVrfI9PZsnM ZraKp59FMG2BqOwR0vP2rSyjstRxnsLfyJKFxIhcfvxu4mK/S10yh2Z7RLRaSXz6UP4k ohTfS/ZSq+aC4j9+D7E5qojiXA29Dnn0LRbXRVEK7RiUcettaEooUXhu/R8R4JzCdbnL Vi9Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:sender:dkim-signature; bh=+DS0DtBclwHzHSMAZjstaKCNqredx4H7iHvNpCWzDbo=; fh=tnBKLP+Cab9VO74Scjv8u5Unca1J6Gt7YBI7xPnWQm4=; b=wTIYB/pCEmzCpJRt+SU+rI8Ftr1gYf7rzTD4erZbcuET9+ruucg5+7Rf1EuvPzQkSt A7NkcI7QGAW0960R/8ozsLKWv7+ZOWnGqM5etlVA3JLEvZmXcR6G6lJmGcsH+jreMYAM OBlokWT49Ut297jN7F5VAMEdrQAxnpYcGav8eIQljTNaIYMOV+o8PK4/XnMIvM749gxe kpc7PWaOkL/qs47x4ACuc/oC71dq8DYPdwvRaE1asRy9d4x2/15HDaKqV4nW4aSOETiR ct+fQEP8uHXjKOkhVV8BhKPPkvtiWhRo/sv2vS5W391/+GibZoEn5TB8xnPAuH2e2PNA AEBg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=gQzNUwIJ; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69122-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69122-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id x7-20020a1f7c07000000b004c0ae4d3fc8si189418vkc.307.2024.02.16.10.07.22 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:07:22 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-69122-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=gQzNUwIJ; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69122-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69122-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 0A0D11C219A6 for ; Fri, 16 Feb 2024 18:07:22 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3727C135A6F; Fri, 16 Feb 2024 18:06:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gQzNUwIJ" Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 783051353FD for ; Fri, 16 Feb 2024 18:06:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106783; cv=none; b=R0BANBPeLyD06Pys2yXsNUjaUSwRrADmqwA8oDzJLPLIoaNfAq+RsViBbxirHGV7pTtEpEXKJ2neL3rEEs9Lc+Ck75nlevjiP552uNL8vD5T0Yceh2BD0S3eN5528nFsvYKxSoYFwd0mJIUd3c8IEQkORJfDbYLf8kMYG/3Najw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106783; c=relaxed/simple; bh=8S4nvCiX9SIcGhxtmUUofNJcQBOMVPXacIdRA1r1JaM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GkNH4nn0ytvHGko4rgDIjkNGaCBV8EC9olGo5kLVfkINxSU1ckFJ8opTbaqecifGljS9yJCN9yTSSoZlgZCMSFW87F6Y8wieUDV8OptizHjI9Pai9N+3L4nHTfKIjhr26EdyYvaItxVzFIyQlplZ5JQUf67pWLFdVRVaeUq1eUc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=gQzNUwIJ; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-1d93edfa76dso8831135ad.1 for ; Fri, 16 Feb 2024 10:06:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708106780; x=1708711580; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=+DS0DtBclwHzHSMAZjstaKCNqredx4H7iHvNpCWzDbo=; b=gQzNUwIJtOqsqMNUMZllVTcJ+u2MFrWH+aUoQn0zdRVKVac/UZmyuSs5pnkMgq+d7k E4PtgNzdBmoLR1Spdqz2KBd4HKr7bS0UdDTe8x/hHLaZcUU8/wgUmFhqstU7ZQqGkDfh tj52XI4CYAcyIJ4Rpku32aRJXZILKiFDtqAcd9jBIhA+CsP9ccs4HWNgrUvQmoqywYzp 6U2OzopvxifY03ioYbdOXjTgqCXHgio3Fl6F98WZ9jwxVxViEwSA3M8FbYu8pfX7y+qN gY2d4q54SA1T6kK/zf6Cu/PJeZER0q8xrE5dtbO/38z7KOlESXFDqJxThtpAhbOA1K74 XUaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708106780; x=1708711580; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=+DS0DtBclwHzHSMAZjstaKCNqredx4H7iHvNpCWzDbo=; b=KqM8SpcOJugr6YaHfCjQntKNghP+domhfutgSDHE5cp3ybEaluezGU66Z3mkP0tt/T yawj59oDJKE13AaIJ2rbtqYxCHcTX4AFJi6IJ+HtFwzTP9iVG1IVsbPs3CzbuFFU5WsQ /qOFqJ2wQdg7LCkzchuSNDapfV6wIMlRgHQ23Y5gBee2Km2l4IWXk9jHCRuO2Phrbfbd 9prPcL9pxGQTNhCozOJE4Pmw1ywyLMUcIxEc3vGlOw+6ux2GF90wR1QDApzi6bTDh2YZ 9G8Ilhe80F5dQhfocbDJnJIPXH0OrLRiR3Pa0BjgGyA/iLRSkKimFDCVI0jxYn3bZoQw VPEw== X-Forwarded-Encrypted: i=1; AJvYcCU0H2Ll2ANiUx1M9heTeEVCVEX860XgMSbojzKx1+CBnEY/ID3oHvZiweed6eY7PsQh8Hm1FwugcsJ6omAHGLt9KLSPL8ZflOlBzFWV X-Gm-Message-State: AOJu0YxjvzmO60WjwQUToR3SjSL7RT26TRZA45COzG5u2YzBoXkYkx4h wt7czOPq3fwmnlg78L5CooFbXyf4e6yWqpwesaqMnYIap9d6z9Sj X-Received: by 2002:a17:903:904:b0:1db:a6b8:6b23 with SMTP id ll4-20020a170903090400b001dba6b86b23mr3323978plb.27.1708106780472; Fri, 16 Feb 2024 10:06:20 -0800 (PST) Received: from localhost (dhcp-141-239-158-86.hawaiiantel.net. [141.239.158.86]) by smtp.gmail.com with ESMTPSA id jc15-20020a17090325cf00b001db54324488sm161468plb.38.2024.02.16.10.06.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:06:20 -0800 (PST) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, allen.lkml@gmail.com, kernel-team@meta.com, Tejun Heo Subject: [PATCH 04/17] workqueue: Reorganize flush and cancel[_sync] functions Date: Fri, 16 Feb 2024 08:04:45 -1000 Message-ID: <20240216180559.208276-5-tj@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240216180559.208276-1-tj@kernel.org> References: <20240216180559.208276-1-tj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791079840590441401 X-GMAIL-MSGID: 1791079840590441401 They are currently a bit disorganized with flush and cancel functions mixed. Reoranize them so that flush functions come first, cancel next and cancel_sync last. This way, we won't have to add prototypes for internal functions for the planned disable/enable support. This is pure code reorganization. No functional changes. Signed-off-by: Tejun Heo --- kernel/workqueue.c | 136 ++++++++++++++++++++++----------------------- 1 file changed, 68 insertions(+), 68 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 7e2af79bfa62..962061dca05c 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -4061,6 +4061,65 @@ bool flush_work(struct work_struct *work) } EXPORT_SYMBOL_GPL(flush_work); +/** + * flush_delayed_work - wait for a dwork to finish executing the last queueing + * @dwork: the delayed work to flush + * + * Delayed timer is cancelled and the pending work is queued for + * immediate execution. Like flush_work(), this function only + * considers the last queueing instance of @dwork. + * + * Return: + * %true if flush_work() waited for the work to finish execution, + * %false if it was already idle. + */ +bool flush_delayed_work(struct delayed_work *dwork) +{ + local_irq_disable(); + if (del_timer_sync(&dwork->timer)) + __queue_work(dwork->cpu, dwork->wq, &dwork->work); + local_irq_enable(); + return flush_work(&dwork->work); +} +EXPORT_SYMBOL(flush_delayed_work); + +/** + * flush_rcu_work - wait for a rwork to finish executing the last queueing + * @rwork: the rcu work to flush + * + * Return: + * %true if flush_rcu_work() waited for the work to finish execution, + * %false if it was already idle. + */ +bool flush_rcu_work(struct rcu_work *rwork) +{ + if (test_bit(WORK_STRUCT_PENDING_BIT, work_data_bits(&rwork->work))) { + rcu_barrier(); + flush_work(&rwork->work); + return true; + } else { + return flush_work(&rwork->work); + } +} +EXPORT_SYMBOL(flush_rcu_work); + +static bool __cancel_work(struct work_struct *work, bool is_dwork) +{ + unsigned long flags; + int ret; + + do { + ret = try_to_grab_pending(work, is_dwork, &flags); + } while (unlikely(ret == -EAGAIN)); + + if (unlikely(ret < 0)) + return false; + + set_work_pool_and_clear_pending(work, get_work_pool_id(work)); + local_irq_restore(flags); + return ret; +} + struct cwt_wait { wait_queue_entry_t wait; struct work_struct *work; @@ -4139,6 +4198,15 @@ static bool __cancel_work_sync(struct work_struct *work, bool is_dwork) return ret; } +/* + * See cancel_delayed_work() + */ +bool cancel_work(struct work_struct *work) +{ + return __cancel_work(work, false); +} +EXPORT_SYMBOL(cancel_work); + /** * cancel_work_sync - cancel a work and wait for it to finish * @work: the work to cancel @@ -4163,74 +4231,6 @@ bool cancel_work_sync(struct work_struct *work) } EXPORT_SYMBOL_GPL(cancel_work_sync); -/** - * flush_delayed_work - wait for a dwork to finish executing the last queueing - * @dwork: the delayed work to flush - * - * Delayed timer is cancelled and the pending work is queued for - * immediate execution. Like flush_work(), this function only - * considers the last queueing instance of @dwork. - * - * Return: - * %true if flush_work() waited for the work to finish execution, - * %false if it was already idle. - */ -bool flush_delayed_work(struct delayed_work *dwork) -{ - local_irq_disable(); - if (del_timer_sync(&dwork->timer)) - __queue_work(dwork->cpu, dwork->wq, &dwork->work); - local_irq_enable(); - return flush_work(&dwork->work); -} -EXPORT_SYMBOL(flush_delayed_work); - -/** - * flush_rcu_work - wait for a rwork to finish executing the last queueing - * @rwork: the rcu work to flush - * - * Return: - * %true if flush_rcu_work() waited for the work to finish execution, - * %false if it was already idle. - */ -bool flush_rcu_work(struct rcu_work *rwork) -{ - if (test_bit(WORK_STRUCT_PENDING_BIT, work_data_bits(&rwork->work))) { - rcu_barrier(); - flush_work(&rwork->work); - return true; - } else { - return flush_work(&rwork->work); - } -} -EXPORT_SYMBOL(flush_rcu_work); - -static bool __cancel_work(struct work_struct *work, bool is_dwork) -{ - unsigned long flags; - int ret; - - do { - ret = try_to_grab_pending(work, is_dwork, &flags); - } while (unlikely(ret == -EAGAIN)); - - if (unlikely(ret < 0)) - return false; - - set_work_pool_and_clear_pending(work, get_work_pool_id(work)); - local_irq_restore(flags); - return ret; -} - -/* - * See cancel_delayed_work() - */ -bool cancel_work(struct work_struct *work) -{ - return __cancel_work(work, false); -} -EXPORT_SYMBOL(cancel_work); - /** * cancel_delayed_work - cancel a delayed work * @dwork: delayed_work to cancel From patchwork Fri Feb 16 18:04:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 202285 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:c619:b0:108:e6aa:91d0 with SMTP id hn25csp689853dyb; Fri, 16 Feb 2024 10:07:45 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWn61zYzCrSwKNTeQEUzqCLuUGieQ5bUWKOi7wesjc+f+x9y9qWEuJquLkVd7ulSTD9TS5ATNwkuSoCHQKPJVMs1QuzXA== X-Google-Smtp-Source: AGHT+IEdwTvd/lEXuJSRO6GNO5tOfi7OO8+7LnHrgj2uZCMcNNzfeyHHlSRsGUOzz4OAjDN8eILv X-Received: by 2002:a17:902:e551:b0:1db:be69:d037 with SMTP id n17-20020a170902e55100b001dbbe69d037mr864047plf.46.1708106864948; Fri, 16 Feb 2024 10:07:44 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708106864; cv=pass; d=google.com; s=arc-20160816; b=Wysp7/Cvq254nHgYg/fAD9SSzGD4yA4GfEOCHA4dR5O612cMTn6pkhm24h2xsUvwRQ FT4MC6xnUp7PzSpeXaYbCdN90PCe44E53BST2DAO21fPw8AFyHS29OkEAdTeA3ZeEhS2 feQMYbEXOwUpcLpwq0dfjY9+oz0wL+TWonuXR+miejAdXF+j4RbycqNfn5YRhEvjTEMW z28usy58VTpfDUh9Xcyzb6bmR23cVHup4MtTKjez3SdMOWG7BI377IezvHYGafo3bKq+ yxknQJoJ4jA2jA5+GYLpJHSFt/EklaX2aYGVziaHfAnc7bmaswEfUVZVOuNKLMTdDSHr umOw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:sender:dkim-signature; bh=fiVo0gbO7Dqzblo5C6ryvAHZfaLX7ADi1ZU8tAGlpEU=; fh=b6TvkTb9MddDGmqy5C0tPq47zAenflsmPG9cND8EIZ0=; b=CacPkf7snkvtnHnccyzprgYO8HFlPqJDR/6cwCmrfsUFAxYfvhne/gGsAyKk1U9q/0 v+qekL0j3ZYcHk61XixPPSi7JkUw+9t2VwQpoBl82YCLaWDxvohGFx65o3eSf2XJdC/5 agu0VSKKIvZza2RUAQ+ogaiJmwKs9kxSV2qBYk3avec+t2B4Yo9ZmLKK/idkLNltjhZS iKnOP7qtWxmQMLj94Zswwoy8prPUAxmY/QUySsUR5nXyGYqb1PAFiZWbu7q6RK+aDX1c n0baq2C486He0x2+dLwZUOkSclNcwCgaJLxLSNtTLKpTf+0Tpk2JyJ47MrbCJbREkQ3q CdJg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=J9STS7GV; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69123-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69123-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id ix13-20020a170902f80d00b001d5457db81dsi183334plb.649.2024.02.16.10.07.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:07:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-69123-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=J9STS7GV; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69123-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69123-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id A2EF6283462 for ; Fri, 16 Feb 2024 18:07:44 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id F24EC1350D5; Fri, 16 Feb 2024 18:06:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="J9STS7GV" Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 04DD413541F for ; Fri, 16 Feb 2024 18:06:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106784; cv=none; b=YQ5203+LoaoHWs6xY83vgp9JOiO9+XpEy9U/pz0QUdrOb7VUeWmcsFu66QDoH5dU4ClfV6fg890pk+K0eSGl/3EBkXJTEOSGttAXi5jyq5AybsXqEsuJINY95NPKTrpEl9priDB/16W5edO+7v82XR8ROkARUsyuitgP6lIPTtM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106784; c=relaxed/simple; bh=tPPHqZjVF21W3HTmNqLMh1L8ACYe5HaB4tCiqIc46fw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Kb37llwC0pGABQJt/OzwKkItc3xH9KJmt1uQ+8Wl/b/dEyeUqblUI0hkbpbx6Rsu6afyKZkg7ifmwkMw+n17sYvrh1+XOt2XAzw1K64dxchRzcJaSGCwqoh38nrk7OiN9a0SIHXxG4RsCZbn817VK09YuR+gMJKKES3gDDx1h/E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=J9STS7GV; arc=none smtp.client-ip=209.85.215.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pg1-f176.google.com with SMTP id 41be03b00d2f7-5cdf76cde78so1930209a12.1 for ; Fri, 16 Feb 2024 10:06:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708106782; x=1708711582; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=fiVo0gbO7Dqzblo5C6ryvAHZfaLX7ADi1ZU8tAGlpEU=; b=J9STS7GV0D7ECNxXOcGvDl2/7XEcjk4X3bhlxoEzbrcBbC1y677AesKv3urnz2DTLf qJT5I6Lwc65Cla+KWxonDcH45WnXer1L48w9cWA42Q2hm0/DF4jYAsEdw1sIyGPO4tpF YyRhi5wO6UwVl2OSZmoTxiF9NmjiUmiPV1ch5EL3JR/S4YaI6/zE7wpWiRXG05yAYhUR +M+uLHejsVt+C0A3qDWekHkNT0trDLjsBqwng90hO/KLP5IoGQcD6lyTJrYoYTi0NF1/ Ce6Qfv/J0Bw4h3klL7XPKragoD0mQ+3q4gaVM5wqAPPXlECZgrbMhxbSu+UDi7pfoKSd NiwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708106782; x=1708711582; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=fiVo0gbO7Dqzblo5C6ryvAHZfaLX7ADi1ZU8tAGlpEU=; b=ImezeF9bxq8J3XpSTI0HSdT3LL1W2WbZ/20HuMbQa4gkwyI8csJ4QRlJ1V1LJL/P7a hymMGq4Hq6BNYvgz6D8Wx9343FJ/byi9RFhxSGAVEAl9OO6K7pmeuzpRnT9YHU6mxrj5 MRce7bWB6q/U1zZWhbzCp2EwhUjl+wxKQHM/Gu8um4YKUEF+e6xXEkdjZMh8nvdzOtv+ sXL0X8BbgwmqeL4/tBfSyNKslE9/bBsegUK2yd+vbRYVNu5Yu4cSrlcqXiBb/yh4/jJs j4+Pzir9KIJOIOf8jQ0NAgWB9tamryKlhMfqhL5p1J80Fd/edmUfyMJzpPkcVK76jfLc TG3g== X-Forwarded-Encrypted: i=1; AJvYcCWWZJWNTYWgushuaqhzMQetETqNtjf6GH51JVVAZRtNVfR8D4U3uDq15SziJ6vd2lYlWhnPMP9+K5DKBpUs7tZW10PEW1rnpmKx57sD X-Gm-Message-State: AOJu0Yzk8R6OurkWg3lxjFsr0UZkTpuKO7K8a7ULkmNr6SMru7xI/VlN 3NAJN5tZx6itZBI7Zx+IHpnd+Ajlleqw4ojteOVrluw/5RCIF+9lVet3sDqS/n8= X-Received: by 2002:a17:90b:234b:b0:299:30eb:f687 with SMTP id ms11-20020a17090b234b00b0029930ebf687mr3073017pjb.16.1708106782092; Fri, 16 Feb 2024 10:06:22 -0800 (PST) Received: from localhost (dhcp-141-239-158-86.hawaiiantel.net. [141.239.158.86]) by smtp.gmail.com with ESMTPSA id qa9-20020a17090b4fc900b002994f848bc9sm99440pjb.43.2024.02.16.10.06.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:06:21 -0800 (PST) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, allen.lkml@gmail.com, kernel-team@meta.com, Tejun Heo Subject: [PATCH 05/17] workqueue: Use variable name irq_flags for saving local irq flags Date: Fri, 16 Feb 2024 08:04:46 -1000 Message-ID: <20240216180559.208276-6-tj@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240216180559.208276-1-tj@kernel.org> References: <20240216180559.208276-1-tj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791079863998315893 X-GMAIL-MSGID: 1791079863998315893 Using the generic term `flags` for irq flags is conventional but can be confusing as there's quite a bit of code dealing with work flags which involves some subtleties. Let's use a more explicit name `irq_flags` for local irq flags. No functional changes. Signed-off-by: Tejun Heo --- kernel/workqueue.c | 76 +++++++++++++++++++++++----------------------- 1 file changed, 38 insertions(+), 38 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 962061dca05c..b590d93d054b 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -2029,7 +2029,7 @@ static void pwq_dec_nr_in_flight(struct pool_workqueue *pwq, unsigned long work_ * try_to_grab_pending - steal work item from worklist and disable irq * @work: work item to steal * @is_dwork: @work is a delayed_work - * @flags: place to store irq state + * @irq_flags: place to store irq state * * Try to grab PENDING bit of @work. This function can handle @work in any * stable state - idle, on timer or on worklist. @@ -2051,17 +2051,17 @@ static void pwq_dec_nr_in_flight(struct pool_workqueue *pwq, unsigned long work_ * irqsafe, ensures that we return -EAGAIN for finite short period of time. * * On successful return, >= 0, irq is disabled and the caller is - * responsible for releasing it using local_irq_restore(*@flags). + * responsible for releasing it using local_irq_restore(*@irq_flags). * * This function is safe to call from any context including IRQ handler. */ static int try_to_grab_pending(struct work_struct *work, bool is_dwork, - unsigned long *flags) + unsigned long *irq_flags) { struct worker_pool *pool; struct pool_workqueue *pwq; - local_irq_save(*flags); + local_irq_save(*irq_flags); /* try to steal the timer if it exists */ if (is_dwork) { @@ -2136,7 +2136,7 @@ static int try_to_grab_pending(struct work_struct *work, bool is_dwork, raw_spin_unlock(&pool->lock); fail: rcu_read_unlock(); - local_irq_restore(*flags); + local_irq_restore(*irq_flags); if (work_is_canceling(work)) return -ENOENT; cpu_relax(); @@ -2344,16 +2344,16 @@ bool queue_work_on(int cpu, struct workqueue_struct *wq, struct work_struct *work) { bool ret = false; - unsigned long flags; + unsigned long irq_flags; - local_irq_save(flags); + local_irq_save(irq_flags); if (!test_and_set_bit(WORK_STRUCT_PENDING_BIT, work_data_bits(work))) { __queue_work(cpu, wq, work); ret = true; } - local_irq_restore(flags); + local_irq_restore(irq_flags); return ret; } EXPORT_SYMBOL(queue_work_on); @@ -2410,7 +2410,7 @@ static int select_numa_node_cpu(int node) bool queue_work_node(int node, struct workqueue_struct *wq, struct work_struct *work) { - unsigned long flags; + unsigned long irq_flags; bool ret = false; /* @@ -2424,7 +2424,7 @@ bool queue_work_node(int node, struct workqueue_struct *wq, */ WARN_ON_ONCE(!(wq->flags & WQ_UNBOUND)); - local_irq_save(flags); + local_irq_save(irq_flags); if (!test_and_set_bit(WORK_STRUCT_PENDING_BIT, work_data_bits(work))) { int cpu = select_numa_node_cpu(node); @@ -2433,7 +2433,7 @@ bool queue_work_node(int node, struct workqueue_struct *wq, ret = true; } - local_irq_restore(flags); + local_irq_restore(irq_flags); return ret; } EXPORT_SYMBOL_GPL(queue_work_node); @@ -2503,17 +2503,17 @@ bool queue_delayed_work_on(int cpu, struct workqueue_struct *wq, { struct work_struct *work = &dwork->work; bool ret = false; - unsigned long flags; + unsigned long irq_flags; /* read the comment in __queue_work() */ - local_irq_save(flags); + local_irq_save(irq_flags); if (!test_and_set_bit(WORK_STRUCT_PENDING_BIT, work_data_bits(work))) { __queue_delayed_work(cpu, wq, dwork, delay); ret = true; } - local_irq_restore(flags); + local_irq_restore(irq_flags); return ret; } EXPORT_SYMBOL(queue_delayed_work_on); @@ -2539,16 +2539,16 @@ EXPORT_SYMBOL(queue_delayed_work_on); bool mod_delayed_work_on(int cpu, struct workqueue_struct *wq, struct delayed_work *dwork, unsigned long delay) { - unsigned long flags; + unsigned long irq_flags; int ret; do { - ret = try_to_grab_pending(&dwork->work, true, &flags); + ret = try_to_grab_pending(&dwork->work, true, &irq_flags); } while (unlikely(ret == -EAGAIN)); if (likely(ret >= 0)) { __queue_delayed_work(cpu, wq, dwork, delay); - local_irq_restore(flags); + local_irq_restore(irq_flags); } /* -ENOENT from try_to_grab_pending() becomes %true */ @@ -4105,18 +4105,18 @@ EXPORT_SYMBOL(flush_rcu_work); static bool __cancel_work(struct work_struct *work, bool is_dwork) { - unsigned long flags; + unsigned long irq_flags; int ret; do { - ret = try_to_grab_pending(work, is_dwork, &flags); + ret = try_to_grab_pending(work, is_dwork, &irq_flags); } while (unlikely(ret == -EAGAIN)); if (unlikely(ret < 0)) return false; set_work_pool_and_clear_pending(work, get_work_pool_id(work)); - local_irq_restore(flags); + local_irq_restore(irq_flags); return ret; } @@ -4137,11 +4137,11 @@ static int cwt_wakefn(wait_queue_entry_t *wait, unsigned mode, int sync, void *k static bool __cancel_work_sync(struct work_struct *work, bool is_dwork) { static DECLARE_WAIT_QUEUE_HEAD(cancel_waitq); - unsigned long flags; + unsigned long irq_flags; int ret; do { - ret = try_to_grab_pending(work, is_dwork, &flags); + ret = try_to_grab_pending(work, is_dwork, &irq_flags); /* * If someone else is already canceling, wait for it to * finish. flush_work() doesn't work for PREEMPT_NONE @@ -4175,7 +4175,7 @@ static bool __cancel_work_sync(struct work_struct *work, bool is_dwork) /* tell other tasks trying to grab @work to back off */ mark_work_canceling(work); - local_irq_restore(flags); + local_irq_restore(irq_flags); /* * Skip __flush_work() during early boot when we know that @work isn't @@ -5381,15 +5381,15 @@ static void wq_adjust_max_active(struct workqueue_struct *wq) activated = false; for_each_pwq(pwq, wq) { - unsigned long flags; + unsigned long irq_flags; /* can be called during early boot w/ irq disabled */ - raw_spin_lock_irqsave(&pwq->pool->lock, flags); + raw_spin_lock_irqsave(&pwq->pool->lock, irq_flags); if (pwq_activate_first_inactive(pwq, true)) { activated = true; kick_pool(pwq->pool); } - raw_spin_unlock_irqrestore(&pwq->pool->lock, flags); + raw_spin_unlock_irqrestore(&pwq->pool->lock, irq_flags); } } while (activated); } @@ -5762,7 +5762,7 @@ EXPORT_SYMBOL_GPL(workqueue_congested); unsigned int work_busy(struct work_struct *work) { struct worker_pool *pool; - unsigned long flags; + unsigned long irq_flags; unsigned int ret = 0; if (work_pending(work)) @@ -5771,10 +5771,10 @@ unsigned int work_busy(struct work_struct *work) rcu_read_lock(); pool = get_work_pool(work); if (pool) { - raw_spin_lock_irqsave(&pool->lock, flags); + raw_spin_lock_irqsave(&pool->lock, irq_flags); if (find_worker_executing_work(pool, work)) ret |= WORK_BUSY_RUNNING; - raw_spin_unlock_irqrestore(&pool->lock, flags); + raw_spin_unlock_irqrestore(&pool->lock, irq_flags); } rcu_read_unlock(); @@ -6006,7 +6006,7 @@ void show_one_workqueue(struct workqueue_struct *wq) { struct pool_workqueue *pwq; bool idle = true; - unsigned long flags; + unsigned long irq_flags; for_each_pwq(pwq, wq) { if (!pwq_is_empty(pwq)) { @@ -6020,7 +6020,7 @@ void show_one_workqueue(struct workqueue_struct *wq) pr_info("workqueue %s: flags=0x%x\n", wq->name, wq->flags); for_each_pwq(pwq, wq) { - raw_spin_lock_irqsave(&pwq->pool->lock, flags); + raw_spin_lock_irqsave(&pwq->pool->lock, irq_flags); if (!pwq_is_empty(pwq)) { /* * Defer printing to avoid deadlocks in console @@ -6031,7 +6031,7 @@ void show_one_workqueue(struct workqueue_struct *wq) show_pwq(pwq); printk_deferred_exit(); } - raw_spin_unlock_irqrestore(&pwq->pool->lock, flags); + raw_spin_unlock_irqrestore(&pwq->pool->lock, irq_flags); /* * We could be printing a lot from atomic context, e.g. * sysrq-t -> show_all_workqueues(). Avoid triggering @@ -6050,10 +6050,10 @@ static void show_one_worker_pool(struct worker_pool *pool) { struct worker *worker; bool first = true; - unsigned long flags; + unsigned long irq_flags; unsigned long hung = 0; - raw_spin_lock_irqsave(&pool->lock, flags); + raw_spin_lock_irqsave(&pool->lock, irq_flags); if (pool->nr_workers == pool->nr_idle) goto next_pool; @@ -6081,7 +6081,7 @@ static void show_one_worker_pool(struct worker_pool *pool) pr_cont("\n"); printk_deferred_exit(); next_pool: - raw_spin_unlock_irqrestore(&pool->lock, flags); + raw_spin_unlock_irqrestore(&pool->lock, irq_flags); /* * We could be printing a lot from atomic context, e.g. * sysrq-t -> show_all_workqueues(). Avoid triggering @@ -7212,10 +7212,10 @@ static DEFINE_PER_CPU(unsigned long, wq_watchdog_touched_cpu) = INITIAL_JIFFIES; static void show_cpu_pool_hog(struct worker_pool *pool) { struct worker *worker; - unsigned long flags; + unsigned long irq_flags; int bkt; - raw_spin_lock_irqsave(&pool->lock, flags); + raw_spin_lock_irqsave(&pool->lock, irq_flags); hash_for_each(pool->busy_hash, bkt, worker, hentry) { if (task_is_running(worker->task)) { @@ -7233,7 +7233,7 @@ static void show_cpu_pool_hog(struct worker_pool *pool) } } - raw_spin_unlock_irqrestore(&pool->lock, flags); + raw_spin_unlock_irqrestore(&pool->lock, irq_flags); } static void show_cpu_pools_hogs(void) From patchwork Fri Feb 16 18:04:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 202286 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:c619:b0:108:e6aa:91d0 with SMTP id hn25csp689913dyb; Fri, 16 Feb 2024 10:07:48 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXMyV9gtFjDS3kvXO/mNkQJeVpFi7E1yqZgDGBxhpddXpEROFhOXt4YXoYb2bfK7ReZNVGP8ALrdRyO2P4C32Da0wwilA== X-Google-Smtp-Source: AGHT+IFAFGORdifgLh40ckGCad19/7ltJ0KkEX+afG7cAPqzOaKlceforH8wSzYSYMoypUKkvEZ1 X-Received: by 2002:a05:6a21:150c:b0:19e:9ad6:9213 with SMTP id nq12-20020a056a21150c00b0019e9ad69213mr7854451pzb.2.1708106868498; Fri, 16 Feb 2024 10:07:48 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708106868; cv=pass; d=google.com; s=arc-20160816; b=HQQ2tmsE/01uB0wnq69wBKOLYNCR3Z3ALy5dsWjXgjwBiQ+x1H1+u1bFwDvcw6v6Lk uxL7NTM4dn9qeCIktlAy40fn7vnQJpTQ05WARJ09TGIz8/HvH2yTlZ7IClvNgal/0kj/ ikYloUXerpTUZssrrT0v/XlcPfs90vlvGV9BQQwgAonKnoKWbXAZ2b/SHHFIyFjSlDyJ 8+qfRe51KFH6eQ9RfxpfaM8idh6bVV/50M2+XUyPDCyn68KTHCpKNF3REdUINNc7ZWv5 1cYGXQOtbZFBTfB6ruD5HaJ8w9ASO8y8I9Uqu0zBgMIsNBP3sXcu666P8e6vedGjiYxo ObJA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:sender:dkim-signature; bh=THTKmyp+zgWOjBKgKcw4Z+riVlmxEgNDNx8pO3NcTsI=; fh=MhyGF2QYleG+BNhRIdBdsEw1bPAyTnGYZWqoSx7ojF8=; b=LQfGlXJ9U0CFZ0ZOfcunBnKXheRte7xjgUki1sXcme2GDyZ5Gmb7C/lrNV1J6h7fvW L6oPJv35KHDG82Pr5xfKKvdotY5+qfkOsoEWHaiAOk2x9GFVXmwfRogCJ8v0K0tpx4bd L4FtX0pPRceLee1dJ7R5S4gfFpY0qsTMxUru8/AUBPhGhJzyW8zv2ks+tPSqfmY+o4g+ gizjySScIvbGdQfFYWnFRFz3yycs9+BKxJMyBurjnFcm+/sFHSH8xJYwlZgjuY4TX8kn aAMbPLYoQtw6JygXIfsriuTVaFFUMMUnMVDNvqnVtndYnE/2Gt6KmDXzVmf7bJf6dYRV cJzQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=gdbbcaFM; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69124-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69124-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id m189-20020a6258c6000000b006e08c4f7d69si256523pfb.191.2024.02.16.10.07.48 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:07:48 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-69124-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=gdbbcaFM; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69124-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69124-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 0F47A282158 for ; Fri, 16 Feb 2024 18:07:48 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 74C57137C49; Fri, 16 Feb 2024 18:06:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gdbbcaFM" Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A7EE11350F6 for ; Fri, 16 Feb 2024 18:06:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106786; cv=none; b=GjZT1E1EKhQhRudX6b148zIteTYdIeKQWw22jLAbfVHzoTDXV/HlHtYnIpdeiTAjVUi8VHBaY+S+6s2AiD0bS8FAWicizGdn4DmdehzP5vJk+l/WGICo7WMq8ER6+f8e4VESeHxF9q03YFoxM9VAIBxpDh+ceB4aud0jGLnDg/w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106786; c=relaxed/simple; bh=EnYPc6hrKRSIFSoUsgaDvii7SmmceMlHcVdpQXbdsuM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IDP84rDzk9drakkuA7PsOPbgBs0tApn40uohfSgXFsNmmEU0if3cPd25lO33XZhN34YaXfw+cjfKFmq6IEfW26YbIaEUgx2TgnnA9b7AGckfOCa8HtrhPfToKavPaMpn2C6RTk1pT144exr81X5pePCWku/cgUKXzQwNO+pTRuY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=gdbbcaFM; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-1d7354ba334so21020085ad.1 for ; Fri, 16 Feb 2024 10:06:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708106784; x=1708711584; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=THTKmyp+zgWOjBKgKcw4Z+riVlmxEgNDNx8pO3NcTsI=; b=gdbbcaFMNY/ebW43+vxgJooQrhcl3/BE94BYK16j+YHCfaGbOuQ5kK6KonUOZIR5qr 4TUhnoSrobYQOarxx/dKlLd/xwlsTvcd/WQFy82fg0jJ/lqpAZjA0+y7s7Byxa+Fxmyz 73zkW5IyM0pMmEDYblGav90tm+sPHkVeFiczjf6/NNWU1hzhWfApmZPNPA06MXvGItpw gYwziVVTIJ9Nv+PYV42SDqX62ZSyHE9NE4p2VJdRGCimxj9pqvp0lKGqIdRrNq1hE21k aJiAC7fGFBv6KhNAk6da8SnWgL3Wv0AfqPG2O0g2LyCVaIQmNJPiVJodaWacahAxCMUZ 1w3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708106784; x=1708711584; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=THTKmyp+zgWOjBKgKcw4Z+riVlmxEgNDNx8pO3NcTsI=; b=RZxj1FTXqETFXVspEjBEY7Q7J267hQAqIKGhA4ES0ZU8Ivwq9u+jNhI12egnY23Tb4 HrAm09okVOShLtUGqP5uk44W4wa5wiTpRj2uh33VFAWnPCT5QoXZQaTiPHHW2MTaoxE/ FGcboGyW3ML1RFzu7GyUtd8SoKPoPAp9K65T93PojRfESjhPE553pHe4/V9LZN4x50qE eoz7c1eiK5M9zPxR3ybU662dMbSpeFZ3KjwWZRTV7lzvCDKweIQOuckPU3WrVbjv7u6z ljO0hVvGOtCSwVQ46Rc1C8YrgcwqxJ6ewhjnKHY6yX4fPElKfkmOQ78Trc3uYbqtRc6D OBUA== X-Forwarded-Encrypted: i=1; AJvYcCWN/gl3nNZP9JcD4so+3ozUCwnlXv5onsypR0O0uWzkc0H1SUakg2MAI9EkmnRPqDEm3vHyfdePQOvdKi1aP+Knbv09/EvRW/Kxr50L X-Gm-Message-State: AOJu0YzQqphPRahM0dBXQoF5mj3455qKsjUjnTbZJfygVFD0rTzM3uJI uMEiXQWJ/6FTog92R7543AryBSkLr9kg2QeESQo7KWHsY8hLciPp X-Received: by 2002:a17:902:6e0f:b0:1d9:167b:8e6c with SMTP id u15-20020a1709026e0f00b001d9167b8e6cmr4774669plk.46.1708106783880; Fri, 16 Feb 2024 10:06:23 -0800 (PST) Received: from localhost (dhcp-141-239-158-86.hawaiiantel.net. [141.239.158.86]) by smtp.gmail.com with ESMTPSA id n4-20020a170902d2c400b001db6e600ae0sm144232plc.267.2024.02.16.10.06.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:06:23 -0800 (PST) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, allen.lkml@gmail.com, kernel-team@meta.com, Tejun Heo Subject: [PATCH 06/17] workqueue: Introduce work_cancel_flags Date: Fri, 16 Feb 2024 08:04:47 -1000 Message-ID: <20240216180559.208276-7-tj@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240216180559.208276-1-tj@kernel.org> References: <20240216180559.208276-1-tj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791079867519139220 X-GMAIL-MSGID: 1791079867519139220 The cancel path used bool @is_dwork to distinguish canceling a regular work and a delayed one. The planned disable/enable support will need passing around another flag in the code path. As passing them around with bools will be confusing, let's introduce named flags to pass around in the cancel path. WORK_CANCEL_DELAYED replaces @is_dwork. No functional changes. Signed-off-by: Tejun Heo --- kernel/workqueue.c | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index b590d93d054b..317c85f051b0 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -96,6 +96,10 @@ enum worker_flags { WORKER_UNBOUND | WORKER_REBOUND, }; +enum work_cancel_flags { + WORK_CANCEL_DELAYED = 1 << 0, /* canceling a delayed_work */ +}; + enum wq_internal_consts { NR_STD_WORKER_POOLS = 2, /* # standard pools per cpu */ @@ -2028,7 +2032,7 @@ static void pwq_dec_nr_in_flight(struct pool_workqueue *pwq, unsigned long work_ /** * try_to_grab_pending - steal work item from worklist and disable irq * @work: work item to steal - * @is_dwork: @work is a delayed_work + * @cflags: %WORK_CANCEL_ flags * @irq_flags: place to store irq state * * Try to grab PENDING bit of @work. This function can handle @work in any @@ -2055,7 +2059,7 @@ static void pwq_dec_nr_in_flight(struct pool_workqueue *pwq, unsigned long work_ * * This function is safe to call from any context including IRQ handler. */ -static int try_to_grab_pending(struct work_struct *work, bool is_dwork, +static int try_to_grab_pending(struct work_struct *work, u32 cflags, unsigned long *irq_flags) { struct worker_pool *pool; @@ -2064,7 +2068,7 @@ static int try_to_grab_pending(struct work_struct *work, bool is_dwork, local_irq_save(*irq_flags); /* try to steal the timer if it exists */ - if (is_dwork) { + if (cflags & WORK_CANCEL_DELAYED) { struct delayed_work *dwork = to_delayed_work(work); /* @@ -2543,7 +2547,8 @@ bool mod_delayed_work_on(int cpu, struct workqueue_struct *wq, int ret; do { - ret = try_to_grab_pending(&dwork->work, true, &irq_flags); + ret = try_to_grab_pending(&dwork->work, WORK_CANCEL_DELAYED, + &irq_flags); } while (unlikely(ret == -EAGAIN)); if (likely(ret >= 0)) { @@ -4103,13 +4108,13 @@ bool flush_rcu_work(struct rcu_work *rwork) } EXPORT_SYMBOL(flush_rcu_work); -static bool __cancel_work(struct work_struct *work, bool is_dwork) +static bool __cancel_work(struct work_struct *work, u32 cflags) { unsigned long irq_flags; int ret; do { - ret = try_to_grab_pending(work, is_dwork, &irq_flags); + ret = try_to_grab_pending(work, cflags, &irq_flags); } while (unlikely(ret == -EAGAIN)); if (unlikely(ret < 0)) @@ -4134,14 +4139,14 @@ static int cwt_wakefn(wait_queue_entry_t *wait, unsigned mode, int sync, void *k return autoremove_wake_function(wait, mode, sync, key); } -static bool __cancel_work_sync(struct work_struct *work, bool is_dwork) +static bool __cancel_work_sync(struct work_struct *work, u32 cflags) { static DECLARE_WAIT_QUEUE_HEAD(cancel_waitq); unsigned long irq_flags; int ret; do { - ret = try_to_grab_pending(work, is_dwork, &irq_flags); + ret = try_to_grab_pending(work, cflags, &irq_flags); /* * If someone else is already canceling, wait for it to * finish. flush_work() doesn't work for PREEMPT_NONE @@ -4203,7 +4208,7 @@ static bool __cancel_work_sync(struct work_struct *work, bool is_dwork) */ bool cancel_work(struct work_struct *work) { - return __cancel_work(work, false); + return __cancel_work(work, 0); } EXPORT_SYMBOL(cancel_work); @@ -4227,7 +4232,7 @@ EXPORT_SYMBOL(cancel_work); */ bool cancel_work_sync(struct work_struct *work) { - return __cancel_work_sync(work, false); + return __cancel_work_sync(work, 0); } EXPORT_SYMBOL_GPL(cancel_work_sync); @@ -4249,7 +4254,7 @@ EXPORT_SYMBOL_GPL(cancel_work_sync); */ bool cancel_delayed_work(struct delayed_work *dwork) { - return __cancel_work(&dwork->work, true); + return __cancel_work(&dwork->work, WORK_CANCEL_DELAYED); } EXPORT_SYMBOL(cancel_delayed_work); @@ -4264,7 +4269,7 @@ EXPORT_SYMBOL(cancel_delayed_work); */ bool cancel_delayed_work_sync(struct delayed_work *dwork) { - return __cancel_work_sync(&dwork->work, true); + return __cancel_work_sync(&dwork->work, WORK_CANCEL_DELAYED); } EXPORT_SYMBOL(cancel_delayed_work_sync); From patchwork Fri Feb 16 18:04:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 202287 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:c619:b0:108:e6aa:91d0 with SMTP id hn25csp690071dyb; Fri, 16 Feb 2024 10:08:00 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXk+GsmOBuxm4IulCuRlwovGfKeUkm4CqCfa0yy8AF7QPWxENGFmPaJnp+mnozakgWA7ZQQuXhEQvx2giC5gmA9Rz1tNA== X-Google-Smtp-Source: AGHT+IE6R+tMrvBy1O9YjsI5iSJALarTJgNeV/HTBcNQBAOJVj1VQCS1mv5wH30CfBNmhPE+bgD5 X-Received: by 2002:a05:6870:a44a:b0:21e:631d:9473 with SMTP id n10-20020a056870a44a00b0021e631d9473mr2795396oal.10.1708106880797; Fri, 16 Feb 2024 10:08:00 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708106880; cv=pass; d=google.com; s=arc-20160816; b=wArPO9Hoy9CPRuy4HLSr7fhr/tn7jKktTRSkvACYtA7/oYztDKFsfEiHSzS4NGFavD Ll10M/SrGlBkwbgHnrRwqEUIu2FjICQCyv2CbMcopl5SRjoAWWXjdaXRitp+oaV0kTIH YHPzuoXCegHQe0K+LIv5Uv0nm+mvq9VOIAXi4oYECV8DbRGwP0HDSEHVCN0UF4+asCHT m5S64hN62lQHINVvvZpRmdmCBNfj8ONxQXMaIua5CZs6fnk7QRVXzCYp5ZF69r93g7ZS i15HYuADHSco7uLwa6wObmqQcEgzQ91EfMkaU0S99BcwZ7zfj4CdJkiKr9V16mCARLXh yRAA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:sender:dkim-signature; bh=40uXzXSqq0DnD+u5GPsgH4bN6ZTIaIFvK8Qv3W13lls=; fh=nI+dsOIsiN+OzCzdMu2TmfKB7GsmWOOTQ5L5/S/M4u0=; b=vilpDKy+HzTVfk2UWxO0JELZUdGbKpY6MjswDWMAn9L/DHU3orXHd/3DmzG8Zdl3AI SvqMY29bl83AtJx81eEiePlD7iRM5qSk2R6qCpc6Y0WCcaUHsTzpQrYefV4JEoBX4XOp rm9rsLUQqs7wlCJgDajpqMRkeSnaJFsDDw5ylvhZkcTgipqwoNAOHdDByN69as1snx3W sR1tjqnHZrS79DOhf0JP6JAiYqnukMlV/dUUEybpgEYQjEv2DVaGQmuvxKGMpX+WoRnx S1cMApgnVbUYML22xLExe2cukcfXqYssDL/6RGZSmJFux6reouVUMpiCBQ5i75DNcrKD verQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=JeznFtxx; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69125-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69125-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id t26-20020ac5c97a000000b004c0293f50d1si197959vkm.249.2024.02.16.10.08.00 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:08:00 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-69125-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=JeznFtxx; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69125-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69125-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 7F1F31C21A0F for ; Fri, 16 Feb 2024 18:08:00 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6FE8D1350E1; Fri, 16 Feb 2024 18:06:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JeznFtxx" Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D4DF135A77 for ; Fri, 16 Feb 2024 18:06:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106788; cv=none; b=LfbPtMSyotTR3/dRl3kWPIcsJQ03G0/Lf17FRn6JEMPVt54cayfc/ujKGYvaYb+TVivtsgMCLEAZrHfn9fFsGht+jWV26JLZQrjtMXmy20nN+dpSM/9cMN2cbMQhoVaQtZOA5mwor0w1JIdQW5WozM5OUqMsxmGC0htl5GX5U8c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106788; c=relaxed/simple; bh=bIcLEIHssCXg2J3Y2ia1/tsZ3YGJRV1EiIfHehEmTOs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lRRAWOS/Cmbbqggat6+g/OWMkkREM5fNmWapQpdyg1SgXHSG8uKltEoTRfSbNy81QW+pZxHwi8SYxUlM2XQT9dGyougwbBgsT/c9ZFSEC6ivhT5sLbexJFYNwRIFsQcxyxTSeXMz6uvCN8UhA6F/uhvTJCeaWSFIwB//cC44+04= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=JeznFtxx; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-1d780a392fdso9993165ad.3 for ; Fri, 16 Feb 2024 10:06:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708106785; x=1708711585; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=40uXzXSqq0DnD+u5GPsgH4bN6ZTIaIFvK8Qv3W13lls=; b=JeznFtxxfTcOzvQuCIb5U9OySI6Q8VfS9TgnrAWArD7PLLiJesM4ELjamePRhFZuZq QoSiPweRdQIc9x5NAJMWnWjALNs80k7KciWEpTvGu8c12JXZPZyYrQado9IRnQ/R2b2e D0KdLq8FlZsi7sR8eYx78XScO9E9Zo/TYZIWB1qNT+hCvgNRvjTC4bmiYX/goSWEhw1q BdpIl1iXfK1ynHQ2ItEfGsc2yUv2QWJEgWQvzATV3YXLrS6v3n0kavR/mHslUe7dwNCl 6iPm9bHkeiKx1XZ9meNt/EQ3PSF1xbwmwH5+WW1ztGTc+w9otev8Tf3/NT8PcT36Qa6d ni1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708106785; x=1708711585; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=40uXzXSqq0DnD+u5GPsgH4bN6ZTIaIFvK8Qv3W13lls=; b=OlZmflhJ6gZqlb52EA+eOM2KCZ2FAQhjykp+CljJL5Iy767DCkA53Dw35mAwfy0W+W tzpj6Y/u99HMg7iRe/FfkFf+TKrArPZKSF75yX6GSdvCHJSyWfsUqk2XZNKrR3wDHq+Z G3RPxvSvNU+zzdZlEYICbdJF0y94lR0RhY4BKlL5VlOh4Ac6SdAIoxh0Cz6MQCAYhsCO ovuzz978EcfajRfvnYccdxKVYBXSLzDTNdxspmPaUFE1y32frfEVkC9EKKGqx4wIL9de HK8cGa2G1DZgsVGlBKsIL2huAoqM2E24oa41FU9j5IIGWU9dAF5N9++5UZR+9JR4yqSU XC+A== X-Forwarded-Encrypted: i=1; AJvYcCUEcJZlBrGBdzl0ro0YXwYmb9oILgsS237J+DN4B2YZc3khdnnVCFfpudqbhYvOPO1EuRjn1nUoLZFTrj4yNh0UWovfxNN65bhVD9RS X-Gm-Message-State: AOJu0YwHgg+PmigzSRH0way3DgJ+9gHvwxwM90Z3TuvpmUrzzJavyCJz 7lMzepi3L746CJH5MmzFRkt79QlNogSJxFsPzEK5jW2k9PckV4FCB6LEmNl4Ygw= X-Received: by 2002:a17:902:d4c3:b0:1d9:cb99:da29 with SMTP id o3-20020a170902d4c300b001d9cb99da29mr7088781plg.14.1708106785546; Fri, 16 Feb 2024 10:06:25 -0800 (PST) Received: from localhost (dhcp-141-239-158-86.hawaiiantel.net. [141.239.158.86]) by smtp.gmail.com with ESMTPSA id v3-20020a170902d08300b001db6c3a192bsm163016plv.26.2024.02.16.10.06.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:06:25 -0800 (PST) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, allen.lkml@gmail.com, kernel-team@meta.com, Tejun Heo Subject: [PATCH 07/17] workqueue: Clean up enum work_bits and related constants Date: Fri, 16 Feb 2024 08:04:48 -1000 Message-ID: <20240216180559.208276-8-tj@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240216180559.208276-1-tj@kernel.org> References: <20240216180559.208276-1-tj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791079880859562065 X-GMAIL-MSGID: 1791079880859562065 The bits of work->data are used for a few different purposes. How the bits are used is determined by enum work_bits. The planned disable/enable support will add another use, so let's clean it up a bit in preparation. - Let WORK_STRUCT_*_BIT's values be determined by enum definition order. - Deliminate different bit sections the same way using SHIFT and BITS values. - Rename __WORK_OFFQ_CANCELING to WORK_OFFQ_CANCELING_BIT for consistency. - Introduce WORK_STRUCT_PWQ_SHIFT and replace WORK_STRUCT_FLAG_MASK and WORK_STRUCT_WQ_DATA_MASK with WQ_STRUCT_PWQ_MASK for clarity. - Improve documentation. No functional changes. Signed-off-by: Tejun Heo --- include/linux/workqueue.h | 58 +++++++++++++++++++++------------------ kernel/workqueue.c | 8 +++--- 2 files changed, 36 insertions(+), 30 deletions(-) diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h index 1565bab9edc8..0ad534fe6673 100644 --- a/include/linux/workqueue.h +++ b/include/linux/workqueue.h @@ -24,41 +24,49 @@ enum work_bits { WORK_STRUCT_PENDING_BIT = 0, /* work item is pending execution */ - WORK_STRUCT_INACTIVE_BIT= 1, /* work item is inactive */ - WORK_STRUCT_PWQ_BIT = 2, /* data points to pwq */ - WORK_STRUCT_LINKED_BIT = 3, /* next work is linked to this one */ + WORK_STRUCT_INACTIVE_BIT, /* work item is inactive */ + WORK_STRUCT_PWQ_BIT, /* data points to pwq */ + WORK_STRUCT_LINKED_BIT, /* next work is linked to this one */ #ifdef CONFIG_DEBUG_OBJECTS_WORK - WORK_STRUCT_STATIC_BIT = 4, /* static initializer (debugobjects) */ - WORK_STRUCT_COLOR_SHIFT = 5, /* color for workqueue flushing */ -#else - WORK_STRUCT_COLOR_SHIFT = 4, /* color for workqueue flushing */ + WORK_STRUCT_STATIC_BIT, /* static initializer (debugobjects) */ #endif + WORK_STRUCT_FLAG_BITS, + /* color for workqueue flushing */ + WORK_STRUCT_COLOR_SHIFT = WORK_STRUCT_FLAG_BITS, WORK_STRUCT_COLOR_BITS = 4, /* - * Reserve 8 bits off of pwq pointer w/ debugobjects turned off. - * This makes pwqs aligned to 256 bytes and allows 16 workqueue - * flush colors. + * When WORK_STRUCT_PWQ is set, reserve 8 bits off of pwq pointer w/ + * debugobjects turned off. This makes pwqs aligned to 256 bytes (512 + * bytes w/ DEBUG_OBJECTS_WORK) and allows 16 workqueue flush colors. + * + * MSB + * [ pwq pointer ] [ flush color ] [ STRUCT flags ] + * 4 bits 4 or 5 bits */ - WORK_STRUCT_FLAG_BITS = WORK_STRUCT_COLOR_SHIFT + - WORK_STRUCT_COLOR_BITS, + WORK_STRUCT_PWQ_SHIFT = WORK_STRUCT_COLOR_SHIFT + WORK_STRUCT_COLOR_BITS, - /* data contains off-queue information when !WORK_STRUCT_PWQ */ - WORK_OFFQ_FLAG_BASE = WORK_STRUCT_COLOR_SHIFT, - - __WORK_OFFQ_CANCELING = WORK_OFFQ_FLAG_BASE, + /* + * data contains off-queue information when !WORK_STRUCT_PWQ. + * + * MSB + * [ pool ID ] [ OFFQ flags ] [ STRUCT flags ] + * 1 bit 4 or 5 bits + */ + WORK_OFFQ_FLAG_SHIFT = WORK_STRUCT_FLAG_BITS, + WORK_OFFQ_CANCELING_BIT = WORK_OFFQ_FLAG_SHIFT, + WORK_OFFQ_FLAG_END, + WORK_OFFQ_FLAG_BITS = WORK_OFFQ_FLAG_END - WORK_OFFQ_FLAG_SHIFT, /* - * When a work item is off queue, its high bits point to the last - * pool it was on. Cap at 31 bits and use the highest number to - * indicate that no pool is associated. + * When a work item is off queue, the high bits encode off-queue flags + * and the last pool it was on. Cap pool ID to 31 bits and use the + * highest number to indicate that no pool is associated. */ - WORK_OFFQ_FLAG_BITS = 1, - WORK_OFFQ_POOL_SHIFT = WORK_OFFQ_FLAG_BASE + WORK_OFFQ_FLAG_BITS, + WORK_OFFQ_POOL_SHIFT = WORK_OFFQ_FLAG_SHIFT + WORK_OFFQ_FLAG_BITS, WORK_OFFQ_LEFT = BITS_PER_LONG - WORK_OFFQ_POOL_SHIFT, WORK_OFFQ_POOL_BITS = WORK_OFFQ_LEFT <= 31 ? WORK_OFFQ_LEFT : 31, - }; enum work_flags { @@ -88,12 +96,10 @@ enum wq_misc_consts { }; /* Convenience constants - of type 'unsigned long', not 'enum'! */ -#define WORK_OFFQ_CANCELING (1ul << __WORK_OFFQ_CANCELING) +#define WORK_OFFQ_CANCELING (1ul << WORK_OFFQ_CANCELING_BIT) #define WORK_OFFQ_POOL_NONE ((1ul << WORK_OFFQ_POOL_BITS) - 1) #define WORK_STRUCT_NO_POOL (WORK_OFFQ_POOL_NONE << WORK_OFFQ_POOL_SHIFT) - -#define WORK_STRUCT_FLAG_MASK ((1ul << WORK_STRUCT_FLAG_BITS) - 1) -#define WORK_STRUCT_WQ_DATA_MASK (~WORK_STRUCT_FLAG_MASK) +#define WORK_STRUCT_PWQ_MASK (~((1ul << WORK_STRUCT_PWQ_SHIFT) - 1)) #define WORK_DATA_INIT() ATOMIC_LONG_INIT((unsigned long)WORK_STRUCT_NO_POOL) #define WORK_DATA_STATIC_INIT() \ diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 317c85f051b0..7c6915e23c5c 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -247,7 +247,7 @@ enum pool_workqueue_stats { }; /* - * The per-pool workqueue. While queued, the lower WORK_STRUCT_FLAG_BITS + * The per-pool workqueue. While queued, bits below WORK_PWQ_SHIFT * of work_struct->data are used for flags and the remaining high bits * point to the pwq; thus, pwqs need to be aligned at two's power of the * number of flag bits. @@ -294,7 +294,7 @@ struct pool_workqueue { */ struct kthread_work release_work; struct rcu_head rcu; -} __aligned(1 << WORK_STRUCT_FLAG_BITS); +} __aligned(1 << WORK_STRUCT_PWQ_SHIFT); /* * Structure used to wait for workqueue flush. @@ -843,7 +843,7 @@ static void clear_work_data(struct work_struct *work) static inline struct pool_workqueue *work_struct_pwq(unsigned long data) { - return (struct pool_workqueue *)(data & WORK_STRUCT_WQ_DATA_MASK); + return (struct pool_workqueue *)(data & WORK_STRUCT_PWQ_MASK); } static struct pool_workqueue *get_work_pwq(struct work_struct *work) @@ -4851,7 +4851,7 @@ static void pwq_release_workfn(struct kthread_work *work) static void init_pwq(struct pool_workqueue *pwq, struct workqueue_struct *wq, struct worker_pool *pool) { - BUG_ON((unsigned long)pwq & WORK_STRUCT_FLAG_MASK); + BUG_ON((unsigned long)pwq & ~WORK_STRUCT_PWQ_MASK); memset(pwq, 0, sizeof(*pwq)); From patchwork Fri Feb 16 18:04:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 202288 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:c619:b0:108:e6aa:91d0 with SMTP id hn25csp690350dyb; Fri, 16 Feb 2024 10:08:18 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWCfkSW1DTUZY9d/9PcKCE5Q1T7qFwFmLbeo31fs8+53CeJCuK/XHDSSyXo2/yFtb+dRN2OXx4RibpzlOof0rdqPKhi2w== X-Google-Smtp-Source: AGHT+IEMBVzw0UaMqZl6IRTWRwhUDTLsJHRiOxUdMGEx2fC/1+LUCJEOKnryTILn0ewv6088uAqW X-Received: by 2002:a17:902:6bc1:b0:1d9:a780:393d with SMTP id m1-20020a1709026bc100b001d9a780393dmr5731815plt.8.1708106898476; Fri, 16 Feb 2024 10:08:18 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708106898; cv=pass; d=google.com; s=arc-20160816; b=qVD1/9/goMrtm4W+dYmtKcUXT26WHosd1NmVq2pLiZFszukBCoepSbJDnN61mqxPWr PwOHQaGhcL0ejeiEDm5exI34nNDG1kla7mc86nrO1gvO3jL9QDYOlCee+esvhiNt6qw6 /H1wRu72N4kpPqhJkzGozQrBSkf6qLkSuMzdJFcigUd2CGmXcbU/P/baQBpF2b2pcFDK GpFPUk576b2Z34YccWwOjnOtxt2ohNZI5PiZl/A9mM4HzgqHf4/v3KoMA8E38HDZjMxI tFRaXVhvb8aG4ib8w50NlVoCxXPW26sLdihb+VHzYBjDs+nw/GrGzyqyUJxnhjR3dAGu ZW9g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:sender:dkim-signature; bh=UWQ++TfiXPPbsr6vtxYlRZHmraPzLnchssMNAsSwn3w=; fh=Td3tT+5OO8cDYnF025SWsy3SrXeu5WqN7JhI36ut0jg=; b=zvtRk9oIXlaUMP5yrW55oHsd53s1BkNLYiCR87JuFfmuYtwk5ZyVjnAc1bg7DO8zX7 SUJwFOz6saZS9Oez3fWdCgYiHHj9ziseJdgnQxD6v5zE5/4k3OTFJQ95RMsIeQQjtOoE Ypiii/OuFD0BQqtbwAs9rx1LYTShykBzonkiuOhkIfn3LRLARx5N8BRe4MIXKrJgocWw QJnN+LtSz7urrjgGQG7CHqevP6XbsOxb6IRI3W3dBTvkF/o6KqDOE5DB840dePDglah9 vz9RxHSn+dNnkwlEQFtMyxXyXObr5CmuftwoSYfWzPW7iyflJpm2HhKvhpALJwwWmA0L qfyg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=QlKqpYad; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69126-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69126-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id lv15-20020a1709032a8f00b001db3bb0647esi200074plb.605.2024.02.16.10.08.18 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:08:18 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-69126-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=QlKqpYad; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69126-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69126-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 45EBA283AB4 for ; Fri, 16 Feb 2024 18:08:18 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E63DE13958F; Fri, 16 Feb 2024 18:06:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QlKqpYad" Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D2E51137C28 for ; Fri, 16 Feb 2024 18:06:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106790; cv=none; b=hGcmYtJidvzlzq4xQvgirOZsU66581OVY0Oiar3PWHnNW/BgtaDmLooLJAp7ou9Om88JJUAFJX4eCTTfIEmnR5nc9aMuvqst/LGIrutET4Mk7ADj9bqmUCWSd47QYM9Jl/Hj+98tY/h5TRE6tTK+lqcppMiSPrh1W1SlX9p4ryE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106790; c=relaxed/simple; bh=ylqgSgPh9dnX0Y5CbOgQcNyBm4zxKG0QVJ5yrKucFnQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=heJTQJAJZpeyN76/iHDwLjbrbnxJdu6VMDggTQ72pgEy/KiR6O8GoDWFo1fHFVQ4NBDkKlIVYz+aI98PEv89EPka0TOJ09QaNjNdY/+SsRE5Ycqa7VFCJeVT3lupTeko6CFHUveD+pzgsUbJntm2s6JXNLRmJHtgvYMvzTVJwWM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=QlKqpYad; arc=none smtp.client-ip=209.85.210.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-6e10303de41so989154b3a.1 for ; Fri, 16 Feb 2024 10:06:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708106788; x=1708711588; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=UWQ++TfiXPPbsr6vtxYlRZHmraPzLnchssMNAsSwn3w=; b=QlKqpYadqdu1OE0HXevKB/xzBHYXZJZw215R7IbTPkTMm6n+5QI8HdlXukJznp6oQX vQNyoWjGF4UW33BmF9a4VM1+snohEsHnoZSADVJ+vdRzwcMoN11k7aeXUgoHI6FuJmd7 reXZ3wjO/IYMBw66xlMtBQSsIpxOHWcUpm0ig7FQoa3kCjzNKI6tzLYmSMzO6i9QimPv bijzNAVTCGNzZMQVXC6ald/74VC/wJchGSdGC2fea0kJ7lxh/KSeSGRCzkdwuIhDBb3n ILJ/LrlJ+p8jwjpgWcNQ3At0lTWRXfMujvwKBWmFmO7t17qndYUX+1VCBVSrUjnNO/DE 6Vdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708106788; x=1708711588; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=UWQ++TfiXPPbsr6vtxYlRZHmraPzLnchssMNAsSwn3w=; b=EMA6DJy4G+y3VybOH5qiX75MjgiTvs57Gd+wsTSbsoEkX2D4JGnqEdnyJS8D7Ubpzh M4PicEi2j+Is5zT6RvoRJ4ceInaq6idZtlCXfxJDosJjK+rlrI6sVJzcSbPVL16J4zs8 w/GYmIvPNLdVkBt8ig9huXYBrYhWeT1imoUVxDrmPeivm3ll4KBvQ7b9154hgNBVGp6V umEIqUAZmeMNtExIfrSREsJjNNBiLMs0mu7XqNQtWeE/nCYJIwY5Jn1SWKyMMaAXJd1G KPj0RtSozqVcMVJGCVlevvGuM7bprccb2LzAaUHGoCQ8UMPV0BasUZUqjcJckd0SuW4z rD2w== X-Forwarded-Encrypted: i=1; AJvYcCUVWaQY6b4CE9WKWFT9f1UAlmyjTkQf3/AftWO+/ABnx0WjVzwHglI1x66pBAwEZ16lYU4DRpwIVIQ10t7wVpg1qHPvzsCjifzxbOJk X-Gm-Message-State: AOJu0YzNltl0isbRZ0hCqrRM4ZapqjOQa74dPnadzDQojbL6kbZOrDpl sF9cP4x8RNxFBBMR9ZV7W4Rs0ZbcV8IY+XzrEvhlaairDY3E45Ia X-Received: by 2002:a05:6a21:3511:b0:19e:bc68:3c67 with SMTP id zc17-20020a056a21351100b0019ebc683c67mr7060998pzb.33.1708106788020; Fri, 16 Feb 2024 10:06:28 -0800 (PST) Received: from localhost (dhcp-141-239-158-86.hawaiiantel.net. [141.239.158.86]) by smtp.gmail.com with ESMTPSA id o23-20020a056a001b5700b006e025ce0beesm232976pfv.168.2024.02.16.10.06.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:06:27 -0800 (PST) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, allen.lkml@gmail.com, kernel-team@meta.com, Tejun Heo Subject: [PATCH 08/17] workqueue: Factor out work_grab_pending() from __cancel_work_sync() Date: Fri, 16 Feb 2024 08:04:49 -1000 Message-ID: <20240216180559.208276-9-tj@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240216180559.208276-1-tj@kernel.org> References: <20240216180559.208276-1-tj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791079899324713922 X-GMAIL-MSGID: 1791079899324713922 The planned disable/enable support will need the same logic. Let's factor it out. No functional changes. Signed-off-by: Tejun Heo --- kernel/workqueue.c | 130 +++++++++++++++++++++++++++------------------ 1 file changed, 78 insertions(+), 52 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 7c6915e23c5c..38e589b6871c 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -484,6 +484,12 @@ static struct workqueue_attrs *unbound_std_wq_attrs[NR_STD_WORKER_POOLS]; /* I: attributes used when instantiating ordered pools on demand */ static struct workqueue_attrs *ordered_wq_attrs[NR_STD_WORKER_POOLS]; +/* + * Used to synchronize multiple cancel_sync attempts on the same work item. See + * work_grab_pending() and __cancel_work_sync(). + */ +static DECLARE_WAIT_QUEUE_HEAD(wq_cancel_waitq); + /* * I: kthread_worker to release pwq's. pwq release needs to be bounced to a * process context while holding a pool lock. Bounce to a dedicated kthread @@ -2147,6 +2153,73 @@ static int try_to_grab_pending(struct work_struct *work, u32 cflags, return -EAGAIN; } +struct cwt_wait { + wait_queue_entry_t wait; + struct work_struct *work; +}; + +static int cwt_wakefn(wait_queue_entry_t *wait, unsigned mode, int sync, void *key) +{ + struct cwt_wait *cwait = container_of(wait, struct cwt_wait, wait); + + if (cwait->work != key) + return 0; + return autoremove_wake_function(wait, mode, sync, key); +} + +/** + * work_grab_pending - steal work item from worklist and disable irq + * @work: work item to steal + * @cflags: %WORK_CANCEL_ flags + * + * Grab PENDING bit of @work. @work can be in any stable state - idle, on timer + * or on worklist. + * + * Must be called in process context. IRQ is disabled on return. The caller is + * responsible for re-enabling it using local_irq_enable(). + * + * Returns %true if @work was pending. %false if idle. + */ +static bool work_grab_pending(struct work_struct *work, u32 cflags, + unsigned long *irq_flags) +{ + struct cwt_wait cwait; + int ret; + + might_sleep(); +repeat: + ret = try_to_grab_pending(work, cflags, irq_flags); + if (likely(ret >= 0)) + return ret; + if (ret != -ENOENT) + goto repeat; + + /* + * Someone is already canceling. Wait for it to finish. flush_work() + * doesn't work for PREEMPT_NONE because we may get woken up between + * @work's completion and the other canceling task resuming and clearing + * CANCELING - flush_work() will return false immediately as @work is no + * longer busy, try_to_grab_pending() will return -ENOENT as @work is + * still being canceled and the other canceling task won't be able to + * clear CANCELING as we're hogging the CPU. + * + * Let's wait for completion using a waitqueue. As this may lead to the + * thundering herd problem, use a custom wake function which matches + * @work along with exclusive wait and wakeup. + */ + init_wait(&cwait.wait); + cwait.wait.func = cwt_wakefn; + cwait.work = work; + + prepare_to_wait_exclusive(&wq_cancel_waitq, &cwait.wait, + TASK_UNINTERRUPTIBLE); + if (work_is_canceling(work)) + schedule(); + finish_wait(&wq_cancel_waitq, &cwait.wait); + + goto repeat; +} + /** * insert_work - insert a work into a pool * @pwq: pwq @work belongs to @@ -4125,60 +4198,13 @@ static bool __cancel_work(struct work_struct *work, u32 cflags) return ret; } -struct cwt_wait { - wait_queue_entry_t wait; - struct work_struct *work; -}; - -static int cwt_wakefn(wait_queue_entry_t *wait, unsigned mode, int sync, void *key) -{ - struct cwt_wait *cwait = container_of(wait, struct cwt_wait, wait); - - if (cwait->work != key) - return 0; - return autoremove_wake_function(wait, mode, sync, key); -} - static bool __cancel_work_sync(struct work_struct *work, u32 cflags) { - static DECLARE_WAIT_QUEUE_HEAD(cancel_waitq); unsigned long irq_flags; - int ret; - - do { - ret = try_to_grab_pending(work, cflags, &irq_flags); - /* - * If someone else is already canceling, wait for it to - * finish. flush_work() doesn't work for PREEMPT_NONE - * because we may get scheduled between @work's completion - * and the other canceling task resuming and clearing - * CANCELING - flush_work() will return false immediately - * as @work is no longer busy, try_to_grab_pending() will - * return -ENOENT as @work is still being canceled and the - * other canceling task won't be able to clear CANCELING as - * we're hogging the CPU. - * - * Let's wait for completion using a waitqueue. As this - * may lead to the thundering herd problem, use a custom - * wake function which matches @work along with exclusive - * wait and wakeup. - */ - if (unlikely(ret == -ENOENT)) { - struct cwt_wait cwait; - - init_wait(&cwait.wait); - cwait.wait.func = cwt_wakefn; - cwait.work = work; - - prepare_to_wait_exclusive(&cancel_waitq, &cwait.wait, - TASK_UNINTERRUPTIBLE); - if (work_is_canceling(work)) - schedule(); - finish_wait(&cancel_waitq, &cwait.wait); - } - } while (unlikely(ret < 0)); + bool ret; - /* tell other tasks trying to grab @work to back off */ + /* claim @work and tell other tasks trying to grab @work to back off */ + ret = work_grab_pending(work, cflags, &irq_flags); mark_work_canceling(work); local_irq_restore(irq_flags); @@ -4197,8 +4223,8 @@ static bool __cancel_work_sync(struct work_struct *work, u32 cflags) * visible there. */ smp_mb(); - if (waitqueue_active(&cancel_waitq)) - __wake_up(&cancel_waitq, TASK_NORMAL, 1, work); + if (waitqueue_active(&wq_cancel_waitq)) + __wake_up(&wq_cancel_waitq, TASK_NORMAL, 1, work); return ret; } From patchwork Fri Feb 16 18:04:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 202289 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:c619:b0:108:e6aa:91d0 with SMTP id hn25csp690465dyb; Fri, 16 Feb 2024 10:08:27 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCX3ZRXBrguAZRYqhkgRs2ypK5h4vEX9fEZWnQJMavuGEBWk/BQapUE286yhzFmIA9c37lwhbf7KJaT8uB/+PWbzuQ5gyw== X-Google-Smtp-Source: AGHT+IGjq1vrj4Oip7uDoeLRxkCFM2I0GQoHJMjISfp1FeZCkpsEqaSOzT+nGgHkvYGpbaNm1yYi X-Received: by 2002:ac2:44a3:0:b0:512:9dcb:f6cc with SMTP id c3-20020ac244a3000000b005129dcbf6ccmr472817lfm.46.1708106907023; Fri, 16 Feb 2024 10:08:27 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708106906; cv=pass; d=google.com; s=arc-20160816; b=uL+tEAmI48FJiyta1bWVDuRnubogov2m553cCx3d7I6Yu5XWWWPw8PFbM/XFAaIcIv iP/n4+Fekmw3nYqkG6uxUB2Mg0Kr7ePIWDHGx6YrCV9F5VfqTJA4ZWTRCOEIA+EcnLNX ZQqaMNEVecB2rxtNibwhsOWXZ6bjS2APXKEK8+TUHJ8fVqIoitlCaGBYgsYtCAlBE7gR m3aj+C2fbNi5SEfPGD3oNvLZuXRiz/Qq/Y/9t7AcSfLvZ2PSFenIo8TAEPiWgdoaz9Va hsakekUcxHtmFzaEJ4c+74UD3bbzGH4zdqZ0nDmkpd07GND8GA/kmZEiup8Agfbd7Yfa KsXQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:sender:dkim-signature; bh=B89hT+vvVK1kAIqQN3SycMfFOJPiZ6Bixp7yRrf8y94=; fh=TgcgF59dym1/qllkDH1MaOSiSPfHfvAx1paTMCCpN/8=; b=xEFs6HvOpic5d4v/D8aK9uEaogGMoeCjFYD6aw5/2sl4GCb3NuVZyvmQ2aW+8z+3OI S6NHNz7kV2M/iN6iXWpjINfjWsV1R7nApqW08cWRwYZRVaKFr1+tRhvzbgontL1jIyth lcxhXa2k+IGJavEKEIp6nedDgL/MuxG8fOnuqCa75JkV7wmYd9gwSL1tnHJAS6ZH4vWY Y1OCY6nLDQ9jUOxXQngj7iuHaT6Q2zvr+uGo0sWQTsg5GKLJgmXH2UOVYUq/FvO1CojW +pvUj6SxeuqQaxQdKnzjQQuQMgRZvQWfuFDC8ZdsWcegw1q/pA/pjnyYYlKyJHZVdbP9 7cDQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=G28AetZ0; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69127-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69127-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id v1-20020a056402184100b0056173831870si171367edy.321.2024.02.16.10.08.26 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:08:26 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-69127-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=G28AetZ0; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69127-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69127-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 787D91F21629 for ; Fri, 16 Feb 2024 18:08:26 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C838A13AA53; Fri, 16 Feb 2024 18:06:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="G28AetZ0" Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8376A1350F6 for ; Fri, 16 Feb 2024 18:06:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106792; cv=none; b=Hs/vZ4cDjH6GD1QcO+FW1RKQw3ZpOSzEii+AdidM6xxhKVdHtIbGy8WuQgP/aHndriXVYLv55cIR/KBEZTAuoCh0K/2D1NbJKBVzcC+pzHLtx1XpL9DCq4ctnYtpp0R+gsmZRs/o4DJm6KoAxkY6om8cDhmOSUHT3WCdvLOw9dY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106792; c=relaxed/simple; bh=FJO8f8DVbJeanF+hpUj3gFVOJ/tGxb7K3WHo8qwoQnU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XV8ZBu3Coju1Ph+diiGFuF7KGCD6GtDP6/0DSLI1SwuyucZ2wbvLZoD5+Z5h+CQctlKF83/xZ7LWRvTAhjLKXxbJOn7YI4MFPrZEU4LGrvRH9JpAzW7zOiQecM8KYFalQW2cKMXChG/qICyRI3TTRlG5x9lEivJsRt+ZImEOBMI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=G28AetZ0; arc=none smtp.client-ip=209.85.210.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-6e0f803d9dfso738749b3a.0 for ; Fri, 16 Feb 2024 10:06:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708106790; x=1708711590; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=B89hT+vvVK1kAIqQN3SycMfFOJPiZ6Bixp7yRrf8y94=; b=G28AetZ05W+dKbDim0E5x30azyX3/ONa4nCzBeWE+BwM1qjO80c7MLvRUrGrii4KcO ywPtZP9M6SlwMrvmr+WDvQnKfZVNCORFlIqJ36/3+dc0/MJVAyD/+lwQg3aexQt4nUz+ 8x4derTTfsMtKW546d1B3TItUX/13kTntra89b46AQ947XqJFibaQU+76KMYh76+JxOv nLvbkmSrt0qaTt5ebFXEMXwmqbwPTweP1x2XDqnDm0NHtMMvrFMg8CMtdn0NfJ0DURKj Kyrz1fghGfAWnLpQiZR+4Mtzs+eSWI2rz6eTa7/p6FES4h8k2Q7VndeYO1z0+2AjBImp mDDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708106790; x=1708711590; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=B89hT+vvVK1kAIqQN3SycMfFOJPiZ6Bixp7yRrf8y94=; b=Pdsc+bgHBLOTxsZVygCc9nu1DNSwixeSdZlMRyrMjCrS4tekrtkSTlbwozg+cL7pSw IP4IhFfaJFMBe7nf2EbEo2loujFHpEE5TG7RX8Uk1zHJ99y3DPwqGGiM+w+YKT/zTM8Y NN2Mrk/GI44UWIwY0NdxCaYNucDRJIJrxqOGw8O+UeM3sLfW4n9hzp82PqMcqJDTQsQU Fakfk3Q/Ph/vTk6GnGj7GpueMknyS7LO2atvtGLCX3+XBsGIQSAA61YpqcU2r/pdg78m +cHkJXr2N9QrvsFN8TSu9JTPqg7KmEuLVgBMkwarEaJ6b9uh3fJza+COMlzcCwI3tnu6 0RoQ== X-Forwarded-Encrypted: i=1; AJvYcCUr5Y93zg2tHlaYW+DBBcsIfNz6bY+gulmQs7JUK4PI+Z1Xr2kk6H8XvAvhZcMbI0Fvc7lDbZPXPDhvijVxfr/JyMTHhCehLqXRQ7rY X-Gm-Message-State: AOJu0YyNtwZBd+gQTmehITJzwEezw/m6MW4IfiBdE46VKRNgfZKWCDNM FpEgcewxNjIREoWoDPjnW2nBrviY1rI/19Mb0N9HtQ9F0leJ7x9NYR6OTP9PcpI= X-Received: by 2002:a05:6a20:be93:b0:1a0:6856:d12c with SMTP id gf19-20020a056a20be9300b001a06856d12cmr4149085pzb.18.1708106789781; Fri, 16 Feb 2024 10:06:29 -0800 (PST) Received: from localhost (dhcp-141-239-158-86.hawaiiantel.net. [141.239.158.86]) by smtp.gmail.com with ESMTPSA id v7-20020aa78507000000b006e13ce6f4d8sm238055pfn.72.2024.02.16.10.06.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:06:29 -0800 (PST) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, allen.lkml@gmail.com, kernel-team@meta.com, Tejun Heo Subject: [PATCH 09/17] workqueue: Remove clear_work_data() Date: Fri, 16 Feb 2024 08:04:50 -1000 Message-ID: <20240216180559.208276-10-tj@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240216180559.208276-1-tj@kernel.org> References: <20240216180559.208276-1-tj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791079907807965277 X-GMAIL-MSGID: 1791079907807965277 clear_work_data() is only used in one place and immediately followed by smp_mb(), making it equivalent to set_work_pool_and_clear_pending() w/ WORK_OFFQ_POOL_NONE for @pool_id. Drop it. No functional changes. Signed-off-by: Tejun Heo --- kernel/workqueue.c | 24 ++++++++---------------- 1 file changed, 8 insertions(+), 16 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 38e589b6871c..ea53b53f8ff9 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -763,10 +763,9 @@ static int work_next_color(int color) * contain the pointer to the queued pwq. Once execution starts, the flag * is cleared and the high bits contain OFFQ flags and pool ID. * - * set_work_pwq(), set_work_pool_and_clear_pending(), mark_work_canceling() - * and clear_work_data() can be used to set the pwq, pool or clear - * work->data. These functions should only be called while the work is - * owned - ie. while the PENDING bit is set. + * set_work_pwq(), set_work_pool_and_clear_pending() and mark_work_canceling() + * can be used to set the pwq, pool or clear work->data. These functions should + * only be called while the work is owned - ie. while the PENDING bit is set. * * get_work_pool() and get_work_pwq() can be used to obtain the pool or pwq * corresponding to a work. Pool is available once the work has been @@ -841,12 +840,6 @@ static void set_work_pool_and_clear_pending(struct work_struct *work, smp_mb(); } -static void clear_work_data(struct work_struct *work) -{ - smp_wmb(); /* see set_work_pool_and_clear_pending() */ - set_work_data(work, WORK_STRUCT_NO_POOL, 0); -} - static inline struct pool_workqueue *work_struct_pwq(unsigned long data) { return (struct pool_workqueue *)(data & WORK_STRUCT_PWQ_MASK); @@ -4215,14 +4208,13 @@ static bool __cancel_work_sync(struct work_struct *work, u32 cflags) if (wq_online) __flush_work(work, true); - clear_work_data(work); - /* - * Paired with prepare_to_wait() above so that either - * waitqueue_active() is visible here or !work_is_canceling() is - * visible there. + * smp_mb() at the end of set_work_pool_and_clear_pending() is paired + * with prepare_to_wait() above so that either waitqueue_active() is + * visible here or !work_is_canceling() is visible there. */ - smp_mb(); + set_work_pool_and_clear_pending(work, WORK_OFFQ_POOL_NONE); + if (waitqueue_active(&wq_cancel_waitq)) __wake_up(&wq_cancel_waitq, TASK_NORMAL, 1, work); From patchwork Fri Feb 16 18:04:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 202290 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:c619:b0:108:e6aa:91d0 with SMTP id hn25csp690639dyb; Fri, 16 Feb 2024 10:08:40 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUPLz04Da81K8leEtbfm34VLJn5EFgyRQEKvexq5U9+trcaLXTp9s4KoJ+nRB8U7m3VYQuEGXeLXpMjvEAS+xxkPmR+wQ== X-Google-Smtp-Source: AGHT+IFrwQxb4x2YQR/dH9fFyq4zMizM6pZOCwIJw02lIMioqMNTzzW37M/7pJ2yL2Df1XEPEdDQ X-Received: by 2002:a17:902:e88f:b0:1db:b16c:1076 with SMTP id w15-20020a170902e88f00b001dbb16c1076mr2015493plg.32.1708106920353; Fri, 16 Feb 2024 10:08:40 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708106920; cv=pass; d=google.com; s=arc-20160816; b=oXu9SMYlDQ8lMgl5ryyq3NlC8XSCcOcOUq4tEaTkbSYko3L2IG37ami9xrolJBg4Z/ HkrxWkw2NIYBFOPH0ujA5uE9Xs8VxTSGyT2656hISpwt8LIj2jab6Ui6bydAcXCbCXG2 5NVtMYiJytueVpuncQnew508KB1UIxE8CxWfAghthRa7tYD+zE+GbLFwsXgdFtSKStlv 5BXwk/6stkZqWK5bBXwx14VzJEAti21Y0T7giH8OnzQwBynOP8msg2GKI3Mx4XET+3/r yPt6LFpLbgCCgcy90XZ1s1EPzKlf0BNkwni+hCBGLGRaQUnbOdoZeQOQosBmrqGEYe0T fcBg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:sender:dkim-signature; bh=41qYLxE4dVgv1c5XbTIQWibRRdLdTaJ138CgAqO4goo=; fh=M0a2ZNEMBl1P9/Zmkmmm44pA+ksttnGSmgaEVg1MgxU=; b=Vg0eUsOjliTqqZl4PLyVLUTy8IngXC45Xt2sEMkekaOhUXO31KipdaEyloXksImaqg MxhTEHhFSeOszxn2sXxpzhErcle77/FHiloWB/q3MPUlGJCGWKs2xImNpkGgHUBrwKtN hc/AGFEsqXPaReK9ksgi4X3d5mkbOJ91O7fSS9bMsuu7HaMtmgKNRUCf3V+hvtaUAOqd gZxne4sTpW/zNPpcI8Z48mYSuqUm5Gz7LcWRXzvYUezroKKr4TbcylSrBx8HpQqtoW1O tnpMsQXcnwyE/cZC4fVQxdsA3ogPgtdoN9QwYTZqosxf1aCgRJz8gSOKPJFraYycbbjU ti+w==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=dwxTxdQm; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69128-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69128-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id h13-20020a170902f7cd00b001d5e9f77cf3si217055plw.79.2024.02.16.10.08.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:08:40 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-69128-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=dwxTxdQm; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69128-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69128-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 2C42E28465A for ; Fri, 16 Feb 2024 18:08:40 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5F76713DB8A; Fri, 16 Feb 2024 18:06:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dwxTxdQm" Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BD4E31384B5 for ; Fri, 16 Feb 2024 18:06:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106794; cv=none; b=a3i3d2SxWbrema/Ki8ZmhjAErNxa4DLg//mec9j4jlPqPNsx8klah1g7vT0Qth9RcrFFpqOpNYtU/WrDaKeoWsk3Ar1jBll0jymcSoAQOQU9sQCyYTGvfOVeI9UXe39s8qNuzs0HgIVgHOAgc/dT+Yk6rCaBD30ONc2XxjO4Awk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106794; c=relaxed/simple; bh=cPP5CIKEOYRT6epnHGsws/oABavjhCk4bIN8XpK22Ek=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WKITA9x0/AMymPSzXmhIVmZ9ifyrJprIT9LmSuMm/Bg7/Sz7vxOF4oljAy+JTNygKV8ImlZOvys9BI47VnGQ+xQJdPdze+Qx2V7FREZ6tOB37d1NINW1EipXfBe8L6+LWrY3S8DRYAC0241OXgu2JHn9RVMZVm012b2jQYKfM/g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=dwxTxdQm; arc=none smtp.client-ip=209.85.215.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pg1-f172.google.com with SMTP id 41be03b00d2f7-5d42e7ab8a9so1749892a12.3 for ; Fri, 16 Feb 2024 10:06:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708106792; x=1708711592; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=41qYLxE4dVgv1c5XbTIQWibRRdLdTaJ138CgAqO4goo=; b=dwxTxdQmJBJC9sl+oXNRB5dqMHScmAwNXDuwQ5HwAxO3dtUEq5WsQeBMNXh5SPj9zU wI9Oxn2M3HAnmYilLY8CBVHNgv248RHoJ3HrY16k8xhq0iXMRys6z0arPy+LZP77i0SM HZkqJ9d30iL/jsQgiANS1GCx6gx38qGIb40f/T3Z8on3WE5bPUumbN9T+PQ0pxBE738w drBFM4C4zx8CWTDZi2umIYSt190aKFPPikoHJ8eAfiP/0tU5s/ZRjeomR/nQb/PlNIxG yafc6s/dKcLI/eHCj7dfehY7SLP/QYN5T4Mxu5Pym4qtp7KGOWcmWCSxJPnWURWI1NuL kR8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708106792; x=1708711592; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=41qYLxE4dVgv1c5XbTIQWibRRdLdTaJ138CgAqO4goo=; b=LakPDhblC1j8SXq/6Z1EVd2P3olj6zCG8Q7rjU7mGOMiJPMcA1b1uBIgd2s4agf8UJ SKgfzAnJnEN7ZUjrkDAPQob6PyoYBs9Y9tuXmLg5XLheAEAugzUKWJ+yKQakClBZsHpn QqBs41f0DCbGNOtoBzU62XR6B7telvmH2rJExOmnIbuTTBJUbT4G66coQrXfijHo2hZR qK9p9ynC79cDxwVk9IYPY0uhtFog0H2k622aAaEorKzVrJ/UXMbysD03I8fJR0uRu2S3 nTYEeng6ZqC4UOs/uv6oex4/SFfrKVz7fM067PrVrYGJl42ShA7c9uNWEoVdgwtS6BQI 4iKg== X-Forwarded-Encrypted: i=1; AJvYcCWBXvdTXi1mQPonJeVY3sgTtnjZhwd3kZ/PcNLB0CKvOFXtRAmYiH3QD+egTTCokqsuJyzNXHdTK3GLQ132bMopKobkqPy1fmsuoQ2/ X-Gm-Message-State: AOJu0YzVjM/aYNBvHVYxVr6bgq0FEIT+Ovj8fIJQgZXiTYeXQ+20scgw kB8U4k8hk3vLmh5VLZUaNMDIBmrKB9HRd7Y5w2jW1Ghez1Yxp/Dx X-Received: by 2002:a17:90b:3ecc:b0:299:4e65:7c10 with SMTP id rm12-20020a17090b3ecc00b002994e657c10mr330669pjb.48.1708106791953; Fri, 16 Feb 2024 10:06:31 -0800 (PST) Received: from localhost (dhcp-141-239-158-86.hawaiiantel.net. [141.239.158.86]) by smtp.gmail.com with ESMTPSA id s22-20020a63af56000000b005dc507e8d13sm183390pgo.91.2024.02.16.10.06.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:06:31 -0800 (PST) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, allen.lkml@gmail.com, kernel-team@meta.com, Tejun Heo Subject: [PATCH 10/17] workqueue: Make @flags handling consistent across set_work_data() and friends Date: Fri, 16 Feb 2024 08:04:51 -1000 Message-ID: <20240216180559.208276-11-tj@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240216180559.208276-1-tj@kernel.org> References: <20240216180559.208276-1-tj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791079922027154366 X-GMAIL-MSGID: 1791079922027154366 - set_work_data() takes a separate @flags argument but just ORs it to @data. This is more confusing than helpful. Just take @data. - Use the name @flags consistently and add the parameter to set_work_pool_and_{keep|clear}_pending(). This will be used by the planned disable/enable support. No functional changes. Signed-off-by: Tejun Heo --- kernel/workqueue.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index ea53b53f8ff9..7aa53a2e41af 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -777,29 +777,28 @@ static int work_next_color(int color) * but stay off timer and worklist for arbitrarily long and nobody should * try to steal the PENDING bit. */ -static inline void set_work_data(struct work_struct *work, unsigned long data, - unsigned long flags) +static inline void set_work_data(struct work_struct *work, unsigned long data) { WARN_ON_ONCE(!work_pending(work)); - atomic_long_set(&work->data, data | flags | work_static(work)); + atomic_long_set(&work->data, data | work_static(work)); } static void set_work_pwq(struct work_struct *work, struct pool_workqueue *pwq, - unsigned long extra_flags) + unsigned long flags) { - set_work_data(work, (unsigned long)pwq, - WORK_STRUCT_PENDING | WORK_STRUCT_PWQ | extra_flags); + set_work_data(work, (unsigned long)pwq | WORK_STRUCT_PENDING | + WORK_STRUCT_PWQ | flags); } static void set_work_pool_and_keep_pending(struct work_struct *work, - int pool_id) + int pool_id, unsigned long flags) { - set_work_data(work, (unsigned long)pool_id << WORK_OFFQ_POOL_SHIFT, - WORK_STRUCT_PENDING); + set_work_data(work, ((unsigned long)pool_id << WORK_OFFQ_POOL_SHIFT) | + WORK_STRUCT_PENDING | flags); } static void set_work_pool_and_clear_pending(struct work_struct *work, - int pool_id) + int pool_id, unsigned long flags) { /* * The following wmb is paired with the implied mb in @@ -808,7 +807,8 @@ static void set_work_pool_and_clear_pending(struct work_struct *work, * owner. */ smp_wmb(); - set_work_data(work, (unsigned long)pool_id << WORK_OFFQ_POOL_SHIFT, 0); + set_work_data(work, ((unsigned long)pool_id << WORK_OFFQ_POOL_SHIFT) | + flags); /* * The following mb guarantees that previous clear of a PENDING bit * will not be reordered with any speculative LOADS or STORES from @@ -909,7 +909,7 @@ static void mark_work_canceling(struct work_struct *work) unsigned long pool_id = get_work_pool_id(work); pool_id <<= WORK_OFFQ_POOL_SHIFT; - set_work_data(work, pool_id | WORK_OFFQ_CANCELING, WORK_STRUCT_PENDING); + set_work_data(work, pool_id | WORK_STRUCT_PENDING | WORK_OFFQ_CANCELING); } static bool work_is_canceling(struct work_struct *work) @@ -2127,7 +2127,7 @@ static int try_to_grab_pending(struct work_struct *work, u32 cflags, * this destroys work->data needed by the next step, stash it. */ work_data = *work_data_bits(work); - set_work_pool_and_keep_pending(work, pool->id); + set_work_pool_and_keep_pending(work, pool->id, 0); /* must be the last step, see the function comment */ pwq_dec_nr_in_flight(pwq, work_data); @@ -3203,7 +3203,7 @@ __acquires(&pool->lock) * PENDING and queued state changes happen together while IRQ is * disabled. */ - set_work_pool_and_clear_pending(work, pool->id); + set_work_pool_and_clear_pending(work, pool->id, 0); pwq->stats[PWQ_STAT_STARTED]++; raw_spin_unlock_irq(&pool->lock); @@ -4186,7 +4186,7 @@ static bool __cancel_work(struct work_struct *work, u32 cflags) if (unlikely(ret < 0)) return false; - set_work_pool_and_clear_pending(work, get_work_pool_id(work)); + set_work_pool_and_clear_pending(work, get_work_pool_id(work), 0); local_irq_restore(irq_flags); return ret; } @@ -4213,7 +4213,7 @@ static bool __cancel_work_sync(struct work_struct *work, u32 cflags) * with prepare_to_wait() above so that either waitqueue_active() is * visible here or !work_is_canceling() is visible there. */ - set_work_pool_and_clear_pending(work, WORK_OFFQ_POOL_NONE); + set_work_pool_and_clear_pending(work, WORK_OFFQ_POOL_NONE, 0); if (waitqueue_active(&wq_cancel_waitq)) __wake_up(&wq_cancel_waitq, TASK_NORMAL, 1, work); From patchwork Fri Feb 16 18:04:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 202291 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:c619:b0:108:e6aa:91d0 with SMTP id hn25csp690864dyb; Fri, 16 Feb 2024 10:08:57 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWpMsj8M9pxU+mRfiv8yslwWn0IKbda/aXVr0tAHy5vLd4IIQMkgRQig9lpplwwbeiSy7raTX+d1EjUMziQY+Qes+HP5g== X-Google-Smtp-Source: AGHT+IGvpdX6I8sO3bK4p2I1ZOr1lxY5/YmkQkXrDtDxSbEod2kGA7AilE2YycWNyv1pCckd6qlX X-Received: by 2002:aa7:c1d2:0:b0:561:e8c:791e with SMTP id d18-20020aa7c1d2000000b005610e8c791emr4252135edp.38.1708106936921; Fri, 16 Feb 2024 10:08:56 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708106936; cv=pass; d=google.com; s=arc-20160816; b=NjwHf6GsBWu1Q61h2L1aSv1DXDp32EQN0NvQEA5cFyzxXOrW3keaXFA+lo8TdPyeB1 4fDqWH/EiEUTRT9rMN/9NZ+8GjyYzKYhu490deB6sZvcvGODp7KJ/MFF2QQneU8JmwQq h1knSscSN3xHGyLKZZB44zMagWBzrDQ3NYUmNSV9vHlOzp9sXWGyqd2pGwVJ6u14bhql 8p/kMl1+FUGbeaxMRJjVfA5ko0gBwtuWXM4P0Cp/9yLEovfwLq+vsDAWVkTrEVQZqCpG U6QrYA8UV9zvFZTV04h1kJQGZ4/pxqQ+58WcGgP3GmaBV8rIW5KZs8sLVDc39Y7Sjchc Ixgg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:sender:dkim-signature; bh=JYw17V2FqKp0C6CYJFYt8WJLZzrh9NJi0Sqb0UNwGU0=; fh=kD78iKNaYxGTJ93myJcTHe6intnCxn9XVL7P3BdkWMA=; b=prIE5AmEDP1TKqasUZF9AxcfUBJbaEvmQ6I4Puv0Vkwf0E4fuM9Ejsfrtrtk04XPKM i5tgI232zKDyPasl8j16Y+Pql+GLY4osHixSGyzgIsrToDTZOglsD6nZWt5bKhAwXMdm 7brOMJdLmiPny2R0g1qyNorONW9woQ7orTQpTiB0o0XSltzGPokePhGh5Y4yJBShBb8a GzTTWjxZU5M9xJwKWVaFxJ7CgXmgLjxwoMM5uLfnEPLkF/a2h7407Y5AWUBwA0cwRrrA PBak4Iilw7/rgvjloDugvpyVnfh+zfxr5FGuFczn9xsZvTUfrw8q29jJqXFmtptqD1Db kGzg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=cGlfhXnC; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69129-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69129-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id d7-20020aa7d5c7000000b00564023a42cesi157960eds.579.2024.02.16.10.08.56 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:08:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-69129-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=cGlfhXnC; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69129-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69129-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 5CF331F2145F for ; Fri, 16 Feb 2024 18:08:56 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 10FF1145B0C; Fri, 16 Feb 2024 18:06:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="cGlfhXnC" Received: from mail-il1-f174.google.com (mail-il1-f174.google.com [209.85.166.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8C1913AA49 for ; Fri, 16 Feb 2024 18:06:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106796; cv=none; b=WmghuzTK7TGPF7kc45SDwp0nb4Y7Q7ngcCTTkTzaEMtDt7Y9sa1NfCYxbaKjU7biAIwaJLyZCMp9LKpRuWowReUcRaTSnXnHdf1nhfMduXgsVcbm0iyaFQYQCIKFhWGQIxCT3tJmvZniZYhkEsH0pafwmB13FvNg2e5QOKogy3A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106796; c=relaxed/simple; bh=kaVesntSH9WtASpo/ITaFDU41OYmgmXbJfft7L0rkZs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sxLsWt0HGy8CBx0ImpfDN+HjZSevPExetTpecUtfhYnlQTjAYuI3pngiM9pjBvYxK1LOatfLief6VLdKsX9oJVYRVTTh5ebfNZTOZSeOUU7xbvEqO7TL1KSeq2GK4EBU+seaAGNERAGHmrWviwvyRhfRkP72yrt/3KsUAPHHebk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=cGlfhXnC; arc=none smtp.client-ip=209.85.166.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-il1-f174.google.com with SMTP id e9e14a558f8ab-363c2edac5aso6694365ab.2 for ; Fri, 16 Feb 2024 10:06:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708106794; x=1708711594; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=JYw17V2FqKp0C6CYJFYt8WJLZzrh9NJi0Sqb0UNwGU0=; b=cGlfhXnCY15ermxqwL3BXB/C52oZ8DhIeuCZTqOFqmTZSosb411H9oY04xxjJlC/Dh yKhpnjHqTshQMtV752vr+2DlOZeRwD/xZPjGnjdzN306ggCBr4tMafR3q3BSlg668Imx aX86ONAzBOJDriYCcEViR/74snj0hLoulqtXD/EYatJerqD7QChLbiRUJwsgaA3Pxp4Z 74+aLGpzLLBSFOCCnBPG5B9kA6jVST8zjmsseTZb40P+PW+vO6/Se+NJQhvkg26tXUMS 8zkhsqnYAx9tkGyEYgTRU3WVuneKhx3bEjI2RZ8WB1d2KF6eHwC0ZohqHqXgJkyDieQ1 GUKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708106794; x=1708711594; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=JYw17V2FqKp0C6CYJFYt8WJLZzrh9NJi0Sqb0UNwGU0=; b=SHYW6l2u3gymBy5IFRWnXgK3YgHMOkeaMdgHU1zv0HNCflvhiivw0xSyzt2OVO7v2i iKZFHGT/mXkmMlgrTv9wrBG20KsuYeYRUIo7mrf6smmIbIlYQo/MByXGF7F4x3aDkuJz fhfvZLf5zrG4pnbGzglLCmMZ88DU2zma+3Z3U0fQYjfhe4nxD/rUQBDey7VjwiXTmqLj 8cbGPYh+OpM2s/EQeIb7u8ymIeTgIStahhZAWVSGOBj0zPbKv2b/7J5Eljb7BeLnj3Xd wYjIroayLsH3VF/QaJ2mn4iXUkRCKmmsA9fI5dztZ2taXm3WTRsutRmem/F0AbbbTY5h e6dQ== X-Forwarded-Encrypted: i=1; AJvYcCXu8oSHa+ZKNBNt4VB/EEWKUKQHTGlhm9DS97VwSNMZYMhLCNgfWi45/+Oaaeu5Vzo61cb8w+41dstEdzlfmjCXatFZwQWsu19Zbv9h X-Gm-Message-State: AOJu0YyM40TsKkG/yuOtr4sQaCflQvC/5d8hhs38uEiV6uGefGAnL2zQ uiqAfPVV2y8SlQOMzazpa7qYNhTTIu4W0Rxiv++uLPc2YSwJa9Zs X-Received: by 2002:a05:6e02:13c8:b0:363:d734:14b with SMTP id v8-20020a056e0213c800b00363d734014bmr6223644ilj.26.1708106793656; Fri, 16 Feb 2024 10:06:33 -0800 (PST) Received: from localhost (dhcp-141-239-158-86.hawaiiantel.net. [141.239.158.86]) by smtp.gmail.com with ESMTPSA id k131-20020a636f89000000b005d8b89bbf20sm190558pgc.63.2024.02.16.10.06.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:06:33 -0800 (PST) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, allen.lkml@gmail.com, kernel-team@meta.com, Tejun Heo Subject: [PATCH 11/17] workqueue: Preserve OFFQ bits in cancel[_sync] paths Date: Fri, 16 Feb 2024 08:04:52 -1000 Message-ID: <20240216180559.208276-12-tj@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240216180559.208276-1-tj@kernel.org> References: <20240216180559.208276-1-tj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791079939869920102 X-GMAIL-MSGID: 1791079939869920102 The cancel[_sync] paths acquire and release WORK_STRUCT_PENDING, and manipulate WORK_OFFQ_CANCELING. However, they assume that all the OFFQ bit values except for the pool ID are statically known and don't preserve them, which is not wrong in the current code as the pool ID and CANCELING are the only information carried. However, the planned disable/enable support will add more fields and need them to be preserved. This patch updates work data handling so that only the bits which need updating are updated. - struct work_offq_data is added along with work_offqd_unpack() and work_offqd_pack_flags() to help manipulating multiple fields contained in work->data. Note that the helpers look a bit silly right now as there isn't that much to pack. The next patch will add more. - mark_work_canceling() which is used only by __cancel_work_sync() is replaced by open-coded usage of work_offq_data and set_work_pool_and_keep_pending() in __cancel_work_sync(). - __cancel_work[_sync]() uses offq_data helpers to preserve other OFFQ bits when clearing WORK_STRUCT_PENDING and WORK_OFFQ_CANCELING at the end. - This removes all users of get_work_pool_id() which is dropped. Note that get_work_pool_id() could handle both WORK_STRUCT_PWQ and !WORK_STRUCT_PWQ cases; however, it was only being called after try_to_grab_pending() succeeded, in which case WORK_STRUCT_PWQ is never set and thus it's safe to use work_offqd_unpack() instead. No behavior changes intended. Signed-off-by: Tejun Heo --- include/linux/workqueue.h | 1 + kernel/workqueue.c | 51 ++++++++++++++++++++++++--------------- 2 files changed, 32 insertions(+), 20 deletions(-) diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h index 0ad534fe6673..e15fc77bf2e2 100644 --- a/include/linux/workqueue.h +++ b/include/linux/workqueue.h @@ -97,6 +97,7 @@ enum wq_misc_consts { /* Convenience constants - of type 'unsigned long', not 'enum'! */ #define WORK_OFFQ_CANCELING (1ul << WORK_OFFQ_CANCELING_BIT) +#define WORK_OFFQ_FLAG_MASK (((1ul << WORK_OFFQ_FLAG_BITS) - 1) << WORK_OFFQ_FLAG_SHIFT) #define WORK_OFFQ_POOL_NONE ((1ul << WORK_OFFQ_POOL_BITS) - 1) #define WORK_STRUCT_NO_POOL (WORK_OFFQ_POOL_NONE << WORK_OFFQ_POOL_SHIFT) #define WORK_STRUCT_PWQ_MASK (~((1ul << WORK_STRUCT_PWQ_SHIFT) - 1)) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 7aa53a2e41af..e8f310aa16d6 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -391,6 +391,11 @@ struct wq_pod_type { int *cpu_pod; /* cpu -> pod */ }; +struct work_offq_data { + u32 pool_id; + u32 flags; +}; + static const char *wq_affn_names[WQ_AFFN_NR_TYPES] = { [WQ_AFFN_DFL] = "default", [WQ_AFFN_CPU] = "cpu", @@ -887,29 +892,23 @@ static struct worker_pool *get_work_pool(struct work_struct *work) return idr_find(&worker_pool_idr, pool_id); } -/** - * get_work_pool_id - return the worker pool ID a given work is associated with - * @work: the work item of interest - * - * Return: The worker_pool ID @work was last associated with. - * %WORK_OFFQ_POOL_NONE if none. - */ -static int get_work_pool_id(struct work_struct *work) +static unsigned long shift_and_mask(unsigned long v, u32 shift, u32 bits) { - unsigned long data = atomic_long_read(&work->data); + return (v >> shift) & ((1 << bits) - 1); +} - if (data & WORK_STRUCT_PWQ) - return work_struct_pwq(data)->pool->id; +static void work_offqd_unpack(struct work_offq_data *offqd, unsigned long data) +{ + WARN_ON_ONCE(data & WORK_STRUCT_PWQ); - return data >> WORK_OFFQ_POOL_SHIFT; + offqd->pool_id = shift_and_mask(data, WORK_OFFQ_POOL_SHIFT, + WORK_OFFQ_POOL_BITS); + offqd->flags = data & WORK_OFFQ_FLAG_MASK; } -static void mark_work_canceling(struct work_struct *work) +static unsigned long work_offqd_pack_flags(struct work_offq_data *offqd) { - unsigned long pool_id = get_work_pool_id(work); - - pool_id <<= WORK_OFFQ_POOL_SHIFT; - set_work_data(work, pool_id | WORK_STRUCT_PENDING | WORK_OFFQ_CANCELING); + return (unsigned long)offqd->flags; } static bool work_is_canceling(struct work_struct *work) @@ -4176,6 +4175,7 @@ EXPORT_SYMBOL(flush_rcu_work); static bool __cancel_work(struct work_struct *work, u32 cflags) { + struct work_offq_data offqd; unsigned long irq_flags; int ret; @@ -4186,19 +4186,26 @@ static bool __cancel_work(struct work_struct *work, u32 cflags) if (unlikely(ret < 0)) return false; - set_work_pool_and_clear_pending(work, get_work_pool_id(work), 0); + work_offqd_unpack(&offqd, *work_data_bits(work)); + set_work_pool_and_clear_pending(work, offqd.pool_id, + work_offqd_pack_flags(&offqd)); local_irq_restore(irq_flags); return ret; } static bool __cancel_work_sync(struct work_struct *work, u32 cflags) { + struct work_offq_data offqd; unsigned long irq_flags; bool ret; /* claim @work and tell other tasks trying to grab @work to back off */ ret = work_grab_pending(work, cflags, &irq_flags); - mark_work_canceling(work); + + work_offqd_unpack(&offqd, *work_data_bits(work)); + offqd.flags |= WORK_OFFQ_CANCELING; + set_work_pool_and_keep_pending(work, offqd.pool_id, + work_offqd_pack_flags(&offqd)); local_irq_restore(irq_flags); /* @@ -4208,12 +4215,16 @@ static bool __cancel_work_sync(struct work_struct *work, u32 cflags) if (wq_online) __flush_work(work, true); + work_offqd_unpack(&offqd, *work_data_bits(work)); + /* * smp_mb() at the end of set_work_pool_and_clear_pending() is paired * with prepare_to_wait() above so that either waitqueue_active() is * visible here or !work_is_canceling() is visible there. */ - set_work_pool_and_clear_pending(work, WORK_OFFQ_POOL_NONE, 0); + offqd.flags &= ~WORK_OFFQ_CANCELING; + set_work_pool_and_clear_pending(work, WORK_OFFQ_POOL_NONE, + work_offqd_pack_flags(&offqd)); if (waitqueue_active(&wq_cancel_waitq)) __wake_up(&wq_cancel_waitq, TASK_NORMAL, 1, work); From patchwork Fri Feb 16 18:04:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 202292 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:c619:b0:108:e6aa:91d0 with SMTP id hn25csp691115dyb; Fri, 16 Feb 2024 10:09:14 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVIHY3oT6B4B//4g31EWHab1K11CIZae4BuYQoUCe0kG0os68eCdoklSkrVxN4VAJlr0LSNV9Qzl2i7dwGBvS7bKL3QKw== X-Google-Smtp-Source: AGHT+IFOTCq58mNjoTo8zUBcAwoozjjxu0P51aR4wwf8uU5ufWcmTZYqNEBwhWtmxHJV1rlCAXs3 X-Received: by 2002:a17:903:1210:b0:1d9:fc3f:5081 with SMTP id l16-20020a170903121000b001d9fc3f5081mr6211077plh.33.1708106954380; Fri, 16 Feb 2024 10:09:14 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708106954; cv=pass; d=google.com; s=arc-20160816; b=aZpeWZ9vqvDwPbR5xH0lgjrWBiuxzRrruc0h9SAiKcMZmy5XPOY80jdCFrEbJCgpef 6ldlzKbOGk5uvGmBF/1QPyxDw4BEpJuz5SupCkDerThdlJ0E54Ni35z22KdQbazb/19L jBXsdGLEB8tlegFX1umpBnqENi1TYaa6raeAd0HZB2O29gnwjzY3y9mRPwP2uBYZOtH9 R93EMFkUdLweOEed5D5kjkBAeKNFWRs50fT9+B+eebl5OwCnBHAMypMOMa+/PDm4q31d d4G9dBuzCVS43X2FElbVrA/F/e4jLgdyXFnkiwJoLxSx50OktY2wteD3cqyz0ldR7e59 L+ow== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:sender:dkim-signature; bh=/eVjuEcfDf9TXnJ3SsBpAJjkkwdry7m0sb4FPgUS9H8=; fh=1srqrKlc28GKZrgm6OvJWklFZtAT+ERN8n6CRrqdUTw=; b=lOM8yNK8HO7ISc40RhnsYsaFVjinV4HkHk8mDrmxj+yJF4rjRKD/wVfg2xnhdZYir0 jN+Hux5IBSenvh8YHj3uG3MaCtSVy7oahEf7Zvo6UPTL9O7JD1+R7vKapsaWPWgQxB8Y uPiHKvAMjkkcqaUs5+i01FTmUIFr6zuo5zalKTv+yFAh0pU/VCUx36KgVZxPGSJliodP g4xN2YAfHQt1oD4AK2WkutXgFdbbEaxYOWqPGsGHk2k4QM0DSiScpEZWP7JAaclYtzLh GJZ3PlI4Phuox/nStoeoF/m6HoOMaNoU7bNN6oZbsw3yWwnDFeYOJzr3QSVQJrShMVbe G3Nw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=hw6at90A; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69130-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69130-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id h13-20020a170902f7cd00b001d5e9f77cf3si217055plw.79.2024.02.16.10.09.14 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:09:14 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-69130-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=hw6at90A; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69130-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69130-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 31EBF2845EC for ; Fri, 16 Feb 2024 18:09:14 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D58681350C6; Fri, 16 Feb 2024 18:06:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hw6at90A" Received: from mail-pg1-f174.google.com (mail-pg1-f174.google.com [209.85.215.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6864C1350E5 for ; Fri, 16 Feb 2024 18:06:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106798; cv=none; b=lMJIyprRH+4rYaoAPf4+j8/f+/ERoyGYp1UAlP6XZdqM7HwJNY4h+5+5dgCnRajuM2tnIwqn5/G/+2v0Pq91ObhZsfahw+dbsPkV+839BCSyzyHicTa6AMcOIXh66bl78bv5ZKPzs4hi2/LrMFYoJOqzpqO6JrwlkDgAaMb5e58= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106798; c=relaxed/simple; bh=wgs2b7US+r9x5FZZPLabAcWsN4g3NqQmXm1FT7Dn4E8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oXE+knvICLEfuJWf59yD5N8xqln5EMuhTD4s2ZYpa5NmVilwrD8RRfh4NQw+W7v4unzGIuF+LVjGuvJQvjh5oURU4irpqa6UfMXXaz+eWe3Dea7ZEM3tKQsZaFfm776UNU/Bj2xiv4JRHPTTsmpCdk59/fCRTMV+JK64tVpaWDc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hw6at90A; arc=none smtp.client-ip=209.85.215.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pg1-f174.google.com with SMTP id 41be03b00d2f7-5d3912c9a83so1731589a12.3 for ; Fri, 16 Feb 2024 10:06:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708106796; x=1708711596; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=/eVjuEcfDf9TXnJ3SsBpAJjkkwdry7m0sb4FPgUS9H8=; b=hw6at90AJedgyT+xkV48ReyYvsVuCfFQnSg8ZzyFM6u5kpMwBUxGoGZKs6kiYFVf68 JrEoyYWn4+osoEjlOFy763YEO+jCVsw4SeezOquuUj+baQsL6nGfh1cC3EVGQ+6bbTpf v0qEjgLWgCkLKJW4C+neDDzdGIa6U6KhyGKh7JDIWw3mDoX0koaQRIYVeTabs7LAWVzx hnAm17udPpru/b/lr/Eel/M1WMkXIY1GKjipA3lPb8svEpBv+vFLif0MYpnXkeoFvYt1 xGYHkDVcMc4nuk7TqZwY6vq+ozsJAHSPK42KvdtRlArLCmQI/EpnJAG3XRmgI6UHvCX1 Ycgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708106796; x=1708711596; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=/eVjuEcfDf9TXnJ3SsBpAJjkkwdry7m0sb4FPgUS9H8=; b=aV8UT+oLdTrhwjfJX/fS03k7y+ickj1SDVyGp68D80Je8QLopm62YuwCkPmDZfejf2 pu+xYpTR2l8y2qZ3JwNxyMGppXtNYzsklRelLt5bdZmPSrglsRe9j8mIPm5w5oHHJm5v yBT3nQEpPMnlXF1X1NdKoTB6VmKKRGRIhBJ6ipgS26nTr8vEuRyHJH3eLbwzwd+qvtyV K2R4mUQKiSqIguDkql9KuJ6dv9qTkeXUBTP1nDRN77wMDLaycu6KZSoaOEAkyI8DHwWb ymAgL4OEXxv9Oiu+f5TK/HgSRULF1hRxe95EVPRC0gaqaSHUItJWiH706ZS/Xgm7MrKN EXDw== X-Forwarded-Encrypted: i=1; AJvYcCUKaTq8t0G2GBk1zAF6rm2Ylw+S3aC9MTSbdecrg71VNOOUod0v8D8XX9IVhxXl13ZWRhGtsDzsZEMPdRopRcrDh7xVby9ZpYiXLq2Z X-Gm-Message-State: AOJu0Yzz2jFeVl2aRwdEk2QDkojdm6ad86pr8fGmY1h1+CYCPdTGAgjQ PPUp6KczAWNXrKx/+YqoI/WEjNzFBrxhedaevL7n5mdvCli7y3wW X-Received: by 2002:a17:90a:af92:b0:295:cfb4:9d4c with SMTP id w18-20020a17090aaf9200b00295cfb49d4cmr5359721pjq.33.1708106795583; Fri, 16 Feb 2024 10:06:35 -0800 (PST) Received: from localhost (dhcp-141-239-158-86.hawaiiantel.net. [141.239.158.86]) by smtp.gmail.com with ESMTPSA id n17-20020a17090ade9100b0029937256b91sm307998pjv.7.2024.02.16.10.06.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:06:35 -0800 (PST) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, allen.lkml@gmail.com, kernel-team@meta.com, Tejun Heo Subject: [PATCH 12/17] workqueue: Implement disable/enable for (delayed) work items Date: Fri, 16 Feb 2024 08:04:53 -1000 Message-ID: <20240216180559.208276-13-tj@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240216180559.208276-1-tj@kernel.org> References: <20240216180559.208276-1-tj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791079957733794550 X-GMAIL-MSGID: 1791079957733794550 While (delayed) work items could be flushed and canceled, there was no way to prevent them from being queued in the future. While this didn't lead to functional deficiencies, it sometimes required a bit more effort from the workqueue users to e.g. sequence shutdown steps with more care. Workqueue is currently in the process of replacing tasklet which does support disabling and enabling. The feature is used relatively widely to, for example, temporarily suppress main path while a control plane operation (reset or config change) is in progress. To enable easy conversion of tasklet users and as it seems like an inherent useful feature, this patch implements disabling and enabling of work items. - A work item carries 10bit disable count in work->data while not queued. The access to the count is synchronized by the PENDING bit like all other parts of work->data. - If the count is non-zero, the work item cannot be queued. Any attempt to queue the work item fails and returns %false. - disable_work[_sync](), enable_work(), disable_delayed_work[_sync]() and enable_delayed_work() are added. Signed-off-by: Tejun Heo --- include/linux/workqueue.h | 18 +++- kernel/workqueue.c | 167 ++++++++++++++++++++++++++++++++++++-- 2 files changed, 174 insertions(+), 11 deletions(-) diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h index e15fc77bf2e2..f25915e47efb 100644 --- a/include/linux/workqueue.h +++ b/include/linux/workqueue.h @@ -51,20 +51,23 @@ enum work_bits { * data contains off-queue information when !WORK_STRUCT_PWQ. * * MSB - * [ pool ID ] [ OFFQ flags ] [ STRUCT flags ] - * 1 bit 4 or 5 bits + * [ pool ID ] [ disable depth ] [ OFFQ flags ] [ STRUCT flags ] + * 16 bits 1 bit 4 or 5 bits */ WORK_OFFQ_FLAG_SHIFT = WORK_STRUCT_FLAG_BITS, WORK_OFFQ_CANCELING_BIT = WORK_OFFQ_FLAG_SHIFT, WORK_OFFQ_FLAG_END, WORK_OFFQ_FLAG_BITS = WORK_OFFQ_FLAG_END - WORK_OFFQ_FLAG_SHIFT, + WORK_OFFQ_DISABLE_SHIFT = WORK_OFFQ_FLAG_SHIFT + WORK_OFFQ_FLAG_BITS, + WORK_OFFQ_DISABLE_BITS = 16, + /* * When a work item is off queue, the high bits encode off-queue flags * and the last pool it was on. Cap pool ID to 31 bits and use the * highest number to indicate that no pool is associated. */ - WORK_OFFQ_POOL_SHIFT = WORK_OFFQ_FLAG_SHIFT + WORK_OFFQ_FLAG_BITS, + WORK_OFFQ_POOL_SHIFT = WORK_OFFQ_DISABLE_SHIFT + WORK_OFFQ_DISABLE_BITS, WORK_OFFQ_LEFT = BITS_PER_LONG - WORK_OFFQ_POOL_SHIFT, WORK_OFFQ_POOL_BITS = WORK_OFFQ_LEFT <= 31 ? WORK_OFFQ_LEFT : 31, }; @@ -98,6 +101,7 @@ enum wq_misc_consts { /* Convenience constants - of type 'unsigned long', not 'enum'! */ #define WORK_OFFQ_CANCELING (1ul << WORK_OFFQ_CANCELING_BIT) #define WORK_OFFQ_FLAG_MASK (((1ul << WORK_OFFQ_FLAG_BITS) - 1) << WORK_OFFQ_FLAG_SHIFT) +#define WORK_OFFQ_DISABLE_MASK (((1ul << WORK_OFFQ_DISABLE_BITS) - 1) << WORK_OFFQ_DISABLE_SHIFT) #define WORK_OFFQ_POOL_NONE ((1ul << WORK_OFFQ_POOL_BITS) - 1) #define WORK_STRUCT_NO_POOL (WORK_OFFQ_POOL_NONE << WORK_OFFQ_POOL_SHIFT) #define WORK_STRUCT_PWQ_MASK (~((1ul << WORK_STRUCT_PWQ_SHIFT) - 1)) @@ -556,6 +560,14 @@ extern bool flush_delayed_work(struct delayed_work *dwork); extern bool cancel_delayed_work(struct delayed_work *dwork); extern bool cancel_delayed_work_sync(struct delayed_work *dwork); +extern bool disable_work(struct work_struct *work); +extern bool disable_work_sync(struct work_struct *work); +extern bool enable_work(struct work_struct *work); + +extern bool disable_delayed_work(struct delayed_work *dwork); +extern bool disable_delayed_work_sync(struct delayed_work *dwork); +extern bool enable_delayed_work(struct delayed_work *dwork); + extern bool flush_rcu_work(struct rcu_work *rwork); extern void workqueue_set_max_active(struct workqueue_struct *wq, diff --git a/kernel/workqueue.c b/kernel/workqueue.c index e8f310aa16d6..87e40e6e14cb 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -98,6 +98,7 @@ enum worker_flags { enum work_cancel_flags { WORK_CANCEL_DELAYED = 1 << 0, /* canceling a delayed_work */ + WORK_CANCEL_DISABLE = 1 << 1, /* canceling to disable */ }; enum wq_internal_consts { @@ -393,6 +394,7 @@ struct wq_pod_type { struct work_offq_data { u32 pool_id; + u32 disable; u32 flags; }; @@ -903,12 +905,15 @@ static void work_offqd_unpack(struct work_offq_data *offqd, unsigned long data) offqd->pool_id = shift_and_mask(data, WORK_OFFQ_POOL_SHIFT, WORK_OFFQ_POOL_BITS); + offqd->disable = shift_and_mask(data, WORK_OFFQ_DISABLE_SHIFT, + WORK_OFFQ_DISABLE_BITS); offqd->flags = data & WORK_OFFQ_FLAG_MASK; } static unsigned long work_offqd_pack_flags(struct work_offq_data *offqd) { - return (unsigned long)offqd->flags; + return ((unsigned long)offqd->disable << WORK_OFFQ_DISABLE_SHIFT) | + ((unsigned long)offqd->flags); } static bool work_is_canceling(struct work_struct *work) @@ -2395,6 +2400,21 @@ static void __queue_work(int cpu, struct workqueue_struct *wq, rcu_read_unlock(); } +static bool clear_pending_if_disabled(struct work_struct *work) +{ + unsigned long data = *work_data_bits(work); + struct work_offq_data offqd; + + if (likely((data & WORK_STRUCT_PWQ) || + !(data & WORK_OFFQ_DISABLE_MASK))) + return false; + + work_offqd_unpack(&offqd, data); + set_work_pool_and_clear_pending(work, offqd.pool_id, + work_offqd_pack_flags(&offqd)); + return true; +} + /** * queue_work_on - queue work on specific cpu * @cpu: CPU number to execute work on @@ -2417,7 +2437,8 @@ bool queue_work_on(int cpu, struct workqueue_struct *wq, local_irq_save(irq_flags); - if (!test_and_set_bit(WORK_STRUCT_PENDING_BIT, work_data_bits(work))) { + if (!test_and_set_bit(WORK_STRUCT_PENDING_BIT, work_data_bits(work)) && + !clear_pending_if_disabled(work)) { __queue_work(cpu, wq, work); ret = true; } @@ -2577,7 +2598,8 @@ bool queue_delayed_work_on(int cpu, struct workqueue_struct *wq, /* read the comment in __queue_work() */ local_irq_save(irq_flags); - if (!test_and_set_bit(WORK_STRUCT_PENDING_BIT, work_data_bits(work))) { + if (!test_and_set_bit(WORK_STRUCT_PENDING_BIT, work_data_bits(work)) && + !clear_pending_if_disabled(work)) { __queue_delayed_work(cpu, wq, dwork, delay); ret = true; } @@ -4173,20 +4195,46 @@ bool flush_rcu_work(struct rcu_work *rwork) } EXPORT_SYMBOL(flush_rcu_work); +static void work_offqd_disable(struct work_offq_data *offqd) +{ + const unsigned long max = (1lu << WORK_OFFQ_DISABLE_BITS) - 1; + + if (likely(offqd->disable < max)) + offqd->disable++; + else + WARN_ONCE(true, "workqueue: work disable count overflowed\n"); +} + +static void work_offqd_enable(struct work_offq_data *offqd) +{ + if (likely(offqd->disable > 0)) + offqd->disable--; + else + WARN_ONCE(true, "workqueue: work disable count underflowed\n"); +} + static bool __cancel_work(struct work_struct *work, u32 cflags) { struct work_offq_data offqd; unsigned long irq_flags; int ret; - do { - ret = try_to_grab_pending(work, cflags, &irq_flags); - } while (unlikely(ret == -EAGAIN)); + if (cflags & WORK_CANCEL_DISABLE) { + ret = work_grab_pending(work, cflags, &irq_flags); + } else { + do { + ret = try_to_grab_pending(work, cflags, &irq_flags); + } while (unlikely(ret == -EAGAIN)); - if (unlikely(ret < 0)) - return false; + if (unlikely(ret < 0)) + return false; + } work_offqd_unpack(&offqd, *work_data_bits(work)); + + if (cflags & WORK_CANCEL_DISABLE) + work_offqd_disable(&offqd); + set_work_pool_and_clear_pending(work, offqd.pool_id, work_offqd_pack_flags(&offqd)); local_irq_restore(irq_flags); @@ -4203,6 +4251,10 @@ static bool __cancel_work_sync(struct work_struct *work, u32 cflags) ret = work_grab_pending(work, cflags, &irq_flags); work_offqd_unpack(&offqd, *work_data_bits(work)); + + if (cflags & WORK_CANCEL_DISABLE) + work_offqd_disable(&offqd); + offqd.flags |= WORK_OFFQ_CANCELING; set_work_pool_and_keep_pending(work, offqd.pool_id, work_offqd_pack_flags(&offqd)); @@ -4302,6 +4354,105 @@ bool cancel_delayed_work_sync(struct delayed_work *dwork) } EXPORT_SYMBOL(cancel_delayed_work_sync); +/** + * disable_work - Disable and cancel a work item + * @work: work item to disable + * + * Disable @work by incrementing its disable count and cancel it if currently + * pending. As long as the disable count is non-zero, any attempt to queue @work + * will fail and return %false. The maximum supported disable depth is 2 to the + * power of %WORK_OFFQ_DISABLE_BITS, currently 65536. + * + * Must be called from a sleepable context. Returns %true if @work was pending, + * %false otherwise. + */ +bool disable_work(struct work_struct *work) +{ + return __cancel_work(work, WORK_CANCEL_DISABLE); +} +EXPORT_SYMBOL_GPL(disable_work); + +/** + * disable_work_sync - Disable, cancel and drain a work item + * @work: work item to disable + * + * Similar to disable_work() but also wait for @work to finish if currently + * executing. + * + * Must be called from a sleepable context. Returns %true if @work was pending, + * %false otherwise. + */ +bool disable_work_sync(struct work_struct *work) +{ + return __cancel_work_sync(work, WORK_CANCEL_DISABLE); +} +EXPORT_SYMBOL_GPL(disable_work_sync); + +/** + * enable_work - Enable a work item + * @work: work item to enable + * + * Undo disable_work[_sync]() by decrementing @work's disable count. @work can + * only be queued if its disable count is 0. + * + * Must be called from a sleepable context. Returns %true if the disable count + * reached 0. Otherwise, %false. + */ +bool enable_work(struct work_struct *work) +{ + struct work_offq_data offqd; + unsigned long irq_flags; + + work_grab_pending(work, 0, &irq_flags); + + work_offqd_unpack(&offqd, *work_data_bits(work)); + work_offqd_enable(&offqd); + set_work_pool_and_clear_pending(work, offqd.pool_id, + work_offqd_pack_flags(&offqd)); + local_irq_enable(); + + return !offqd.disable; +} +EXPORT_SYMBOL_GPL(enable_work); + +/** + * disable_delayed_work - Disable and cancel a delayed work item + * @dwork: delayed work item to disable + * + * disable_work() for delayed work items. + */ +bool disable_delayed_work(struct delayed_work *dwork) +{ + return __cancel_work(&dwork->work, + WORK_CANCEL_DELAYED | WORK_CANCEL_DISABLE); +} +EXPORT_SYMBOL_GPL(disable_delayed_work); + +/** + * disable_delayed_work_sync - Disable, cancel and drain a delayed work item + * @dwork: delayed work item to disable + * + * disable_work_sync() for delayed work items. + */ +bool disable_delayed_work_sync(struct delayed_work *dwork) +{ + return __cancel_work_sync(&dwork->work, + WORK_CANCEL_DELAYED | WORK_CANCEL_DISABLE); +} +EXPORT_SYMBOL_GPL(disable_delayed_work_sync); + +/** + * enable_delayed_work - Enable a delayed work item + * @dwork: delayed work item to enable + * + * enable_work() for delayed work items. + */ +bool enable_delayed_work(struct delayed_work *dwork) +{ + return enable_work(&dwork->work); +} +EXPORT_SYMBOL_GPL(enable_delayed_work); + /** * schedule_on_each_cpu - execute a function synchronously on each online CPU * @func: the function to call From patchwork Fri Feb 16 18:04:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 202303 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:c619:b0:108:e6aa:91d0 with SMTP id hn25csp705309dyb; Fri, 16 Feb 2024 10:30:09 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXMskh4MYv1NWO78wDFScnn1tRWwo7RtX/O8UUiv47jMKBDJvLnT82fVgce31tHAg6PE46wX4fizL9KHm3N1k0XNLXGVg== X-Google-Smtp-Source: AGHT+IGdLr2/YippKz2CXCrfiHVkC6kaANW7oq+LRPJFLNiG5j1siuFFNN++bjcKIs/L2Mymssgm X-Received: by 2002:a17:903:11c3:b0:1d9:7ebe:431f with SMTP id q3-20020a17090311c300b001d97ebe431fmr7180165plh.25.1708108209491; Fri, 16 Feb 2024 10:30:09 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708108209; cv=pass; d=google.com; s=arc-20160816; b=Ib/WchS9pe0LUx+eOfjrq1Q8VJhnCCHxvuFD0fBAFuK8CRcYaCKqJzaWbzXgZzVxWC UdmwC5GYwtdusuBqHH9ibdampyFRBPMXeHvW6pYWIoycyFEhuUblRmj6q89MKJNmBSTv Q/HLwJTdyhYcbVQpHR+qYR0uZysaRputFas2axK+VIBmgCEZvGoye6mQQjq4C3/ASFlw 0x48MwSt5cOdAofZRVvlglnT7XpAtznrf+g8ePjH6wlLzw79+BpIFNLEermyqMK5hOoK 6bVQ058Iu7D1i1t/vZyPJ56mXPEgQS+AnIseuhqwUlJwsWDkaLeSzUNrzJ7GFmPhUpLW o8nA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:sender:dkim-signature; bh=4xKQ1PCdJY3gCpADUngBtrwEqa69ZLfcY2hROuE2NTw=; fh=UI9ONgqpav0+zBYTYqeiWrq+fXjZi1Rd9Qf0xVuvDOM=; b=X+HnoG40ByXab61bQhqIJtDt0JGQDMYS9S26Shl1JdKsstBwLsxL0POGm6iJeJPf5E 92zvaHuiJNHtkerr9UyYFsahPijGiQeTAG6Sfi340YT7IpcPS7ERnaEeok2xc5Nen2ZB 0YBRVR8Aq3kkF302kxLsLFvTjyYxzRh7OKJRJ4dj+XpuyONUlrDgYfFu+kPcRjY3Wnwl 4Ook/5TUW0ytoiVWXtGuNa0ZOF7kIfmqMT6No9vjA8sNU/cBN+dHQB4gxGgMVCITIUC/ RDkZ6VzWEl945Ef8VlVyqaMQwSjNGlzaOelxpGMFvYhjP97BECFuHC3/fridMe7DOlmd gG4g==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=VG4jf8ex; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69131-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69131-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id lw16-20020a1709032ad000b001db9cab6c8asi214447plb.603.2024.02.16.10.30.09 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:30:09 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-69131-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=VG4jf8ex; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69131-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69131-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 07E62B24010 for ; Fri, 16 Feb 2024 18:09:41 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 58F2D146905; Fri, 16 Feb 2024 18:06:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="VG4jf8ex" Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 881E41420B8 for ; Fri, 16 Feb 2024 18:06:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106800; cv=none; b=Xgn1LoodJKrWg1lowEw1NQGHcdHZHPnu24JkpN2yQbuIUWMXoQBRdN2993iBwGXgKTI+kCY7C1IkU4g4hpi8rBQirMmGVgLzyPgQnocYMwImX/ZUuIaoHhOB7cW8zG6c7ca+/bbdL7dvF5cZCHeOvKBV223f6ASseFmkChc7RVo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106800; c=relaxed/simple; bh=GqG/7G0WK8IUjrbaHzU3XCy5XoAdxvldaNBS7e5eaXg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=B3awvVLICbaXpOMNlfVFPr2HOfhmUR0/LDkWSFdTPhcc48jSxpgSnl6hcfyxF1O39pH1byaAFnR2uFZSHpagET+JsgDN/7qf90d9A8G9m9mQkErAyxgtP2dFu0QXRkq6DgNNAUmTLiw0pb31aM5O78gyGauHKmJkpeW2Jhso7pE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=VG4jf8ex; arc=none smtp.client-ip=209.85.210.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-6e1358dd3a1so1239454b3a.1 for ; Fri, 16 Feb 2024 10:06:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708106798; x=1708711598; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=4xKQ1PCdJY3gCpADUngBtrwEqa69ZLfcY2hROuE2NTw=; b=VG4jf8ex07e5f0VvEbDTj3A+1yonaGquGgu7CjEmrNV1Fg7lb9iiZHYFkXjsVDQRz5 SSFI1oDRRFE4VElwUAu9jcEmXcGNZ5ls52VD79OEntD/h19ZE86P9ZMDISDv4jnXK3rf txz65P6JTNmwlRfmyPyM6D3ZUAf634CMeXLGpwOAtQyzL1EqslUictAiijcBPNGgNvXS bFJJtR+03YccNzX8vMVu6rrPi1CekIz8hWFSXQI2nr+apa37KSdOlR7KRoqd9CmSfUpk osZ4dn+4rwRz3LH/XZRyf1S1F8OWjQeEdnUzU1OvUlAtsjxeysm1lddzmzikk44JLLGu 9kFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708106798; x=1708711598; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=4xKQ1PCdJY3gCpADUngBtrwEqa69ZLfcY2hROuE2NTw=; b=xH1wzLJ3CDWoqt3Cm7e4sv10h8iMqIhHobM7IZVEIv6wuFP8FettBs5J5RN3n9aKZq veudlkpHv6Tsos2JrtbdGmj5gm1+ghGZTtigsvI/nBGTVMHLIr8W9ARFV06Cv+LXqW6g /enAAwT0DNutm4v1rP+RuKAdPvXyZKQCnumt/ZyUX5tI+kQMd9p8bv8nrVcySssrfyUx QMnQndo0JSGmN5AwDEJEFGloiRLBGvdUmIewL23D/thNzRIPvs3dgS0XJ4hPF4MV602e e9y9FDAIv+77vAOLOXLQWlUBFLRY/3CA8mrsigBifzciGTDRLcAUQcknXeGJxGPZ2nfa 9A2A== X-Forwarded-Encrypted: i=1; AJvYcCWUwnXZlmzeFlfXmtYh94fnMOjarNwYYFLvExt/nxTnrnAhIYHIY14/mdWc+2/sa/PqneXtbLKadKlmuL8TAL2Wgz+OJ1zvxoLppeZO X-Gm-Message-State: AOJu0Yz+BEoV3GnZpddfx5mNuhd02PsrnDiPdsLjb65TbWfYuRJwb1aj imC6unKEei/Io0oBPFObITk9nqq5SFSqgj3vfv/z8vcPWr5LY95ynxnOITf0vAg= X-Received: by 2002:a05:6a00:1d06:b0:6e1:33ca:aa78 with SMTP id a6-20020a056a001d0600b006e133caaa78mr4426647pfx.26.1708106797472; Fri, 16 Feb 2024 10:06:37 -0800 (PST) Received: from localhost (dhcp-141-239-158-86.hawaiiantel.net. [141.239.158.86]) by smtp.gmail.com with ESMTPSA id h6-20020a63c006000000b0059b2316be86sm198856pgg.46.2024.02.16.10.06.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:06:37 -0800 (PST) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, allen.lkml@gmail.com, kernel-team@meta.com, Tejun Heo Subject: [PATCH 13/17] workqueue: Remove WORK_OFFQ_CANCELING Date: Fri, 16 Feb 2024 08:04:54 -1000 Message-ID: <20240216180559.208276-14-tj@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240216180559.208276-1-tj@kernel.org> References: <20240216180559.208276-1-tj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791081273897381873 X-GMAIL-MSGID: 1791081273897381873 cancel[_delayed]_work_sync() guarantees that it can shut down self-requeueing work items. To achieve that, it grabs and then holds WORK_STRUCT_PENDING bit set while flushing the currently executing instance. As the PENDING bit is set, all queueing attempts including the self-requeueing ones fail and once the currently executing instance is flushed, the work item should be idle as long as someone else isn't actively queueing it. This means that the cancel_work_sync path may hold the PENDING bit set while flushing the target work item. This isn't a problem for the queueing path - it can just fail which is the desired effect. It doesn't affect flush. It doesn't matter to cancel_work either as it can just report that the work item has successfully canceled. However, if there's another cancel_work_sync attempt on the work item, it can't simply fail or report success and that would breach the guarantee that it should provide. cancel_work_sync has to wait for and grab that PENDING bit and go through the motions. WORK_OFFQ_CANCELING and wq_cancel_waitq are what implement this cancel_work_sync to cancel_work_sync wait mechanism. When a work item is being canceled, WORK_OFFQ_CANCELING is also set on it and other cancel_work_sync attempts wait on the bit to be cleared using the wait queue. While this works, it's an isolated wart which doesn't jive with the rest of flush and cancel mechanisms and forces enable_work() and disable_work() to require a sleepable context, which hampers their usability. Now that a work item can be disabled, we can use that to block queueing while cancel_work_sync is in progress. Instead of holding PENDING the bit, it can temporarily disable the work item, flush and then re-enable it as that'd achieve the same end result of blocking queueings while canceling and thus enable canceling of self-requeueing work items. - WORK_OFFQ_CANCELING and the surrounding mechanims are removed. - work_grab_pending() is now simpler, no longer has to wait for a blocking operation and thus can be called from any context. - With work_grab_pending() simplified, no need to use try_to_grab_pending() directly. All users are converted to use work_grab_pending(). - __cancel_work_sync() is updated to __cancel_work() with WORK_CANCEL_DISABLE to cancel and plug racing queueing attempts. It then flushes and re-enables the work item if necessary. - These changes allow disable_work() and enable_work() to be called from any context. Signed-off-by: Tejun Heo --- include/linux/workqueue.h | 4 +- kernel/workqueue.c | 139 +++++--------------------------------- 2 files changed, 19 insertions(+), 124 deletions(-) diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h index f25915e47efb..86483743ad28 100644 --- a/include/linux/workqueue.h +++ b/include/linux/workqueue.h @@ -52,10 +52,9 @@ enum work_bits { * * MSB * [ pool ID ] [ disable depth ] [ OFFQ flags ] [ STRUCT flags ] - * 16 bits 1 bit 4 or 5 bits + * 16 bits 0 bits 4 or 5 bits */ WORK_OFFQ_FLAG_SHIFT = WORK_STRUCT_FLAG_BITS, - WORK_OFFQ_CANCELING_BIT = WORK_OFFQ_FLAG_SHIFT, WORK_OFFQ_FLAG_END, WORK_OFFQ_FLAG_BITS = WORK_OFFQ_FLAG_END - WORK_OFFQ_FLAG_SHIFT, @@ -99,7 +98,6 @@ enum wq_misc_consts { }; /* Convenience constants - of type 'unsigned long', not 'enum'! */ -#define WORK_OFFQ_CANCELING (1ul << WORK_OFFQ_CANCELING_BIT) #define WORK_OFFQ_FLAG_MASK (((1ul << WORK_OFFQ_FLAG_BITS) - 1) << WORK_OFFQ_FLAG_SHIFT) #define WORK_OFFQ_DISABLE_MASK (((1ul << WORK_OFFQ_DISABLE_BITS) - 1) << WORK_OFFQ_DISABLE_SHIFT) #define WORK_OFFQ_POOL_NONE ((1ul << WORK_OFFQ_POOL_BITS) - 1) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 87e40e6e14cb..e9d25e1f79d7 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -491,12 +491,6 @@ static struct workqueue_attrs *unbound_std_wq_attrs[NR_STD_WORKER_POOLS]; /* I: attributes used when instantiating ordered pools on demand */ static struct workqueue_attrs *ordered_wq_attrs[NR_STD_WORKER_POOLS]; -/* - * Used to synchronize multiple cancel_sync attempts on the same work item. See - * work_grab_pending() and __cancel_work_sync(). - */ -static DECLARE_WAIT_QUEUE_HEAD(wq_cancel_waitq); - /* * I: kthread_worker to release pwq's. pwq release needs to be bounced to a * process context while holding a pool lock. Bounce to a dedicated kthread @@ -778,11 +772,6 @@ static int work_next_color(int color) * corresponding to a work. Pool is available once the work has been * queued anywhere after initialization until it is sync canceled. pwq is * available only while the work item is queued. - * - * %WORK_OFFQ_CANCELING is used to mark a work item which is being - * canceled. While being canceled, a work item may have its PENDING set - * but stay off timer and worklist for arbitrarily long and nobody should - * try to steal the PENDING bit. */ static inline void set_work_data(struct work_struct *work, unsigned long data) { @@ -916,13 +905,6 @@ static unsigned long work_offqd_pack_flags(struct work_offq_data *offqd) ((unsigned long)offqd->flags); } -static bool work_is_canceling(struct work_struct *work) -{ - unsigned long data = atomic_long_read(&work->data); - - return !(data & WORK_STRUCT_PWQ) && (data & WORK_OFFQ_CANCELING); -} - /* * Policy functions. These define the policies on how the global worker * pools are managed. Unless noted otherwise, these functions assume that @@ -2047,8 +2029,6 @@ static void pwq_dec_nr_in_flight(struct pool_workqueue *pwq, unsigned long work_ * 1 if @work was pending and we successfully stole PENDING * 0 if @work was idle and we claimed PENDING * -EAGAIN if PENDING couldn't be grabbed at the moment, safe to busy-retry - * -ENOENT if someone else is canceling @work, this state may persist - * for arbitrarily long * ======== ================================================================ * * Note: @@ -2144,26 +2124,9 @@ static int try_to_grab_pending(struct work_struct *work, u32 cflags, fail: rcu_read_unlock(); local_irq_restore(*irq_flags); - if (work_is_canceling(work)) - return -ENOENT; - cpu_relax(); return -EAGAIN; } -struct cwt_wait { - wait_queue_entry_t wait; - struct work_struct *work; -}; - -static int cwt_wakefn(wait_queue_entry_t *wait, unsigned mode, int sync, void *key) -{ - struct cwt_wait *cwait = container_of(wait, struct cwt_wait, wait); - - if (cwait->work != key) - return 0; - return autoremove_wake_function(wait, mode, sync, key); -} - /** * work_grab_pending - steal work item from worklist and disable irq * @work: work item to steal @@ -2180,41 +2143,14 @@ static int cwt_wakefn(wait_queue_entry_t *wait, unsigned mode, int sync, void *k static bool work_grab_pending(struct work_struct *work, u32 cflags, unsigned long *irq_flags) { - struct cwt_wait cwait; int ret; - might_sleep(); -repeat: - ret = try_to_grab_pending(work, cflags, irq_flags); - if (likely(ret >= 0)) - return ret; - if (ret != -ENOENT) - goto repeat; - - /* - * Someone is already canceling. Wait for it to finish. flush_work() - * doesn't work for PREEMPT_NONE because we may get woken up between - * @work's completion and the other canceling task resuming and clearing - * CANCELING - flush_work() will return false immediately as @work is no - * longer busy, try_to_grab_pending() will return -ENOENT as @work is - * still being canceled and the other canceling task won't be able to - * clear CANCELING as we're hogging the CPU. - * - * Let's wait for completion using a waitqueue. As this may lead to the - * thundering herd problem, use a custom wake function which matches - * @work along with exclusive wait and wakeup. - */ - init_wait(&cwait.wait); - cwait.wait.func = cwt_wakefn; - cwait.work = work; - - prepare_to_wait_exclusive(&wq_cancel_waitq, &cwait.wait, - TASK_UNINTERRUPTIBLE); - if (work_is_canceling(work)) - schedule(); - finish_wait(&wq_cancel_waitq, &cwait.wait); - - goto repeat; + while (true) { + ret = try_to_grab_pending(work, cflags, irq_flags); + if (ret >= 0) + return ret; + cpu_relax(); + } } /** @@ -2631,19 +2567,13 @@ bool mod_delayed_work_on(int cpu, struct workqueue_struct *wq, struct delayed_work *dwork, unsigned long delay) { unsigned long irq_flags; - int ret; + bool ret; - do { - ret = try_to_grab_pending(&dwork->work, WORK_CANCEL_DELAYED, - &irq_flags); - } while (unlikely(ret == -EAGAIN)); + ret = work_grab_pending(&dwork->work, WORK_CANCEL_DELAYED, &irq_flags); - if (likely(ret >= 0)) { - __queue_delayed_work(cpu, wq, dwork, delay); - local_irq_restore(irq_flags); - } + __queue_delayed_work(cpu, wq, dwork, delay); - /* -ENOENT from try_to_grab_pending() becomes %true */ + local_irq_restore(irq_flags); return ret; } EXPORT_SYMBOL_GPL(mod_delayed_work_on); @@ -4219,16 +4149,7 @@ static bool __cancel_work(struct work_struct *work, u32 cflags) unsigned long irq_flags; int ret; - if (cflags & WORK_CANCEL_DISABLE) { - ret = work_grab_pending(work, cflags, &irq_flags); - } else { - do { - ret = try_to_grab_pending(work, cflags, &irq_flags); - } while (unlikely(ret == -EAGAIN)); - - if (unlikely(ret < 0)) - return false; - } + ret = work_grab_pending(work, cflags, &irq_flags); work_offqd_unpack(&offqd, *work_data_bits(work)); @@ -4243,22 +4164,9 @@ static bool __cancel_work(struct work_struct *work, u32 cflags) static bool __cancel_work_sync(struct work_struct *work, u32 cflags) { - struct work_offq_data offqd; - unsigned long irq_flags; bool ret; - /* claim @work and tell other tasks trying to grab @work to back off */ - ret = work_grab_pending(work, cflags, &irq_flags); - - work_offqd_unpack(&offqd, *work_data_bits(work)); - - if (cflags & WORK_CANCEL_DISABLE) - work_offqd_disable(&offqd); - - offqd.flags |= WORK_OFFQ_CANCELING; - set_work_pool_and_keep_pending(work, offqd.pool_id, - work_offqd_pack_flags(&offqd)); - local_irq_restore(irq_flags); + ret = __cancel_work(work, cflags | WORK_CANCEL_DISABLE); /* * Skip __flush_work() during early boot when we know that @work isn't @@ -4267,19 +4175,8 @@ static bool __cancel_work_sync(struct work_struct *work, u32 cflags) if (wq_online) __flush_work(work, true); - work_offqd_unpack(&offqd, *work_data_bits(work)); - - /* - * smp_mb() at the end of set_work_pool_and_clear_pending() is paired - * with prepare_to_wait() above so that either waitqueue_active() is - * visible here or !work_is_canceling() is visible there. - */ - offqd.flags &= ~WORK_OFFQ_CANCELING; - set_work_pool_and_clear_pending(work, WORK_OFFQ_POOL_NONE, - work_offqd_pack_flags(&offqd)); - - if (waitqueue_active(&wq_cancel_waitq)) - __wake_up(&wq_cancel_waitq, TASK_NORMAL, 1, work); + if (!(cflags & WORK_CANCEL_DISABLE)) + enable_work(work); return ret; } @@ -4363,8 +4260,8 @@ EXPORT_SYMBOL(cancel_delayed_work_sync); * will fail and return %false. The maximum supported disable depth is 2 to the * power of %WORK_OFFQ_DISABLE_BITS, currently 65536. * - * Must be called from a sleepable context. Returns %true if @work was pending, - * %false otherwise. + * Can be called from any context. Returns %true if @work was pending, %false + * otherwise. */ bool disable_work(struct work_struct *work) { @@ -4395,8 +4292,8 @@ EXPORT_SYMBOL_GPL(disable_work_sync); * Undo disable_work[_sync]() by decrementing @work's disable count. @work can * only be queued if its disable count is 0. * - * Must be called from a sleepable context. Returns %true if the disable count - * reached 0. Otherwise, %false. + * Can be called from any context. Returns %true if the disable count reached 0. + * Otherwise, %false. */ bool enable_work(struct work_struct *work) { From patchwork Fri Feb 16 18:04:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 202293 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:c619:b0:108:e6aa:91d0 with SMTP id hn25csp691327dyb; Fri, 16 Feb 2024 10:09:30 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCXxrhZSd7vbafqwGi0WsKpRNHtgNtIVqWdTmfgLaFFdvIVii4ymK5LBwbMSlCbFLb78hHpZN3SH6B5RqpRAchYBUfneIw== X-Google-Smtp-Source: AGHT+IFUghXZuTI9ZOqDCOJtkrta3kYcFhCMzj/D88ipbpWlLTwr26im2YqHt+jqPJoHUjP+ibDn X-Received: by 2002:a17:90a:6f41:b0:299:466c:dc0d with SMTP id d59-20020a17090a6f4100b00299466cdc0dmr1113750pjk.46.1708106969977; Fri, 16 Feb 2024 10:09:29 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708106969; cv=pass; d=google.com; s=arc-20160816; b=de51A/WOigKD2ozCj5CH5gCF6/6MkmlLOR7ySoeMUsZKCpDHdY5QDDymxfOlCHmhYo j6A4r8csYJzKJeX27DMqvvkA9zX8js3a+5Z66P6FcYRsUWixXe6ljE8F6I9IZMmzniLd /fU48DIECRyRv48FGYiiN1COo5i2MHJ9FzHXbZcpzwuUGFsNU4/SDWRLLZReiUyv53WK yuxsX6Dfg/1oMFsXHrTk4+f0jUdRYt08Na4q42GQl+FR5kfcrPzYQzoTqv/IGeoBZIpi f+RxXMa2PZ4mXnlbHz+h/HQTX3+NaqyAQ0av44Yr3Lk4DP37hh2Rx/FHWt6i6hOFTpQc dexQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:sender:dkim-signature; bh=Zk4Yw7otsNk9FpXH6NBaGmePt/Pu9b2Fl+Zht6UJW3M=; fh=ybhW3CNWd9P8h+lCFsSy3/9b6oByce2DZYLVIcDMdYA=; b=Mz3h8DDMmtRX6A7PDuE/itfwDhc9r6m/LMmCEGASrS2nCS1IBekFSC6AruprVaFkki BiGbOOf3hMQG+X1iwdgzmeN2rjeK8cyqKSKoRij9H0NOumCvyX5AvRZHb4U8lWdNk0rV aUJHvntwm34+LqtwC9EXrYxFXE0Dibw6yYhWNfsRhwV5er7RExx5qUGwMkw8IiH7GLaz xSIrkpyKYCoLOcZF2nlLjYXchSC5aNBlv8xSqUzz5zWXEM0d4moeSaLCe1FG73ZTR7ew sFpygk+GNRav6Nq65V1iagsK//Y7CSYWP2s0HBaFM4r2c53L9WGGG9CZl2c00neewobV /9+Q==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=GqMkOVwh; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69132-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69132-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id gv22-20020a17090b11d600b0029929e66a2fsi293132pjb.147.2024.02.16.10.09.29 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:09:29 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-69132-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=GqMkOVwh; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69132-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69132-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id C8506282072 for ; Fri, 16 Feb 2024 18:09:29 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 86B7A1350FF; Fri, 16 Feb 2024 18:06:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GqMkOVwh" Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E31DE145B09 for ; Fri, 16 Feb 2024 18:06:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106801; cv=none; b=i9wCor4jx/Qk4Whf4zbNmLaiAVXM2qt2FcwoY37IqBrzF2yw8K+jT5QpO3Hwz1LFLglMFStm7v8YMzUTdHpXj63A39Pvyx4DSvoL3cRwIZt3/v+wDx5iCIF7cW+RDxEJYCvpcftFOfxx+ytQjsGmVDA/dtC2gwInAyt6GVbAERE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106801; c=relaxed/simple; bh=z0GyY03vKK4q1rF1q8Nt5aQw7DEoslNs5/981jnsvW0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ukRPaNZm/m85+VzzBxy1XqlUGpLZ/zKpOsdCWII83fQTXmRhXysZ9tnhF2gfWu4uMYv/tXlvGGrKsCRrMP3/Yiav9b7bNQrFxPRTKCZG+cjx000ksy6hu+XrqdnqBtgdm2fOMYEyPA9doB7FRbW2gkCLHlcYPhciG97O0olL02E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=GqMkOVwh; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-1db51e55023so15744985ad.1 for ; Fri, 16 Feb 2024 10:06:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708106799; x=1708711599; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=Zk4Yw7otsNk9FpXH6NBaGmePt/Pu9b2Fl+Zht6UJW3M=; b=GqMkOVwhuaK15WeXyfpOf/8VwkZGR2kY0Pi5tX1jOXiKhotyFxd8IZFaHztDbW4Ahx Qi+4C+E0IW8NX1OAnrqhmHfJDdXcuY4s0IAsc9n+mRizGErC88xxh3CdIWgxk/dTHrdR p8JgkSM5EnnVbffELdxkQuktFjv5ccsppWCtcj0EL9Cx6VrEJ2yEFXH1mmp98Vh82RRF 5UOlmYionERCyBo6p/JFEs0ItoYjIqpaqkSrVmPxjkpe3STnh/lmgddaLOdk7+qr7mMA Ik8I1bPTWmb7tI8WC8YtkZr4KaGPoM9PV5wEM6TBsxxykrXkMi8uv075xgs6XiTX0bo1 30LQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708106799; x=1708711599; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Zk4Yw7otsNk9FpXH6NBaGmePt/Pu9b2Fl+Zht6UJW3M=; b=jH16EgS00kl2XUyEq35Wpor8w15oPV2KKlemr6qPRayuo4mybQxq1BvbcscYu2rIUy 6L+luN5h9FEwjJ0vz7ODRNAT9Tp8K4vuA1W3LiOoNFbpcCmF7Jf/qvPPsxBHSzDr3ghR 66pVelxaniyOnXI33tkt6NTTBMigexoeFzQGUx1EtZ/4HyagyhJ+pbuliDAeH9A4FM2P EnXCtfIRlHjsFxkVcqFbLp+9NFm3hM3yuiHerse/KigzY/sPRvRs8JFlIENpTnfES2dY D+3HJz006xV7S5liRYNwxhqZ7pJvBgQHgV98tAzw31aUzqr8E0UwZO+516S5uoAi5eu0 qIUQ== X-Forwarded-Encrypted: i=1; AJvYcCVCCATgZk+O+jH0vRk5aflmFAtk3OkXRmDoRReRQIkGY+sxkCTxBErxIfpoNbUterXDq0B3zIpfrc9Yf7jNsiBfA6MFrl3rxuY82cA2 X-Gm-Message-State: AOJu0Ywu8MitfxioC8v9SO+Vel3akTNqGhrTDzcP038lE1auOBkVGnBB hDWuMvxJqnSA9hnCXUhRYSyJXmXfW81FHirQMrqeEYNerAD1hrXM X-Received: by 2002:a17:903:2593:b0:1d9:9c67:58eb with SMTP id jb19-20020a170903259300b001d99c6758ebmr5435411plb.30.1708106799261; Fri, 16 Feb 2024 10:06:39 -0800 (PST) Received: from localhost (dhcp-141-239-158-86.hawaiiantel.net. [141.239.158.86]) by smtp.gmail.com with ESMTPSA id kn14-20020a170903078e00b001d8f82c61cdsm146168plb.231.2024.02.16.10.06.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:06:38 -0800 (PST) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, allen.lkml@gmail.com, kernel-team@meta.com, Tejun Heo Subject: [PATCH 14/17] workqueue: Remember whether a work item was on a BH workqueue Date: Fri, 16 Feb 2024 08:04:55 -1000 Message-ID: <20240216180559.208276-15-tj@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240216180559.208276-1-tj@kernel.org> References: <20240216180559.208276-1-tj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791079973775349360 X-GMAIL-MSGID: 1791079973775349360 Add an off-queue flag, WORK_OFFQ_BH, that indicates whether the last workqueue the work item was on was a BH one. This will be used to sanity check the context cancel_sync operations can be called from for the work item. Signed-off-by: Tejun Heo --- include/linux/workqueue.h | 4 +++- kernel/workqueue.c | 10 ++++++++-- 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h index 86483743ad28..7710cd52f7f0 100644 --- a/include/linux/workqueue.h +++ b/include/linux/workqueue.h @@ -52,9 +52,10 @@ enum work_bits { * * MSB * [ pool ID ] [ disable depth ] [ OFFQ flags ] [ STRUCT flags ] - * 16 bits 0 bits 4 or 5 bits + * 16 bits 1 bit 4 or 5 bits */ WORK_OFFQ_FLAG_SHIFT = WORK_STRUCT_FLAG_BITS, + WORK_OFFQ_BH_BIT = WORK_OFFQ_FLAG_SHIFT, WORK_OFFQ_FLAG_END, WORK_OFFQ_FLAG_BITS = WORK_OFFQ_FLAG_END - WORK_OFFQ_FLAG_SHIFT, @@ -98,6 +99,7 @@ enum wq_misc_consts { }; /* Convenience constants - of type 'unsigned long', not 'enum'! */ +#define WORK_OFFQ_BH (1ul << WORK_OFFQ_BH_BIT) #define WORK_OFFQ_FLAG_MASK (((1ul << WORK_OFFQ_FLAG_BITS) - 1) << WORK_OFFQ_FLAG_SHIFT) #define WORK_OFFQ_DISABLE_MASK (((1ul << WORK_OFFQ_DISABLE_BITS) - 1) << WORK_OFFQ_DISABLE_SHIFT) #define WORK_OFFQ_POOL_NONE ((1ul << WORK_OFFQ_POOL_BITS) - 1) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index e9d25e1f79d7..6a2abc81ae2b 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -759,6 +759,11 @@ static int work_next_color(int color) return (color + 1) % WORK_NR_COLORS; } +static unsigned long pool_offq_flags(struct worker_pool *pool) +{ + return (pool->flags & POOL_BH) ? WORK_OFFQ_BH : 0; +} + /* * While queued, %WORK_STRUCT_PWQ is set and non flag bits of a work's data * contain the pointer to the queued pwq. Once execution starts, the flag @@ -2111,7 +2116,8 @@ static int try_to_grab_pending(struct work_struct *work, u32 cflags, * this destroys work->data needed by the next step, stash it. */ work_data = *work_data_bits(work); - set_work_pool_and_keep_pending(work, pool->id, 0); + set_work_pool_and_keep_pending(work, pool->id, + pool_offq_flags(pool)); /* must be the last step, see the function comment */ pwq_dec_nr_in_flight(pwq, work_data); @@ -3154,7 +3160,7 @@ __acquires(&pool->lock) * PENDING and queued state changes happen together while IRQ is * disabled. */ - set_work_pool_and_clear_pending(work, pool->id, 0); + set_work_pool_and_clear_pending(work, pool->id, pool_offq_flags(pool)); pwq->stats[PWQ_STAT_STARTED]++; raw_spin_unlock_irq(&pool->lock); From patchwork Fri Feb 16 18:04:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 202294 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:c619:b0:108:e6aa:91d0 with SMTP id hn25csp691618dyb; Fri, 16 Feb 2024 10:09:50 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWnRI4mMrPq0adoGNyTYYze3HYmN+D+LbD1bS1HJ08DcI8i07Ttb8TzTAyBPuPtlFHvUOI94bfPHWMoChvr/9BiV7eHZA== X-Google-Smtp-Source: AGHT+IGUZ65s66kXDYqcT+jMa6nha0eeWX0xkFFDJMNXGIUdHBf2+CPQJhNer7UJ+uY5dpD+5sVT X-Received: by 2002:a67:e3cd:0:b0:46e:d77f:65cc with SMTP id k13-20020a67e3cd000000b0046ed77f65ccmr6134625vsm.28.1708106990009; Fri, 16 Feb 2024 10:09:50 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708106989; cv=pass; d=google.com; s=arc-20160816; b=k1TEv43l+ad5OzyscQlkG6gqod2A8qaPoM+jgc6nQCPOKviTQOkFIQGYmR+0QSniPX 6G2QvEx4HmCoYGcQA2iXIm4/WprrZHi2Fv5YLAxux0nivP6pHQ+f3IoQK8huNFD8daRv ZJb/9+3CA2QN5pbBIExLyj22lHu+6XUSYiAhTFCs40z9ubK60CNSb+b3PbdJTQ5/KTYT wn30tshC5qsZ42RFiNmAalJzbWTRgz4VTmG/YQl++8/8xP0L6GqhHjQ9XaB2YQ9X5aSH hyxhTcJQZHKXm4YmpN1S2G9+jckuGrk8nri0WhrYUMYeQJUUd5DXTSpqbMx3oocfzckh tG0A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:sender:dkim-signature; bh=4xh5X4jP++uuYjnXCzqnuN7YFRsni8pg3IhpUUiVDrs=; fh=3xcE0Yd7si6KXTUg1D2wjIX/6P5lVhhHypU0OOCNbsM=; b=B1MvzIa2L+XWe1VQnzmc3itStdyI/HzNoYx4lAyh0uHOEwTEcAARO2QCsMPdVxrseH 0eQsqJqDOff41ObhI54Jj6V6nLojFWEHwE0QDpp5nOugtUPTBRNSBInKRqtFZjj0sZWT YVc7hZZcSx3PYFEp5VAAkLDtKnU80QuMKKlSPCWqpP8DYM4p7OUIdN7ozos0iJKbb8/+ /OuTuKrt7LIDLkZ2XbDH/6JqCce+Xbyj3t131nYxn03b1pVcs+glhAnp2bCe+rcJ9CJq nAmv3aOLK3T3W31ZCMiglx8u9fZ+ouxUf/z8Nf58sIr7RPZQsDyxPqFB33MoqC1C3l9h S8JA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b="HxQg/DY1"; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69133-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69133-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id b7-20020a0561023a0700b0046d3399b832si121254vsu.305.2024.02.16.10.09.49 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:09:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-69133-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b="HxQg/DY1"; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69133-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69133-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 7F4091C21E86 for ; Fri, 16 Feb 2024 18:09:49 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 91B871468E9; Fri, 16 Feb 2024 18:06:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="HxQg/DY1" Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D01CC145FE7 for ; Fri, 16 Feb 2024 18:06:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106803; cv=none; b=gRtk5Hw4EiGKFBGje4Wb6SD5ipWy0LuKyAQ3bybpWUXy+I3NKiBapk8EW+w5MWLnJufN0uu8pi2dUavvdurrfi3PssmMF/EtHbSjdXK/082F0sSVoxxKgJoH1JdPX6TvbmlAS4UrH0P1oVfHfiyy+bhUC+3io8T++pSH8BjGoaw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106803; c=relaxed/simple; bh=GeKVjnbdns5bK22n91xVMXpiQjcDzrlmZrUwQuwmJNg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XLfaZ14KT9vainHJ73McKL8UTRy68bxtYdUMM13/+rnu7gjdxO8bsSUDPM7VaSpaIDPOV3VZc2ByPrNjM/Xw+VAM5nhh7ChFfjxRlsiW8+CJP5vIQlden4kSYWXV9Vb6JNf2Upk5FO6ze4daG+Vl8oiSXqqSMjH1wmJdGmYLLYk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=HxQg/DY1; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-1d934c8f8f7so24377895ad.2 for ; Fri, 16 Feb 2024 10:06:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708106801; x=1708711601; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=4xh5X4jP++uuYjnXCzqnuN7YFRsni8pg3IhpUUiVDrs=; b=HxQg/DY19kgVTQYYQeSe163DCTvpVZfhgbSGL/0kk9ctmF1v6fkVYnDdLfsaU3+b5o +k3vsAg7RiTb6QX1rHLwxajUgMVCW5B09mgV7e5y+7qPOKZJXeTLQsDaBzGLb+deHZdu 9s+LZayjhU6Roy+Q1/ZdN8jEKQzHHB4B/mncFzMTu9w5N5gMsO01AMm7bdCyUU2Qj+o5 rEkUzYbH8KSI3hlE8DCl6k9ZkklGAU3KbuNMs7Vxzj8IDbaLCH0QFVO1ZSuhcuzCdmCa 81TuMerbDPv9tVo1iRdkf/EToAzoVOw2LuzmYTQO0nOCmTJ8iY3qmS33d7CDEWp8IzOE un6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708106801; x=1708711601; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=4xh5X4jP++uuYjnXCzqnuN7YFRsni8pg3IhpUUiVDrs=; b=WSrd3vj7wsPPiF2fS/Vqy9dKBXR97W4l/awTOZQgN4rzu5O6DU6a4gG/OikpA3VoN+ myNXovh0oR4+s8rmLbQFhk+aWYNy9AOrWaitLU5625OvK5NvdjNwNLvylu/RuBcD9TjU 6BbZVbJiGoPudOppkrJw27j4NFqdZ0aRneEc5ClOFGwwJtmDsMJIxKPTPQL5tJ0c+gK9 dGxYWRnUrsqffdnGQLvkHYgDttJlyDoYqGOYmyKv6xwxbpDTE+ngF0FbMSxzmdutxIR6 1TvGzsNtboGKDXtNu13ivXp7cwb1w63WKH8QpAresbYAhOwSCQ1J/Jgc8PVRw3772tVM ynTw== X-Forwarded-Encrypted: i=1; AJvYcCUZp4PESHbAmnCBIp5/gCiK+4dzWbnuwxSxNadOlaexb+lN8dzzTfOJNZlg4XMQpaI9/yNzdfNRLJ10cWCu+CCVojGDi/W0vw8wXw6v X-Gm-Message-State: AOJu0Yzt5a6iK1FddYS/pPZttM1Ay3hBT8j3nCraCOgmKTU0G1qtsujz nb5/qUOC5GP3p/Px2JdGLTijJR/R9n73VjphnFinjcxPy8PhlcwH X-Received: by 2002:a17:903:1c2:b0:1db:a2bc:22bb with SMTP id e2-20020a17090301c200b001dba2bc22bbmr3223151plh.41.1708106800941; Fri, 16 Feb 2024 10:06:40 -0800 (PST) Received: from localhost (dhcp-141-239-158-86.hawaiiantel.net. [141.239.158.86]) by smtp.gmail.com with ESMTPSA id e17-20020a17090301d100b001d720a7a616sm146700plh.165.2024.02.16.10.06.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:06:40 -0800 (PST) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, allen.lkml@gmail.com, kernel-team@meta.com, Tejun Heo Subject: [PATCH 15/17] workqueue: Update how start_flush_work() is called Date: Fri, 16 Feb 2024 08:04:56 -1000 Message-ID: <20240216180559.208276-16-tj@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240216180559.208276-1-tj@kernel.org> References: <20240216180559.208276-1-tj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791079995125190634 X-GMAIL-MSGID: 1791079995125190634 In prepartion of in-BH canceling of BH work items, update start_flush_work() so that: - rcu_read_lock()'ing is moved to the caller. - Instead of true or false, it now returns the worker_pool associated with the work item if the work item needs to be waited for. NULL if waiting is not needed. - Add a WARN if it encounters a queued work item when @from_cancel. This shouldn't happen. No behavior changes are intended. Signed-off-by: Tejun Heo --- kernel/workqueue.c | 39 ++++++++++++++++++++++----------------- 1 file changed, 22 insertions(+), 17 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 6a2abc81ae2b..f6ea25628701 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -3995,8 +3995,9 @@ void drain_workqueue(struct workqueue_struct *wq) } EXPORT_SYMBOL_GPL(drain_workqueue); -static bool start_flush_work(struct work_struct *work, struct wq_barrier *barr, - bool from_cancel) +static struct worker_pool *start_flush_work(struct work_struct *work, + struct wq_barrier *barr, + bool from_cancel) { struct worker *worker = NULL; struct worker_pool *pool; @@ -4005,12 +4006,9 @@ static bool start_flush_work(struct work_struct *work, struct wq_barrier *barr, might_sleep(); - rcu_read_lock(); pool = get_work_pool(work); - if (!pool) { - rcu_read_unlock(); - return false; - } + if (!pool) + return NULL; raw_spin_lock_irq(&pool->lock); /* see the comment in try_to_grab_pending() with the same code */ @@ -4018,6 +4016,12 @@ static bool start_flush_work(struct work_struct *work, struct wq_barrier *barr, if (pwq) { if (unlikely(pwq->pool != pool)) goto already_gone; + /* + * Cancel path should already have removed @work from worklist + * in try_to_grab_pending(). Control should get here iff we need + * to wait for the current execution to finish. + */ + WARN_ON_ONCE(from_cancel); } else { worker = find_worker_executing_work(pool, work); if (!worker) @@ -4045,17 +4049,16 @@ static bool start_flush_work(struct work_struct *work, struct wq_barrier *barr, if (!from_cancel && (wq->saved_max_active == 1 || wq->rescuer)) touch_wq_lockdep_map(wq); - rcu_read_unlock(); - return true; + return pool; already_gone: raw_spin_unlock_irq(&pool->lock); - rcu_read_unlock(); - return false; + return NULL; } static bool __flush_work(struct work_struct *work, bool from_cancel) { struct wq_barrier barr; + struct worker_pool *pool; if (WARN_ON(!wq_online)) return false; @@ -4063,13 +4066,15 @@ static bool __flush_work(struct work_struct *work, bool from_cancel) if (WARN_ON(!work->func)) return false; - if (start_flush_work(work, &barr, from_cancel)) { - wait_for_completion(&barr.done); - destroy_work_on_stack(&barr.work); - return true; - } else { + rcu_read_lock(); + pool = start_flush_work(work, &barr, from_cancel); + rcu_read_unlock(); + if (!pool) return false; - } + + wait_for_completion(&barr.done); + destroy_work_on_stack(&barr.work); + return true; } /** From patchwork Fri Feb 16 18:04:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 202295 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:c619:b0:108:e6aa:91d0 with SMTP id hn25csp691744dyb; Fri, 16 Feb 2024 10:09:59 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWjnGXzapF4i9K6QuMckwjwGnCQj0aF58L8zqkF0uGsvf8djofkO0bfm3LZZI9NlJO/Fg54UGuYV0ItF8KjmHRuLLw8CA== X-Google-Smtp-Source: AGHT+IHA6syisaw4OXMiCVQ+WNnXZBR5xbcLaqi2IirEZ2f83tFiqrNDcXf8RD/FQU7v/BeXhnwp X-Received: by 2002:a05:6102:2a70:b0:46e:c5c5:6e00 with SMTP id hp16-20020a0561022a7000b0046ec5c56e00mr5083556vsb.33.1708106999251; Fri, 16 Feb 2024 10:09:59 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708106999; cv=pass; d=google.com; s=arc-20160816; b=QvLqG3b8XvkQV7Y0DEhCOi8Z6FCwQw5ZrjvjzX8ngFcRQL92l1S6THMb9YCyIkONXD 8NXOY7yOEbeVQOQ8ShbG+qTYyD2sUMVVx6LUZZYQCKkTk3SK1fvbgbLqAAQiOHqf9GJK tPNM8q10k12WytRoH+R7L102e2RTSIiKP2kvJwJjCqTDSqlaNSm06Ryz8Zgx8Xo7TvE5 7qIMEkhgd/a8IwfXzsXW9mqXj7+qgUtSBXdLBDSR7N/EYeP85h9NPloZ+sMfi6dK8qER u5nzQXlBnKVYzLc2AcMA1/Gyum3LRaDluVnlRaNZ+9cuAIl0NsBt37n7sESThlsk1Nr7 NQ+A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:sender:dkim-signature; bh=N9bk7H/R6dZtCT0NyxymPB9Me8HfbuqTGpGRRxjeyPY=; fh=hPPepoEKa3aQxzY69A4XiFblvkPIJsKxMx/UnNtDBzs=; b=F04KEq7eAuuu8ie34ndtQrBWS3q4QbGyCSHTN//tQTcLN2Lnri+9bEvw0iJgZVQhkn eL/1JByrzsTxcDObnAPZnAu5qD+BWyLLjGkvqHB8gdEnvWSKwxg8o0Zo7hNfS18Jzol5 AbQpqZAAnukv0e5YAHuiFo7KU84ZVF43buVVh8ehWNIMbXdEMvlhQfabvYndaa4sE8cA 1acBXB0BgOktezfa5WrrmtVe1qM4MLAQn2DzTH6xgyVVmd/EDw+GSWTEo4mwfgkFih/s mURqFepg36Ri0jD0qCXsfssjSGFa4+7ABY12qsmbUpScqQ0LTQA01cFbC47qREqVl0PX 60jA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=Q5FE57mY; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69134-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69134-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id o42-20020a0561023faa00b0046d13cef75fsi117843vsv.293.2024.02.16.10.09.59 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:09:59 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-69134-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=Q5FE57mY; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69134-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69134-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 67B411C21F82 for ; Fri, 16 Feb 2024 18:09:58 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7E720146915; Fri, 16 Feb 2024 18:06:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Q5FE57mY" Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 45848146008 for ; Fri, 16 Feb 2024 18:06:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106804; cv=none; b=SDKxA8kHjtGtIaGZUuFMmgSKorozY3mB290kcT7m+CwArKFHGjyxggAizn52hcFHwt4QZ8avkw3Wv+jcYkQu2zfojhkrun+Fvo43VdoU3gQoqNehtS5mzuviqOII/R95YGpU1oAY0XOdr2PSlaOOf+vhyVGY9WSMqhhYqgoPm8I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106804; c=relaxed/simple; bh=AGf1jpF8ueWGlL1VPqRKidiHEBIaTbTQYlw7Y3Hn+5I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZcxjgudJPpzeyzfxzjXuVK/8Hb6BbJ7NyEpeXFiIznB4J5/5i7KbUv0szhsvHMhjUpAfEP0kM58n0oSxBlf1GXNdAN9cLUunqST93ca4nfATIT/puhyUTkX3ZXQ8p25j+Av3xopvVvmot7cZ0bKirSap5vjhudB9a/1Kzeh6q+4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Q5FE57mY; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-1d93edfa76dso8835175ad.1 for ; Fri, 16 Feb 2024 10:06:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708106803; x=1708711603; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=N9bk7H/R6dZtCT0NyxymPB9Me8HfbuqTGpGRRxjeyPY=; b=Q5FE57mYI4SwF31ybzAyFyLBzpjr+kn5mAiyU5FySGAqaBTDoJZuxjDb+wrb+dvPVb id2ZJksRE1FDB4roeR8mkB7KGmRdAz0zhnDSrM9rd2BWphAoZaS/Z3XYc1uKbFEJ5CyR JyPn8iQft2Ihwu5KkICUPo2M52u3zfijOslGq87rvrBIMV3ETGKX64Fwc6fh7FfuTAu4 3f3BC7u4du8VD+ivqE3ubSp/NH24lTNXz4oE49NWOhsGPkSiCNiPIIlRnEHbOpVbxEps yB3Ofb0PU3YAHKZjGHr6PmD7v5k3HO02hmh3upFat1oR3l8nbqMjyJoNlG6J67QjqC7U FzUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708106803; x=1708711603; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=N9bk7H/R6dZtCT0NyxymPB9Me8HfbuqTGpGRRxjeyPY=; b=i22hNDd10vr5uhrgAg/UHcBImto7hsv4+NUWZRwssErcnf6XHHZMf535Xgyp4Ujx5r 4ysxoRxptWvJ1pVRQFWzNTLHP8f/06bHibVdxHQZdi/uZKojk6tuRSZwwk92aYrx8Or3 jyaGuNDUwM1BTTGeIf9vpY9Rkj0ZQpjQa29fAqe0Q4YcW/jfyA3iojAKt+zduOR9mpzC 6qrLQ8Np2/I8R11DTE6lOc8v+JoeRpdrXE0nLctQ01FnvgRv44pTYbHQkex5XS7bKku+ PAfKPd9LRO/UVYlmFup+ILaZXT/+Jb4WnUHSpZ/a3FkcP+XZ0fVejve8pPIi3poilCzW HOQA== X-Forwarded-Encrypted: i=1; AJvYcCU1dE+XI/oNaZ/KX+EgjMmBvJERpB6HOulqaVCESVQ2iKjm07he7meviDIA3iy99RJhGAoD5FXRo+FkK04+Pp+RK+HIOPRKLPyq+6yP X-Gm-Message-State: AOJu0Yy2A9ZasqgwUt7Qy69VUYjqGb3EnTE94f+9PQcY8ZI4NHcQh6sz CK54cGVBlDeLVMIBpzFbJqGZw1F1MM/p00vTg+855V+mjia57aXCcDD9MJm/UP4= X-Received: by 2002:a17:902:dac1:b0:1d9:3101:c46d with SMTP id q1-20020a170902dac100b001d93101c46dmr7033665plx.55.1708106802661; Fri, 16 Feb 2024 10:06:42 -0800 (PST) Received: from localhost (dhcp-141-239-158-86.hawaiiantel.net. [141.239.158.86]) by smtp.gmail.com with ESMTPSA id u12-20020a170903308c00b001da11733477sm160054plc.75.2024.02.16.10.06.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:06:42 -0800 (PST) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, allen.lkml@gmail.com, kernel-team@meta.com, Tejun Heo Subject: [PATCH 16/17] workqueue: Allow cancel_work_sync() and disable_work() from atomic contexts on BH work items Date: Fri, 16 Feb 2024 08:04:57 -1000 Message-ID: <20240216180559.208276-17-tj@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240216180559.208276-1-tj@kernel.org> References: <20240216180559.208276-1-tj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791080004759250528 X-GMAIL-MSGID: 1791080004759250528 Now that work_grab_pending() can always grab the PENDING bit without sleeping, the only thing that prevents allowing cancel_work_sync() of a BH work item from an atomic context is the flushing of the in-flight instance. When we're flushing a BH work item for cancel_work_sync(), we know that the work item is not queued and must be executing in a BH context, which means that it's safe to busy-wait for its completion from a non-hardirq atomic context. This patch updates __flush_work() so that it busy-waits when flushing a BH work item for cancel_work_sync(). might_sleep() is pushed from start_flush_work() to its callers - when operating on a BH work item, __cancel_work_sync() now enforces !in_hardirq() instead of might_sleep(). This allows cancel_work_sync() and disable_work() to be called from non-hardirq atomic contexts on BH work items. Signed-off-by: Tejun Heo --- kernel/workqueue.c | 62 +++++++++++++++++++++++++++++++++++----------- 1 file changed, 47 insertions(+), 15 deletions(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index f6ea25628701..00eac314e62a 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -4004,8 +4004,6 @@ static struct worker_pool *start_flush_work(struct work_struct *work, struct pool_workqueue *pwq; struct workqueue_struct *wq; - might_sleep(); - pool = get_work_pool(work); if (!pool) return NULL; @@ -4072,7 +4070,32 @@ static bool __flush_work(struct work_struct *work, bool from_cancel) if (!pool) return false; - wait_for_completion(&barr.done); + if ((pool->flags & POOL_BH) && from_cancel) { + /* + * We're flushing a BH work item which is being canceled. It + * must have been executing during start_flush_work() and can't + * currently be queued. If @work is still executing, we know it + * is running in the BH context and thus can be busy-waited. + * + * On RT, prevent a live lock when current preempted soft + * interrupt processing or prevents ksoftirqd from running by + * keeping flipping BH. If the tasklet runs on a different CPU + * then this has no effect other than doing the BH + * disable/enable dance for nothing. This is copied from + * kernel/softirq.c::tasklet_unlock_spin_wait(). + */ + while (!try_wait_for_completion(&barr.done)) { + if (IS_ENABLED(CONFIG_PREEMPT_RT)) { + local_bh_disable(); + local_bh_enable(); + } else { + cpu_relax(); + } + } + } else { + wait_for_completion(&barr.done); + } + destroy_work_on_stack(&barr.work); return true; } @@ -4090,6 +4113,7 @@ static bool __flush_work(struct work_struct *work, bool from_cancel) */ bool flush_work(struct work_struct *work) { + might_sleep(); return __flush_work(work, false); } EXPORT_SYMBOL_GPL(flush_work); @@ -4179,6 +4203,11 @@ static bool __cancel_work_sync(struct work_struct *work, u32 cflags) ret = __cancel_work(work, cflags | WORK_CANCEL_DISABLE); + if (*work_data_bits(work) & WORK_OFFQ_BH) + WARN_ON_ONCE(in_hardirq()); + else + might_sleep(); + /* * Skip __flush_work() during early boot when we know that @work isn't * executing. This allows canceling during early boot. @@ -4205,19 +4234,19 @@ EXPORT_SYMBOL(cancel_work); * cancel_work_sync - cancel a work and wait for it to finish * @work: the work to cancel * - * Cancel @work and wait for its execution to finish. This function - * can be used even if the work re-queues itself or migrates to - * another workqueue. On return from this function, @work is - * guaranteed to be not pending or executing on any CPU. + * Cancel @work and wait for its execution to finish. This function can be used + * even if the work re-queues itself or migrates to another workqueue. On return + * from this function, @work is guaranteed to be not pending or executing on any + * CPU as long as there aren't racing enqueues. * - * cancel_work_sync(&delayed_work->work) must not be used for - * delayed_work's. Use cancel_delayed_work_sync() instead. + * cancel_work_sync(&delayed_work->work) must not be used for delayed_work's. + * Use cancel_delayed_work_sync() instead. * - * The caller must ensure that the workqueue on which @work was last - * queued can't be destroyed before this function returns. + * Must be called from a sleepable context if @work was last queued on a non-BH + * workqueue. Can also be called from non-hardirq atomic contexts including BH + * if @work was last queued on a BH workqueue. * - * Return: - * %true if @work was pending, %false otherwise. + * Returns %true if @work was pending, %false otherwise. */ bool cancel_work_sync(struct work_struct *work) { @@ -4287,8 +4316,11 @@ EXPORT_SYMBOL_GPL(disable_work); * Similar to disable_work() but also wait for @work to finish if currently * executing. * - * Must be called from a sleepable context. Returns %true if @work was pending, - * %false otherwise. + * Must be called from a sleepable context if @work was last queued on a non-BH + * workqueue. Can also be called from non-hardirq atomic contexts including BH + * if @work was last queued on a BH workqueue. + * + * Returns %true if @work was pending, %false otherwise. */ bool disable_work_sync(struct work_struct *work) { From patchwork Fri Feb 16 18:04:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 202296 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:c619:b0:108:e6aa:91d0 with SMTP id hn25csp692107dyb; Fri, 16 Feb 2024 10:10:26 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCWOcYDVXONX8FmaOP1sv4Nr3djzQqY24fNXjFXaSiaN2YgK84ak1EwoVxWo1dwgcIfaNEQAna4qX059SI2BfxjlkxcRsQ== X-Google-Smtp-Source: AGHT+IGSchHhLZ6NL6WCmTag96nGP2mxGNiED4nQ94lWignPj0LQtTaf/6zSSMR7to2TWQIeSbSz X-Received: by 2002:a05:6214:21ec:b0:68f:160c:fd8a with SMTP id p12-20020a05621421ec00b0068f160cfd8amr5854768qvj.0.1708107026060; Fri, 16 Feb 2024 10:10:26 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708107026; cv=pass; d=google.com; s=arc-20160816; b=hj0y4SjQt4EhgAa7bf8eifvazMJvXnS8ZvZcPJxZij9amHHOSQ695aCKQKeiNCajPP VyLuufIYoLGRfWQ7nnoZibv/haFdxmjMInZYSl3BL+PALp8UOhfptQE9C1GuNPaNhdsv Qq5L039vyKbsbCgwikHTtQJ2Fw9ebFE/wJuc0c3O52R3graCB98BPzSD8QtK4P5h1p0A 8YklW4eUQT1ibGAstFFDja/Znu3muCjNCtI8qCEOW8q8WAfjgRDLL2CkeVmlSwQkxIsa wTvKPuqcTWzIvJk1wX66WZvMf6MOE0VmU7Jg0wpUmESj5lsSYIO/v7oRcHfN43GzJF/N JjcA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:sender:dkim-signature; bh=znKiBDk/+YyZPZxI19sZt/T09kGaOnS9hAUBe89oizg=; fh=+N0wesq5NoKMYgmaYM67HKUlD8CLvz6PRLMqCdLuH/o=; b=d2cBfe2/JuR6OfiR1kKAmPnAAdYMZpzTGfOpEe6o2gpmQswagwwnj96AEBXCy6RK1D V5MaFLwhkAW2XtSdn8k0ZCVy59vC+OjK5dbtqqYTosSLAZfJ3NROgMHHy64oWgissXfy K5KUE+/8VfldHHdG7Mm8hCGuov0eFnpJmRCwhGN3/ypPo2PH0PYaCPEmF15+k+HT59Wm C+gvLEtb40WYwKRJQzR6C39okmOElcGo8XJXoYOfgr/buzeTTkt0rPrbSzPND9c1LawY 55i0bYKWDqfc9OKS7MYg1098RliTGg+QZr4T/XfduNgeQ3/CAEDA7eo2tZQpwlOkSZpS /Q1Q==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b="k9pIx/ug"; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69135-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69135-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id jz8-20020a0562140e6800b0068c8aaec623si295739qvb.81.2024.02.16.10.10.25 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:10:26 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-69135-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b="k9pIx/ug"; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-69135-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-69135-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 8A3001C21F8F for ; Fri, 16 Feb 2024 18:10:11 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 289301474CB; Fri, 16 Feb 2024 18:06:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="k9pIx/ug" Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 14B9C1468F5 for ; Fri, 16 Feb 2024 18:06:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106808; cv=none; b=ALdeQdmWMdEyyla9Y6gSC5topGx2sMMKaxb/4EAqYOFQiQr0lsRQYKnR5OJn6YheZ3J770gBboRXcxONzbZsuy/jRlBvYDPp/GvnWw8uskc9p9nFdenzbVcc+xy87KMFNOqNCr4AcOikVCi6LRPSlKbMII4DW3JFh40FJ+TiZNg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708106808; c=relaxed/simple; bh=nDGxDIbhTaR2385iAsGzN/VEKZ6fYazF2B4H7Rt3wtk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Bd3OA9LS0eLf1fIUqqHgLfNlAtP4x+Nh5wYDPS90uJAsetLf3iZKJU2TX+5Lysudh4cX/VS9o6OM6GZ97dYl2FEtrEZNQytEiqC1EnSF4ZiDCr4SPAwotvg+n/KGrOyoe95HrMsHSKarLroMAcFNSxVQ+OUvWzv8w8y+HR7ku3I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=k9pIx/ug; arc=none smtp.client-ip=209.85.210.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-6e146dcf0b0so412648b3a.1 for ; Fri, 16 Feb 2024 10:06:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708106805; x=1708711605; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=znKiBDk/+YyZPZxI19sZt/T09kGaOnS9hAUBe89oizg=; b=k9pIx/ugoIt20QG9bkv9o6CbPC13piaH+sCfvcGIXrrGjvxeQ/WHOXdzR0M+2uMnmJ cExWXXtM7A7e9dzpCxs+ZlSAonEb0U/doDO8e6FEZ2bc4sPnSEEWTf+rGwVtQfAtshT5 jpyCk5nJBDabiC/rrsGbggD8umjGY6n+1StRYIsZ5zaGdmJvXDLQf/bUSGSSfmbMCtPO D5IhLpjE01TZvZWBk3nMkI2btOJjTqq1fiihIt4Z4L4bgY7A43zJHpPj/IgkuuiGprGg 7M80rPhD4QqG7kNBFo8Pr+YZCqG8LTdU7mQOHeYcgxcPtPd1e6fUdRT2+FUBEvGkZ2Jv CVmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708106805; x=1708711605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=znKiBDk/+YyZPZxI19sZt/T09kGaOnS9hAUBe89oizg=; b=e/K1nxoUTsngxr9JVhlDwvSp/mBnS/P1lklEFNUoELjpoUVvnzls9Upqg1isyshtXV zeDOJaYrPRGWYs055fCeYUHbkSlVa3fsxnAC3GeT0pwtOfJy+qpivvECDrH4aYwC0Ayf 7P0F0ABfnE3xbsQKz9EQcSRQ0seL9sOEacEwV+Xh1+vSOlEizxOp9aC7rY4lQposL33C EGzPM+bwqr0eqQsU8ntmSbsFimNyuHfUiz04T16PJ2dXuvqsuV46NXLkNKYENn5heMka nqTxxFe12Ph9EvmZ8y9HuTFgNg/yCRhRet5NCHb4yR+1OcvF55E6gujq9N1QwXlcipuW Fisg== X-Forwarded-Encrypted: i=1; AJvYcCVJo4TCNzkOEaDWsfsU/OsNhjDxrI3g6tFSIAbMhT2s0qSmpvBqI5gHU74LKP1cTGZaHAZLla0SNXCt9nABxj836rnhd8IAMxpWvwue X-Gm-Message-State: AOJu0Yx4h85gAf4pWOQONjNn0Iuqb0rMt/5BfcPJ7lCoRJBOeThMzzqC JLVzWL01kv9rO9/go3fB7VvISTOxltSR9/fWSQC74UURKZb1YzOmqrv53zdmigY= X-Received: by 2002:aa7:824f:0:b0:6e1:1e05:e0f5 with SMTP id e15-20020aa7824f000000b006e11e05e0f5mr5430985pfn.12.1708106804488; Fri, 16 Feb 2024 10:06:44 -0800 (PST) Received: from localhost (dhcp-141-239-158-86.hawaiiantel.net. [141.239.158.86]) by smtp.gmail.com with ESMTPSA id x40-20020a056a0018a800b006e0503f467bsm243908pfh.39.2024.02.16.10.06.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Feb 2024 10:06:44 -0800 (PST) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, allen.lkml@gmail.com, kernel-team@meta.com, Tejun Heo Subject: [PATCH 17/17] r8152: Convert from tasklet to BH workqueue Date: Fri, 16 Feb 2024 08:04:58 -1000 Message-ID: <20240216180559.208276-18-tj@kernel.org> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240216180559.208276-1-tj@kernel.org> References: <20240216180559.208276-1-tj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1791080033174666367 X-GMAIL-MSGID: 1791080033174666367 tasklet is being replaced by BH workqueue. No noticeable behavior or performance changes are expected. The following is how the two APIs map: - tasklet_setup/init() -> INIT_WORK() - tasklet_schedule() -> queue_work(system_bh_wq, ...) - tasklet_hi_schedule() -> queue_work(system_bh_highpri_wq, ...) - tasklet_disable_nosync() -> disable_work() - tasklet_disable[_in_atomic]() -> disable_work_sync() - tasklet_enable() -> enable_work() + queue_work() - tasklet_kill() -> cancel_work_sync() Note that unlike tasklet_enable(), enable_work() doesn't queue the work item automatically according to whether the work item was queued while disabled. While the caller can track this separately, unconditionally scheduling the work item after enable_work() returns %true should work for most users. r8152 conversion has been tested by repeatedly forcing the device to go through resets using usbreset under iperf3 generated traffic. Signed-off-by: Tejun Heo --- drivers/net/usb/r8152.c | 44 ++++++++++++++++++++++------------------- 1 file changed, 24 insertions(+), 20 deletions(-) diff --git a/drivers/net/usb/r8152.c b/drivers/net/usb/r8152.c index 9bf2140fd0a1..24e284b9eb38 100644 --- a/drivers/net/usb/r8152.c +++ b/drivers/net/usb/r8152.c @@ -882,7 +882,7 @@ struct r8152 { #ifdef CONFIG_PM_SLEEP struct notifier_block pm_notifier; #endif - struct tasklet_struct tx_tl; + struct work_struct tx_work; struct rtl_ops { void (*init)(struct r8152 *tp); @@ -1948,7 +1948,7 @@ static void write_bulk_callback(struct urb *urb) return; if (!skb_queue_empty(&tp->tx_queue)) - tasklet_schedule(&tp->tx_tl); + queue_work(system_bh_wq, &tp->tx_work); } static void intr_callback(struct urb *urb) @@ -2746,9 +2746,9 @@ static void tx_bottom(struct r8152 *tp) } while (res == 0); } -static void bottom_half(struct tasklet_struct *t) +static void bottom_half(struct work_struct *work) { - struct r8152 *tp = from_tasklet(tp, t, tx_tl); + struct r8152 *tp = container_of(work, struct r8152, tx_work); if (test_bit(RTL8152_INACCESSIBLE, &tp->flags)) return; @@ -2942,7 +2942,7 @@ static netdev_tx_t rtl8152_start_xmit(struct sk_buff *skb, schedule_delayed_work(&tp->schedule, 0); } else { usb_mark_last_busy(tp->udev); - tasklet_schedule(&tp->tx_tl); + queue_work(system_bh_wq, &tp->tx_work); } } else if (skb_queue_len(&tp->tx_queue) > tp->tx_qlen) { netif_stop_queue(netdev); @@ -6824,11 +6824,12 @@ static void set_carrier(struct r8152 *tp) } else { if (netif_carrier_ok(netdev)) { netif_carrier_off(netdev); - tasklet_disable(&tp->tx_tl); + disable_work_sync(&tp->tx_work); napi_disable(napi); tp->rtl_ops.disable(tp); napi_enable(napi); - tasklet_enable(&tp->tx_tl); + enable_work(&tp->tx_work); + queue_work(system_bh_wq, &tp->tx_work); netif_info(tp, link, netdev, "carrier off\n"); } } @@ -6864,7 +6865,7 @@ static void rtl_work_func_t(struct work_struct *work) /* don't schedule tasket before linking */ if (test_and_clear_bit(SCHEDULE_TASKLET, &tp->flags) && netif_carrier_ok(tp->netdev)) - tasklet_schedule(&tp->tx_tl); + queue_work(system_bh_wq, &tp->tx_work); if (test_and_clear_bit(RX_EPROTO, &tp->flags) && !list_empty(&tp->rx_done)) @@ -6971,7 +6972,7 @@ static int rtl8152_open(struct net_device *netdev) goto out_unlock; } napi_enable(&tp->napi); - tasklet_enable(&tp->tx_tl); + enable_work(&tp->tx_work); mutex_unlock(&tp->control); @@ -6999,7 +7000,7 @@ static int rtl8152_close(struct net_device *netdev) #ifdef CONFIG_PM_SLEEP unregister_pm_notifier(&tp->pm_notifier); #endif - tasklet_disable(&tp->tx_tl); + disable_work_sync(&tp->tx_work); clear_bit(WORK_ENABLE, &tp->flags); usb_kill_urb(tp->intr_urb); cancel_delayed_work_sync(&tp->schedule); @@ -8421,7 +8422,7 @@ static int rtl8152_pre_reset(struct usb_interface *intf) return 0; netif_stop_queue(netdev); - tasklet_disable(&tp->tx_tl); + disable_work_sync(&tp->tx_work); clear_bit(WORK_ENABLE, &tp->flags); usb_kill_urb(tp->intr_urb); cancel_delayed_work_sync(&tp->schedule); @@ -8466,7 +8467,8 @@ static int rtl8152_post_reset(struct usb_interface *intf) } napi_enable(&tp->napi); - tasklet_enable(&tp->tx_tl); + enable_work(&tp->tx_work); + queue_work(system_bh_wq, &tp->tx_work); netif_wake_queue(netdev); usb_submit_urb(tp->intr_urb, GFP_KERNEL); @@ -8625,12 +8627,13 @@ static int rtl8152_system_suspend(struct r8152 *tp) clear_bit(WORK_ENABLE, &tp->flags); usb_kill_urb(tp->intr_urb); - tasklet_disable(&tp->tx_tl); + disable_work_sync(&tp->tx_work); napi_disable(napi); cancel_delayed_work_sync(&tp->schedule); tp->rtl_ops.down(tp); napi_enable(napi); - tasklet_enable(&tp->tx_tl); + enable_work(&tp->tx_work); + queue_work(system_bh_wq, &tp->tx_work); } return 0; @@ -9387,11 +9390,12 @@ static int rtl8152_change_mtu(struct net_device *dev, int new_mtu) if (netif_carrier_ok(dev)) { netif_stop_queue(dev); napi_disable(&tp->napi); - tasklet_disable(&tp->tx_tl); + disable_work_sync(&tp->tx_work); tp->rtl_ops.disable(tp); tp->rtl_ops.enable(tp); rtl_start_rx(tp); - tasklet_enable(&tp->tx_tl); + enable_work(&tp->tx_work); + queue_work(system_bh_wq, &tp->tx_work); napi_enable(&tp->napi); rtl8152_set_rx_mode(dev); netif_wake_queue(dev); @@ -9819,8 +9823,8 @@ static int rtl8152_probe_once(struct usb_interface *intf, mutex_init(&tp->control); INIT_DELAYED_WORK(&tp->schedule, rtl_work_func_t); INIT_DELAYED_WORK(&tp->hw_phy_work, rtl_hw_phy_work_func_t); - tasklet_setup(&tp->tx_tl, bottom_half); - tasklet_disable(&tp->tx_tl); + INIT_WORK(&tp->tx_work, bottom_half); + disable_work(&tp->tx_work); netdev->netdev_ops = &rtl8152_netdev_ops; netdev->watchdog_timeo = RTL8152_TX_TIMEOUT; @@ -9954,7 +9958,7 @@ static int rtl8152_probe_once(struct usb_interface *intf, unregister_netdev(netdev); out1: - tasklet_kill(&tp->tx_tl); + cancel_work_sync(&tp->tx_work); cancel_delayed_work_sync(&tp->hw_phy_work); if (tp->rtl_ops.unload) tp->rtl_ops.unload(tp); @@ -10010,7 +10014,7 @@ static void rtl8152_disconnect(struct usb_interface *intf) rtl_set_unplug(tp); unregister_netdev(tp->netdev); - tasklet_kill(&tp->tx_tl); + cancel_work_sync(&tp->tx_work); cancel_delayed_work_sync(&tp->hw_phy_work); if (tp->rtl_ops.unload) tp->rtl_ops.unload(tp);