From patchwork Wed Feb 7 01:19:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 20031 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:168b:b0:106:860b:bbdd with SMTP id ma11csp1931430dyb; Tue, 6 Feb 2024 17:21:56 -0800 (PST) X-Google-Smtp-Source: AGHT+IFGxTfnmMXieTmTiAwy//aUwsYha1sJjC2yMj5NkJNEOk5vbLfJBAkO/4KrDqyTrp+6822D X-Received: by 2002:a05:6358:6a50:b0:178:9b16:5bbe with SMTP id c16-20020a0563586a5000b001789b165bbemr1457310rwh.9.1707268916730; Tue, 06 Feb 2024 17:21:56 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1707268916; cv=pass; d=google.com; s=arc-20160816; b=xV9t5KlkASmkMqRhIfzuRzWx/i6hN+r+DFPOdbE+DR65AofiK5x8DeAqxIOXLs4RA/ JmVsRnoNX8goZ9xJ4epHUAiOC6PyN//ob7AS6C3ZUALf77E8uki9MWxXLz7BrY74YJI6 VKuHCFotHkZlOq7Di+iHFOQYU1c8b/FEQ0smUg/pxadfv41OFW3/c2/ofQRaugmqgiox fEMEAkEJm4xGRvqtKN9bU9NqO3b10wpfFt71YaHsTZzNdawyQgcXLz6uvqQgMDyapdEv Jot8iX4kygzWXVD4HS+hmkMYrZWTp6rOMiNLo3aBhRAl7vMATUrz5pEKdJorDW8c2RUt HtGg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:message-id:date:subject:cc:to :from:dkim-signature; bh=plEIwUaygIL7gqVklYFDo8bIs1ikW11gzA9fRUjic6s=; fh=EVa/e4OdFxi47V1BUiW+d0PcfsrjehnmXz1+W4IGaMc=; b=rMs+zlX2kIJDmcgnPorNg301bawm0evVxPUYlGH9MIrYpD+GFjbcDT+vaEkdTJ66jq g2sJN8k5XiuX8vn/4NCcRTMP3QsWTwW6tZugrl5MUdd97wB8tZH66XyKVzfTgsoOwZq6 yujP7xC1zir4lGsfiO3mLDvlS9/1QlizvodoU/vvhl151juf/FbuTcB3Ff0/nCNxVV9M opxSbGCxrC574hgDyoRL6Y7dPGQhpwKmiS0Xm7FMjjytuOlzMxDdATmjMOyCRgdHvT21 LJk4wMFzykKm48+ITT5llwt+UX6ylyyRp06QePPy3K8qH/XSZquM0cSdXGrJGtDbDZd7 ixqg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=etul6Y8d; arc=pass (i=1 spf=pass spfdomain=redhat.com dkim=pass dkdomain=redhat.com dmarc=pass fromdomain=redhat.com); spf=pass (google.com: domain of linux-kernel+bounces-55792-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-55792-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Forwarded-Encrypted: i=2; AJvYcCVaw6VrOLc+PLP1TBwvinaNUXeLrAnZLgHv14J39jdB8Ap7CwxCjXhA4iUO+rYy6Fu/RlM6BJbGgu/ac5abBOsWgaiZ0w== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id ea12-20020a056a004c0c00b006e03b590869si275175pfb.241.2024.02.06.17.21.56 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Feb 2024 17:21:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-55792-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=etul6Y8d; arc=pass (i=1 spf=pass spfdomain=redhat.com dkim=pass dkdomain=redhat.com dmarc=pass fromdomain=redhat.com); spf=pass (google.com: domain of linux-kernel+bounces-55792-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-55792-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 5ACF9B241F4 for ; Wed, 7 Feb 2024 01:20:51 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CF8D063CF; Wed, 7 Feb 2024 01:19:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="etul6Y8d" Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 365E712E63 for ; Wed, 7 Feb 2024 01:19:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707268778; cv=none; b=L5QQUQgTF0wSig/JFu75wPrNCvoyxKWJ5ITsRLGkTur6F98vGLgAf/RhSP6M4ILmfwDEUIgV9RmD1/zwsre06cxYSMGqof9B/3eaYqtH1G0Und/dr32U0oRV+10a0K4GGHh3ecCehRQC73CBYkJ1NERzo2BeXrD3tA9le69MAk8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707268778; c=relaxed/simple; bh=V376dAOHiR5NDvb/JoHxzkqr2Lx52cRJPyLcJzBT4tw=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=XBvyYCPmb8/hW4Zh3jtIe1QfMORoXRB87GpGISZT9a6nkmVS/AQ8Btwid75eMAtFpC1klPd7wJLKBbv5NunDUC90YLNPhjqi5JnCVQa8BJ/LG2P7dEtLc97K7EwoNbX4Ekm/gJJrNRGFv7y1kHNeHNwvTppV4qi4XYqid4T/A9k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=etul6Y8d; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1707268776; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=plEIwUaygIL7gqVklYFDo8bIs1ikW11gzA9fRUjic6s=; b=etul6Y8dGwhn5ouYP30ED7Xbxi3/6Vc/yvF85j/YLlq+JSjK/MJI7c/c/BoxXgRzRVpeQl 1zOQh7N2qLaFkvk7gZjjBWsDI5SEHhz7eoUw6Q0+93blHj0rLaFkCYT0CZiKX88afbwWRo 0oF+A0F8feVoUyB8ym4R1hCLFX19LsI= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-467-KBcKGl_rNbGKEBFM2mM0WA-1; Tue, 06 Feb 2024 20:19:32 -0500 X-MC-Unique: KBcKGl_rNbGKEBFM2mM0WA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 201F7280A9A1; Wed, 7 Feb 2024 01:19:32 +0000 (UTC) Received: from llong.com (unknown [10.22.16.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9300040C9444; Wed, 7 Feb 2024 01:19:31 +0000 (UTC) From: Waiman Long To: Tejun Heo , Lai Jiangshan Cc: linux-kernel@vger.kernel.org, Juri Lelli , Cestmir Kalina , Alex Gladkov , Phil Auld , Costa Shulyupin , Waiman Long Subject: [PATCH wq/for-6.9 v4 0/4] workqueue: Enable unbound cpumask update on ordered workqueues Date: Tue, 6 Feb 2024 20:19:07 -0500 Message-Id: <20240207011911.975608-1-longman@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1790201166681394862 X-GMAIL-MSGID: 1790201211457250750 v4: - [v3] https://lore.kernel.org/lkml/20240205194602.871505-1-longman@redhat.com/ - Rebase on the latest for-6.9 branch again & discard the use of __WQ_ORDERED_EXPLICIT and resetting of __WQ_ORDERED. - Add a new patch 1 to change the ordering of pwq's in wq->pwqs from oldest to newest. - Change the terminalogy from freeze/thaw to plug/unplug. - Allow more than 2 pwq's in wq->pwqs of ordered workqueue but only the oldest one is unplugged. This eliminates the need to wait for the draining of extra pwq in workqueue_apply_unbound_cpumask(). v3: - [v2] https://lore.kernel.org/lkml/20240203154334.791910-1-longman@redhat.com/ - Drop patch 1 as it has been merged into the for-6.9 branch. - Use rcu_access_pointer() to access wq->dfl_pwq. - Use RCU protection instead of acquiring wq->mutex in apply_wqattrs_cleanup(). v2: - [v1] https://lore.kernel.org/all/20240130183336.511948-1-longman@redhat.com/ - Rebased on top of wq's for-v6.9 branch. - Use the new pwq_tryinc_nr_active() mechanism to freeze the new pwq of an ordered workqueue until the old pwq has been properly drained to maintain ordering. - Make rescuer follow changes in workqueue unbound cpumask as well as its sysfs cpumask, if available. Ordered workqueues does not currently follow changes made to the global unbound cpumask because per-pool workqueue changes may break the ordering guarantee. IOW, a work function in an ordered workqueue may run on a cpuset isolated CPU. This series enables ordered workqueues to follow changes made to the global unbound cpumask by temporaily suspending (plugging) the execution of work items in the newly allocated pool_workqueue until the old pwq has been properly drained. The cpumask of the rescuer task of each workqueue is also made to follow changes in workqueue unbound cpumask as well as its sysfs cpumask, if available. Juri Lelli (1): kernel/workqueue: Let rescuers follow unbound wq cpumask changes Waiman Long (3): workqueue: Link pwq's into wq->pwqs from oldest to newest workqueue: Enable unbound cpumask update on ordered workqueues workqueue: Bind unbound workqueue rescuer to wq_unbound_cpumask kernel/workqueue.c | 111 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 99 insertions(+), 12 deletions(-)