From patchwork Fri Apr 21 02:50:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86118 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp781208vqo; Thu, 20 Apr 2023 20:09:23 -0700 (PDT) X-Google-Smtp-Source: AKy350ZEd7+e0jCe27U15lNqLmxKl7zPIv285GuTlvkpzca0kE0Dp5PgAoIjxtFlpLJW0JXw4e/P X-Received: by 2002:a05:6a20:729a:b0:ef:ead5:6fdf with SMTP id o26-20020a056a20729a00b000efead56fdfmr5192501pzk.33.1682046563403; Thu, 20 Apr 2023 20:09:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682046563; cv=none; d=google.com; s=arc-20160816; b=zmM4Nl1jhBM8Iltn3SwGnY6+8RwTu+ENmctiZiAbLU/jH15LcD2gIjVuWMex5WbphH V8nugx8ceW+nrKnnfYZ+0GtI/MdITEEypq0fMjgzZwmoekVmfZE0LE9q9RGOT2194YsA xIzSLBx+f6IgkEAJyAyxt3eEILEzYo6/4WCW5RUaXPtakNXPDAa0OM2KYDuSinnDl9x2 nXVncvrNLZ5yyvAucTc2rDfADmEOySvMtNo1F6KpHIE8mf/ImqcIw6Hy8j6ghxbnR/xM a1huqLub70e1k4cWnKN8SSiEj9Fv9jf4tXdyFNIrVW9pavVwHF4aOR2py4utqfQ/Z5A6 Mv7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=QmtrNohZohvwwvoXrwix45PfrI45pVOsg46nypoG6vA=; b=Jbw3p4wMdEf916ebDLGKmoB+mXRoaesaeEhzTHC9CAEM866bfkP6uTh7M+khLHSoYL fLspKE+GXF7pPgzUotsE5Nxr5h11bD42OTC/UVpNnjesTaMD9UuqFXif/CBZYzskeNhT grB2tdrklcT1wAfq2DVXjxpLJl2hnchOm0vkl7G86ZrF7gW9UksI4w6gXToX7oAXZ0MP Rl5dNp4p0KQKLZsOkPvSPhLfCBwnGIjXReXJmnp0hDEY+ySsPEk8u/bPj/KIv9LfJ3kn XPKl4oOlfGAgESAaECUajjwSdTr2b+inPSJDel1iTASfaEL1pQkWeMQmKajzmnsarLph fBog== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=ZmpAEUq+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 4-20020a630704000000b00524b56d9e63si3238231pgh.324.2023.04.20.20.09.10; Thu, 20 Apr 2023 20:09:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=ZmpAEUq+; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233647AbjDUCu4 (ORCPT + 99 others); Thu, 20 Apr 2023 22:50:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43018 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233642AbjDUCuy (ORCPT ); Thu, 20 Apr 2023 22:50:54 -0400 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A862E75 for ; Thu, 20 Apr 2023 19:50:53 -0700 (PDT) Received: by mail-pj1-x102b.google.com with SMTP id 98e67ed59e1d1-247048f86c7so1235313a91.2 for ; Thu, 20 Apr 2023 19:50:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045453; x=1684637453; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=QmtrNohZohvwwvoXrwix45PfrI45pVOsg46nypoG6vA=; b=ZmpAEUq+gjuPKgzNFcuAAdXwmm2GgiMJlJhP1UqCHPzgFmzgT/Z6Ri4oGf+vaelLbF bo4eVpMsoWtjWBTio6FfNgAKkq3C5sXwv2quyHuNUvOTEYUuw9ww0QuVP4DrckZQtoJC 3ADJOE5YEviK6cYzpxW9M25DRSL9ZsUGz+3uPJsgTfoqmZtPSEX+tuMrLqWj81i7An02 nxmDWw1C2GX+ThleqlvVhVu0Q6NmK4HTO567n+ZW5hevASnd4N1RERG2XiuEe7P45kwI zmc4aPBM2h1Bl0gmIJeKUixs5hDOsBbS6DfSJXPSEgahAgLQ1PeLj0Mud/zYDJ2Z+2Zb lx7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045453; x=1684637453; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=QmtrNohZohvwwvoXrwix45PfrI45pVOsg46nypoG6vA=; b=Sj6toxp1CfQb4PGuKKkPIMOom0AH94S3E3/OY3Qy9zxiuNcOBa9nRlQKYJEUXdNfzL 3hYTfU4PQjgXqtq8fWxJVhoTGKSyL2xehPkcrYV8hWNnjnyKqHkQILwjTfS6JXcvEebn gGNuAyaGfQfPZ+ThJXhnPGL9ZIdUjDeQAPW3PsGYgPoKmBxR5zAyFXvlHoSgEySldo0r j4kuRgbHOc4DyZn57jcWCAZx8HiI8bZf9yBXVY2N+ewahmq9VSmHQiVLFxIqRNdg5SNf Qe0MDa9mub/hEsHkz2U1VUmorXPQa/LZHAMI5gdgmwJrQYxFo1yuTq61a2/hfKXMl1p7 OchQ== X-Gm-Message-State: AAQBX9eEpCUKoZVAPPu+rgZKw2P+HdyQKm1KDd6gh/lGv2cLGkGMndNP GiHf0sHnc5ZyHEK4gfbJ3NI= X-Received: by 2002:a17:90a:d985:b0:247:193b:ce84 with SMTP id d5-20020a17090ad98500b00247193bce84mr3540463pjv.15.1682045452656; Thu, 20 Apr 2023 19:50:52 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id gg6-20020a17090b0a0600b0023d386e4806sm1742354pjb.57.2023.04.20.19.50.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:50:52 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Nathan Lynch , linuxppc-dev@lists.ozlabs.org Subject: [PATCH 01/22] powerpc, workqueue: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:25 -1000 Message-Id: <20230421025046.4008499-2-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763753657543414105?= X-GMAIL-MSGID: =?utf-8?q?1763753657543414105?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: Michael Ellerman Cc: Nicholas Piggin Cc: Christophe Leroy Cc: Nathan Lynch Cc: linuxppc-dev@lists.ozlabs.org Acked-by: Michael Ellerman (powerpc) --- arch/powerpc/kernel/tau_6xx.c | 2 +- arch/powerpc/platforms/pseries/dlpar.c | 3 +-- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/powerpc/kernel/tau_6xx.c b/arch/powerpc/kernel/tau_6xx.c index 828d0f4106d2..cba6dd15de3b 100644 --- a/arch/powerpc/kernel/tau_6xx.c +++ b/arch/powerpc/kernel/tau_6xx.c @@ -200,7 +200,7 @@ static int __init TAU_init(void) tau_int_enable = IS_ENABLED(CONFIG_TAU_INT) && !strcmp(cur_cpu_spec->platform, "ppc750"); - tau_workq = alloc_workqueue("tau", WQ_UNBOUND, 1); + tau_workq = alloc_ordered_workqueue("tau", 0); if (!tau_workq) return -ENOMEM; diff --git a/arch/powerpc/platforms/pseries/dlpar.c b/arch/powerpc/platforms/pseries/dlpar.c index 75ffdbcd2865..e9117b03807e 100644 --- a/arch/powerpc/platforms/pseries/dlpar.c +++ b/arch/powerpc/platforms/pseries/dlpar.c @@ -564,8 +564,7 @@ int __init dlpar_workqueue_init(void) if (pseries_hp_wq) return 0; - pseries_hp_wq = alloc_workqueue("pseries hotplug workqueue", - WQ_UNBOUND, 1); + pseries_hp_wq = alloc_ordered_workqueue("pseries hotplug workqueue", 0); return pseries_hp_wq ? 0 : -ENOMEM; } From patchwork Fri Apr 21 02:50:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86119 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp781252vqo; Thu, 20 Apr 2023 20:09:29 -0700 (PDT) X-Google-Smtp-Source: AKy350arDrK6+aVUkQGsZJFrDArVGOewV6SZ1N4fmt6I+tyq+N6c5ozh/5aVgEbTVBMwgxAWOPAx X-Received: by 2002:a17:90b:3a88:b0:247:6023:f884 with SMTP id om8-20020a17090b3a8800b002476023f884mr3862701pjb.15.1682046569381; Thu, 20 Apr 2023 20:09:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682046569; cv=none; d=google.com; s=arc-20160816; b=dNV5T7a0+XuCtO6KcTcnplCH6OvrBwBoyBwNqY1U2Cx7HMGC8mPjwijlFPtECTndw7 XrgJ/RyRXa3NMB8dACyHH56W5vTVFJp09pcYL5Qp9daeK/WTcP/SEn7c7oruF9j+1uzD U8SGwDWx9k0/F2g2Y2gKU83i3iA6LoBnm+CGJnZLcXUWiuSnuTcMGzS4Snt9q3HyEZOW BPE0o/OqxXRePMxrvQY5Muir2hm6qUXR1qk2uSHFfYGNIyb0LhLWSRI3ZLhVibgPSi5e tONfoyN3iH7KdEhp9mLEJne3HRx8MQZVOH0CNimRKb8g/vlXnc8ClBzxN7fC/UaYWWNY NYyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=iZd1f2A/dHMRVFWDYKTVlcM/K09Kp/NZDEzNt1pICiQ=; b=rHT5hW4cvEnLv0cTb4MskCkqBz2oA9RH4CX+7l8zm3fepGJblSwr8HDh3Wc5getRBr /Emm2g2Dj5DcBMdcmf/k5h/S9xtPQ6meN7yGDBfkPVudyg2umAl+NCfLF7tMu/vBGY7v 4CNXYxoB2njEYEOUt/TFSCn/Ards1v1ADESEjvF363nti6IjSMDAKNjQYLJOfazx4piu hkxgqX+70LXxY8iU3u79z0pQLvxv4TNbKFmXSx2tPLLlaWwJj/D+rlCrj8Y46Bbvp1An jEV5/MK5wmGtudh3DKqZimnMRoX9mRWR2f6OTxdXr28qinVHrvw7bp7x+fkjccwK9bJv c2pw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=JjUnbLxt; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id c5-20020a6566c5000000b0051a6292309fsi3112888pgw.894.2023.04.20.20.09.17; Thu, 20 Apr 2023 20:09:29 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=JjUnbLxt; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233689AbjDUCvD (ORCPT + 99 others); Thu, 20 Apr 2023 22:51:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43030 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233049AbjDUCu4 (ORCPT ); Thu, 20 Apr 2023 22:50:56 -0400 Received: from mail-pf1-x42e.google.com (mail-pf1-x42e.google.com [IPv6:2607:f8b0:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3A058E75 for ; Thu, 20 Apr 2023 19:50:55 -0700 (PDT) Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-63b62d2f729so1525812b3a.1 for ; Thu, 20 Apr 2023 19:50:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045455; x=1684637455; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=iZd1f2A/dHMRVFWDYKTVlcM/K09Kp/NZDEzNt1pICiQ=; b=JjUnbLxt1gWtb1QGpTKbuG9EbQaQbfiV1lE2b9kWYLdkgiKRAQlLXv/MlBL38U2wRb mBGgmlt+dD/c6WSgRZFG2bcSK6CvQ3/oO4UoB6f8TfRZyPKcb3tc1LHQUAr3puB9TpkQ rfVwglC+yLSTxnSj9AH/RfNxy3RMR0J08NTMiE5Pecdr10VBHBvSIzUTXC0/FK5a4B9P cppn3vl/c5qH8w0RDslEkXuvgEjsLbFiFCf6McyoknkZgZTRH0ib+0nQMUBnbNKuuDpM Njeu075OowgSpTzoifm4E5WTbkrHaWa4sR2SbaiuXIXOTpipzsgh+R9jwYK1IETsIatN APFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045455; x=1684637455; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=iZd1f2A/dHMRVFWDYKTVlcM/K09Kp/NZDEzNt1pICiQ=; b=FrZ5kzgo646Z2pQbQbTp6/eGF7vngx+VTpdkQBjuoHK/G6xF1kOGVULO5EgMQ3U4hD vSNctp/DBjH5jWbmty68vkW4GsBQcwleRP5mGg5PxU5+JtPBCYgsekvyzx+i10C3shdp Cq2ISnaILfxrifAtW4oHkLrJaG6mjp1M7gtyD0yZLQ81XdD//EJoKcMPW/X/6TkJjSP8 +UdCc0zRmeZV/QlhOne3HeFQ5DtIabV+7M/8ISNim7CpIJPCFqW6nTun96l2LrYro0u3 i0BpqE8JLgTIn4v6Oij/N4HIvnzaMJk1riNhUVk20cQVz+YdWHanNh+nBGS518CgjXgK FDIA== X-Gm-Message-State: AAQBX9cl6KlIGYZBj0UDVETQZoV9tY2nQVOz388AWOzLA/lFristSkWK 7A4MGauR+524A7cZKCowGbo= X-Received: by 2002:a05:6a20:e619:b0:f0:4664:ad53 with SMTP id my25-20020a056a20e61900b000f04664ad53mr4774783pzb.48.1682045454474; Thu, 20 Apr 2023 19:50:54 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id a22-20020a056a000c9600b00628e9871c24sm1886589pfv.183.2023.04.20.19.50.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:50:54 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , Johan Hovold , Alex Elder , Greg Kroah-Hartman , greybus-dev@lists.linaro.org Subject: [PATCH 02/22] greybus: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:26 -1000 Message-Id: <20230421025046.4008499-3-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763753663549488873?= X-GMAIL-MSGID: =?utf-8?q?1763753663549488873?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: Johan Hovold Cc: Alex Elder Cc: Greg Kroah-Hartman Cc: greybus-dev@lists.linaro.org Acked-by: Greg Kroah-Hartman Acked-by: Johan Hovold Acked-by: Alex Elder --- drivers/greybus/connection.c | 4 ++-- drivers/greybus/svc.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/greybus/connection.c b/drivers/greybus/connection.c index e3799a53a193..9c88861986c8 100644 --- a/drivers/greybus/connection.c +++ b/drivers/greybus/connection.c @@ -187,8 +187,8 @@ _gb_connection_create(struct gb_host_device *hd, int hd_cport_id, spin_lock_init(&connection->lock); INIT_LIST_HEAD(&connection->operations); - connection->wq = alloc_workqueue("%s:%d", WQ_UNBOUND, 1, - dev_name(&hd->dev), hd_cport_id); + connection->wq = alloc_ordered_workqueue("%s:%d", 0, dev_name(&hd->dev), + hd_cport_id); if (!connection->wq) { ret = -ENOMEM; goto err_free_connection; diff --git a/drivers/greybus/svc.c b/drivers/greybus/svc.c index 16cced80867a..0d7e749174a4 100644 --- a/drivers/greybus/svc.c +++ b/drivers/greybus/svc.c @@ -1318,7 +1318,7 @@ struct gb_svc *gb_svc_create(struct gb_host_device *hd) if (!svc) return NULL; - svc->wq = alloc_workqueue("%s:svc", WQ_UNBOUND, 1, dev_name(&hd->dev)); + svc->wq = alloc_ordered_workqueue("%s:svc", 0, dev_name(&hd->dev)); if (!svc->wq) { kfree(svc); return NULL; From patchwork Fri Apr 21 02:50:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86117 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp781207vqo; Thu, 20 Apr 2023 20:09:23 -0700 (PDT) X-Google-Smtp-Source: AKy350ai+UOfwIWj41LBbqq2u+OL5Oj3F1QnaGKbY+3kFR95F4jroNzovy2GdH2s8n2+X2HzjIk6 X-Received: by 2002:a17:902:ef8a:b0:1a1:dd2a:fe6c with SMTP id iz10-20020a170902ef8a00b001a1dd2afe6cmr3370270plb.53.1682046563373; Thu, 20 Apr 2023 20:09:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682046563; cv=none; d=google.com; s=arc-20160816; b=EGtegwJvOJy/fOXJYcI9bR2yQL3owAW00YD5zMi72co0X7eASYgOlB76k5EebTMNNo tvDLxNIXlt5zq9N9SEFWZkewNNFxD5Vlo8GPNuQ4VswZl/8Kd8uOEhhqOyvZVnpBEo+n ZucVFNZIgQI1X09PvIHsmD5r47otjuxsxJJPnnKMphjXrkMTG3ETp3JpRfd/0vXm+Wm0 U8dTuv2a2atIYeDv0T5aA6r4dgkdQxofJY3LVFWLnSNcF3FUq5gakF8KkPjvHbT/GT8b Yvm/LEqPf2gBmiraozfjWpNkCgBFZdVi8snRmPUwYkvGDHA1+LyDX4FRQVyvUVehl4nu xyJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=BrkvCxlxcddagK2yaLOOMG4Y/CE9pr0ilngxK0X5Vcs=; b=XXO7an0GtIadXIXVAq3sQysUMuL2sP88uPiC60vtCpB/zjO0mBRsKh2UkRiNtfsDW6 yruyAiJJx6sHhQ6BW1MfCISC4jjj5S4yJGzlxGf+pgxXQEDQ6LHlbSKOhEduWHpRgTTU JB8wRCFsMY7pITHEuJ97cuTLpVby2USAVyJlfZD0M36g4Sd6WJs1/afBD/bCdd1EaBqF mHqU30d4D9yGS0VmGCdcUbXT8GFXnb7gPklkpw0RaOmoM4EoQJUzVxZvrTWYDwSVZVxk Dz0bu53sXkDJSTuMyCEffUeHjnpRKLE1PDNO+AC765QX2PSLl5OqIqnopMXEQOlfaVgr eqCA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=c1D0l4VF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u14-20020a170902e80e00b001a6fe422894si3499731plg.200.2023.04.20.20.09.11; Thu, 20 Apr 2023 20:09:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=c1D0l4VF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233678AbjDUCvB (ORCPT + 99 others); Thu, 20 Apr 2023 22:51:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233642AbjDUCu6 (ORCPT ); Thu, 20 Apr 2023 22:50:58 -0400 Received: from mail-pf1-x42e.google.com (mail-pf1-x42e.google.com [IPv6:2607:f8b0:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 148246EB5; Thu, 20 Apr 2023 19:50:57 -0700 (PDT) Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-63b62d2f729so1525824b3a.1; Thu, 20 Apr 2023 19:50:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045457; x=1684637457; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=BrkvCxlxcddagK2yaLOOMG4Y/CE9pr0ilngxK0X5Vcs=; b=c1D0l4VFEN5et1lJc400wBpLI6YQQVyklfQAmpDQO+ssteaWRX5ExAfvZQgLZSRrJa Au338bpjVqaaLNIxrfSdVfpOOfA0sNglhpQTH+d+b1QilXqPfQJOeRD53PhFSZMsVrNb NnO/3jFONXgbh178zX3WuTlIOLys/IdfcGImmOaLEF42bs7ol79o+fbLnZt+N8EJlGNz mZHg9yN6oOr9mT7ymCCx46vyHyyx99zQ6K43mun+15EhQXa9/hHtmPEjPEZJcUJdC1mU 5gkeblIz6Sz+4r8myGRTABi+4XfyUq7rtsoZdDeXD9EPwo6oNc3aQ9YhK1IdGHbWzEKv Jh4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045457; x=1684637457; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=BrkvCxlxcddagK2yaLOOMG4Y/CE9pr0ilngxK0X5Vcs=; b=FrsoDnvUUdUT7LpypTruQXBtS1M2Y6Ddq4nqRiXWfengRvxnE+hWdGYnlSTNANxMrb MJvJRhRYCkRBTrVoaghaVfcJb8IdMnqbiQkIXFsBhPeI6Rq/9ETXbmZSqcRMZOLwn/Ii bygt87HyqNkbsA3ONG2tzc/TMwZMvgrtZuqnJ7mqLjmj2E/oo70uW7MV2a7vDYI5wPgV ++DgfSehZ4ps6hC2eSRVZJ0Kw0wpmzoip4HbsN09HCKfQUnpolyQ/5wNMaXyJTjuliU3 viWXZlaHI+8ZJ/Mz5DNqQcHrD1PyVW5ibSWHXlhfip40SHCbszp4TD3rHwTvUWMJbbmx Kzmw== X-Gm-Message-State: AAQBX9cE1awT4HMuQQxe6jKpzIqhpErZ6R2ZqE60ofDLOEc2aLmGz3iH AwwXM4PKh7RYnk2fYuw3/Rc= X-Received: by 2002:a17:902:da8f:b0:19d:138b:7c4a with SMTP id j15-20020a170902da8f00b0019d138b7c4amr4624417plx.3.1682045456373; Thu, 20 Apr 2023 19:50:56 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id jo13-20020a170903054d00b0019acd3151d0sm1250013plb.114.2023.04.20.19.50.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:50:56 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , Dennis Dalessandro , Jason Gunthorpe , Leon Romanovsky , linux-rdma@vger.kernel.org Subject: [PATCH 03/22] IB/hfi1: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:27 -1000 Message-Id: <20230421025046.4008499-4-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763753657282269366?= X-GMAIL-MSGID: =?utf-8?q?1763753657282269366?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: Dennis Dalessandro Cc: Jason Gunthorpe Cc: Leon Romanovsky Cc: linux-rdma@vger.kernel.org Acked-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/init.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/init.c b/drivers/infiniband/hw/hfi1/init.c index 62b6c5020039..e03d867cda13 100644 --- a/drivers/infiniband/hw/hfi1/init.c +++ b/drivers/infiniband/hw/hfi1/init.c @@ -755,14 +755,13 @@ static int create_workqueues(struct hfi1_devdata *dd) } if (!ppd->link_wq) { /* - * Make the link workqueue single-threaded to enforce + * Make the link workqueue ordered to enforce * serialization. */ ppd->link_wq = - alloc_workqueue( + alloc_ordered_workqueue( "hfi_link_%d_%d", - WQ_SYSFS | WQ_MEM_RECLAIM | WQ_UNBOUND, - 1, /* max_active */ + WQ_SYSFS | WQ_MEM_RECLAIM, dd->unit, pidx); if (!ppd->link_wq) goto wq_error; From patchwork Fri Apr 21 02:50:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86106 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp778146vqo; Thu, 20 Apr 2023 20:01:59 -0700 (PDT) X-Google-Smtp-Source: AKy350bBQkNJ3XarLNeuL106SWo9Z5PyrW/FpbNAXGn/BxyeWK1ZHl0VlT1tP1F1iv6DpZyb22do X-Received: by 2002:a17:902:b097:b0:1a6:6b30:2b3b with SMTP id p23-20020a170902b09700b001a66b302b3bmr3232678plr.64.1682046119440; Thu, 20 Apr 2023 20:01:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682046119; cv=none; d=google.com; s=arc-20160816; b=yKu5kIcTcq58VVmK0sXw5nkhWqUKy9QJevasLbK9pvOAzH/+JCs62dz87xFuSwO9l0 YBs8CnOkoGMmdyC86eS4SZPdcOSTsaVyYHEfmoh8Yxll3G6etFRBAJv7Azg2UKN6iTCC U7gB/T/yK4/fv4lN8lEGmc2uDUvvpZ3pJBISdTlRu9GliBOEEquDU+OW0q5YuehAMHD4 QNjk15cen25iW6srLP6ljHegBQun4B0xJOmhJ+x/dbwXL4LY088PJQcRZYpC8hNji9qt 2m39Guhf6BHyLPTI/TYoYIpvRR77EI3bR+NCic0+DtVBkk75rc32GIVP9hrMSslkKcTl g66w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=Kmxq/N/H4rwg524LDGOZfm1KVVcee8vyb3VSvQP8F5Y=; b=EaztSlUd25zEuxVs6kp1qvb71N2kQWjRj1uuOP0eLyY4tIXWUAzgitSD9WWsw6vwud VJnB6TCtcTZipCeorGmCp0xIhX9R/x54TQ6Ea8yKInqSZXD+RzQJ6qwYc0VLXVMRX8hW T4Fl04BxjYY34tfbg0d2c4MKAzfkVDNBsbin0GYbPI+2YsDxKlKiSbT5rdZ746cqw2rO VH9Dd+TGdcUNDB388fH8GKwQkaqhlYpO9JUL82Xv/xNlE5k1S6thQt6WDDExF3QcxOGF 1KP7X1Uxuxok21K3daHWAElCoLkWQfm8NofKaCdc3Fx11YEMvjjx+VGcuWz86TqTBcPr V74Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=ZTCY7NIZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id kt6-20020a170903088600b001a6555fb909si3030491plb.183.2023.04.20.20.01.46; Thu, 20 Apr 2023 20:01:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=ZTCY7NIZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233732AbjDUCvN (ORCPT + 99 others); Thu, 20 Apr 2023 22:51:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43124 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233679AbjDUCvB (ORCPT ); Thu, 20 Apr 2023 22:51:01 -0400 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 24DF17292 for ; Thu, 20 Apr 2023 19:50:59 -0700 (PDT) Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-1a686260adcso19604905ad.0 for ; Thu, 20 Apr 2023 19:50:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045458; x=1684637458; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=Kmxq/N/H4rwg524LDGOZfm1KVVcee8vyb3VSvQP8F5Y=; b=ZTCY7NIZjAIZ4DB+G2pVkVPay5EJSfc1C25/4dQzO6rEOb8dWk2a3sBo7DktdfJC97 +6Y19aUSKlqAQ1mrHDXlyfdbr7ESauDOqjG9aJuiFMb23wcfOtuYeQVA+G4lBMt3Itl4 6kq4kWs1IIMirSKOI61ReX5aSk44XK4NBpomNlGY1wfROaGTo9/a8jUugvs3En0+3f48 5397J+WfVgEwnsKWZRbgoyqEYm3oXoe5EoHpItEzTyNGQREY+jYnykahtAsU5y4rRGa6 gNqB3SLSAwLDMmLTyRyiIqfPRPjnxKP3Xg3IIu0IR58pTzBDxorMVtYBWjc2+1cxy+9W J2Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045458; x=1684637458; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Kmxq/N/H4rwg524LDGOZfm1KVVcee8vyb3VSvQP8F5Y=; b=I//PYJUoI21dbsRBH832qzE2c/VJcfnzqZd1sTjC0TIM24FgWUlQUlVursjkd2HEVK 4OffjUsIvG1iDn3/ipPUpV3xLh+MjqLK65C4YgW+73bDn8PzAD5evYEQMJ7qXtB1cFOu mEnuLMNxncOA8ClOV9KI4bD30VULAjI4J7vmKVSC4/24XDq7MoFP4QlsbU20Mx3Kd0GR 5U1NfjHE1c12OL+YyPNI/2UtG+0lZe7DhkzuUvz/lc/LK+eheW+Hb1PbIxp3XkBlAtoM L/DVynthaipotjGnLEYdbuo6eQc9FGDFVrRUiZmedBsQY8MUkY0hA5heHDXnNHItmN9F 20pg== X-Gm-Message-State: AAQBX9dR5DFurMu0uoocMLGt1f2jT2CwVHpMic6t4Yb/yzE7HvDlToX0 DIx+uHenQKsa1fUNjcdnzuA= X-Received: by 2002:a17:902:ceca:b0:19e:665b:2514 with SMTP id d10-20020a170902ceca00b0019e665b2514mr4741371plg.43.1682045458279; Thu, 20 Apr 2023 19:50:58 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id q92-20020a17090a756500b00246f73a994esm1767145pjk.32.2023.04.20.19.50.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:50:57 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , Alasdair Kergon , Mike Snitzer , dm-devel@redhat.com Subject: [PATCH 04/22] dm integrity: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:28 -1000 Message-Id: <20230421025046.4008499-5-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763753191276754690?= X-GMAIL-MSGID: =?utf-8?q?1763753191276754690?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: Alasdair Kergon Cc: Mike Snitzer Cc: dm-devel@redhat.com Cc: linux-kernel@vger.kernel.org --- drivers/md/dm-integrity.c | 4 ++-- drivers/md/dm.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/md/dm-integrity.c b/drivers/md/dm-integrity.c index b0d5057fbdd9..6b36554f9103 100644 --- a/drivers/md/dm-integrity.c +++ b/drivers/md/dm-integrity.c @@ -4268,10 +4268,10 @@ static int dm_integrity_ctr(struct dm_target *ti, unsigned int argc, char **argv } /* - * If this workqueue were percpu, it would cause bio reordering + * If this workqueue weren't ordered, it would cause bio reordering * and reduced performance. */ - ic->wait_wq = alloc_workqueue("dm-integrity-wait", WQ_MEM_RECLAIM | WQ_UNBOUND, 1); + ic->wait_wq = alloc_ordered_workqueue("dm-integrity-wait", WQ_MEM_RECLAIM); if (!ic->wait_wq) { ti->error = "Cannot allocate workqueue"; r = -ENOMEM; diff --git a/drivers/md/dm.c b/drivers/md/dm.c index dfde0088147a..bd3342613e4c 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -207,7 +207,7 @@ static int __init local_init(void) if (r) return r; - deferred_remove_workqueue = alloc_workqueue("kdmremove", WQ_UNBOUND, 1); + deferred_remove_workqueue = alloc_ordered_workqueue("kdmremove", 0); if (!deferred_remove_workqueue) { r = -ENOMEM; goto out_uevent_exit; From patchwork Fri Apr 21 02:50:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86123 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp781816vqo; Thu, 20 Apr 2023 20:10:59 -0700 (PDT) X-Google-Smtp-Source: AKy350aT//dt5BK0hLkr1ryrstzuoIp/vNeS/t0YIxhwMMPWOXo4/6RGnFy9D5tjlHWorWpkIyEv X-Received: by 2002:a17:902:e84e:b0:1a6:ca7c:de05 with SMTP id t14-20020a170902e84e00b001a6ca7cde05mr4506808plg.16.1682046659322; Thu, 20 Apr 2023 20:10:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682046659; cv=none; d=google.com; s=arc-20160816; b=0KhgiUCw9kZayJ+BtLiE5O6zrDWrFqvqonjd9gOFG9eHrdy+qOmWjsBArCrZGq31XF sqQ1+lzFRhDnieuWIJU82bGCLZMl9xvHSIgveZZvn4P9HI5F0v5AFGLsLSAYZkAi8R4D ixf4hxETRaZeM3ngXwlGVciludm2xswuaWynNT2+blIRpNzLdxB8gSC/gOx1RI8nt3dl RGTrdJ8w2ZffTCihZ8vjpEZPksBUgX3rehsQyJU3I75KbhA7U6DziuUoVZaprAhsg9bm WMFAm8SZ4WE0hT0WRxieNDEmc0gCMTGkMQq7sA4r2yKZOf7yBIJYxG1NplgEMPOk38Fs LgOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=67LgqNT8XF4vwy6yP32eHkU2I1u0Y3vukwyzktySQbU=; b=qAMDiURD1fwETn27BTuT8qyGItjB0v2oY6hqnhnguep2p5owX8HSwKCOAFL7sQM5sz xXu5soCWi9PMDmTctvvTL6zT0bYXS0vvK0Xc10GRVNgax6xdH2w/wxE8JiERJt7tRSho EWyjALuCviWFZTYubWo6g6n9FbA5zaPRGZwBCzw3OzdD7fHyaiE4MYgZ7sV++D3aKvZA wGPijKzb+DHLHi3ol7IWNqJWFcpWADeFenX3kbzpxDX0lWYq0K0Ab+u7qAQKLH2iTUa4 y94XlO3w4XcyDfJhI5yrPuzdN2c2ay5PoQOFuNuKwl2tuAzpz1NDBlBWz6vKFmCT6GUH UakA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=ZiAbVoUZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u14-20020a170902e80e00b001a6fe422894si3499731plg.200.2023.04.20.20.10.46; Thu, 20 Apr 2023 20:10:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=ZiAbVoUZ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233049AbjDUCvQ (ORCPT + 99 others); Thu, 20 Apr 2023 22:51:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43272 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233699AbjDUCvG (ORCPT ); Thu, 20 Apr 2023 22:51:06 -0400 Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2FCD86EB9; Thu, 20 Apr 2023 19:51:01 -0700 (PDT) Received: by mail-pj1-x1034.google.com with SMTP id 98e67ed59e1d1-247122e9845so1180197a91.0; Thu, 20 Apr 2023 19:51:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045460; x=1684637460; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=67LgqNT8XF4vwy6yP32eHkU2I1u0Y3vukwyzktySQbU=; b=ZiAbVoUZ8IJAvox8IcFlDfaPh+VjoBXLmk6VAgIov93QLpk2pLax3y6rXcov36Ldwu vpOMojneq8Km6ua5QR8wSfoPVEPpscDPScZeepJ7zfN+3badZWA3SHeJXMjD2pwdkhGb vPeMUQ7KlKoxz1V2kEeNA2ueeYAYmySfRamPHAPEckyonnguEcP0WtfyngHomfQ4alcJ 1PYEaUKSL9FghM7WlQernDRjNWvawyRwvBLatXB/MZENORCKFR+W2Zp7opGS1hMYGR1H Xj4v9NlzxiP+br94U+xXJp6izZ/+DT1OvWx7LelKejYjbPDhd3qUUJipzuxJ03KHGBV+ pUsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045460; x=1684637460; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=67LgqNT8XF4vwy6yP32eHkU2I1u0Y3vukwyzktySQbU=; b=cj6ug0K2Lj0LvW3pZooOj8/iE/SHIVL+3D4EKcEjcwc2OFR0+jUpTSr+UoCzbeHPSE Dk3Z1IjIqeYoy8TLjaXs2lrj47vtid2NcwxocdbToeiuFVXVQUcfj0qNT/jq+wo7qhdB LIFWTRLgvoF40umkhVHhbL1o38UNQrxUPWUNwpGkkglPUEwwoxqfsusGcgPicJ68tiEA wKoeyVsLwez0UOjeFcEgwUusK9R9M3QlZstjLTv9N7VPKx0yji1arfVi8VSG2Wox9lqS hV5V94F/iJC8jb0jLsETWcYRDJBS8vcOjFyyTcAgNM+3ox/AT65Gd7uXMN33DXQbsgL5 TbSg== X-Gm-Message-State: AAQBX9cREaDSwj1V+Afk1hvHmFEzmUILZvfo+QDaFjDJCfQeu9jBcWwi LnyOfHEfO7sthaoJbDZ+5OI= X-Received: by 2002:a17:90a:d14f:b0:24b:6349:4f87 with SMTP id t15-20020a17090ad14f00b0024b63494f87mr1686085pjw.6.1682045460086; Thu, 20 Apr 2023 19:51:00 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id d4-20020a170902b70400b001a681d123a0sm1717302pls.297.2023.04.20.19.50.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:50:59 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , Ming Qian , Shijie Qin , Zhou Peng , Mauro Carvalho Chehab , linux-media@vger.kernel.org Subject: [PATCH 05/22] media: amphion: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:29 -1000 Message-Id: <20230421025046.4008499-6-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763753757856572405?= X-GMAIL-MSGID: =?utf-8?q?1763753757856572405?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: Ming Qian Cc: Shijie Qin Cc: Zhou Peng Cc: Mauro Carvalho Chehab Cc: linux-media@vger.kernel.org --- drivers/media/platform/amphion/vpu_core.c | 2 +- drivers/media/platform/amphion/vpu_v4l2.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/media/platform/amphion/vpu_core.c b/drivers/media/platform/amphion/vpu_core.c index f9ec1753f7c8..57d56c680c07 100644 --- a/drivers/media/platform/amphion/vpu_core.c +++ b/drivers/media/platform/amphion/vpu_core.c @@ -254,7 +254,7 @@ static int vpu_core_register(struct device *dev, struct vpu_core *core) if (vpu_core_is_exist(vpu, core)) return 0; - core->workqueue = alloc_workqueue("vpu", WQ_UNBOUND | WQ_MEM_RECLAIM, 1); + core->workqueue = alloc_ordered_workqueue("vpu", WQ_MEM_RECLAIM); if (!core->workqueue) { dev_err(core->dev, "fail to alloc workqueue\n"); return -ENOMEM; diff --git a/drivers/media/platform/amphion/vpu_v4l2.c b/drivers/media/platform/amphion/vpu_v4l2.c index 6773b885597c..a48edb445eea 100644 --- a/drivers/media/platform/amphion/vpu_v4l2.c +++ b/drivers/media/platform/amphion/vpu_v4l2.c @@ -740,7 +740,7 @@ int vpu_v4l2_open(struct file *file, struct vpu_inst *inst) inst->fh.ctrl_handler = &inst->ctrl_handler; file->private_data = &inst->fh; inst->state = VPU_CODEC_STATE_DEINIT; - inst->workqueue = alloc_workqueue("vpu_inst", WQ_UNBOUND | WQ_MEM_RECLAIM, 1); + inst->workqueue = alloc_ordered_workqueue("vpu_inst", WQ_MEM_RECLAIM); if (inst->workqueue) { INIT_WORK(&inst->msg_work, vpu_inst_run_work); ret = kfifo_init(&inst->msg_fifo, From patchwork Fri Apr 21 02:50:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86121 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp781713vqo; Thu, 20 Apr 2023 20:10:39 -0700 (PDT) X-Google-Smtp-Source: AKy350ZCQQtvtPy8TJLnKm6rFUJbFmlbhfzt2k4uM8rr1yrm37MmabXdHYpVUHVRoEQQyauC1nZS X-Received: by 2002:aa7:88cf:0:b0:63b:873d:bdb3 with SMTP id k15-20020aa788cf000000b0063b873dbdb3mr4219150pff.13.1682046638867; Thu, 20 Apr 2023 20:10:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682046638; cv=none; d=google.com; s=arc-20160816; b=o7k5bte8owI86u6n3FD9KBlg33uXncOREgG0d2OXMZGwESXD8IZp7bFXEF0fXJxdju 0c5YU+GzzctPU3O3tCg15lOJI7X+ikbahlml8aepyUqCFnV/aE6gOSlgoQljx0EbWHkI fDUrfC6mo5L2jtGSPtYG6M+dNDEOpXhX6ELY+/OgyaD8p9zxBTMn60cAvv/GKLe08hkm I/lcHfrlU+LGZN4PlAwkZnkeLstOOUMAxFeIQ+7tjBrBex9m4OZX47l3PlXeRp5Bl/J+ 83rckDA7aQrp9x24W3O0LPhWCQHN4HjSVBMG1aQ2VkGA8hwb1eZUPOfrdGGpwqtt7IQ2 YwRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=34/Qii7FhGNnTA9R8bpTQlpj4RZtJcqmBQ/JN/Zhs64=; b=iOfdHIeNDHZaLBFe7je5iJPtPLS3/sn87XnnLewj+kaeM1K6iooMRgCuPjOK95WYo0 tGdE7jNUvDNqEn+JZq0z69/5A1g0/BtFclX1SK7b9v1hWn0p2HTLZlofVqsf+GF6zed0 ++aBFspSokSo8TfYi8MHVE7EzpuLQe1Z62tZz3nNmFuW2MHJ41sESzgXleXdxCSWtYMB Ul+/ZeETMZmWowCa6bZF4tJJMfl5A3qt2c9MWDayXlZRQ7VS7aVbtBuLZaToXcSWtYEK +wmnN66X6So5ldhpa+kqwGFq8x3n7eeB2rWHIMbC51Gxl4ccNTFtKF0RwnIBTkjbJdE3 jXBQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=rloF2mVA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q24-20020aa78438000000b0063d2aa67a8esi3174889pfn.325.2023.04.20.20.10.26; Thu, 20 Apr 2023 20:10:38 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=rloF2mVA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233786AbjDUCvU (ORCPT + 99 others); Thu, 20 Apr 2023 22:51:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233674AbjDUCvL (ORCPT ); Thu, 20 Apr 2023 22:51:11 -0400 Received: from mail-pf1-x434.google.com (mail-pf1-x434.google.com [IPv6:2607:f8b0:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CBF9A72B6; Thu, 20 Apr 2023 19:51:02 -0700 (PDT) Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-63b5c48ea09so1600864b3a.1; Thu, 20 Apr 2023 19:51:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045462; x=1684637462; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=34/Qii7FhGNnTA9R8bpTQlpj4RZtJcqmBQ/JN/Zhs64=; b=rloF2mVAYBrDqM/6XuYQZv7gDt2EIV8lhLGgEEJ3aoXgc0Wvlus+anMCNBx6PUd0+k HQ/kOTFvj3/q6z85JtIlfUU1j5z/ZAJb1YnJW+JewjxZYuQ9kGRTHS0ZaV4/9UqSTjKI QLjvN+BgQzSGhKEztoAQ1YtgyBnLS5Wv1sOGtn0cPnlrfmm809JWs4QNV1B/DookEJq6 T9ZRYKVAVUELaRb+Bcp/Rx2wIDO8bd210XgXcNEYG9OV/dX2QvYlUyALg25HkejBfB3w bsGYcbipu6KkSio+dJCNrD0lXpUWNTwNW6g5iTJ4+ar3ZmOFXg72Yqbx5uUpF3uQrVEH vY9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045462; x=1684637462; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=34/Qii7FhGNnTA9R8bpTQlpj4RZtJcqmBQ/JN/Zhs64=; b=MX1FnsfEI8JOCDxvS0Lggm7r6at/QHp2e1f1jZy8dXeorr9RX3H5CZp2+Dd9wBmRY5 Tm9/3tjfi+P5yYvujjIIbGBSIfuaMJnrArXkl/9INbnoVmCrt+tuKawluCxZZgGiIytc ypgzp/jQ9BeWGVJjD2S8EJhPPgtRHWVKbyzC8oBgSjSjEsDIsobaDlQhX+mgTWwBU5gk AFCLnVnzo2wzTR5IkmJD2bWzOZjxYn7wnZO/alAm8iBJCPgHjomQrcSO2TT0t02DXmV3 U47arc9tHdQFV1a4qtvznJfOKuqCAX4fEqNI8zu5PAAMrlza5fMzP0CmmX+8uJbnMWb+ 1T1Q== X-Gm-Message-State: AAQBX9dWFeRt+ljUfUyattLGMLiD3yNkevskQfN/toYMIfrjjZlxIGQ7 07Gq616o+QrvNPK0mM0FjCE= X-Received: by 2002:a05:6a21:6817:b0:f0:164b:fa5d with SMTP id wr23-20020a056a21681700b000f0164bfa5dmr4235235pzb.15.1682045462096; Thu, 20 Apr 2023 19:51:02 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id z12-20020a63c04c000000b0050f74d435e6sm1676172pgi.18.2023.04.20.19.51.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:51:01 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , Sunil Goutham , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , linux-arm-kernel@lists.infradead.org, netdev@vger.kernel.org Subject: [PATCH 06/22] net: thunderx: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:30 -1000 Message-Id: <20230421025046.4008499-7-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763753736177723500?= X-GMAIL-MSGID: =?utf-8?q?1763753736177723500?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: Sunil Goutham Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: linux-arm-kernel@lists.infradead.org Cc: netdev@vger.kernel.org Reviewed-by: Sunil Goutham Acked-by: Jakub Kicinski --- drivers/net/ethernet/cavium/thunder/thunder_bgx.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/net/ethernet/cavium/thunder/thunder_bgx.c b/drivers/net/ethernet/cavium/thunder/thunder_bgx.c index 7eb2ddbe9bad..a317feb8decb 100644 --- a/drivers/net/ethernet/cavium/thunder/thunder_bgx.c +++ b/drivers/net/ethernet/cavium/thunder/thunder_bgx.c @@ -1126,8 +1126,7 @@ static int bgx_lmac_enable(struct bgx *bgx, u8 lmacid) } poll: - lmac->check_link = alloc_workqueue("check_link", WQ_UNBOUND | - WQ_MEM_RECLAIM, 1); + lmac->check_link = alloc_ordered_workqueue("check_link", WQ_MEM_RECLAIM); if (!lmac->check_link) return -ENOMEM; INIT_DELAYED_WORK(&lmac->dwork, bgx_poll_for_link); From patchwork Fri Apr 21 02:50:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86115 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp780651vqo; Thu, 20 Apr 2023 20:07:52 -0700 (PDT) X-Google-Smtp-Source: AKy350ZkI+rWohL8NDsO8ySHLc6NsqPiK1W0WvHtiBzhDF5vY2by+D1MWmLKzQ5/xRRiKEiVNFmM X-Received: by 2002:a05:6a21:6d89:b0:ee:d266:32be with SMTP id wl9-20020a056a216d8900b000eed26632bemr4799571pzb.11.1682046471611; Thu, 20 Apr 2023 20:07:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682046471; cv=none; d=google.com; s=arc-20160816; b=mmSZ9D8Lt4hjR8kTs3Etir5cRmviQK3qxC76/k658CNRuncvlDck5I2kvanL01hopg Lrx49uRwB0Dl52KXe53gBI238aVEmnZUHM+WR9yi8i8wBOmi/WL4jLARumiv8EYdok5o CUcB4raIi7q4qgP9uN6JdYf9bCxK47ehJZzIROyKg56NMdT+zz+qxy4roiDL06+/T6v6 Z/FpZUlTMcb4ijXUWyV6qH16eb79RMB9HpEqCpVsy/u1Fw9erNhVuLPELhc/H8iuI3zO jO9U4VaKjpNqmwF/RuhMaF2N1gyubPlDGim2BrOZ/g2Vw4Epo/vVzVEekHCFeReDvdU7 Ru4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=AwtUCKMDpoTm9KXX0yGrXQSd2H3Fgy9ECPbB0fz8oGo=; b=zFaT39XPhCNHDFgcEHkUEynQXe50v+G8hrQGSpQpj2tkZVM+VysxfV45BrnwWmQ+kp R4QF+HqTmvjv5wlvNx+MupVGOKvdI2oHX0Pc1kDlhmGyZ76iO0wmTTMyjR4vZnRuqepF 8zRXCBe9obNwUvBT+/eTJvgGjZ8X24w9hC4PNC8XJKyMdG3CMOi/QB/76L0aguRQeQ5x 6gN5ishYDziXI891CvGfDfj1uC/MTpYeO3TF2+JlqC6k4GNtMWPOUpcyIhicrGtrZIz2 E8JapnjNfCZ6ZRacku9Jc/k78nJlQ/9vVT3TqFtWyht2A7ypqp84RgcenHw/6sJj2XAA GTPA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=cRJhfKuP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z19-20020a63e113000000b00521e4b138fcsi3367576pgh.148.2023.04.20.20.07.38; Thu, 20 Apr 2023 20:07:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=cRJhfKuP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233803AbjDUCvX (ORCPT + 99 others); Thu, 20 Apr 2023 22:51:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233719AbjDUCvM (ORCPT ); Thu, 20 Apr 2023 22:51:12 -0400 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E132D76AA; Thu, 20 Apr 2023 19:51:04 -0700 (PDT) Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-63b35789313so1362748b3a.3; Thu, 20 Apr 2023 19:51:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045464; x=1684637464; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=AwtUCKMDpoTm9KXX0yGrXQSd2H3Fgy9ECPbB0fz8oGo=; b=cRJhfKuPjaED++LchZsJzv3xv2rfNzoPZmErb4pfag5q1r7/g5SiDmE5lJmQZdSg0C Y6H3eUOTI/N7cG2u7vHVCwtPtgx76TWwDthHKaUOdDXhqudG4ncOx5I+9HxyuIZoZXUA l5fM8XwbhWUdkztqx8ommuepL9dKuw0aAlRQW1czU4ZdFsz/H4Zw7Tuh5Ku5izmM2d9q Z7969/wHOFjZxsOGYWplKZaPsPhyWaXjQTglVTt1+pL5RbrritkAgmQUq7LLwmquHLnz i4X6DFh92x3Wyahf1P+N6mOzZwxfZUpIpurZWY0cweTU2+OjGoMijp8Qsydoc98mIrsh kt7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045464; x=1684637464; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=AwtUCKMDpoTm9KXX0yGrXQSd2H3Fgy9ECPbB0fz8oGo=; b=QVi1A6bCmvREN7v5pKkB0br0UliXxXlTLtYKBsT9g11V/U/lOBycP5zcA/bJEIEuCG QhbqR6F2cJSXT0ISLUNLmWfTNOJeIjztxLE92rjAggzf9DUl7B60XTsVJ/YGBAmddTUD I7Vd+fKYAvHgEyyR7rLWD04qiyCGfVAybRf4KoOxOnI0dHVpvPs6l+T/HyNodpr+ThYA MtXPDLEWGfRFmWApQfrG3Rl7HBGLnDYT5dSYwfepf1EqE+we21TlTfDARd6RW1314CRd 0/8Xj5+rHh0NTLgtzZH2o60y517wscuvdHhL0M8+VW9nIatvTKGvOSkPUuZ1LeDlW5O2 R3+Q== X-Gm-Message-State: AAQBX9frGwRCgePWPDYhZ9fNICwlzSmCA2wHIiJdRL0zDogUYhHb8Inh o0v7XYR99Kk6dvUSbfJlEpw= X-Received: by 2002:a05:6a21:6da7:b0:f2:6984:b8d with SMTP id wl39-20020a056a216da700b000f269840b8dmr1152992pzb.29.1682045463979; Thu, 20 Apr 2023 19:51:03 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id q10-20020a63d60a000000b0051eff0a70d7sm1647561pgg.94.2023.04.20.19.51.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:51:03 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Sunil Goutham , Ratheesh Kannoth , Srujana Challa , Geetha sowjanya , netdev@vger.kernel.org Subject: [PATCH 07/22] net: octeontx2: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:31 -1000 Message-Id: <20230421025046.4008499-8-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763753561075909337?= X-GMAIL-MSGID: =?utf-8?q?1763753561075909337?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: Sunil Goutham Cc: Ratheesh Kannoth Cc: Srujana Challa Cc: Geetha sowjanya Cc: netdev@vger.kernel.org Reviewed-by: Sunil Goutham --- drivers/net/ethernet/marvell/octeontx2/af/rvu.c | 5 ++--- .../net/ethernet/marvell/octeontx2/nic/otx2_pf.c | 13 +++++-------- .../net/ethernet/marvell/octeontx2/nic/otx2_vf.c | 5 ++--- 3 files changed, 9 insertions(+), 14 deletions(-) diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c index 8683ce57ed3f..207041c81184 100644 --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.c +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.c @@ -3013,9 +3013,8 @@ static int rvu_flr_init(struct rvu *rvu) cfg | BIT_ULL(22)); } - rvu->flr_wq = alloc_workqueue("rvu_afpf_flr", - WQ_UNBOUND | WQ_HIGHPRI | WQ_MEM_RECLAIM, - 1); + rvu->flr_wq = alloc_ordered_workqueue("rvu_afpf_flr", + WQ_HIGHPRI | WQ_MEM_RECLAIM); if (!rvu->flr_wq) return -ENOMEM; diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c index 179433d0a54a..7b3114105757 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c @@ -271,8 +271,7 @@ static int otx2_pf_flr_init(struct otx2_nic *pf, int num_vfs) { int vf; - pf->flr_wq = alloc_workqueue("otx2_pf_flr_wq", - WQ_UNBOUND | WQ_HIGHPRI, 1); + pf->flr_wq = alloc_ordered_workqueue("otx2_pf_flr_wq", WQ_HIGHPRI); if (!pf->flr_wq) return -ENOMEM; @@ -593,9 +592,8 @@ static int otx2_pfvf_mbox_init(struct otx2_nic *pf, int numvfs) if (!pf->mbox_pfvf) return -ENOMEM; - pf->mbox_pfvf_wq = alloc_workqueue("otx2_pfvf_mailbox", - WQ_UNBOUND | WQ_HIGHPRI | - WQ_MEM_RECLAIM, 1); + pf->mbox_pfvf_wq = alloc_ordered_workqueue("otx2_pfvf_mailbox", + WQ_HIGHPRI | WQ_MEM_RECLAIM); if (!pf->mbox_pfvf_wq) return -ENOMEM; @@ -1063,9 +1061,8 @@ static int otx2_pfaf_mbox_init(struct otx2_nic *pf) int err; mbox->pfvf = pf; - pf->mbox_wq = alloc_workqueue("otx2_pfaf_mailbox", - WQ_UNBOUND | WQ_HIGHPRI | - WQ_MEM_RECLAIM, 1); + pf->mbox_wq = alloc_ordered_workqueue("otx2_pfaf_mailbox", + WQ_HIGHPRI | WQ_MEM_RECLAIM); if (!pf->mbox_wq) return -ENOMEM; diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c index ab126f8706c7..1f16e0dcbb3e 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c @@ -297,9 +297,8 @@ static int otx2vf_vfaf_mbox_init(struct otx2_nic *vf) int err; mbox->pfvf = vf; - vf->mbox_wq = alloc_workqueue("otx2_vfaf_mailbox", - WQ_UNBOUND | WQ_HIGHPRI | - WQ_MEM_RECLAIM, 1); + vf->mbox_wq = alloc_ordered_workqueue("otx2_vfaf_mailbox", + WQ_HIGHPRI | WQ_MEM_RECLAIM); if (!vf->mbox_wq) return -ENOMEM; From patchwork Fri Apr 21 02:50:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86114 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp780346vqo; Thu, 20 Apr 2023 20:07:05 -0700 (PDT) X-Google-Smtp-Source: AKy350ZX6yHxC6MSCtabVf2dfP3oGxGEHa5oifKsKPyJz7u+VR5lEENG0mnQHI/kbwgncLyhscO4 X-Received: by 2002:a05:6a20:4296:b0:ef:1457:6cdf with SMTP id o22-20020a056a20429600b000ef14576cdfmr5362749pzj.19.1682046425537; Thu, 20 Apr 2023 20:07:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682046425; cv=none; d=google.com; s=arc-20160816; b=uOUMfqUcHZta7xySlZwgTJEldh+4KSMXEVgS3Z6Yhs+BzlXCzv8XruJ2sYuHGo1DwA fNax/QDBeeSslKGO3QGNTyQdPwxLOn9i8TX/2jMVyJv99K5fBOhI64pFYqMtVTyDAws2 h6BmkFmGxpk/4IAgNfHAg6rLaShPA65KLJ8BTA8VXciduMDocEmq6ZQ1JEhiqOP7oXsV ude7MVjmMaVWQTOozOm4S8lrllcX+9JgOevKRTZPrxY/EFiDfnPDDdt5E8lN80SAsRsh n7aS7/gRH/Hqhk94vQOAQPf8ilAy3Smv8Hm2O1ku52pGOQQha6HOjfFOFMJUIE3mrnXp mwUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=2OMa4H5B7cQ31o0iO4EHjR7g/GmtbQFAxNrqAS+m3rA=; b=sIocGgcHCfT/5v+ep9F0AYfgWgHolX25Zey7jN/wG+gtUSp7RUGkK0+wZF+N+IT9Cb 1V6R0LcX99qxVJKadbXcmkfhTmKyCdHaghAD+F03qeUYS/Wyv74KLcFcJQRft4SEkPDA ZsW+IR330475kaPFKutVoGXswq3uoi2RwujRIs/dHXdc1SCzaDEi31rvdrI0NiNDt9U5 /4J1j+ulxoG5FfXd4WpJzVInIIopIaQViAMZi5KbVL7wAEYhf19XiWLuMhDHfpFvCHG/ LJYNYKS54g8lEZEQb94MIUbVVFrdhbL0CZFxux3u92+X2L/9sWj7cs6gPcYaIlxKSqf0 ojHA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=ed6rzYNu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 4-20020a630704000000b00524b56d9e63si3238231pgh.324.2023.04.20.20.06.49; Thu, 20 Apr 2023 20:07:05 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=ed6rzYNu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233818AbjDUCv2 (ORCPT + 99 others); Thu, 20 Apr 2023 22:51:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43406 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233728AbjDUCvN (ORCPT ); Thu, 20 Apr 2023 22:51:13 -0400 Received: from mail-pg1-x535.google.com (mail-pg1-x535.google.com [IPv6:2607:f8b0:4864:20::535]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 32C8E7692; Thu, 20 Apr 2023 19:51:07 -0700 (PDT) Received: by mail-pg1-x535.google.com with SMTP id 41be03b00d2f7-51f597c975fso1744062a12.0; Thu, 20 Apr 2023 19:51:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045466; x=1684637466; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=2OMa4H5B7cQ31o0iO4EHjR7g/GmtbQFAxNrqAS+m3rA=; b=ed6rzYNuCkWaolQBt1qf/IaUHcwfXxuC5SbNdlIaF08ctYw+9Z4paHRTmDAW4FtsAi N8RRJRPFzktovjo7fpVlYA/p6AaCY2AgPdiXX26Trrodn72gYdhzSan2w7mBlOMVDpUJ Tr6WoSC7BQ3lAtWHPpOrAcdMolmjkqZdiwhEkvWcfAyx5mxkKHBVpV29sanYgaENW+iB +qOzsE1CiFbnqXwPWFDS3+KT81w7mLwRl1srwhsJW8DNKiJeKXelf8/Y8MuXTaCSEATO 1sp3CO6Lcx7iYnFy96NZbqYOjwpuPOb4qGltDAHHFWPwW5qvBsQYFUJnI1/aDJL9yayh unVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045466; x=1684637466; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=2OMa4H5B7cQ31o0iO4EHjR7g/GmtbQFAxNrqAS+m3rA=; b=GEMcwc5ZnkdVqUCwhYmulVUNujuN8xZVGiU06LHJmc9GUc0B2oBYbs5xtgL0gm4QcS U2NIO7O3ZHRA+xFDq0tNwBk8s3xfa0i5mVjTQ82QaKmN53XOQRXJ3/8mxCLAItSwCBuQ 16v3jIuHc142Kos3kR7s0a8V/f6Errf2ouevAflZcqBDgMqTI/6lEjBCCnZtWxu4GDph 34hU23Xkn9ZQOeN0vFy1pNZXmSPoVgWz7jfThsPOAlLaD5/pUe+3nzL7bGwA/UZesiRs PPdiZMs9+rKmVSq2Vh+WIxBHGTfhYOzu68Tl/Rp8TlplvQRcfImLSixbyV3X8GUDieec tTAw== X-Gm-Message-State: AAQBX9cNaWqTcwL6PVXsLpnebGR7mDb47aPDAk8QXjYZE/29ysL6CF/k llvAAKU0GPvb2RQMYf0RRDD8Gs1HXuU= X-Received: by 2002:a17:90a:ea09:b0:23f:7d05:8762 with SMTP id w9-20020a17090aea0900b0023f7d058762mr3389480pjy.23.1682045465785; Thu, 20 Apr 2023 19:51:05 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id jj2-20020a170903048200b001a6d08dc847sm1734028plb.173.2023.04.20.19.51.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:51:05 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , Kalle Valo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , linux-wireless@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH 08/22] wifi: ath10/11/12k: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:32 -1000 Message-Id: <20230421025046.4008499-9-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763753512382848838?= X-GMAIL-MSGID: =?utf-8?q?1763753512382848838?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: Kalle Valo Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: linux-wireless@vger.kernel.org Cc: netdev@vger.kernel.org --- drivers/net/wireless/ath/ath10k/qmi.c | 3 +-- drivers/net/wireless/ath/ath11k/qmi.c | 3 +-- drivers/net/wireless/ath/ath12k/qmi.c | 3 +-- 3 files changed, 3 insertions(+), 6 deletions(-) diff --git a/drivers/net/wireless/ath/ath10k/qmi.c b/drivers/net/wireless/ath/ath10k/qmi.c index 90f457b8e1fe..ebedef8767cd 100644 --- a/drivers/net/wireless/ath/ath10k/qmi.c +++ b/drivers/net/wireless/ath/ath10k/qmi.c @@ -1082,8 +1082,7 @@ int ath10k_qmi_init(struct ath10k *ar, u32 msa_size) if (ret) goto err; - qmi->event_wq = alloc_workqueue("ath10k_qmi_driver_event", - WQ_UNBOUND, 1); + qmi->event_wq = alloc_ordered_workqueue("ath10k_qmi_driver_event", 0); if (!qmi->event_wq) { ath10k_err(ar, "failed to allocate workqueue\n"); ret = -EFAULT; diff --git a/drivers/net/wireless/ath/ath11k/qmi.c b/drivers/net/wireless/ath/ath11k/qmi.c index ab923e24b0a9..26b252e62909 100644 --- a/drivers/net/wireless/ath/ath11k/qmi.c +++ b/drivers/net/wireless/ath/ath11k/qmi.c @@ -3256,8 +3256,7 @@ int ath11k_qmi_init_service(struct ath11k_base *ab) return ret; } - ab->qmi.event_wq = alloc_workqueue("ath11k_qmi_driver_event", - WQ_UNBOUND, 1); + ab->qmi.event_wq = alloc_ordered_workqueue("ath11k_qmi_driver_event", 0); if (!ab->qmi.event_wq) { ath11k_err(ab, "failed to allocate workqueue\n"); return -EFAULT; diff --git a/drivers/net/wireless/ath/ath12k/qmi.c b/drivers/net/wireless/ath/ath12k/qmi.c index 979a63f2e2ab..471810877eed 100644 --- a/drivers/net/wireless/ath/ath12k/qmi.c +++ b/drivers/net/wireless/ath/ath12k/qmi.c @@ -3054,8 +3054,7 @@ int ath12k_qmi_init_service(struct ath12k_base *ab) return ret; } - ab->qmi.event_wq = alloc_workqueue("ath12k_qmi_driver_event", - WQ_UNBOUND, 1); + ab->qmi.event_wq = alloc_ordered_workqueue("ath12k_qmi_driver_event", 0); if (!ab->qmi.event_wq) { ath12k_err(ab, "failed to allocate workqueue\n"); return -EFAULT; From patchwork Fri Apr 21 02:50:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86111 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp778871vqo; Thu, 20 Apr 2023 20:03:45 -0700 (PDT) X-Google-Smtp-Source: AKy350YWzeR6VyMuUrI98f+4BuwCrax0gl0o+jx6FJ5n6BSueAIyt/VAOI87jQhFbT3S8/mtWVZn X-Received: by 2002:a17:90b:603:b0:247:bd63:c3c6 with SMTP id gb3-20020a17090b060300b00247bd63c3c6mr3725122pjb.33.1682046224764; Thu, 20 Apr 2023 20:03:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682046224; cv=none; d=google.com; s=arc-20160816; b=kaqSN6/JWpld2586awFspypANx3TwvUi/w7qzoekqfFyqqOV4df+inb7ZBfq8uGEoT MEW01ZNrnEyLTeRRhXvAhuZ78niwpJ9ZBeLNcS3KZ7j6zKph+BzHtNS1UeMr0O+cuFK/ 62NG4A9OQ7ueycKZUG92yUIDWOamwu92KpL52kgzoGEyO2//Ax6tcClQMDTF1mE2nN11 cniuCGF7JOtGZfzZmmKLjM849syuiP2PRrYRcikDKN1xY4M4c3dlrjkTdOwtNbu9bsol +NdpQDg+lETgZ5y8TVRuKUMNA0A46iCb3T+e4vuTFD3nrXGWtgT26wZMqxGypBOWqPDN 5n8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=vXy2LlryeqFYuG6VIcWnTi0Y22Hml5WbUl7g3QZBsxI=; b=P0yjmnmyIDfHg0t/V3ZU4Jjzjujd84Plm/IeGmjkqOIf8RmMVP1uSNwjxkMPcdjNaQ dsutRfZLuHaXwkNtDlXUIegLGKBERLME02EUtqUFrAORMVAsCrtSOeepCg+ALmvb9pzs uWp2v/ng0bRQ59atPCNU4YgTPbFp2k04tFIgFk5ccpQ2aBgcT6px1OK9xXJF63CZ8fYF DFyFBgxpLG/tyn1p1JH/VpRImdaUh0XlK+8IztfpOvl44OqlZ2sAGTnpfFDGVLhqGBqZ yky4KtZyYM+0efq9sHBj6dCynW/yuOnwd9FmbDOg4duMWbi7tjO9jT7x2f2bAwvM99Sf Eflw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=DNXPUdu3; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id o5-20020a17090ac70500b0024b5a028e7asi1779794pjt.47.2023.04.20.20.03.32; Thu, 20 Apr 2023 20:03:44 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=DNXPUdu3; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233846AbjDUCvn (ORCPT + 99 others); Thu, 20 Apr 2023 22:51:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43410 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233729AbjDUCvN (ORCPT ); Thu, 20 Apr 2023 22:51:13 -0400 Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B45576B8; Thu, 20 Apr 2023 19:51:08 -0700 (PDT) Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-63b4dfead1bso1584816b3a.3; Thu, 20 Apr 2023 19:51:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045468; x=1684637468; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=vXy2LlryeqFYuG6VIcWnTi0Y22Hml5WbUl7g3QZBsxI=; b=DNXPUdu3So6Ly099msBkeS7QG2IIR0hElah4MDZwf8MWR1gh2Qq/qQbimHClsh938b 823fOETKFYsI5gKPXdc5lvyzkHjWyvJxTZM7XPzbXB8kMGcGMUJpUgIL0LfhkKobISuC OA5UP91DpFYekIyJbBQshJbMEeWYiJhA0onfRS4M+Y8wioMvq6Cx+cres2NEdLWHpDdr 8A9XAwa0cIu9GLFDCoRmnhNGnjNQJTgvHNQAAR9V8t1ZQomU3z7aYsnBlQSuk+QFicie YA64dM6wNqV6F4c3hHdtw1bllG2Zwd0XCD3/VoVVN+drc4zNkIsTdTgQXJ03D1fM5dCf usPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045468; x=1684637468; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=vXy2LlryeqFYuG6VIcWnTi0Y22Hml5WbUl7g3QZBsxI=; b=a2+60fq2LfuQvHrJsd3e9WDOhW6wLAdgABo4UVaboNY0IwZvWxZukHtXkgrFWxVbtr oTFgk/JPA3/MbWLEHkxW/YG17dc8biq6IohSRO79BwWqe5CuetfYZkexscuNnX2IIPXx qrpvBJgWx8IrN5Avzy4h2dyMQ65nX0CsKKQQ5MJtMhIL/+lF9pcV7uBMRA1dbqUGvtZX BCGgTopsVSw8wflk0DmP1F+6DPvN+Rb//+8foCORlDbJsyqTtSUZkNPgjmUeqVfPwjjn qI0jHWQyM49UBodvHaD1GyIxLlnDbQcZ7SKQJ7q0Zw9HoKRXkOxwP2cnKYvzg/lorDJJ cHPQ== X-Gm-Message-State: AAQBX9dXOgfv4ioUH96EZEcfyMrYwWIJ53j7ievMbYhnoqID7mnx1QOw /LZhndiZeaxDjjyoF40h8CU= X-Received: by 2002:a05:6a20:a1a0:b0:de:247e:d1fe with SMTP id r32-20020a056a20a1a000b000de247ed1femr3442160pzk.1.1682045467914; Thu, 20 Apr 2023 19:51:07 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id o64-20020a62cd43000000b0063d642dcd12sm1972276pfg.16.2023.04.20.19.51.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:51:07 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , Kalle Valo , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Gregory Greenman , Johannes Berg , Avraham Stern , Kees Cook , Mordechay Goodstein , "Haim, Dreyfuss" , linux-wireless@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH 09/22] wifi: iwlwifi: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:33 -1000 Message-Id: <20230421025046.4008499-10-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763753302116698913?= X-GMAIL-MSGID: =?utf-8?q?1763753302116698913?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: Kalle Valo Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: Gregory Greenman Cc: Johannes Berg Cc: Avraham Stern Cc: Kees Cook Cc: Mordechay Goodstein Cc: "Haim, Dreyfuss" Cc: linux-wireless@vger.kernel.org Cc: netdev@vger.kernel.org --- drivers/net/wireless/intel/iwlwifi/pcie/trans.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/trans.c b/drivers/net/wireless/intel/iwlwifi/pcie/trans.c index 0a9af1ad1f20..cd17b601b172 100644 --- a/drivers/net/wireless/intel/iwlwifi/pcie/trans.c +++ b/drivers/net/wireless/intel/iwlwifi/pcie/trans.c @@ -3576,8 +3576,8 @@ struct iwl_trans *iwl_trans_pcie_alloc(struct pci_dev *pdev, init_waitqueue_head(&trans_pcie->fw_reset_waitq); init_waitqueue_head(&trans_pcie->imr_waitq); - trans_pcie->rba.alloc_wq = alloc_workqueue("rb_allocator", - WQ_HIGHPRI | WQ_UNBOUND, 1); + trans_pcie->rba.alloc_wq = alloc_ordered_workqueue("rb_allocator", + WQ_HIGHPRI); if (!trans_pcie->rba.alloc_wq) { ret = -ENOMEM; goto out_free_trans; From patchwork Fri Apr 21 02:50:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86108 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp778308vqo; Thu, 20 Apr 2023 20:02:23 -0700 (PDT) X-Google-Smtp-Source: AKy350aj6YWfcAETY9GaBDfD6s8ekTtPUFadx/ZDkJNTnO/2+txWCjVGG1mJfPGgpv/cF7H/uFk7 X-Received: by 2002:a17:902:e547:b0:1a6:87f7:2dbf with SMTP id n7-20020a170902e54700b001a687f72dbfmr4235202plf.62.1682046143489; Thu, 20 Apr 2023 20:02:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682046143; cv=none; d=google.com; s=arc-20160816; b=t8+H+1Dl9IXo8/pOvvzQi9x7OP2poPTN/EkcZxkVU0hou4aBDhoOFlvJlFnDcurrrY PquDv8s64z3LlIH+SxqGRTSMPvzEQHl9TIDnZbpjNOcNMJaHtwS4SaiB33e2cVPc95gr cnRBcJDOBxpAcboqXXA36CEzOrBKQ0ym/aerejoVSdeWM8Czd5xtiw9dx0icT8z+6fV+ DwSGQfL/jR3Lc/pYs6xylJVoWh63BKCN/DvjGEgb06pccMibuhCdZE+p6sRLsGAD3nBJ /JEF14FrxfeUVltTfcwFD5rfURdpGspzKl3MH0PErZzZw6sk+rH6FX5YOmd/N/mp63Ek uDVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=KIwAxCs+xB0lesjbp62PO3r6+mEINYjwnW6XfaYSSdQ=; b=yTSrpGG9yBdRY9wfXUOHWgoNXBMDDZjDFFrzFZ6ZTfxbPC25aUXzK05vC9VCs64Jd6 SXBkRcYQzb2ET0C2zZS54UYDCLpEUGx9BCvA7QyUWUC7E3F5EAncM9gVHGER6RzLrizJ oxTWq+KYvOsFZuXZqewR8SxCVxxfVAIHN+rz+Y+YweARfWvTHQOOwKqLc0LYuykO3Ijf MCgl5NegnYj4cEHQVHIg/8b3IWdlveGWdGVxGFJKeGs5RkaFnU7/c2mTIIvJKtiPfktQ bjCkZjrArUmfUj1oRwc6VeeLkUz+9xQD40tLg7BEj9/eSM/80DaK306uPHcRCqw2fMiQ F14Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=kuAIo58v; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x11-20020a17090abc8b00b00240aff612f9si5907377pjr.140.2023.04.20.20.02.11; Thu, 20 Apr 2023 20:02:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=kuAIo58v; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233919AbjDUCvw (ORCPT + 99 others); Thu, 20 Apr 2023 22:51:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43260 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233750AbjDUCvO (ORCPT ); Thu, 20 Apr 2023 22:51:14 -0400 Received: from mail-pf1-x430.google.com (mail-pf1-x430.google.com [IPv6:2607:f8b0:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 66C9049C2; Thu, 20 Apr 2023 19:51:12 -0700 (PDT) Received: by mail-pf1-x430.google.com with SMTP id d2e1a72fcca58-63b8b19901fso2243040b3a.3; Thu, 20 Apr 2023 19:51:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045472; x=1684637472; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=KIwAxCs+xB0lesjbp62PO3r6+mEINYjwnW6XfaYSSdQ=; b=kuAIo58vZB1+qpKj9vHhK3ZK1gJVwSkzhEOvNl61GQlj9K2oUor4omQ++jwf+nzfIi sCkhXcZ9qrATu4V6fTz5ckRvJl/CyasxWjZDPEXg8GYjZmgbeS6xmY6yHeaQt/TIOnuF Y21XyflrnxJ7BtEGDyTFEY4FGU5Z8U49UJD8FleH6XmqEmGhvo9TJ49rr+7WSW0V4QCl p7TjHk3RBvGRQOGCk/mKI8cREbP3gqVF2EMcZvug7EWVSxneWt7O8LjN5nQrlG3QPL36 2PhChguV2Naq0J3hSqUYEd1ZBwa7RpfO3v2JpkN21Ck1qtJCepDiTW0IWcmdIbYYrMkf kpDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045472; x=1684637472; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=KIwAxCs+xB0lesjbp62PO3r6+mEINYjwnW6XfaYSSdQ=; b=JL0o241HNZOWZAzXvYhOI6OowGepWDYnMXDC0u4noZ3QQeRRUgV0Gk8Jzfyy3+t+3m jLpPY1PJxm3VFL3VEYHyx6tyM70pIHlv8M6B6eFwFI0suJMCAytjl6e69FDpr3Q2GZjD lHzIo3KQg6tqXzyT2UqNdPr31syTWw5YFvAh2QkuzmLoNfE6BhpiVQgKk9P6vINuF+G6 Dijs9f3U4Pp/hYsDyoebx3mEq+x57E7MBAsOacMww2bt2j5WQdJ289D9jLEhQ6j9u2pn 65R/dSIuqzHuU0qIziPB1XYFb6eMyVz/rnSMv3bdJ8omNwhnUC1/YE+0UJ6/doeaXTa3 7KZw== X-Gm-Message-State: AAQBX9eVSp7qLxo6RGaS64pudwYuklgLZZPSK0zx+DeYA4F41fh0LLKN E26iFDP7xig1K2GssMpUQC0= X-Received: by 2002:a05:6a21:29cb:b0:f0:2893:8a3d with SMTP id tv11-20020a056a2129cb00b000f028938a3dmr3457859pzb.45.1682045471653; Thu, 20 Apr 2023 19:51:11 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id bl11-20020a056a00280b00b0063b1b84d54csm1881841pfb.213.2023.04.20.19.51.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:51:11 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , Chandrashekar Devegowda , Intel Corporation , Chiranjeevi Rapolu , Liu Haijun , M Chetan Kumar , Ricardo Martinez , Loic Poulain , Sergey Ryazanov , Johannes Berg , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , netdev@vger.kernel.org Subject: [PATCH 11/22] net: wwan: t7xx: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:35 -1000 Message-Id: <20230421025046.4008499-12-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763753216490865992?= X-GMAIL-MSGID: =?utf-8?q?1763753216490865992?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: Chandrashekar Devegowda Cc: Intel Corporation Cc: Chiranjeevi Rapolu Cc: Liu Haijun Cc: M Chetan Kumar Cc: Ricardo Martinez Cc: Loic Poulain Cc: Sergey Ryazanov Cc: Johannes Berg Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: netdev@vger.kernel.org --- drivers/net/wwan/t7xx/t7xx_hif_cldma.c | 13 +++++++------ drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c | 5 +++-- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/drivers/net/wwan/t7xx/t7xx_hif_cldma.c b/drivers/net/wwan/t7xx/t7xx_hif_cldma.c index aec3a18d44bd..7162bf38a8c9 100644 --- a/drivers/net/wwan/t7xx/t7xx_hif_cldma.c +++ b/drivers/net/wwan/t7xx/t7xx_hif_cldma.c @@ -1293,9 +1293,9 @@ int t7xx_cldma_init(struct cldma_ctrl *md_ctrl) for (i = 0; i < CLDMA_TXQ_NUM; i++) { md_cd_queue_struct_init(&md_ctrl->txq[i], md_ctrl, MTK_TX, i); md_ctrl->txq[i].worker = - alloc_workqueue("md_hif%d_tx%d_worker", - WQ_UNBOUND | WQ_MEM_RECLAIM | (i ? 0 : WQ_HIGHPRI), - 1, md_ctrl->hif_id, i); + alloc_ordered_workqueue("md_hif%d_tx%d_worker", + WQ_MEM_RECLAIM | (i ? 0 : WQ_HIGHPRI), + md_ctrl->hif_id, i); if (!md_ctrl->txq[i].worker) goto err_workqueue; @@ -1306,9 +1306,10 @@ int t7xx_cldma_init(struct cldma_ctrl *md_ctrl) md_cd_queue_struct_init(&md_ctrl->rxq[i], md_ctrl, MTK_RX, i); INIT_WORK(&md_ctrl->rxq[i].cldma_work, t7xx_cldma_rx_done); - md_ctrl->rxq[i].worker = alloc_workqueue("md_hif%d_rx%d_worker", - WQ_UNBOUND | WQ_MEM_RECLAIM, - 1, md_ctrl->hif_id, i); + md_ctrl->rxq[i].worker = + alloc_ordered_workqueue("md_hif%d_rx%d_worker", + WQ_MEM_RECLAIM, + md_ctrl->hif_id, i); if (!md_ctrl->rxq[i].worker) goto err_workqueue; } diff --git a/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c b/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c index 46514208d4f9..8dab025a088a 100644 --- a/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c +++ b/drivers/net/wwan/t7xx/t7xx_hif_dpmaif_tx.c @@ -618,8 +618,9 @@ int t7xx_dpmaif_txq_init(struct dpmaif_tx_queue *txq) return ret; } - txq->worker = alloc_workqueue("md_dpmaif_tx%d_worker", WQ_UNBOUND | WQ_MEM_RECLAIM | - (txq->index ? 0 : WQ_HIGHPRI), 1, txq->index); + txq->worker = alloc_ordered_workqueue("md_dpmaif_tx%d_worker", + WQ_MEM_RECLAIM | (txq->index ? 0 : WQ_HIGHPRI), + txq->index); if (!txq->worker) return -ENOMEM; From patchwork Fri Apr 21 02:50:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86120 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp781686vqo; Thu, 20 Apr 2023 20:10:35 -0700 (PDT) X-Google-Smtp-Source: AKy350YCh+AOOoKDMGRmc/m3y6fnU2XqUpUt7y7lrHN8zLg9a9cBkaI6EflEqyhPvwq+0rZmZ2LE X-Received: by 2002:a17:903:1249:b0:1a6:b140:9d2 with SMTP id u9-20020a170903124900b001a6b14009d2mr4354907plh.61.1682046635374; Thu, 20 Apr 2023 20:10:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682046635; cv=none; d=google.com; s=arc-20160816; b=xcWZ+bbouNwyz95rAS9Y9iIBFc3m18MCTAezkS1/+SjogocXkNcmqjaYHrBfMNat9w OROhHXRyLHA5gFPeSGdklt9pyk5WfRa0o0G2wtboH7R9wg8SADTKmDAf0MVtsmqMhPyP WNsSqL+N8KLK1oF/fm8LHMkT4Ylk7Nxb7mNT/vazkrVMQrpcpzBOPHL8BWLK9PCEKGUC SUtDblxpSwd/VvUQCc/JEjhf7c4OKVKGejktOIIkp92PtMhUn25XcWhVsIEJd/vUGfQA zSjw71ydyX09+IJBmyJWN1IKcQY+t0d5kfSh4Cy5HYB/eD7zgsakv5U+eRf5D6vEEsvQ 9oIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=QmaMAKRoH6NBJdRRp6jUqykqTsJRakI7Ct89GV+DOWM=; b=XiOV0BWPG7vlsSYQMxc3D1tWmJmc8Luy+7IHr7R3mN5V7C8MbaXp83aEDUgWttwOBp GnLHyVrAq2uwenxTQA9AvxAfovgQbDmnHEAOea6BhUGGATDlyVnKMxTM1Kh5umDQml94 PnKPIQ1wsqvIIkYl4AVkZY2ApxGOJghiqJsHL9fwyRMpKIJf2Xudn+ol5KZOThMIjfw6 G6WmxQ0GSAtGuUkhoQNwrb4wX05Zv+JdAO2pcDbX85wbqSNXqMzZtuYTxiBr1cQmP6sm DF4qtjB11vDjcXF8+Y6MNu3rJGaephg0NlrJcchiLvGIi4A5+lMJO2l3jhBeZfU+PqNJ 9jiA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=fyAXCpyz; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u14-20020a170902e80e00b001a6fe422894si3499731plg.200.2023.04.20.20.10.22; Thu, 20 Apr 2023 20:10:35 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=fyAXCpyz; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233804AbjDUCvz (ORCPT + 99 others); Thu, 20 Apr 2023 22:51:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43396 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233760AbjDUCvP (ORCPT ); Thu, 20 Apr 2023 22:51:15 -0400 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0FF996A4B; Thu, 20 Apr 2023 19:51:14 -0700 (PDT) Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-63b5c4c769aso2391515b3a.3; Thu, 20 Apr 2023 19:51:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045473; x=1684637473; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=QmaMAKRoH6NBJdRRp6jUqykqTsJRakI7Ct89GV+DOWM=; b=fyAXCpyzGql2Wiskio6/QvXSwDZANZf0aIFTTvDyv6iBT8ihou/1lc/E6vC3XoW1jN 8RMPaUOGEXxnKfWrLjFPhgwSkfcz6tYV57+iXnIHOScYfYIYV9UmvaZ9hw/hImSrP2qg GXuZ1ipRmDeb+uUAVZkDAcEIHtubeaEFky5VRoAeVHfaQuGfDPrXTX0NQDfDWIWAfr5s 72ukvBfhXHbmcCI/7hLEfRTkYcH8+aApWnPD8gBqKRSEmJYRloXlDCIvGs+l77Jppl8t G8NDgfDTh14VTU5Q52df3MKfV+WmAycbjoBCuxkdbWWgBYzDa/vBoDh05MBCxTgszo8e 1J4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045473; x=1684637473; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=QmaMAKRoH6NBJdRRp6jUqykqTsJRakI7Ct89GV+DOWM=; b=GA2w5gniAXUOEs0Q1Un6kp+RsR9b5mfKjMwpK5uAnO3Ly1aFP1xT5eAtBz1WBtVbmf OY9X9HLYDDOh7uNyLQaIkhvM1UXkA50g1p7XfarQbIUPPORbYAQa9bUh7VCLUn7z8GOm EKPtGFjOeRtKsC513vAOs5UrNbQU9js+4RdZg4BEMc1zb9CEefEqIapWYi++uqHSmvHP vEu0p8USHy0jZ7Iz5h5XQItIDXzt8asK0bEf77v5mJ6JD1Jqghft4Sjhzi30GPTcGfgu idU7rLEXJ4wQ2Ms8qlDoJzFhhzUNdEr1dWhPOdqH52zg7oZwW38JtAtLBQLtT46aCM9H x+XA== X-Gm-Message-State: AAQBX9fqndA+8FPPQbOUg2FguRFncFZoAEZXNpmugwyyYSeZYjIAVZcC c0H1bsU9BH2ddZmTxHNMKCg= X-Received: by 2002:a05:6a00:1796:b0:634:7ba3:d140 with SMTP id s22-20020a056a00179600b006347ba3d140mr4929709pfg.15.1682045473354; Thu, 20 Apr 2023 19:51:13 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id o70-20020a62cd49000000b0062dcf5c01f9sm1916148pfg.36.2023.04.20.19.51.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:51:13 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , "James E.J. Bottomley" , "Martin K. Petersen" , linux-scsi@vger.kernel.org Subject: [PATCH 12/22] scsi: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:36 -1000 Message-Id: <20230421025046.4008499-13-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763753732744074265?= X-GMAIL-MSGID: =?utf-8?q?1763753732744074265?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: "James E.J. Bottomley" Cc: "Martin K. Petersen" Cc: linux-scsi@vger.kernel.org Reviewed-by: Benjamin Block --- drivers/scsi/NCR5380.c | 5 ++--- drivers/scsi/hosts.c | 12 ++++++------ drivers/scsi/libiscsi.c | 5 ++--- 3 files changed, 10 insertions(+), 12 deletions(-) diff --git a/drivers/scsi/NCR5380.c b/drivers/scsi/NCR5380.c index ca85bddb582b..b18dd4591492 100644 --- a/drivers/scsi/NCR5380.c +++ b/drivers/scsi/NCR5380.c @@ -415,9 +415,8 @@ static int NCR5380_init(struct Scsi_Host *instance, int flags) hostdata->flags = flags; INIT_WORK(&hostdata->main_task, NCR5380_main); - hostdata->work_q = alloc_workqueue("ncr5380_%d", - WQ_UNBOUND | WQ_MEM_RECLAIM, - 1, instance->host_no); + hostdata->work_q = alloc_ordered_workqueue("ncr5380_%d", + WQ_MEM_RECLAIM, instance->host_no); if (!hostdata->work_q) return -ENOMEM; diff --git a/drivers/scsi/hosts.c b/drivers/scsi/hosts.c index 9b6fbbe15d92..30bf9f49ca6c 100644 --- a/drivers/scsi/hosts.c +++ b/drivers/scsi/hosts.c @@ -294,9 +294,9 @@ int scsi_add_host_with_dma(struct Scsi_Host *shost, struct device *dev, if (shost->transportt->create_work_queue) { snprintf(shost->work_q_name, sizeof(shost->work_q_name), "scsi_wq_%d", shost->host_no); - shost->work_q = alloc_workqueue("%s", - WQ_SYSFS | __WQ_LEGACY | WQ_MEM_RECLAIM | WQ_UNBOUND, - 1, shost->work_q_name); + shost->work_q = alloc_ordered_workqueue("%s", + WQ_SYSFS | __WQ_LEGACY | WQ_MEM_RECLAIM, + shost->work_q_name); if (!shost->work_q) { error = -EINVAL; @@ -510,9 +510,9 @@ struct Scsi_Host *scsi_host_alloc(struct scsi_host_template *sht, int privsize) goto fail; } - shost->tmf_work_q = alloc_workqueue("scsi_tmf_%d", - WQ_UNBOUND | WQ_MEM_RECLAIM | WQ_SYSFS, - 1, shost->host_no); + shost->tmf_work_q = alloc_ordered_workqueue("scsi_tmf_%d", + WQ_MEM_RECLAIM | WQ_SYSFS, + shost->host_no); if (!shost->tmf_work_q) { shost_printk(KERN_WARNING, shost, "failed to create tmf workq\n"); diff --git a/drivers/scsi/libiscsi.c b/drivers/scsi/libiscsi.c index 127f3d7f19dc..d0eba590dc69 100644 --- a/drivers/scsi/libiscsi.c +++ b/drivers/scsi/libiscsi.c @@ -2907,9 +2907,8 @@ struct Scsi_Host *iscsi_host_alloc(struct scsi_host_template *sht, ihost = shost_priv(shost); if (xmit_can_sleep) { - ihost->workq = alloc_workqueue("iscsi_q_%d", - WQ_SYSFS | __WQ_LEGACY | WQ_MEM_RECLAIM | WQ_UNBOUND, - 1, shost->host_no); + ihost->workq = alloc_ordered_workqueue("iscsi_q_%d", + WQ_SYSFS | __WQ_LEGACY | WQ_MEM_RECLAIM, shost->host_no); if (!ihost->workq) goto free_host; } From patchwork Fri Apr 21 02:50:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86110 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp778458vqo; Thu, 20 Apr 2023 20:02:43 -0700 (PDT) X-Google-Smtp-Source: AKy350apceh9G3NJtelvVzjjw6z+oD6QPhd2OdFZJk3urDP0A6Bo9jJi8tgiKvGLeV0vsQGWgHbe X-Received: by 2002:a05:6a00:1948:b0:63e:9e1f:6d06 with SMTP id s8-20020a056a00194800b0063e9e1f6d06mr3885520pfk.17.1682046163330; Thu, 20 Apr 2023 20:02:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682046163; cv=none; d=google.com; s=arc-20160816; b=p8tJldnO9nl2L8kIqbXSi7WOMaJzfv2lB+oc9F4Pu0A+gMhVkFQiiX0mxfJVJuBBGF grkY31eZZ9v2/St00KPBzebImqwaW1jEADopjC3ML+/lsRjRku9vRXismjkxaBl/XbRi LBalYDEmirxk3mldBejoVf2xRUOOVfLLSCn33lufpPDnMDPatIGaxfrbq1dgHAEhBwUw 24dC8o+3IbdUi2gffhhEVbgrnWPXneucDd/CGap5dc5YVOcm4/shoCQlOsN94y5NtQNn 4eckeE5LZea+L92Kj5S9VDCLCT0TzjPxnl+YJKKMH8vTaAxTC47jHdSoxIf0rkPYFDHy 7Zkw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=qNxWNsNkiW6dC2MPWOAlJ0DZEAiQsjxz/0ttEihBSFM=; b=yz5DyVjdXjwAyYLLi1tFwrnZVqkNMVtgKySr6kYLCcEGfvm1zwGxcQZs9JpSwzgW1n BWU1igxjzscb6UDlR/B8bBY/EV3ttgJXl9lrMl2hadpGVvy1WZMdWujM0TFWcQH4wgdg Tr7GRc2JurdE6v4vZxap0CMvOXmQEIjCozx5EhtdbIJWjiyxrpbd0B92rxoMQl83KS+S HIIa8YH0KarhOf8CsWCYkrNkMSNE9QDLIsZOjehJGwZE10eCsL3BjsUMQ/EglGgh6GP2 6yD8EmRxSp0RxAAb1Fyz7nYLz0F9FMHjduJE0xFONt++t+lU2+Wg9pcHcCRScWfyO6hD NpjA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=mk8IgkQX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k15-20020aa7972f000000b0063b8695da40si3098024pfg.238.2023.04.20.20.02.31; Thu, 20 Apr 2023 20:02:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=mk8IgkQX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233955AbjDUCv5 (ORCPT + 99 others); Thu, 20 Apr 2023 22:51:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43400 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233765AbjDUCvR (ORCPT ); Thu, 20 Apr 2023 22:51:17 -0400 Received: from mail-pl1-x62d.google.com (mail-pl1-x62d.google.com [IPv6:2607:f8b0:4864:20::62d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DB9AF7681 for ; Thu, 20 Apr 2023 19:51:15 -0700 (PDT) Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-1a677dffb37so16334345ad.2 for ; Thu, 20 Apr 2023 19:51:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045475; x=1684637475; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=qNxWNsNkiW6dC2MPWOAlJ0DZEAiQsjxz/0ttEihBSFM=; b=mk8IgkQXApz1FaufEiZfvC15wekz3UMd4Nl3Mr2cB9VJ/qBecpD1Bpr+n5KabJWu6G cmrHDbaznnLLk6ScZLN/q0opj00Kkp0tiX9FxxwSaVFu7nOGXMg2aPYHHok0gmMBXZK7 /1D4K/5afm3qRf9oUoK5OWZlDM0Vk7G1EyqSMAdhIPMPDFXpHItCtLA9Jd0bSBnlCrFV ghN652WLJTDjKK1GjX7VkJEIbiKB887uO9up9rx/DVXUrW4BRmBBWfFAUVftAvZ/9Jgw LGYOmv7h/LLENqWfbT5qujjToajeflNFPmZN8wmNF4xr1voiRREWbW8VVrgPUq5UMcBI 2Spg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045475; x=1684637475; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=qNxWNsNkiW6dC2MPWOAlJ0DZEAiQsjxz/0ttEihBSFM=; b=M1oY0hE6Bbe6KBnQzfjBCLCOVi6wpU4HngRqnRTJXHAh22JQ4fJKIYOfj20fsjPWYN /7Uk+siQ7XxA0AQfsgvzU+VzkKl0eZKd6Dg7QVGJO04G7mgBFfmQyDew0uebqVMrPplr CN+NC6eRWaPMcG4/pkWv6qNcnbWPtWlOtFb3xw2RNBhEOSWaQFNTzAsTO4d60tdfMVG7 Zlh4Cj+kRojSEF2DQT25x4rJjCobMLV9gRrkuwi2cDNUUK9sjSRT7qXlFcF2tic8tY9k SrF2buYHjE6j8yc3sLTca+fxJOzeESufla1TreDSmnQWzeXq7UFf6BTPwcXhbpJ/Y9SN woBQ== X-Gm-Message-State: AAQBX9fWrDEMk3gqUjkzNIdSvNgAxEjRhMayQOl7Zs7wVRHPFXXTz0Mm YmSyUnFI3Zae3xEfEuaYxBE= X-Received: by 2002:a17:902:ecc4:b0:1a6:9f85:9437 with SMTP id a4-20020a170902ecc400b001a69f859437mr4245549plh.67.1682045475058; Thu, 20 Apr 2023 19:51:15 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id x9-20020a1709029a4900b001a6388ce38bsm1724167plv.240.2023.04.20.19.51.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:51:14 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , Fei Li , Greg Kroah-Hartman Subject: [PATCH 13/22] virt: acrn: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:37 -1000 Message-Id: <20230421025046.4008499-14-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763753237813245950?= X-GMAIL-MSGID: =?utf-8?q?1763753237813245950?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: Fei Li Cc: Greg Kroah-Hartman Acked-by: Greg Kroah-Hartman --- drivers/virt/acrn/ioreq.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/virt/acrn/ioreq.c b/drivers/virt/acrn/ioreq.c index d75ab3f66da4..cecdc1c13af7 100644 --- a/drivers/virt/acrn/ioreq.c +++ b/drivers/virt/acrn/ioreq.c @@ -576,8 +576,8 @@ static void ioreq_resume(void) int acrn_ioreq_intr_setup(void) { acrn_setup_intr_handler(ioreq_intr_handler); - ioreq_wq = alloc_workqueue("ioreq_wq", - WQ_HIGHPRI | WQ_MEM_RECLAIM | WQ_UNBOUND, 1); + ioreq_wq = alloc_ordered_workqueue("ioreq_wq", + WQ_HIGHPRI | WQ_MEM_RECLAIM); if (!ioreq_wq) { dev_err(acrn_dev.this_device, "Failed to alloc workqueue!\n"); acrn_remove_intr_handler(); From patchwork Fri Apr 21 02:50:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86105 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp776920vqo; Thu, 20 Apr 2023 19:59:13 -0700 (PDT) X-Google-Smtp-Source: AKy350aQ6KOSO3Hbc0LLdlByXw/hgiQyh7FR194UpZEQuqNEFefdvnq6aPLVhe7g9mhzkEJo2gc3 X-Received: by 2002:a05:6a00:2e06:b0:63a:33d5:9224 with SMTP id fc6-20020a056a002e0600b0063a33d59224mr4016853pfb.18.1682045953478; Thu, 20 Apr 2023 19:59:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682045953; cv=none; d=google.com; s=arc-20160816; b=WxGPWq4PP/0JMwWFthgDEGGkvsr3apsHxcSI6+aMO7at0Ay4OEday+Hr43jOpE7sk4 MXUvlayx9RUtWmd2XsvJTSzFeUBTJlCosvCaVMFGsaHvw0PTWGfY644NNYxzy9GsLlx4 71EL+9rY4dsnyAkmtkJ93waFVFMQ9kxdITJQ+NFN/RmumonBstRxhpks/SzkqPm61wuB BFMdaLMs1IhLI2dklglTxTmWO1+qn51WwdB7FMXRaHefI5m4u+PLhm6WzcPtTK+WKZ5o rFwmqKhpejbrBSKnJkAKEACsOIu53dwf292SSDiQxSMxLlihrBlVwhCdYxSyATd88kvk wbzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=dorQ2qlHIQqiNBUoiQdfsVbgm5wQDWFtkN4YHEiGiW0=; b=OSqwK5Ob1UjluJTbhGk8FLGtKuTTaoLXVRMkqqUrDfFl54gIJYBPT1N2ls0uFCXfno uDGh0zpiWZFUaJWTT1Hh8F5rdZR0GRIcyS2RVVSeZL6nZpeS+xIY5tecd5upXZ/a0x1m v7dp2Miw9seCJlK6x3o6Ct990g0KcQu238VONftp1hNLfANFgpVbvOl3kz6RKXj56/7A UeqwmcCeWE0M77T6BYUQVJPgYl0kOKYQBp/Me8lDE8euFxa/kIWMhS6maKEfex2tvfXo ccCLSH+uwRau6/FVJbQl71Il5zseGB+HyQ/WUQmPfaB+gKM19fZ1SdPzsYb32qVxHLK+ BcCQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=qcJ3g04q; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s64-20020a625e43000000b00625c2bd736asi3154323pfb.20.2023.04.20.19.58.58; Thu, 20 Apr 2023 19:59:13 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=qcJ3g04q; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233982AbjDUCwF (ORCPT + 99 others); Thu, 20 Apr 2023 22:52:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43512 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233767AbjDUCvS (ORCPT ); Thu, 20 Apr 2023 22:51:18 -0400 Received: from mail-pl1-x62a.google.com (mail-pl1-x62a.google.com [IPv6:2607:f8b0:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 65F057292; Thu, 20 Apr 2023 19:51:17 -0700 (PDT) Received: by mail-pl1-x62a.google.com with SMTP id d9443c01a7336-1a92513abebso19694875ad.2; Thu, 20 Apr 2023 19:51:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045477; x=1684637477; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=dorQ2qlHIQqiNBUoiQdfsVbgm5wQDWFtkN4YHEiGiW0=; b=qcJ3g04qdI5nJnF0RwVis4v7EzX0fPL613I0miXb8OBRVtnPZi9rsPKFiOjvtxzM9D k4Lc3kzcMkqM07Gz7lPfr65+OH1ShvUvlu3vKZQ0fksXbAXYpykFjcdCl1ksoIWsEFYw rlrADdfGdmznF4OYk1+zclIZmpe23JU/7PO2So1p+ZwS90tZHUeLdizz1m7pulXEmZhP JGKQXVuxT0LKxJS228sfpj/zMqKIox+ydHEQeloOIjfeqjSuZ8kOkXhW9pxOklPjfL00 93syjdjXSPpvWDcKqeuDfLzjyTy/SCOqiAy9ukPDb2LF4fKF1DQoMLXT/IpOgRa9GTEE ZPkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045477; x=1684637477; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=dorQ2qlHIQqiNBUoiQdfsVbgm5wQDWFtkN4YHEiGiW0=; b=Z1AzSHRfN+nxvvhvI5oyr2CNGwfYyBndxOq7C9SVHs5YalzuNgEjTTUGHFFn5aFMQb F4iIKSLvBJ6dzDc8D7G7th0Ue+kTOEwxu10WMObZfaRN6SewWIxtnXtc65HqQZiCiBHM P9/VvrhzIR2AoK8E/2xdU/b3Jt6IA834RDdvJNJurfkp8p62tU6K7KZ28aH+9EJdq+5x mMgzX1pqtWVLsR0HTvh5/J1frGjoJAx49kF7s5bKvemcNehNyobwwMlK4XY5Q8tUzloH /FqTAXBW1WfvUs/lCkRM+Wkqy6N0CHvz2IrNvsUuGnOW8dBK9flCdN8I6whkOTdBQ2T/ ypJw== X-Gm-Message-State: AAQBX9cIRvImqgRtXtcU+8SMJq1857xxBswCuoWNn2J/T0yZOVHBMz13 kEY2a+8HpRuuNAFdVEhKaX2+h+rUlTY= X-Received: by 2002:a17:902:748c:b0:1a6:be37:22e1 with SMTP id h12-20020a170902748c00b001a6be3722e1mr3102959pll.15.1682045476738; Thu, 20 Apr 2023 19:51:16 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id k1-20020a170902760100b001a1adbe215asm1741130pll.142.2023.04.20.19.51.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:51:16 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , Andy Gross , Bjorn Andersson , Konrad Dybcio , linux-arm-msm@vger.kernel.org Subject: [PATCH 14/22] soc: qcom: qmi: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:38 -1000 Message-Id: <20230421025046.4008499-15-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763753017374073151?= X-GMAIL-MSGID: =?utf-8?q?1763753017374073151?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: Andy Gross Cc: Bjorn Andersson Cc: Konrad Dybcio Cc: linux-arm-msm@vger.kernel.org --- drivers/soc/qcom/qmi_interface.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/soc/qcom/qmi_interface.c b/drivers/soc/qcom/qmi_interface.c index 820bdd9f8e46..78d7361fdcf2 100644 --- a/drivers/soc/qcom/qmi_interface.c +++ b/drivers/soc/qcom/qmi_interface.c @@ -650,7 +650,7 @@ int qmi_handle_init(struct qmi_handle *qmi, size_t recv_buf_size, if (!qmi->recv_buf) return -ENOMEM; - qmi->wq = alloc_workqueue("qmi_msg_handler", WQ_UNBOUND, 1); + qmi->wq = alloc_ordered_workqueue("qmi_msg_handler", 0); if (!qmi->wq) { ret = -ENOMEM; goto err_free_recv_buf; From patchwork Fri Apr 21 02:50:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86107 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp778245vqo; Thu, 20 Apr 2023 20:02:12 -0700 (PDT) X-Google-Smtp-Source: AKy350bu21CxXe9NEOuj8DLeI3USPZHOBRxpHQv2BdfRimq2uckaQT1JLAZHIIfsrPUxytEB6pOn X-Received: by 2002:a05:6a21:6d88:b0:f0:36a3:706a with SMTP id wl8-20020a056a216d8800b000f036a3706amr5530862pzb.10.1682046132549; Thu, 20 Apr 2023 20:02:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682046132; cv=none; d=google.com; s=arc-20160816; b=BL302AKNS3+ig3+eUzxcDHtwnZheYVIST82+zV5DpnW+zjXNPCzKR8Hwy0CRyGo9GC Hm0mlYAZP/4a3YePW4oJKNZONFrnrQp1+USDie8Qn3zb+d17x+14ISacwdyQbhpBvT+Q 59WBW1DNBLNFTPBAONE6eMMp5fhrgcMNece8Ngxx+i5Nczr7wVpv6bsuIqUTYe47QEaa f6m6jWMnmnp6v+0J3Kp8EVTBwgelQPMRVQaGg+rtE6BXlJuj8OMFDhUK3t6EcpkZc0uG hGfy31EMiPzGnIysjeptDV/Dc9TPQ5weu23II0fKEeV/IDWpEgzHgOaDPV69yQ7QGWIS 7MAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=HCPjQIn4S0PQfuCNwvELUc3S8eh+s9mUvwJw7/d5uLU=; b=dFZcEO6u9yPeOWw4y35uzlK5BnCHMkT4zHeVp4jAaRMKzWZrC1Z2mh7P06YG6UmGFm xKqfyAtKcR7yKPqwVcEpL0vpXNs+t8jmolwMVUZi82P28rmUiqWJ0FtzzGKKF5sy1Rxo 5CoZa0xbjMWbu1ipMMgY4rLbHCexyEk6igd9gdjc61ZV5kzAv4mtUIBtYlvhpO0ouQ29 OQLzCRB0UIFBlmLvrdtsvkSVcRO2GFtFYMCBIDDoeaFZUUV7WJn78aUVC7dxZEjDVuMB N3mLYJQbVgRQyFDEjaN4Rl2pkAafipoWiMzhnCD61ECniJ+/X8+yHuQl35r8THDbfTQ/ yj3A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=XMLFtb9w; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l188-20020a6325c5000000b0050bf0f1b79csi3588054pgl.699.2023.04.20.20.01.59; Thu, 20 Apr 2023 20:02:12 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=XMLFtb9w; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233973AbjDUCwD (ORCPT + 99 others); Thu, 20 Apr 2023 22:52:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43566 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233674AbjDUCvV (ORCPT ); Thu, 20 Apr 2023 22:51:21 -0400 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 51F827692 for ; Thu, 20 Apr 2023 19:51:19 -0700 (PDT) Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-63b4bf2d74aso1496380b3a.2 for ; Thu, 20 Apr 2023 19:51:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045479; x=1684637479; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=HCPjQIn4S0PQfuCNwvELUc3S8eh+s9mUvwJw7/d5uLU=; b=XMLFtb9wYP9gqpPSKTS7T/p74xQvZs2nOgKFEigpBUk7eM6Kb8bvSh/sAWoXK1dcSj KWvLP7zD7yOkPXgcobpT2jsCaQldUniFt2NYg1yoc43TcDmS4stE8W3P1iJVksjOSFgn jFjMkwjkXmYgdw0wK5uHCXH21wcr+eFz6d3UdFMZxRfvvPHtseSbL2d8RWYeNIn0heuJ FLmKIh3j/9iUx4YIIjpr7XgLkSBxR4PUF5gMWpa5OT7mIXmR9L4DowA/AYb/EeXijmVO hy4tJiy1od2XsdGkrSh8kcquhPEtsLG9Z6cwResFqBkNejkqYJI+lXHC9L6wZvRHe583 m/KQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045479; x=1684637479; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=HCPjQIn4S0PQfuCNwvELUc3S8eh+s9mUvwJw7/d5uLU=; b=HxsX7UzHRoMmP6ltvzBzL7/BGtdpNQWoCNiw3nEwb8YewtJAiQvpZxyIFfe+C4ijW0 nsk12f+1SQrqvro+D9Dfgbc7IayAVNqT55eCtBz3ZYEdRSJEJkDMJyBjSV7jQxyfK5BD OnPI/CY9HmcuSzt/KaVCEEGWQtKQUqQMSlUEZP3kbZjekt3ejePAx2E3nMTWsxUbZfMF fOcE7zRkaeIbWrK/+Iv60V0/Zlw4ENJz+ygeCe7sJfKX5k8786Qn3BTA/5za19U2A6Yy TwiY5S8fDJWgPxvRnZ6l/tgPN7ZNrRUzJGPbCl2F8tW4gmtMME2Kkqi8yTpbo3EnNhaY vwmQ== X-Gm-Message-State: AAQBX9coJbHwaV0L6b+LgZnJ01AsGimwwMOZrt/wGwH+WM4g6mrQ8zCx VCkerv8VGmII16ihWzVKveJqS+gCCUE= X-Received: by 2002:a05:6a00:15ce:b0:63d:3411:f9e3 with SMTP id o14-20020a056a0015ce00b0063d3411f9e3mr5061818pfu.19.1682045478499; Thu, 20 Apr 2023 19:51:18 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id n11-20020a056a00212b00b0063f167b41bdsm366523pfj.38.2023.04.20.19.51.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:51:18 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , xen-devel@lists.xenproject.org Subject: [PATCH 15/22] xen/pvcalls: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:39 -1000 Message-Id: <20230421025046.4008499-16-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763753205436887120?= X-GMAIL-MSGID: =?utf-8?q?1763753205436887120?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: Juergen Gross Cc: Stefano Stabellini Cc: Oleksandr Tyshchenko Cc: xen-devel@lists.xenproject.org Acked-by: Juergen Gross --- drivers/xen/pvcalls-back.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/xen/pvcalls-back.c b/drivers/xen/pvcalls-back.c index 1f5219e12cc3..b41516f3f84a 100644 --- a/drivers/xen/pvcalls-back.c +++ b/drivers/xen/pvcalls-back.c @@ -361,7 +361,7 @@ static struct sock_mapping *pvcalls_new_active_socket( map->data.in = map->bytes; map->data.out = map->bytes + XEN_FLEX_RING_SIZE(map->ring_order); - map->ioworker.wq = alloc_workqueue("pvcalls_io", WQ_UNBOUND, 1); + map->ioworker.wq = alloc_ordered_workqueue("pvcalls_io", 0); if (!map->ioworker.wq) goto out; atomic_set(&map->io, 1); @@ -637,7 +637,7 @@ static int pvcalls_back_bind(struct xenbus_device *dev, INIT_WORK(&map->register_work, __pvcalls_back_accept); spin_lock_init(&map->copy_lock); - map->wq = alloc_workqueue("pvcalls_wq", WQ_UNBOUND, 1); + map->wq = alloc_ordered_workqueue("pvcalls_wq", 0); if (!map->wq) { ret = -ENOMEM; goto out; From patchwork Fri Apr 21 02:50:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86109 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp778442vqo; Thu, 20 Apr 2023 20:02:41 -0700 (PDT) X-Google-Smtp-Source: AKy350a4t8U32deQ5nV7lEgfGAkLWY6t+x53jMghUxj43wvq9+78rGiqVf2bZzNZ+eX4GiTPFmmV X-Received: by 2002:a17:90b:1906:b0:23d:54e8:3bb7 with SMTP id mp6-20020a17090b190600b0023d54e83bb7mr4170857pjb.17.1682046161567; Thu, 20 Apr 2023 20:02:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682046161; cv=none; d=google.com; s=arc-20160816; b=EemIYIg0I4Ra2TWJMIroLFGuYjaBH2ut6yFGcHhGVzIC1OhAB6rl/w97IQ5NCh3zpB X2JSQc6xkDbqpm1GHkYNpbNv1oVggtRaswWzU4ffL+o+pus7YtYgKok5/4lNpO+SSa5s EWCYC/IVS0QNObBXL0XdaurfvqfTepw2uL5+FPqQ74b3QZU7mer9aMqqfrMDMXCMg/6k Q54GQYMJ+E73Z/Re8M7QA9/Op0r/ABzAgys24ewUWvom4TsJ5zBA1WxsPEVRxs1c34pV B4os2mP3gSoQSQiOPsJ4bpEXtLekJYUUcvV1GRTV3sGkundQZL5fIqf4M0gfm74fwy1l LcXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=TJ7Cn95EWZ5JETSsGI/LC39uqQkeGr0Ng7A7gd766EA=; b=OzCmXImoNGRv/zU25ESXl7zJ7ywpnnsG3uM+4gVC6AE8OU3VJcu7HjCL2W4td4F4C8 jiwb24A2fVcmC/WTjeNSlTe9b7utPWca8wkVzdkV+vx7ERQh+D7+QVN470YH4o9RMC8F Xg/vl1wcH2X6NzicjkeEPzP2fTnUlTB5y+FJZR94Cv6X79WYQwjpmGc5ncC3SrrqFMVq WnZ1ozPZuPSTk09xCjMhS8wDxl8JY9pJxtPEGSbdSnO9bAXnZIbIGIS/hN3585twtpnF p8IVaVdvQEwSxkTf6PCKTbwTzj8XIFMl7x60syuSJBGNcUzlF4IATyzJVSs292+wDapc 2qAw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=CFt5GCPc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id hg5-20020a17090b300500b0023409b3983fsi3369136pjb.99.2023.04.20.20.02.29; Thu, 20 Apr 2023 20:02:41 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=CFt5GCPc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233964AbjDUCwB (ORCPT + 99 others); Thu, 20 Apr 2023 22:52:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43582 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233799AbjDUCvW (ORCPT ); Thu, 20 Apr 2023 22:51:22 -0400 Received: from mail-pf1-x436.google.com (mail-pf1-x436.google.com [IPv6:2607:f8b0:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E6CAD3ABC; Thu, 20 Apr 2023 19:51:20 -0700 (PDT) Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-63d4595d60fso11556768b3a.0; Thu, 20 Apr 2023 19:51:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045480; x=1684637480; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=TJ7Cn95EWZ5JETSsGI/LC39uqQkeGr0Ng7A7gd766EA=; b=CFt5GCPcXiSYIqNG/mYRk8ghpeyEkcNq4qwBUDPRaEev1fEYOmm7mjIY9zy86NaTe7 nalXKDKQ7iFVeuyyWtCByf1EZSR1+vl9feDKy3HKhBve7m38ctf3Yq0GU5GnLY7HZUqK falqItKIfqZQl4XGg01+vfIx2rS5cLzS1FsU5nWY9mUomVBErSS7TvYIix0qczmQxegi sW3l5bEHmOJkEsywt0ZDgn2navgiwLgYpaHRea1OAZbuKVDsCHNZaD5FJAo7V746ARoH SJuBclkFJ1SKVXtZZMIZ73j2KPhC4P8iZYttA+XsYJ85BbO6wxy+1lWaTjvQeSTp6eez K5Yw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045480; x=1684637480; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=TJ7Cn95EWZ5JETSsGI/LC39uqQkeGr0Ng7A7gd766EA=; b=O0Neetxt5xGTTuTJWc2w7amfvhAF5lMJIECQuo7mAAAFdMdVCQ4SlvhmNceMlxmb7c N7CSYumKVYmb7RQGFMOLKHYr5WelhIitaCEeh2ClGOLkwjeLdEcZvSnBpVKnIy1SEpsA E9dltHp+fctaxg7NgPE49Mqr8+f88K4pUhiA9eY0VQpp/fXhfyJpWU6XuTl2Sz8yJGRH kNs861/HtPIV6EcfEesm+USh6J13dvnU3BXYD2gZMpvFmjmfWMr4GcQLx+OmaitOQU1Y GLqR+LVrEcRgHlnxFEMiOKJqKxmuOJch1KQfdkWoAMfNkLNO6Ww0fNiOUheG/XzswkZY P2Pg== X-Gm-Message-State: AAQBX9ecDE9/O+TX71yg08TTvGjAarnRreKD/eqTNRTj1EaPC6N40uo4 EeXtRKXQsZvMiF9VLLgK/PU= X-Received: by 2002:a17:903:22c7:b0:1a6:3737:750c with SMTP id y7-20020a17090322c700b001a63737750cmr9367527plg.21.1682045480309; Thu, 20 Apr 2023 19:51:20 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id x9-20020a1709029a4900b001a6388ce38bsm1724224plv.240.2023.04.20.19.51.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:51:19 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , Chris Mason , Josef Bacik , David Sterba , linux-btrfs@vger.kernel.org Subject: [PATCH 16/22] btrfs: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:40 -1000 Message-Id: <20230421025046.4008499-17-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763753236106144718?= X-GMAIL-MSGID: =?utf-8?q?1763753236106144718?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: Chris Mason Cc: Josef Bacik Cc: David Sterba Cc: linux-btrfs@vger.kernel.org --- fs/btrfs/disk-io.c | 2 +- fs/btrfs/scrub.c | 6 ++++-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 9e1596bb208d..b1f6ff69dbe1 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -2218,7 +2218,7 @@ static int btrfs_init_workqueues(struct btrfs_fs_info *fs_info) fs_info->qgroup_rescan_workers = btrfs_alloc_workqueue(fs_info, "qgroup-rescan", flags, 1, 0); fs_info->discard_ctl.discard_workers = - alloc_workqueue("btrfs_discard", WQ_UNBOUND | WQ_FREEZABLE, 1); + alloc_ordered_workqueue("btrfs_discard", WQ_FREEZABLE); if (!(fs_info->workers && fs_info->hipri_workers && fs_info->delalloc_workers && fs_info->flush_workers && diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index 69c93ae333f6..70882358bdb0 100644 --- a/fs/btrfs/scrub.c +++ b/fs/btrfs/scrub.c @@ -4245,8 +4245,10 @@ static noinline_for_stack int scrub_workers_get(struct btrfs_fs_info *fs_info, if (refcount_inc_not_zero(&fs_info->scrub_workers_refcnt)) return 0; - scrub_workers = alloc_workqueue("btrfs-scrub", flags, - is_dev_replace ? 1 : max_active); + if (is_dev_replace) + scrub_workers = alloc_ordered_workqueue("btrfs-scrub", flags); + else + scrub_workers = alloc_workqueue("btrfs-scrub", flags, max_active); if (!scrub_workers) goto fail_scrub_workers; From patchwork Fri Apr 21 02:50:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86116 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp780825vqo; Thu, 20 Apr 2023 20:08:16 -0700 (PDT) X-Google-Smtp-Source: AKy350YrDeVIC+A6X8LnILyIj9sj/46fYUE/ds0gHByvqywbSBscFVNKDlC6KLthqqO55VjBUe8L X-Received: by 2002:a05:6a21:6811:b0:ef:d23f:d7c3 with SMTP id wr17-20020a056a21681100b000efd23fd7c3mr3859592pzb.41.1682046495899; Thu, 20 Apr 2023 20:08:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682046495; cv=none; d=google.com; s=arc-20160816; b=RA3B4FEKSYdy+bNqng8U9wJoCZJtTGQWv3FlrrLgY9fs51z/a8fdq2NMIL5WhIKJPy LTzWhod9jhpYcGqpPHvgHw9SeAmzbnKIy3a9esuoZ8r/82wOysfbVZrtqFA+B+ibe+du oowRiDlQKosl7XVSqrGmF9wu9t/dflBePKQpAAUUNGGqnOtcnB8C6A5HwyBrz9e7UIpz fX5kWJVywVUHEjW2zMUtbkGILjCoPE/JYUrnfixOm5Wf/aYZl94OJe+C4W9M3tXoEXEG nT7J2yM/K2q8GFNnNfjjr4yxa7LVlt9ugqzZtgdyrIjIFGPOSrtKb6NQCa6k2Hre9n80 NjcA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=akTWRPzzBJ0wdmeG4+QDU8Yy42VUO1hiVrnvCUyERAQ=; b=wvwsntJgy9z3zEq4iqbVsvX5qq/84D3Orb+QRzrAjk04zBlVz/yw8z7kCk1a9bS8+4 itF6cyuQLxCd9vDFSn+OGNZ8xPHdfNCe9qGUlWxHalZrlIopEKHoc7xLemFFTQQS2vEp uR2SrYNT6TtVRP8i5RGfotgitA6Y4M/FSfGUFP8vmTXkw/9IueeIxtgHeZAZcFaZOQsf PS2Su4UtLhecUdXI53DMbVDtD5CFoT4IrrFcZoDJsgUKHd6aMNuN/MDZhg0F5yKtaURF Bs6SQRuJMB2q7uC9dAOTi4V4poWQgOv/6dsiJfBhrLZA+C94OUDfFLr2ny18O+zN56Ss CHXw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=EygXMbRM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bv189-20020a632ec6000000b005139bf86110si3194840pgb.673.2023.04.20.20.08.03; Thu, 20 Apr 2023 20:08:15 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=EygXMbRM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234005AbjDUCwL (ORCPT + 99 others); Thu, 20 Apr 2023 22:52:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44092 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233822AbjDUCvl (ORCPT ); Thu, 20 Apr 2023 22:51:41 -0400 Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 11013769B; Thu, 20 Apr 2023 19:51:23 -0700 (PDT) Received: by mail-pj1-x1031.google.com with SMTP id 98e67ed59e1d1-24b3451b2fcso1230404a91.3; Thu, 20 Apr 2023 19:51:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045482; x=1684637482; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=akTWRPzzBJ0wdmeG4+QDU8Yy42VUO1hiVrnvCUyERAQ=; b=EygXMbRMlcMPeJ20d4+Tkyht4zHzW+SkyJpVIWTlQyx71Cwsz2OY+unwD9T5JAydrD T6KMsfIN4B8EPW8iytzfJnaBRbHq4HHGXitYTxBANO7ekg0TmwXgnlsu7gMbycoc4zLj kIVpqIAP6C2mhgzLGKVBnxOU+dyM82smGgFhXOH3vERe2GNc+hKACo1HzQcmZXk2V6iz GD9l4zn8XR0lumYvsiZUcOcxLJlysZhVaz+EVzwts3iObxQ4HwvyNbwGIxux3gEC/5zC pcWDbYX9KYXykYpEY7iNY4+8Ls9dGaw54xrvCOK7BTs8I5C1h+lZe/sAeiD9tCLH2Dj/ bQ9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045482; x=1684637482; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=akTWRPzzBJ0wdmeG4+QDU8Yy42VUO1hiVrnvCUyERAQ=; b=ZABb3fD18osOCzPHcClRebDXbrOEJi3z1hL9piKu2+uYwRFCq+IJOskmxN24I0KKos 9cBn4z03UvxX3lcBfCeKKeylJFppdevpWLFVgPXuk2Hcvb7WgbyVZIQaVCSX6yersq7x M4d+NNwoGf48FDlMz0FJYK1c+45xwqUFj3HsXErcvlkFP93JH9/ap+/pS3JbZsAJzGtB qurMSZt+MFqJbynZl+KWxnbB4XOEJeZ4mf/ig+LrjVrdqZlN0cAJR49zxbB8r+o40zWa RTIFyUHxOsFsD7gCcj5XDNPx2nK/TMRIzADgu3QJZ7ctRXLvVlZn3o4rV92tWldBf+ZD q+0w== X-Gm-Message-State: AAQBX9eCCny4AcEByHHYIT79BvDONiiSUsclJwLgGraiybIOiCDqbbT+ iH2VRvgxlMYAoo67d/tNPOw= X-Received: by 2002:a17:90a:880b:b0:240:3dc7:4939 with SMTP id s11-20020a17090a880b00b002403dc74939mr3682529pjn.27.1682045482177; Thu, 20 Apr 2023 19:51:22 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id f9-20020a17090a638900b00246fbf416dasm1787580pjj.14.2023.04.20.19.51.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:51:21 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , Steve French , Paulo Alcantara , Ronnie Sahlberg , Shyam Prasad N , Tom Talpey , linux-cifs@vger.kernel.org, samba-technical@lists.samba.org Subject: [PATCH 17/22] cifs: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:41 -1000 Message-Id: <20230421025046.4008499-18-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763753586537369587?= X-GMAIL-MSGID: =?utf-8?q?1763753586537369587?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: Steve French Cc: Paulo Alcantara Cc: Ronnie Sahlberg Cc: Shyam Prasad N Cc: Tom Talpey Cc: linux-cifs@vger.kernel.org Cc: samba-technical@lists.samba.org Acked-by: Paulo Alcantara (SUSE) --- fs/cifs/dfs_cache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/cifs/dfs_cache.c b/fs/cifs/dfs_cache.c index 30cbdf8514a5..aece6d51bce7 100644 --- a/fs/cifs/dfs_cache.c +++ b/fs/cifs/dfs_cache.c @@ -290,7 +290,7 @@ int dfs_cache_init(void) int rc; int i; - dfscache_wq = alloc_workqueue("cifs-dfscache", WQ_FREEZABLE | WQ_UNBOUND, 1); + dfscache_wq = alloc_ordered_workqueue("cifs-dfscache", WQ_FREEZABLE); if (!dfscache_wq) return -ENOMEM; From patchwork Fri Apr 21 02:50:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86112 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp779032vqo; Thu, 20 Apr 2023 20:04:09 -0700 (PDT) X-Google-Smtp-Source: AKy350aG5/eJYcCdqxPdN17OFfjDEokDOIicAI//cy1xMy9JN62j+ycpIEWymudnGLCnOVmUsO4Y X-Received: by 2002:a17:902:d50f:b0:1a1:f70c:c81a with SMTP id b15-20020a170902d50f00b001a1f70cc81amr4976004plg.9.1682046249077; Thu, 20 Apr 2023 20:04:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682046249; cv=none; d=google.com; s=arc-20160816; b=S2IoHewcFmsufjV+Nfl+OZ1pQ8nUFBMifDXFx+0NhmV0cotbHhDvbGfHNymRNTSlvX LWv+z3xpCnjJGZoqqq1pgK4BJ1je89guIaQHFj9AqgroctPnhIESe2uhpv0t5zupc8BZ NUXSyUei3tEB+PkC4yFEKh8uDeBe0onD/LawLEV9s5ku+JfYp7mGpYWO1hxXxFkuX7uF H1og7qFFpROUs/S7/tZQdWHx7yR9ab4Q+8Re6aMCA/QRXvxfkf0OG4hKj0x0wszPkB/A e1KE+Lel/PDqepQvkjR0LoV0dTwm2K9aCFSNjO9L8RFqVTVXyr27E4B1lgJ5hxtYkujw U4Xg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=mzGS+UdzXek+xJHu4+K3eBKPYOUbqdAU3uJzg+J1eL4=; b=zjHMktdDvS7wS4vW9yBnlBG2FUHVuK0DbKdOP1ErO23OzC2f3DPwzZAxT2l3oBOmGe qt3bGOUl4K2AG9TeTQrc/Kucnx+5mOqTlA7alHjI6MleLteCAVvCcZ4AOBFzPyvcLEwb JyscJFUX1+HBWjjMlYSRaAcI59P2r9pssO+ndslOXB29BQgdUK6d+vXxCbHzMmGJTYR3 nKYDxFG7FaPDz9pMLPgIKB0g5GBsVqn9vRYiq2ae+ooFlUbwAGYFMZk22Dh+tkIyjtg8 LDlYP1wkEKwDWRhAPA4WZcL9rTW5ayNgEKqvUXgNCYGZ11D45+0NPe7mLVvAFNlvoXMZ vZ9Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=poUS2eXv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id o5-20020a17090ac70500b0024b5a028e7asi1779794pjt.47.2023.04.20.20.03.56; Thu, 20 Apr 2023 20:04:09 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=poUS2eXv; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234030AbjDUCwg (ORCPT + 99 others); Thu, 20 Apr 2023 22:52:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44200 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233869AbjDUCvq (ORCPT ); Thu, 20 Apr 2023 22:51:46 -0400 Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C27507A9D; Thu, 20 Apr 2023 19:51:24 -0700 (PDT) Received: by mail-pl1-x636.google.com with SMTP id d9443c01a7336-1a66911f5faso15952255ad.0; Thu, 20 Apr 2023 19:51:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045484; x=1684637484; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=mzGS+UdzXek+xJHu4+K3eBKPYOUbqdAU3uJzg+J1eL4=; b=poUS2eXvbKbyutb8Y/OZuu5rSs7SDpvdzJfbPtqCXkQr6zq/VoRNZSPZCmxBfUDsQP GkvMCSIZ3YzfPb7O9lTcuV1wf4zJPgioZ8kgl9goI5X6NIqHC37zHNuiWwN/Rm21qFIg +4Uu92wJQZjpsHFfV9M/FHgmiFGfRNM+dyDTQEGyshzxNKqs8CN5u3KNWKrLA20MKEMX oA39jTCD0hjwjpz6UvCFLWTy4p9590VZUhoKq4upOa+p17MccNMEUDRHX/+ZhgHYVbtX 18NdWSkK/S1DCFXwiAX6pxTUtLyTocrWQxuJkP22PyZW3ekvE+Xp0YUGFWTnnGrqfQ9O wt2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045484; x=1684637484; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=mzGS+UdzXek+xJHu4+K3eBKPYOUbqdAU3uJzg+J1eL4=; b=LxcZmpPaDpjNtJ2Dwpe1Pyr7mLLne7VOmBf9+Wg5gn5592a7y9l6r3LFnktz8RTeyH slqyhH3818sHNcwZV7/McT7/YNBjQcI8aNFqvYz4fa9qgZyuhBRn4ckYZ8t+ksrcYFI0 4KsofOCYCzqB0bg1AchlOoRUkLKlmgl1OTVHE4Bi1NxshziFdkIe8/mOCqXW/9vaaRTC skEFbxKI/0/GakPIocRLVPJWyoZe/NADU+badq962HwQ483LDKOAeB+oulv1RzN8XV3B rsHtV+RbqFqH2EZY6ksjbRbu2JSYqfKExNLUjlC1G8d9ZtF9QS9FZzTGqGdmhN7+cI5S YFyQ== X-Gm-Message-State: AAQBX9fKokeN39xEghey1MsrsbOqC1jrt8Jmt5SoHztMeszXrAUC737K YcFor45IalMPnbkvWqfL53eTmFL03FA= X-Received: by 2002:a17:902:f683:b0:1a8:17db:e252 with SMTP id l3-20020a170902f68300b001a817dbe252mr3875983plg.34.1682045483982; Thu, 20 Apr 2023 19:51:23 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id a13-20020a1709027d8d00b001a647709860sm1736293plm.157.2023.04.20.19.51.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:51:23 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , Manivannan Sadhasivam , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , linux-arm-msm@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH 18/22] net: qrtr: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:42 -1000 Message-Id: <20230421025046.4008499-19-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763753327868115435?= X-GMAIL-MSGID: =?utf-8?q?1763753327868115435?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: Manivannan Sadhasivam Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: linux-arm-msm@vger.kernel.org Cc: netdev@vger.kernel.org --- net/qrtr/ns.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/qrtr/ns.c b/net/qrtr/ns.c index 0f25a386138c..0f7a729f1a1f 100644 --- a/net/qrtr/ns.c +++ b/net/qrtr/ns.c @@ -783,7 +783,7 @@ int qrtr_ns_init(void) goto err_sock; } - qrtr_ns.workqueue = alloc_workqueue("qrtr_ns_handler", WQ_UNBOUND, 1); + qrtr_ns.workqueue = alloc_ordered_workqueue("qrtr_ns_handler", 0); if (!qrtr_ns.workqueue) { ret = -ENOMEM; goto err_sock; From patchwork Fri Apr 21 02:50:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86122 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp781807vqo; Thu, 20 Apr 2023 20:10:57 -0700 (PDT) X-Google-Smtp-Source: AKy350ZggQmmpshL/tvjcYzha3SKo6wUVpKIE5u1FM1mlm+uX1qWy3Kxneyz1tMQyvDf0YTKDJBG X-Received: by 2002:a17:902:eb89:b0:1a2:3108:5cc9 with SMTP id q9-20020a170902eb8900b001a231085cc9mr4068531plg.40.1682046657014; Thu, 20 Apr 2023 20:10:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682046656; cv=none; d=google.com; s=arc-20160816; b=Mul8RHQOdTYg4XuQx/7cMAm79FNVqBbcfEhTWWbFHX+LrYQWIFf5FpDMbJLQoQ8eSP v9mK2U0xf+ywA5W8fjU8/FCTg6zVJBT4Cgyf0E64qOvtwDnx+tcYSxo9pgUdTXc7UCCc g6BMi31Zw7S2kpZNW0O6lnzVtlt8phNjZqj15qw3JklodfFyZwdYVqFP90DAAk4nzdYz nI+GQYysjMtFwhi/8zYo8c32bIppJhGQxlGB475CaSIiWZ3QAh60V9W/16jWyCY3Dru1 BSWETDecEFPaKJ5VWk4CHdMNWYs11frjLJ94giudUNmPUDbWXKE+sM+fq0JhuVi6+2bF mAHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=iCIuGqiDIfKTQJqJOMYJZ+fdDTSyv1xrRS/ONUv2yRA=; b=ChVjtqzpV9LWig9xCDsFslRFgosYa1cOUAz/9fJj7MhObbUPsGFY6ieHB3YNsrFXMr wATB0NStp2TmrgS0sdNO7VFmXgi6i8cqtyRqXiOMIY7msQPXar36VEXt88XS+kCEde66 AZEzJA/BvuupNLPL9V+inozVPr5NXbnxdEyg07SdmA6M/ntGMF6MoIYvNX3JJdRaOWCW MIrdDqJjEr51AyjH7eGlRSHr/POECOIryQUZCTtdMdxZfhcXpVDJw/stowJOeuO61eqd +NX+esQI50hX0juh9ucR+f5phAnxB8o5vAomA1cm6hUC6rDs45ag30OAPBnknuy27Lz8 3V+w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=gezxl302; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id nk15-20020a17090b194f00b00247023fdbcbsi5903978pjb.186.2023.04.20.20.10.44; Thu, 20 Apr 2023 20:10:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=gezxl302; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233936AbjDUCwj (ORCPT + 99 others); Thu, 20 Apr 2023 22:52:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44328 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233893AbjDUCvv (ORCPT ); Thu, 20 Apr 2023 22:51:51 -0400 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7480172B8; Thu, 20 Apr 2023 19:51:26 -0700 (PDT) Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-63b4bf2d74aso1496422b3a.2; Thu, 20 Apr 2023 19:51:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045486; x=1684637486; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=iCIuGqiDIfKTQJqJOMYJZ+fdDTSyv1xrRS/ONUv2yRA=; b=gezxl302dxv42n+zCHus+E5E/8VfIO1QqtasOji91jFxm4HbDDyC/T8wrTYvihAkIV tAIybYBeQQ94t0LI8KHGbVzEDjpQQdu55bK1twCEYMc2yKVoEzj/S5YJlGc+cOK25kMR 6WK7KlJ01Bhj8pv5zXg7vWs4gSLSaLVI1fgW3ZUPXlt9cpWfGRUvASCXkeiLL93vzvrl sEIvA1dtXYyriy0pzL0jk4esCON3xEefzhYJ1jRj4SAyw+6/GegiEDA20CNyY9yVD7oj RLJx8FdAVg9nv1XzlKVR7HQ77y9YPDxI/MnTdqsHS2CdSSbFBaRh46Z8Kf8/JRiWZPTJ JxnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045486; x=1684637486; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=iCIuGqiDIfKTQJqJOMYJZ+fdDTSyv1xrRS/ONUv2yRA=; b=k0BRzxzp/4BTcZnDIz2qHY70NWIy6HiPIjJd3N0fknhHjkszk5ZM4WefdD6PA7nFkC LbWVU9a6FPMnwhctzlpoxgh6fICp2RC95ZckSoI7Oef3rX5eH0TnW58sZl9DxUgLIK28 ekvGUSwcw1iCZC23W0Ls0yBHDagV84gc1gEABl5fq8nfuJAl4cUgQdTpboI7elYjQayv MhLBTR79mdnFXY6Btw32T+9pkKDiHj7sy7bY05tsgOYPqhFmg6CDFTe6YWXIYko//9Fd gpslH7EqKy2nQpjn8TOvj6X+vGobRqzDxBy+0rwSjHaZjQ5rxG8theJPOV0C+4O7YYKS m8Gw== X-Gm-Message-State: AAQBX9cXOBRquwUiVVoAqZHs3DhlIVcT2mDT9aE6dK5RhXjskID7MqU/ ZXNwwRmFJSH3958VZdDOVdA= X-Received: by 2002:a05:6a21:9011:b0:f0:69db:ebea with SMTP id tq17-20020a056a21901100b000f069dbebeamr4516265pzb.30.1682045485928; Thu, 20 Apr 2023 19:51:25 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id x125-20020a636383000000b00513ec871c01sm1701365pgb.16.2023.04.20.19.51.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:51:25 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , David Howells , Marc Dionne , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , linux-afs@lists.infradead.org, netdev@vger.kernel.org Subject: [PATCH 19/22] rxrpc: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:43 -1000 Message-Id: <20230421025046.4008499-20-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763753755086194734?= X-GMAIL-MSGID: =?utf-8?q?1763753755086194734?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: David Howells Cc: Marc Dionne Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: linux-afs@lists.infradead.org Cc: netdev@vger.kernel.org --- net/rxrpc/af_rxrpc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/rxrpc/af_rxrpc.c b/net/rxrpc/af_rxrpc.c index 102f5cbff91a..e1822c12990d 100644 --- a/net/rxrpc/af_rxrpc.c +++ b/net/rxrpc/af_rxrpc.c @@ -972,7 +972,7 @@ static int __init af_rxrpc_init(void) goto error_call_jar; } - rxrpc_workqueue = alloc_workqueue("krxrpcd", WQ_HIGHPRI | WQ_MEM_RECLAIM | WQ_UNBOUND, 1); + rxrpc_workqueue = alloc_ordered_workqueue("krxrpcd", WQ_HIGHPRI | WQ_MEM_RECLAIM); if (!rxrpc_workqueue) { pr_notice("Failed to allocate work queue\n"); goto error_work_queue; From patchwork Fri Apr 21 02:50:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86129 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp792360vqo; Thu, 20 Apr 2023 20:41:25 -0700 (PDT) X-Google-Smtp-Source: AKy350aig7ybvJ7tr8QYFGIPdBk6DapRvsdgpo95HnBjLXG0lRSWOlfB2E2k6JN/dtPffa5M6pTM X-Received: by 2002:a05:6a00:88e:b0:63b:7fa1:f1b1 with SMTP id q14-20020a056a00088e00b0063b7fa1f1b1mr4215394pfj.26.1682048484848; Thu, 20 Apr 2023 20:41:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682048484; cv=none; d=google.com; s=arc-20160816; b=Yu5mW+EI5P0lpX4fzyCnr+cS5W9P6sl6dlRuLRcCOXRmSG8xYPvAAYUP76noeVAYwz /PAOhPemX4arDsRnyVHqfvlchzl6cqibAriG+zfjlaK6Lsh3DHhuFv2PKgOZcg4uLZCE UZYmNKr6e3I6tQfxX/8SsMGfZIMozOjPyyka3jx9O/ulm2d4Fxakt0iD02s9wh7MTkPA G4kWh8eUqLouGjov7VUfb4g88zoUpe4rF8fqpo8P5OuJ2BVRWG5toeOsgDd62YfQpqMf nOpccLatn88sQEtwqLksQ1/DAkbjVBjtsbTj2/KNYSiZTgFqcfbZk4nGeZ5RTxMqLtVc AzFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=sUw0DkZoq2nQ/bLNbKEG/cpbsutsaVMiCTIAKGrHlvQ=; b=lInP6ZiI7uIJI6/HKJUWZC4WpSuKT8qNvaxTUsbHBEEPLmjKY8Gt+pJCV6qpzEXDc7 YceD0V5xgmbKu8ekXslEO7okDu47ZXTXLR2OkZZ6UQevcdRC5PSwI0sYevt36QxBlazI USCn3tTNWuQWjQztABwVyfkCTnaDk2E3ewSIaKmFzDe2fg9ToFaMflqzWAe2g4fou8Kn X0u2SsMr4ymuHUz+ilO0iNeAkaamyuVO0UgFaXWgoV8XkSueg0jtxThB+xely9NPZq39 ln54wQNYLwwack/9QqrzcKmPwpBbONTx53Pm1T9rWXBXJMVIUD7ZN6o6G42VuGItONC7 gK7g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=fVspGee2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id e30-20020a056a0000de00b0063b21029fc5si333487pfj.12.2023.04.20.20.41.09; Thu, 20 Apr 2023 20:41:24 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=fVspGee2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234071AbjDUCwq (ORCPT + 99 others); Thu, 20 Apr 2023 22:52:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44392 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233918AbjDUCvw (ORCPT ); Thu, 20 Apr 2023 22:51:52 -0400 Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6B8437AB7; Thu, 20 Apr 2023 19:51:28 -0700 (PDT) Received: by mail-pl1-x636.google.com with SMTP id d9443c01a7336-1a6862e47b1so19837785ad.0; Thu, 20 Apr 2023 19:51:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045488; x=1684637488; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=sUw0DkZoq2nQ/bLNbKEG/cpbsutsaVMiCTIAKGrHlvQ=; b=fVspGee2WguejmyW9SsJEePERYXvLnqkuEJr6C3DOdX9+QMwzZgHjx45Tnd08CWhEv 96/NJgI1wU4AZjV16sLDqCDuoewN0EZ4u/eznQMLu8vqcywZYQBrtiwri5P+JxzYSAmm Ulbq145l6+P+dzYJ8T9UBS7ZKcLuiUYQIJ+bjmgOqjwZL3KXz0l1NdjfuyZsVrp3/mUc uz2pe37gx+xMom5o4Ffo6xjNyVFQkODv98yxJS7VH80xPFWWR8x1U9gPKhzcMJLIhU4Z 4VRcQKegQexAg/0aqGIXZEaalQPyXpSPRyfwa+wSOVV9x5KHQpZnqrOs+ZmR7CcN/Fxd LAGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045488; x=1684637488; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=sUw0DkZoq2nQ/bLNbKEG/cpbsutsaVMiCTIAKGrHlvQ=; b=j726AK/0utIZPY3jyFCfeLH+hU9in4EgeS4n+sbBPAGtZhL5g4fAQrl4ND3iLusI3Z JlBkLd0EonfY5dnBtFJr5cmzhD1d/sLm33COtzy7tIdw/UeaTzxiZT1GlmquyRUtEcp6 r39DbU4dFhb1RAoavxgnqkkdY03de1ncZg68QA4PC/u7nYp0Uibg34g3ROOgNU2eNkGQ Ga0NIriP69osRC9Hg9J4/R8zYaIcr6cbD73txez2iqj+SNW/wL0djb+7r/S8SKpl85+b z5BFjQphe75XsZSEhyUm/6hRhA2AgADLr/h9TdF7DgWyw55WVqaB80IWPpMjWQP5PMhy AvjQ== X-Gm-Message-State: AAQBX9eOyDRQ2TgWOSTFWF3eJsEL5zhj/oK8lSZnsrkfZYWtrnO8TAHP vcFa4J1w+1BLSFnj69ZvPG0= X-Received: by 2002:a17:902:e751:b0:1a9:433e:41d5 with SMTP id p17-20020a170902e75100b001a9433e41d5mr1550826plf.56.1682045487627; Thu, 20 Apr 2023 19:51:27 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id hi17-20020a17090b30d100b002471deb13fcsm3676695pjb.6.2023.04.20.19.51.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:51:27 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , Boris Brezillon , Arnaud Ebalard , Srujana Challa , Herbert Xu , "David S. Miller" , Shijith Thotton , Vladis Dronov , Vincent Mailhol , Wolfram Sang , Alexander Lobakin , Minghao Chi , ye xingchen , linux-crypto@vger.kernel.org Subject: [PATCH 20/22] crypto: octeontx2: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:44 -1000 Message-Id: <20230421025046.4008499-21-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763755671662516366?= X-GMAIL-MSGID: =?utf-8?q?1763755671662516366?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: Boris Brezillon Cc: Arnaud Ebalard Cc: Srujana Challa Cc: Herbert Xu Cc: "David S. Miller" Cc: Shijith Thotton Cc: Vladis Dronov Cc: Vincent Mailhol Cc: Wolfram Sang Cc: Alexander Lobakin Cc: Minghao Chi Cc: ye xingchen Cc: linux-crypto@vger.kernel.org --- drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c | 12 ++++++------ drivers/crypto/marvell/octeontx2/otx2_cptvf_main.c | 6 +++--- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c b/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c index ddf6e913c1c4..30e6acfc93d9 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c @@ -357,9 +357,9 @@ static int cptpf_vfpf_mbox_init(struct otx2_cptpf_dev *cptpf, int num_vfs) u64 vfpf_mbox_base; int err, i; - cptpf->vfpf_mbox_wq = alloc_workqueue("cpt_vfpf_mailbox", - WQ_UNBOUND | WQ_HIGHPRI | - WQ_MEM_RECLAIM, 1); + cptpf->vfpf_mbox_wq = + alloc_ordered_workqueue("cpt_vfpf_mailbox", + WQ_HIGHPRI | WQ_MEM_RECLAIM); if (!cptpf->vfpf_mbox_wq) return -ENOMEM; @@ -453,9 +453,9 @@ static int cptpf_afpf_mbox_init(struct otx2_cptpf_dev *cptpf) resource_size_t offset; int err; - cptpf->afpf_mbox_wq = alloc_workqueue("cpt_afpf_mailbox", - WQ_UNBOUND | WQ_HIGHPRI | - WQ_MEM_RECLAIM, 1); + cptpf->afpf_mbox_wq = + alloc_ordered_workqueue("cpt_afpf_mailbox", + WQ_HIGHPRI | WQ_MEM_RECLAIM); if (!cptpf->afpf_mbox_wq) return -ENOMEM; diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptvf_main.c b/drivers/crypto/marvell/octeontx2/otx2_cptvf_main.c index 392e9fee05e8..6023a7adb70c 100644 --- a/drivers/crypto/marvell/octeontx2/otx2_cptvf_main.c +++ b/drivers/crypto/marvell/octeontx2/otx2_cptvf_main.c @@ -75,9 +75,9 @@ static int cptvf_pfvf_mbox_init(struct otx2_cptvf_dev *cptvf) resource_size_t offset, size; int ret; - cptvf->pfvf_mbox_wq = alloc_workqueue("cpt_pfvf_mailbox", - WQ_UNBOUND | WQ_HIGHPRI | - WQ_MEM_RECLAIM, 1); + cptvf->pfvf_mbox_wq = + alloc_ordered_workqueue("cpt_pfvf_mailbox", + WQ_HIGHPRI | WQ_MEM_RECLAIM); if (!cptvf->pfvf_mbox_wq) return -ENOMEM; From patchwork Fri Apr 21 02:50:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86126 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp788173vqo; Thu, 20 Apr 2023 20:29:55 -0700 (PDT) X-Google-Smtp-Source: AKy350ZAM1lK99lZ1QgBKVceB0/WWxGqUxYCtUNrxJljW+rXZo6PmXZtwJ9DzhX6hxH2P5s71jDX X-Received: by 2002:a05:6a20:6a1c:b0:ef:b02a:b35b with SMTP id p28-20020a056a206a1c00b000efb02ab35bmr6444092pzk.0.1682047795100; Thu, 20 Apr 2023 20:29:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682047795; cv=none; d=google.com; s=arc-20160816; b=0WcuLQ0zuT5U+WrC/SkF8HvcGY/f93rDCwAwHbQfd6QPV1sJUHE69c51ZfK572HWOJ cr3CHqA2yB8/Y02jwFOoEw3nc7aR3748USqZqDfRXLMf5rRIfosR8zrcJXERnB3C4TdH Wjq846LJ4TdrCSggqPgpX7zkEXYq0y2E0lgCqLVRjNLBhEBEy4mleXdNCC3JA4aJsOI5 KZjN8MgH77tQ4Xr6W57+H3CxUd5DG8bFtIHJMvkveMkAwkFoyhCVYF3h8fjOWQW5LK4R /3QbLMymqGlEP4/HjmjV8F7XwXQASDpycU1mJjP/mEzodHESIxVXFdf/GrkFoT9tzX7Z oU+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=8AHaFd45rL0q5b4AB3LAX24oOzN6BIEVC1RaqbCPXVU=; b=Oc68xrxw1wlSKQUVVuy1P58j67gbjjii06gW8m6hZgTdnFeWJE6dW6YJ4bTEcNjqVr NRKphTchOtZa/KWAcd1+KN6+tA7yoxNCXcpt1S66P+/PwsB3Xx3YPjZxpwV6IEdu7EkZ MJJD6zavujg6rBVqtoZnhg09jJf2kHhVYxL76+Mo9CT9bspOnBS7HYfgo6+1BQGRGpw7 9W8KOeaTJzmFxQU9tPsEqzTvriAfDdzd72kmAh2KNAJS+jGNiTWcWdzfi0sDLRpt2PE1 6oxG0CjW3NejTV0wv/OnQwhgFy68jR3HNTQbh9zqv4JyHe8QwtQm8qpB+BA6+j8/4MWU WkqQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=kilWJuIJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s12-20020a63450c000000b00524b3efc31asi3425055pga.737.2023.04.20.20.29.39; Thu, 20 Apr 2023 20:29:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=kilWJuIJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233909AbjDUCwn (ORCPT + 99 others); Thu, 20 Apr 2023 22:52:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43566 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233750AbjDUCvx (ORCPT ); Thu, 20 Apr 2023 22:51:53 -0400 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0DF417D84; Thu, 20 Apr 2023 19:51:30 -0700 (PDT) Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-63b60366047so1513112b3a.1; Thu, 20 Apr 2023 19:51:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045489; x=1684637489; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=8AHaFd45rL0q5b4AB3LAX24oOzN6BIEVC1RaqbCPXVU=; b=kilWJuIJweTRgBTwgFCwGu9lFsKgCOqZCxPqrZnLjD1QVK+XFkDNp2XuD+d81qgDhP GbCxA77d9pzcU4CDGX2f32KQ2wxlODcfSZQqzWPbjPQG/XVb4BMk+OoPVaeu8yBqD/nZ tKxdqKZuZ2+JLyTixbOJLEL/V8Oj72FHGk5nMRvlLUjbDuVxhI2GoJtMhWmtLjxTfRc2 7iVSR6lx3r2gH32/oKvSkD9XTXegBICwHtrtzLq3MXu+QT52PIgVSCJZbgy1V/tPKUl6 bt9GvindBpEKhbuYIGtwTF74485YSuFrlxtVVXfCIX+lc6JqiCeks/82b8HKH9024NzJ qY1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045489; x=1684637489; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=8AHaFd45rL0q5b4AB3LAX24oOzN6BIEVC1RaqbCPXVU=; b=Z/FoNZ7umK6QzhQo7OrgReDm1S1hOT4gys1wB3jdgO6Wsrv2xTWHsutWThUdnEbwST 4Au30H0tr5WZqi6jBoRdGibax5mcOLFabrOlcLsGwm/ys6w0UcRYOMEYWY851eLAPUOe 93oEhGvX+13xuQkZiQ4ABCC8OLzB44Ss7ZMY7O8rFkmDoVw3UAw1ykvX4CMM9WSG12b9 h0j2xVaCx4ZLb2Qrzh49wciGY69KMOLxDnfMtkpOytn81EMC8piPLvGEeXvwce/ZxNEM 1uGmii5haV84EHpuAVVYgDvmY08vP51Powc6Uc1QsuaXvTBcMWGWETa6i1idyfk2xXA4 /Jjg== X-Gm-Message-State: AAQBX9cCacuF0ByUtkyTUR4eLncD1V96yY6DnWIqGrnyA2jHqRq4O8Gn X2lTCuT1w6H4KB7AMAHXLUUZm6Tmohg= X-Received: by 2002:a05:6a00:1582:b0:63d:3ddb:5f3f with SMTP id u2-20020a056a00158200b0063d3ddb5f3fmr4982153pfk.0.1682045489387; Thu, 20 Apr 2023 19:51:29 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id z21-20020a62d115000000b0063d24fcc2b7sm1908694pfg.1.2023.04.20.19.51.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:51:29 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , Philipp Zabel , Mauro Carvalho Chehab , linux-media@vger.kernel.org Subject: [PATCH 21/22] media: coda: Use alloc_ordered_workqueue() to create ordered workqueues Date: Thu, 20 Apr 2023 16:50:45 -1000 Message-Id: <20230421025046.4008499-22-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763754948385298220?= X-GMAIL-MSGID: =?utf-8?q?1763754948385298220?= BACKGROUND ========== When multiple work items are queued to a workqueue, their execution order doesn't match the queueing order. They may get executed in any order and simultaneously. When fully serialized execution - one by one in the queueing order - is needed, an ordered workqueue should be used which can be created with alloc_ordered_workqueue(). However, alloc_ordered_workqueue() was a later addition. Before it, an ordered workqueue could be obtained by creating an UNBOUND workqueue with @max_active==1. This originally was an implementation side-effect which was broken by 4c16bd327c74 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered"). Because there were users that depended on the ordered execution, 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") made workqueue allocation path to implicitly promote UNBOUND workqueues w/ @max_active==1 to ordered workqueues. While this has worked okay, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue updates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. This patch series audits all callsites that create an UNBOUND workqueue w/ @max_active==1 and converts them to alloc_ordered_workqueue() as necessary. WHAT TO LOOK FOR ================ The conversions are from alloc_workqueue(WQ_UNBOUND | flags, 1, args..) to alloc_ordered_workqueue(flags, args...) which don't cause any functional changes. If you know that fully ordered execution is not ncessary, please let me know. I'll drop the conversion and instead add a comment noting the fact to reduce confusion while conversion is in progress. If you aren't fully sure, it's completely fine to let the conversion through. The behavior will stay exactly the same and we can always reconsider later. As there are follow-up workqueue core changes, I'd really appreciate if the patch can be routed through the workqueue tree w/ your acks. Thanks. Signed-off-by: Tejun Heo Cc: Philipp Zabel Cc: Mauro Carvalho Chehab Cc: linux-media@vger.kernel.org --- drivers/media/platform/chips-media/coda-common.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/media/platform/chips-media/coda-common.c b/drivers/media/platform/chips-media/coda-common.c index af71eea04dbd..c8ecfe760028 100644 --- a/drivers/media/platform/chips-media/coda-common.c +++ b/drivers/media/platform/chips-media/coda-common.c @@ -3268,7 +3268,7 @@ static int coda_probe(struct platform_device *pdev) &dev->iram.blob); } - dev->workqueue = alloc_workqueue("coda", WQ_UNBOUND | WQ_MEM_RECLAIM, 1); + dev->workqueue = alloc_ordered_workqueue("coda", WQ_MEM_RECLAIM); if (!dev->workqueue) { dev_err(&pdev->dev, "unable to alloc workqueue\n"); ret = -ENOMEM; From patchwork Fri Apr 21 02:50:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tejun Heo X-Patchwork-Id: 86113 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp779110vqo; Thu, 20 Apr 2023 20:04:21 -0700 (PDT) X-Google-Smtp-Source: AKy350aTmYsW2yP/pcWjZFNU40U0myBwBwZsJxgFFGQrWTm4mRykKqjD5N2tj25BB0kTfOgUNqon X-Received: by 2002:a05:6a20:1609:b0:da:d4eb:9e07 with SMTP id l9-20020a056a20160900b000dad4eb9e07mr4570551pzj.30.1682046261074; Thu, 20 Apr 2023 20:04:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682046261; cv=none; d=google.com; s=arc-20160816; b=fyezoUXTrXqsD54zrinsXdK7qkyuSHNY+G98wplLYOl19utPpEA+Gm4BJ8KaR60lP7 Q/E0+fUrgvnJHD5jNFki7HEKYygE64gZ2gbH5AvNg+cZ0zyfnRGTiVPNufWwneLOlOxx Px7i35o+64NksWFLrDo7BsSd2bAOCBLVi3LJlUlMgzKhsnkf8UvrtpYK9ykRSgWcGHSC ySR+e22X6RNzuZqo6FHWNrKc0Dzv/WvZkaM0SGLR58NlNLACQaAEIp5fcw7gBfpF3S2M yjo391tI0hv4n6Zhzs5vW+JsVquzsIlpSYEQV74vua7NX8548B0giZF+8dzY25VdoNhz tW1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from:sender :dkim-signature; bh=W1WHvs8FWd3S03N28+vd/xDG2iC9YdMvjBgSLrt0Ra8=; b=U3GA8/6z9ooXntnnm1JjGOgZ+zCikJr8jl7otiACF3GlRGeqN1vEzBdGUZQLHxbzhW MJTHrAQk3RI+auIwbeP4AOGC7sjJwHc7v308xH4Qy/AwpALepHuIe7PxCjNQFxr2AwI1 OZ2xYNY01v7bacZ63/z7AzqwmogJTyDgwiqw+6xl5IY4GpyvEw6s6Rtg7EIBugsm6jvv cDEbssW9Ms+Ady+bLOvguMCFnv1YqtKwupgDvOWoC6JDtw6B94x6yDj+2AGiIT+PhD1G siKIGVWPFNZdKzULHiRbtxN/cG4jHPbLMbvjsTzDGrzZ/AlA+0709591d42RSAVifWUS 59iQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=aHIS0UOb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id c5-20020a6566c5000000b0051a6292309fsi3112888pgw.894.2023.04.20.20.04.08; Thu, 20 Apr 2023 20:04:21 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=aHIS0UOb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233801AbjDUCwt (ORCPT + 99 others); Thu, 20 Apr 2023 22:52:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44488 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233795AbjDUCvy (ORCPT ); Thu, 20 Apr 2023 22:51:54 -0400 Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E1C6D7ABC for ; Thu, 20 Apr 2023 19:51:31 -0700 (PDT) Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-63b73203e0aso11426066b3a.1 for ; Thu, 20 Apr 2023 19:51:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682045491; x=1684637491; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:from:to:cc:subject:date :message-id:reply-to; bh=W1WHvs8FWd3S03N28+vd/xDG2iC9YdMvjBgSLrt0Ra8=; b=aHIS0UObijyj/uf3kCDBFk1KumJsx6sc+4Wzeq8S20U8TZCO1UOHgxF3CUqNqhUiAp WV6LRLfcODt2FTkt9HCpcRbZ+J0f8Mzoc+/uwsZS2InzL/EKUULFjc49bFHYHNYsY/9G yYYeDqd/2nf02/HwWoT7xyYME0DKm2NlpyOTtpujpIz7qW8uiKFn1S5wCwGqrFhPppv+ 7NdvXkJwOZDwzTnj197PrkN0aSq+tpWNxMblkPKeUa+tHKxoT1bD4UcOBAyNFJxsBuH6 HB+o/aVjuknxO0PJ7Zdg5Ox0wQCdNDX7lJTIfsMpQ+R9ZnQYXtCe6Ur0+QWCl53Q/efe 6bkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682045491; x=1684637491; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=W1WHvs8FWd3S03N28+vd/xDG2iC9YdMvjBgSLrt0Ra8=; b=KzCkVxTgf5wkh2pnKOIX+Qc7Rq60guN0m4aWZYO4E30je5goObbuFYUt3X9ByiqoHP +bsV45rQ9nsGUYNuibn+eN3Wh8XkH0y3eh/o91fTO3YhGAK1UGTjjbss1fJgki59W5Pl qOB3ZySa8nKzo2jxyKkx0LtAIKLuLzhZrNUUy05pKEeZlylL4LnHp3PGm5uV+jiApIDF sVtwHy4zLXgBsA18EE+hPa/Ml7mx41yghiOhMl11Wf8ifVLFTMjRBBmJzDTaEz15+St3 1O8EWlXBT5IBUMq9bdzPimHUvF2nKbyE+C7hqAYixq1g7QO0CX/HlrFv92QQ2DbGggQV ya4w== X-Gm-Message-State: AAQBX9cEhOmryF8jCJGPEnn63Cd5sBxKyROM/EoZey0UmCANztOuV4Oj 5AKBqDHa5RRxUmHBuWkVY/4= X-Received: by 2002:a17:903:188:b0:1a9:2a9e:30a8 with SMTP id z8-20020a170903018800b001a92a9e30a8mr6228740plg.9.1682045491106; Thu, 20 Apr 2023 19:51:31 -0700 (PDT) Received: from localhost (2603-800c-1a02-1bae-a7fa-157f-969a-4cde.res6.spectrum.com. [2603:800c:1a02:1bae:a7fa:157f:969a:4cde]) by smtp.gmail.com with ESMTPSA id f21-20020a17090ace1500b002469a865810sm3630905pju.28.2023.04.20.19.51.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Apr 2023 19:51:30 -0700 (PDT) Sender: Tejun Heo From: Tejun Heo To: jiangshanlai@gmail.com Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, Tejun Heo , Christoph Hellwig Subject: [PATCH 22/22] workqueue: Don't implicitly make UNBOUND workqueues w/ @max_active==1 ordered Date: Thu, 20 Apr 2023 16:50:46 -1000 Message-Id: <20230421025046.4008499-23-tj@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230421025046.4008499-1-tj@kernel.org> References: <20230421025046.4008499-1-tj@kernel.org> MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1763753340354238850?= X-GMAIL-MSGID: =?utf-8?q?1763753340354238850?= 5c0338c68706 ("workqueue: restore WQ_UNBOUND/max_active==1 to be ordered") automoatically promoted UNBOUND workqueues w/ @max_active==1 to ordered workqueues because UNBOUND workqueues w/ @max_active==1 used to be the way to create ordered workqueues and the new NUMA support broke it. These problems can be subtle and the fact that they can only trigger on NUMA machines made them even more difficult to debug. However, overloading the UNBOUND allocation interface this way creates other issues. It's difficult to tell whether a given workqueue actually needs to be ordered and users that legitimately want a min concurrency level wq unexpectedly gets an ordered one instead. With planned UNBOUND workqueue udpates to improve execution locality and more prevalence of chiplet designs which can benefit from such improvements, this isn't a state we wanna be in forever. There aren't that many UNBOUND w/ @max_active==1 users in the tree and the preceding patches audited all and converted them to alloc_ordered_workqueue() as appropriate. This patch removes the implicit promotion of UNBOUND w/ @max_active==1 workqueues to ordered ones. Workqueue will also add a debug option to make all unordered UNBOUND workqueues to use per-cpu pool_workqueues so that these problems can be surfaced easier on most machines. Signed-off-by: Tejun Heo Cc: Christoph Hellwig --- include/linux/workqueue.h | 4 +--- kernel/workqueue.c | 23 ++++------------------- 2 files changed, 5 insertions(+), 22 deletions(-) diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h index ac551b8ee7d9..e547a90f0328 100644 --- a/include/linux/workqueue.h +++ b/include/linux/workqueue.h @@ -339,7 +339,6 @@ enum { __WQ_DRAINING = 1 << 16, /* internal: workqueue is draining */ __WQ_ORDERED = 1 << 17, /* internal: workqueue is ordered */ __WQ_LEGACY = 1 << 18, /* internal: create*_workqueue() */ - __WQ_ORDERED_EXPLICIT = 1 << 19, /* internal: alloc_ordered_workqueue() */ WQ_MAX_ACTIVE = 512, /* I like 512, better ideas? */ WQ_MAX_UNBOUND_PER_CPU = 4, /* 4 * #cpus for unbound wq */ @@ -417,8 +416,7 @@ alloc_workqueue(const char *fmt, unsigned int flags, int max_active, ...); * Pointer to the allocated workqueue on success, %NULL on failure. */ #define alloc_ordered_workqueue(fmt, flags, args...) \ - alloc_workqueue(fmt, WQ_UNBOUND | __WQ_ORDERED | \ - __WQ_ORDERED_EXPLICIT | (flags), 1, ##args) + alloc_workqueue(fmt, WQ_UNBOUND | __WQ_ORDERED | (flags), 1, ##args) #define create_workqueue(name) \ alloc_workqueue("%s", __WQ_LEGACY | WQ_MEM_RECLAIM, 1, (name)) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index b8b541caed48..00bdcc3c5b36 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -4180,12 +4180,8 @@ static int apply_workqueue_attrs_locked(struct workqueue_struct *wq, return -EINVAL; /* creating multiple pwqs breaks ordering guarantee */ - if (!list_empty(&wq->pwqs)) { - if (WARN_ON(wq->flags & __WQ_ORDERED_EXPLICIT)) - return -EINVAL; - - wq->flags &= ~__WQ_ORDERED; - } + if (WARN_ON(wq->flags & __WQ_ORDERED)) + return -EINVAL; ctx = apply_wqattrs_prepare(wq, attrs, wq_unbound_cpumask); if (!ctx) @@ -4408,16 +4404,6 @@ struct workqueue_struct *alloc_workqueue(const char *fmt, struct workqueue_struct *wq; struct pool_workqueue *pwq; - /* - * Unbound && max_active == 1 used to imply ordered, which is no - * longer the case on NUMA machines due to per-node pools. While - * alloc_ordered_workqueue() is the right way to create an ordered - * workqueue, keep the previous behavior to avoid subtle breakages - * on NUMA. - */ - if ((flags & WQ_UNBOUND) && max_active == 1) - flags |= __WQ_ORDERED; - /* see the comment above the definition of WQ_POWER_EFFICIENT */ if ((flags & WQ_POWER_EFFICIENT) && wq_power_efficient) flags |= WQ_UNBOUND; @@ -4625,14 +4611,13 @@ void workqueue_set_max_active(struct workqueue_struct *wq, int max_active) struct pool_workqueue *pwq; /* disallow meddling with max_active for ordered workqueues */ - if (WARN_ON(wq->flags & __WQ_ORDERED_EXPLICIT)) + if (WARN_ON(wq->flags & __WQ_ORDERED)) return; max_active = wq_clamp_max_active(max_active, wq->flags, wq->name); mutex_lock(&wq->mutex); - wq->flags &= ~__WQ_ORDERED; wq->saved_max_active = max_active; for_each_pwq(pwq, wq) @@ -5868,7 +5853,7 @@ int workqueue_sysfs_register(struct workqueue_struct *wq) * attributes breaks ordering guarantee. Disallow exposing ordered * workqueues. */ - if (WARN_ON(wq->flags & __WQ_ORDERED_EXPLICIT)) + if (WARN_ON(wq->flags & __WQ_ORDERED)) return -EINVAL; wq->wq_dev = wq_dev = kzalloc(sizeof(*wq_dev), GFP_KERNEL);