From patchwork Wed Nov 29 04:31:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 171057 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a5a7:0:b0:403:3b70:6f57 with SMTP id d7csp116399vqn; Tue, 28 Nov 2023 20:32:31 -0800 (PST) X-Google-Smtp-Source: AGHT+IHT6zunbM/zJ4WgcBiSwYv3wrFACXy+14H9Z+jJV48ptPD8pq1C/fQoWwZk8T6ipMJ2PSd/ X-Received: by 2002:a05:6359:2e17:b0:16e:29cf:4e57 with SMTP id ro23-20020a0563592e1700b0016e29cf4e57mr11069006rwb.29.1701232351074; Tue, 28 Nov 2023 20:32:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701232351; cv=none; d=google.com; s=arc-20160816; b=SOmZvtuC8hAaSfyT6+bPc5xhkRqTfzXt2YMProoRirlKj3xS7ySPLCJIIODSni5s31 BPJdyZwurCZ91C00HocuX7c47qf/JpoV2PDQAmitKOc+O+JdOpNWNabWsTzQePWgKJPw xePX+yXvSZyEYbHZtM88GUZv1/+EhV5YTyqLSNVQDO6XKwcSmWVqJKaVO0Y1pZ/DhMyX kPP2OQC4gj2WT6PZNRalxgrB1HacILBjKcdNP5B4dB9oqrrF8XPJrsp+2mFiW2KtxqV4 17WDS9CMiczI2QWqGPEnQ0wnOEnlfX1wW9Pizs9fmVESvMXFoDjxGM1KET0Srucyqi6F yokA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=q1Cx6P/mVqk+czKLhxSNfP2GrJ4iC4IHJZKEv2bo9N4=; fh=zVIHjnfjYXjU+hYbxPUw8z+QX7mV2ayyrBgf4ZvrzN8=; b=hKqdNRecHDOdxxzKPEfY0aJ/SKT9ot4TUjqKbe2mGnqulATvxu5K1rnOz6FMqL0Z7R W5+NmmlSaW48Hhashq/xXLnCed03G+MJxk3rJUWq3wumA22wsIKVEYCgXEdMmu6T95WG n0Z+FN7ooGoC49ydSb53jzlEBPnnzZ6zj3jFf13Yzir1U35uO3tmbSvKE5MU/osK+Gxd efSAtUplM5HDgHHdha8vp5oG6X21SH+hpqzeOURs9CV1bdtk+O3hxet4fkvFb7o+9yqO LQNB9K9eNZZr/3flKhWkc1dFOhttHP5eiEizz6SpLmE0LhhwktO7BpZx508sXGrrvy92 AuvA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id t25-20020a6564d9000000b005b915369815si13054022pgv.155.2023.11.28.20.32.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Nov 2023 20:32:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 08DE380B01F4; Tue, 28 Nov 2023 20:32:30 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231465AbjK2EcO (ORCPT + 99 others); Tue, 28 Nov 2023 23:32:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57094 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232982AbjK2EcI (ORCPT ); Tue, 28 Nov 2023 23:32:08 -0500 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9D9CD19AE; Tue, 28 Nov 2023 20:32:13 -0800 (PST) Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4Sg5yJ0rScz4f3kGH; Wed, 29 Nov 2023 12:32:08 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 0EBB51A0A80; Wed, 29 Nov 2023 12:32:10 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgDn6xHHvmZldxHwCA--.18598S5; Wed, 29 Nov 2023 12:32:09 +0800 (CST) From: Yu Kuai To: xni@redhat.com, song@kernel.org, yukuai3@huawei.com Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v3 1/3] md: fix missing flush of sync_work Date: Wed, 29 Nov 2023 12:31:25 +0800 Message-Id: <20231129043127.2245901-2-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231129043127.2245901-1-yukuai1@huaweicloud.com> References: <20231129043127.2245901-1-yukuai1@huaweicloud.com> MIME-Version: 1.0 X-CM-TRANSID: cCh0CgDn6xHHvmZldxHwCA--.18598S5 X-Coremail-Antispam: 1UD129KBjvJXoW7tr18WF13Cr18Cr4kJw13Jwb_yoW8XF4rpa yfAa45ArW8Aay5tryUGa4qvFyrWw18t3yDtrW3W345JF1Yqr45G3WY93WjqFyDJF93Xwnx Za10ya9xZa40vr7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUU9v14x267AKxVW5JVWrJwAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_Jr4l82xGYIkIc2 x26xkF7I0E14v26r1I6r4UM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_Ar0_tr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr1UM2 8EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AI xVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20x vE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xv r2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7MxAIw28IcxkI7VAKI48JMxC20s 026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAFwI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_ JrI_JrWlx4CE17CEb7AF67AKxVWUtVW8ZwCIc40Y0x0EwIxGrwCI42IY6xIIjxv20xvE14 v26r1j6r1xMIIF0xvE2Ix0cI8IcVCY1x0267AKxVW8JVWxJwCI42IY6xAIw20EY4v20xva j40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Jr0_Gr1lIxAIcVC2z280aVCY1x0267AKxVW8JV W8JrUvcSsGvfC2KfnxnUUI43ZEXa7VUbec_DUUUUU== X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 28 Nov 2023 20:32:30 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783871413674960200 X-GMAIL-MSGID: 1783871413674960200 From: Yu Kuai Commit ac619781967b ("md: use separate work_struct for md_start_sync()") use a new sync_work to replace del_work, however, stop_sync_thread() and __md_stop_writes() was trying to wait for sync_thread to be done, hence they should switch to use sync_work as well. Noted that md_start_sync() from sync_work will grab 'reconfig_mutex', hence other contex can't held the same lock to flush work, and this will be fixed in later patches. Fixes: ac619781967b ("md: use separate work_struct for md_start_sync()") Signed-off-by: Yu Kuai Acked-by: Xiao Ni --- drivers/md/md.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index c94373d64f2c..5640a948086b 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -4857,7 +4857,7 @@ static void stop_sync_thread(struct mddev *mddev) return; } - if (work_pending(&mddev->del_work)) + if (work_pending(&mddev->sync_work)) flush_workqueue(md_misc_wq); set_bit(MD_RECOVERY_INTR, &mddev->recovery); @@ -6265,7 +6265,7 @@ static void md_clean(struct mddev *mddev) static void __md_stop_writes(struct mddev *mddev) { set_bit(MD_RECOVERY_FROZEN, &mddev->recovery); - if (work_pending(&mddev->del_work)) + if (work_pending(&mddev->sync_work)) flush_workqueue(md_misc_wq); if (mddev->sync_thread) { set_bit(MD_RECOVERY_INTR, &mddev->recovery); From patchwork Wed Nov 29 04:31:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 171058 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a5a7:0:b0:403:3b70:6f57 with SMTP id d7csp116427vqn; Tue, 28 Nov 2023 20:32:36 -0800 (PST) X-Google-Smtp-Source: AGHT+IGCPUHUR9JgXU8/+mNewSk0Di4cd6ILFNPhSyAAnKnxWaH4FCFbExbarTrzoAzPHaHmryii X-Received: by 2002:a05:6a20:3d13:b0:18c:b081:4c32 with SMTP id y19-20020a056a203d1300b0018cb0814c32mr7688899pzi.4.1701232355544; Tue, 28 Nov 2023 20:32:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701232355; cv=none; d=google.com; s=arc-20160816; b=g+cvEH0bnSoTkTblwQjewtU95eB+akpgc09Gw8n2H0sbN0LhskZOFWA7VVC7LuxdND oL9Uh6DpZULqRvIuz5W6tGga2oa3p7Ac5fdGnQ8pPjcLI2Aud/c094Fo3oaUtguGmqHY H4KzBciNvUp6Ha5onJkdNLak1JfL7C1ig1Gr/aggk8Kvde82I/i9MPPhfSM3fZj2hWjr UTqJBpwZcT6tx5avgzZDWLvSMv/64LUZw+zAMsbStb0nLeylOwwiBCs69UccZd8Fpf5o 7Giv3PtfChXFDopqg/UG0bPhrAB3Wb0VCkpxfnKQ13OcLSVp2swnc0tKO9eLfsFBNPGt nriQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=+bCK5/jiB172XuZ5dbmZwfSZOdZQqyZdx+d7VRHvc00=; fh=zVIHjnfjYXjU+hYbxPUw8z+QX7mV2ayyrBgf4ZvrzN8=; b=SQbogRpaT/dkV0SX1LCon03PXNzr1P8rgPcsKvozpZ8sAmjMnKv6OmhbO7dwbvwnK/ vxDs1tEDdclv7pC/tAymkcro8HNlcNtKDfF7QzMRpb8W7ZvS6Nq2CezRtpvaHDrl3amV 3kJx7R8M/AWFUQYs5eOLPyePRFlZESnCGmRA9SqAZWIuZ+H5L5bqr5vdI38tG8NJJ3E/ 01h/71I1BaC7im3iLHozHzmsNhNlM3/mm+Dvev2jmKV9kxe7FtPrWpYg8MqTOjvHirYL DOYBbG11TWaWGbGI9BKJ7a1+zVyrrbAUhss0arcTEJvEuWXeXbE438FK931zFC3NMsv/ W+Pw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id z8-20020a1709027e8800b001cfdd2fe63csi4503858pla.312.2023.11.28.20.32.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Nov 2023 20:32:35 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 0C725804AD36; Tue, 28 Nov 2023 20:32:30 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1376904AbjK2EcL (ORCPT + 99 others); Tue, 28 Nov 2023 23:32:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230198AbjK2EcI (ORCPT ); Tue, 28 Nov 2023 23:32:08 -0500 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A934B19B1; Tue, 28 Nov 2023 20:32:13 -0800 (PST) Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4Sg5yH3wkYz4f3kk1; Wed, 29 Nov 2023 12:32:07 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 689AA1A0B64; Wed, 29 Nov 2023 12:32:10 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgDn6xHHvmZldxHwCA--.18598S6; Wed, 29 Nov 2023 12:32:10 +0800 (CST) From: Yu Kuai To: xni@redhat.com, song@kernel.org, yukuai3@huawei.com Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v3 2/3] md: don't leave 'MD_RECOVERY_FROZEN' in error path of md_set_readonly() Date: Wed, 29 Nov 2023 12:31:26 +0800 Message-Id: <20231129043127.2245901-3-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231129043127.2245901-1-yukuai1@huaweicloud.com> References: <20231129043127.2245901-1-yukuai1@huaweicloud.com> MIME-Version: 1.0 X-CM-TRANSID: cCh0CgDn6xHHvmZldxHwCA--.18598S6 X-Coremail-Antispam: 1UD129KBjvJXoW7uFWfGr48uw4ktF45Cr1rJFb_yoW8KFWxp3 yxtF98Cry8JFy3Zr4Dta4DZa45Zw1jqrWqyFy3u34rJF1ftrsxCFyY9348JrWvkas2v3y5 Xw4rGrW7ua4IgaUanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUU9m14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_Jryl82xGYIkIc2 x26xkF7I0E14v26r4j6ryUM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_Ar0_tr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJw A2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq3wAS 0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2 IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0 Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwCF04k20xvY0x0EwIxGrwCFx2 IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v2 6r106r1rMI8E67AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67 AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k26cxKx2IY s7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Gr 0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x0JUc6pPUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Tue, 28 Nov 2023 20:32:30 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783871418748256183 X-GMAIL-MSGID: 1783871418748256183 From: Yu Kuai If md_set_readonly() failed, the array could still be read-write, however 'MD_RECOVERY_FROZEN' could still be set, which leave the array in an abnormal state that sync or recovery can't continue anymore. Hence make sure the flag is cleared after md_set_readonly() returns. Signed-off-by: Yu Kuai Acked-by: Xiao Ni --- drivers/md/md.c | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 5640a948086b..2d8e45a1af23 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -6355,6 +6355,9 @@ static int md_set_readonly(struct mddev *mddev, struct block_device *bdev) int err = 0; int did_freeze = 0; + if (mddev->external && test_bit(MD_SB_CHANGE_PENDING, &mddev->sb_flags)) + return -EBUSY; + if (!test_bit(MD_RECOVERY_FROZEN, &mddev->recovery)) { did_freeze = 1; set_bit(MD_RECOVERY_FROZEN, &mddev->recovery); @@ -6369,8 +6372,6 @@ static int md_set_readonly(struct mddev *mddev, struct block_device *bdev) */ md_wakeup_thread_directly(mddev->sync_thread); - if (mddev->external && test_bit(MD_SB_CHANGE_PENDING, &mddev->sb_flags)) - return -EBUSY; mddev_unlock(mddev); wait_event(resync_wait, !test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)); @@ -6383,29 +6384,30 @@ static int md_set_readonly(struct mddev *mddev, struct block_device *bdev) mddev->sync_thread || test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)) { pr_warn("md: %s still in use.\n",mdname(mddev)); - if (did_freeze) { - clear_bit(MD_RECOVERY_FROZEN, &mddev->recovery); - set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); - md_wakeup_thread(mddev->thread); - } err = -EBUSY; goto out; } + if (mddev->pers) { __md_stop_writes(mddev); - err = -ENXIO; - if (mddev->ro == MD_RDONLY) + if (mddev->ro == MD_RDONLY) { + err = -ENXIO; goto out; + } + mddev->ro = MD_RDONLY; set_disk_ro(mddev->gendisk, 1); + } + +out: + if ((mddev->pers && !err) || did_freeze) { clear_bit(MD_RECOVERY_FROZEN, &mddev->recovery); set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); md_wakeup_thread(mddev->thread); sysfs_notify_dirent_safe(mddev->sysfs_state); - err = 0; } -out: + mutex_unlock(&mddev->open_mutex); return err; } From patchwork Wed Nov 29 04:31:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 171059 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a5a7:0:b0:403:3b70:6f57 with SMTP id d7csp121600vqn; Tue, 28 Nov 2023 20:51:06 -0800 (PST) X-Google-Smtp-Source: AGHT+IE19KtYbE3jfw56To9F72hC0vwCmkEhb8Qc3CyGj1hsnC+zoAiAwFNceRPVar6oilH5xSit X-Received: by 2002:a17:90b:1894:b0:280:23e4:4326 with SMTP id mn20-20020a17090b189400b0028023e44326mr27659806pjb.14.1701233466348; Tue, 28 Nov 2023 20:51:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1701233466; cv=none; d=google.com; s=arc-20160816; b=vuZvBTGNBkyP4Cj3h5M3HJrbnypXtA1KqDQFGSaeW5HhZlbhh8onWwf9kB+3bGp8JT 9sxPJCTQc+RLk7jf6eeV73raZuOKiVFZL5CqYW16ld2QBZXVshq5lrJLS0zmXsl+DWZY jtJ8xePJfpHy6PhzT5QLujil8SxcXF39Td3la3dj+DaD6dvB/cvS59Mtov9Wo9McbDzJ zsIvcJcfMIrBO3Hb0kAtw5qBeQbddP3/hkOFHdUz36frxqanKeGNjA4sgSp46XS84+M+ 3jgu9fIfG0wNbxiMhmdiajE8oKk3W163jWTrp9j5roKuM2kaLLfaVW8G4RM0TsrNe2BM 3o7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=xihZIMK+vrjI9FerHWad8npcx0ZKtkfF4Kino3hKLcQ=; fh=zVIHjnfjYXjU+hYbxPUw8z+QX7mV2ayyrBgf4ZvrzN8=; b=QJGE/I8uxlJoL85J74k+qjFxoS68PYDK/AIqKy/5h7Qq+4lu3fqUGHy24fDwl6NP4+ Ie82p0VdvnIcAPtfeO7B9BnNPgNrqzfNda6n4lUlDBrvBXWcC7Tl4o3LtdQPflpQtBKd LELS4IuKBuJZV/caX4uCMxzmROLfHATtbbIi1X36wPa23lVE7C57JG/2ev1rBS/eQvgT ZIGZa/sUDKPYczCl6w9kh6JOru7YEvDSSwJUzA7cvUGwOyxJqEKEpCy54/RJBl6IgNIu bNgorXCcgGstZEtSDXla1uVbmZYEGyNfilzMyY3rqC9/Yg9EfDmIx9y0ExAuzgmBbrCV hSEA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id mh17-20020a17090b4ad100b002736ff3cc79si580655pjb.23.2023.11.28.20.51.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Nov 2023 20:51:06 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 03DCF81CCD9F; Tue, 28 Nov 2023 20:50:59 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234938AbjK2EcS (ORCPT + 99 others); Tue, 28 Nov 2023 23:32:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234638AbjK2EcI (ORCPT ); Tue, 28 Nov 2023 23:32:08 -0500 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9D8A719AD; Tue, 28 Nov 2023 20:32:13 -0800 (PST) Received: from mail.maildlp.com (unknown [172.19.93.142]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4Sg5yG25Fvz4f3lfL; Wed, 29 Nov 2023 12:32:06 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id BF3231A086D; Wed, 29 Nov 2023 12:32:10 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgDn6xHHvmZldxHwCA--.18598S7; Wed, 29 Nov 2023 12:32:10 +0800 (CST) From: Yu Kuai To: xni@redhat.com, song@kernel.org, yukuai3@huawei.com Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v3 3/3] md: fix stopping sync thread Date: Wed, 29 Nov 2023 12:31:27 +0800 Message-Id: <20231129043127.2245901-4-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231129043127.2245901-1-yukuai1@huaweicloud.com> References: <20231129043127.2245901-1-yukuai1@huaweicloud.com> MIME-Version: 1.0 X-CM-TRANSID: cCh0CgDn6xHHvmZldxHwCA--.18598S7 X-Coremail-Antispam: 1UD129KBjvJXoW3JF43Xr13GF48ZFy5tw1DKFg_yoWxXFWxp3 yftF98JrW8ArW3ZrW7K3WDZayrZw1jvrZrtry3Wa4fJw1ftr43KFyY9FyUAFykta4Fyr4U ZayrJFWfZFy0gr7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUU9m14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JrWl82xGYIkIc2 x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_Ar0_tr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJw A2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq3wAS 0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2 IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0 Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwCF04k20xvY0x0EwIxGrwCFx2 IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v2 6r106r1rMI8E67AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67 AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k26cxKx2IY s7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Gr 0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x0JUd8n5UUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Tue, 28 Nov 2023 20:51:00 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1783871417582820253 X-GMAIL-MSGID: 1783872583011410080 From: Yu Kuai Currently sync thread is stopped from multiple contex: - idle_sync_thread - frozen_sync_thread - __md_stop_writes - md_set_readonly - do_md_stop And there are some problems: 1) sync_work is flushed while reconfig_mutex is grabbed, this can deadlock because the work function will grab reconfig_mutex as well. 2) md_reap_sync_thread() can't be called directly while md_do_sync() is not finished yet, for example, commit 130443d60b1b ("md: refactor idle/frozen_sync_thread() to fix deadlock"). 3) If MD_RECOVERY_RUNNING is not set, there is no need to stop sync_thread at all because sync_thread must not be registered. Factor out a helper prepare_to_stop_sync_thread(), so that above contex will behave the same. Fix 1) by flushing sync_work after reconfig_mutex is released, before waiting for sync_thread to be done; Fix 2) bt letting daemon thread to unregister sync_thread; Fix 3) by always checking MD_RECOVERY_RUNNING first. Fixes: db5e653d7c9f ("md: delay choosing sync action to md_start_sync()") Acked-by: Xiao Ni Signed-off-by: Yu Kuai --- drivers/md/md.c | 96 +++++++++++++++++++++++-------------------------- 1 file changed, 45 insertions(+), 51 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 2d8e45a1af23..05902e36db66 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -4840,26 +4840,9 @@ action_show(struct mddev *mddev, char *page) return sprintf(page, "%s\n", type); } -static void stop_sync_thread(struct mddev *mddev) +static void prepare_to_stop_sync_thread(struct mddev *mddev) + __releases(&mddev->reconfig_mutex) { - if (!test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)) - return; - - if (mddev_lock(mddev)) - return; - - /* - * Check again in case MD_RECOVERY_RUNNING is cleared before lock is - * held. - */ - if (!test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)) { - mddev_unlock(mddev); - return; - } - - if (work_pending(&mddev->sync_work)) - flush_workqueue(md_misc_wq); - set_bit(MD_RECOVERY_INTR, &mddev->recovery); /* * Thread might be blocked waiting for metadata update which will now @@ -4868,6 +4851,8 @@ static void stop_sync_thread(struct mddev *mddev) md_wakeup_thread_directly(mddev->sync_thread); mddev_unlock(mddev); + if (work_pending(&mddev->sync_work)) + flush_work(&mddev->sync_work); } static void idle_sync_thread(struct mddev *mddev) @@ -4876,10 +4861,20 @@ static void idle_sync_thread(struct mddev *mddev) mutex_lock(&mddev->sync_mutex); clear_bit(MD_RECOVERY_FROZEN, &mddev->recovery); - stop_sync_thread(mddev); - wait_event(resync_wait, sync_seq != atomic_read(&mddev->sync_seq) || + if (mddev_lock(mddev)) { + mutex_unlock(&mddev->sync_mutex); + return; + } + + if (test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)) { + prepare_to_stop_sync_thread(mddev); + wait_event(resync_wait, + sync_seq != atomic_read(&mddev->sync_seq) || !test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)); + } else { + mddev_unlock(mddev); + } mutex_unlock(&mddev->sync_mutex); } @@ -4888,10 +4883,19 @@ static void frozen_sync_thread(struct mddev *mddev) { mutex_lock(&mddev->sync_mutex); set_bit(MD_RECOVERY_FROZEN, &mddev->recovery); - stop_sync_thread(mddev); - wait_event(resync_wait, mddev->sync_thread == NULL && + if (mddev_lock(mddev)) { + mutex_unlock(&mddev->sync_mutex); + return; + } + + if (test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)) { + prepare_to_stop_sync_thread(mddev); + wait_event(resync_wait, !test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)); + } else { + mddev_unlock(mddev); + } mutex_unlock(&mddev->sync_mutex); } @@ -6265,11 +6269,11 @@ static void md_clean(struct mddev *mddev) static void __md_stop_writes(struct mddev *mddev) { set_bit(MD_RECOVERY_FROZEN, &mddev->recovery); - if (work_pending(&mddev->sync_work)) - flush_workqueue(md_misc_wq); - if (mddev->sync_thread) { - set_bit(MD_RECOVERY_INTR, &mddev->recovery); - md_reap_sync_thread(mddev); + if (test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)) { + prepare_to_stop_sync_thread(mddev); + wait_event(resync_wait, !test_bit(MD_RECOVERY_RUNNING, + &mddev->recovery)); + mddev_lock_nointr(mddev); } del_timer_sync(&mddev->safemode_timer); @@ -6363,18 +6367,15 @@ static int md_set_readonly(struct mddev *mddev, struct block_device *bdev) set_bit(MD_RECOVERY_FROZEN, &mddev->recovery); md_wakeup_thread(mddev->thread); } - if (test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)) - set_bit(MD_RECOVERY_INTR, &mddev->recovery); - /* - * Thread might be blocked waiting for metadata update which will now - * never happen - */ - md_wakeup_thread_directly(mddev->sync_thread); + if (test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)) { + prepare_to_stop_sync_thread(mddev); + wait_event(resync_wait, !test_bit(MD_RECOVERY_RUNNING, + &mddev->recovery)); + } else { + mddev_unlock(mddev); + } - mddev_unlock(mddev); - wait_event(resync_wait, !test_bit(MD_RECOVERY_RUNNING, - &mddev->recovery)); wait_event(mddev->sb_wait, !test_bit(MD_SB_CHANGE_PENDING, &mddev->sb_flags)); mddev_lock_nointr(mddev); @@ -6428,20 +6429,13 @@ static int do_md_stop(struct mddev *mddev, int mode, set_bit(MD_RECOVERY_FROZEN, &mddev->recovery); md_wakeup_thread(mddev->thread); } - if (test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)) - set_bit(MD_RECOVERY_INTR, &mddev->recovery); - /* - * Thread might be blocked waiting for metadata update which will now - * never happen - */ - md_wakeup_thread_directly(mddev->sync_thread); - - mddev_unlock(mddev); - wait_event(resync_wait, (mddev->sync_thread == NULL && - !test_bit(MD_RECOVERY_RUNNING, - &mddev->recovery))); - mddev_lock_nointr(mddev); + if (test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)) { + prepare_to_stop_sync_thread(mddev); + wait_event(resync_wait, !test_bit(MD_RECOVERY_RUNNING, + &mddev->recovery)); + mddev_lock_nointr(mddev); + } mutex_lock(&mddev->open_mutex); if ((mddev->pers && atomic_read(&mddev->openers) > !!bdev) ||