From patchwork Tue Jan 30 02:18:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 193800 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp956561dyb; Mon, 29 Jan 2024 18:24:21 -0800 (PST) X-Google-Smtp-Source: AGHT+IHzCiI1Qj4qfqHPlXmkkh7R4yHQcryXva3d6KUFzXkJ3OE2HUIQsiiyR2f/nTcykc/WYEGU X-Received: by 2002:ad4:5f46:0:b0:686:a3fd:45bf with SMTP id p6-20020ad45f46000000b00686a3fd45bfmr7047764qvg.36.1706581460843; Mon, 29 Jan 2024 18:24:20 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706581460; cv=pass; d=google.com; s=arc-20160816; b=qWU/8vcUegmtTFUOeGqpIUuCR6DfGbyovMcZYmzRdylHCT7x4eXt3lA5YqmTpD6llD M2dh+H/G+OylBz0inNQZ9Vkuy3rnbKdbXoTrsYgJQ2Mc9/hbZj16AcEXPLn/bS8mBVsn Er6uWRnIgWlVtTcQ624hW0uktHc6NCKkVAsPpx2iBRRkeNPMI5ir0qS55AfUKr9oI4ji kqxwXl1HlmL1uHcl+NdHZ9bDBdk2skyz4QlQPvrF4aqfQMpLRApLhAtKfkmaPNcT8cHa 78MTHSlIRrleL14CofmR3NxGkbIb12zARXoh7tPBD9wQXYweQIw7gxX88QhHVzzC7mfO /xmQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=dBaWySK1O6HNfOrqRqopNpWnbd2nAQFt9nqs3kj8Nq8=; fh=FtirvJJ13L8gehLQ375a0lCTkAroffAi0JfnF40Fa78=; b=1FJiqlnYz3w0Yu7MjrlvqeukedLH34GUQ76+Ok7qmRzSB92xuiDnp3E0eM7iQM5Ext UTPc68I1PGjbFr4IIhW6X4UeEkaIlOSWnIlLRkLS7XyeAgVFjeryZTgHMfmov8QrDYf/ iTo6KcvsFUb2F0+u/IHx90bAeZdMYQ/PbW8nDKyW8vk3CiPwVdNil670UtYLzTrprjJq j9y2YG+JM6KF7IRznfkcjq01fiMpo5lvWXhi5ZOn1aFm6BLpZ6hUDQlY6jra0nKALUAd nKoQPvXAFrOzhB0Laif1eojiAzoR8vHv9wCYeJFdcvJ9jNLml+h/1EmAhd4ZJuqlPghi zSOQ== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43806-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43806-ouuuleilei=gmail.com@vger.kernel.org" Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id f25-20020a05620a20d900b00783f4c38aaesi5196317qka.247.2024.01.29.18.24.20 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 18:24:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43806-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43806-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43806-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id A00CC1C24552 for ; Tue, 30 Jan 2024 02:24:20 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 941A1381B9; Tue, 30 Jan 2024 02:23:09 +0000 (UTC) Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2524A364BF; Tue, 30 Jan 2024 02:23:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581387; cv=none; b=cb7uGF55QYEsOvankhQAu3mPCmJhffMdT+DI87AI9qjuyd86w3NIp0GQYRoGVw8M7cBDHXO5bf+hAACDyFZOKnsF8xl07CmtxGWoXtGjyMTc5N7SNF4ohbSotY6RUjqa/BL1ZAYn6sELvZ+icMVSu6Zzxdak+O8eYdn6FAFzOYk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581387; c=relaxed/simple; bh=niaIFrprWvPwC2Lv0RN5Cl1tFCytXopzeriQpGSoAMg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=CLB4pmlyoHjcEi/KypomQKqbfQNKU9ge4nGayqOvo/ajm/8SzeEKCrO2GFDG/drOXMvuClzF7tDJGwYmdUNLoZiechy1lvtQVDwFc8NUZVQLAh5Lp0endV54FaTMtdLPWVpawV7KbLYXiFEWfwbnTqDwuykVR4qzwtUE1tQkzKg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.93.142]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4TP88b58hNz4f3m72; Tue, 30 Jan 2024 10:22:55 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 317431A0172; Tue, 30 Jan 2024 10:23:02 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgBHGBGBXbhlrAigCQ--.55484S5; Tue, 30 Jan 2024 10:23:00 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v4 01/14] md: don't ignore suspended array in md_check_recovery() Date: Tue, 30 Jan 2024 10:18:30 +0800 Message-Id: <20240130021843.3608859-2-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240130021843.3608859-1-yukuai1@huaweicloud.com> References: <20240130021843.3608859-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgBHGBGBXbhlrAigCQ--.55484S5 X-Coremail-Antispam: 1UD129KBjvJXoWxJryDAF43ArW7JrWrZr1xGrg_yoW8ZFy7pa yIkF1YyrWjyFZ7Aa4qka4UZa4rAr1jqrW5AFy3u34rCa4fKw43CrWYgFyDXFyqkFyxKrZY vw4rJa15uw18KF7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBE14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_Jr4l82xGYIkIc2 x26xkF7I0E14v26r4j6ryUM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJw A2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq3wAS 0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2 IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0 Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2kIc2 xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v2 6r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_GFv_WrylIxkGc2 Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_ Gr0_Cr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMI IF0xvEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x0JU4T5dUUUUU = X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789480361583538218 X-GMAIL-MSGID: 1789480361583538218 From: Yu Kuai mddev_suspend() never stop sync_thread, hence it doesn't make sense to ignore suspended array in md_check_recovery(), which might cause sync_thread can't be unregistered. After commit f52f5c71f3d4 ("md: fix stopping sync thread"), following hang can be triggered by test shell/integrity-caching.sh: 1) suspend the array: raid_postsuspend mddev_suspend 2) stop the array: raid_dtr md_stop __md_stop_writes stop_sync_thread set_bit(MD_RECOVERY_INTR, &mddev->recovery); md_wakeup_thread_directly(mddev->sync_thread); wait_event(..., !test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)) 3) sync thread done: md_do_sync set_bit(MD_RECOVERY_DONE, &mddev->recovery); md_wakeup_thread(mddev->thread); 4) daemon thread can't unregister sync thread: md_check_recovery if (mddev->suspended) return; -> return directly md_read_sync_thread clear_bit(MD_RECOVERY_RUNNING, &mddev->recovery); -> MD_RECOVERY_RUNNING can't be cleared, hence step 2 hang; This problem is not just related to dm-raid, fix it by ignoring suspended array in md_check_recovery(). And follow up patches will improve dm-raid better to frozen sync thread during suspend. Reported-by: Mikulas Patocka Closes: https://lore.kernel.org/all/8fb335e-6d2c-dbb5-d7-ded8db5145a@redhat.com/ Fixes: 68866e425be2 ("MD: no sync IO while suspended") Fixes: f52f5c71f3d4 ("md: fix stopping sync thread") Signed-off-by: Yu Kuai --- drivers/md/md.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 2266358d8074..07b80278eaa5 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -9469,9 +9469,6 @@ static void md_start_sync(struct work_struct *ws) */ void md_check_recovery(struct mddev *mddev) { - if (READ_ONCE(mddev->suspended)) - return; - if (mddev->bitmap) md_bitmap_daemon_work(mddev); From patchwork Tue Jan 30 02:18:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 193810 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp960960dyb; Mon, 29 Jan 2024 18:39:21 -0800 (PST) X-Google-Smtp-Source: AGHT+IGklL4fQy7rNTWqnUOmvBSkBToNmqnmRZLN1HwGRTzHPWKjyV7ZyRDs9FAegjyMTsk7s/J9 X-Received: by 2002:a62:cf43:0:b0:6de:10a1:35c1 with SMTP id b64-20020a62cf43000000b006de10a135c1mr4842214pfg.5.1706582360823; Mon, 29 Jan 2024 18:39:20 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706582360; cv=pass; d=google.com; s=arc-20160816; b=Fged3GLOR3WWGQuVAzoHcp8PzGAMv/1P0U5CkiN5dc0TYayuCrKbq2bVooWnk+L+Mz ikcuNljivwXlGeGoJ56Yut34+Ky0bWWkJGq2TGPI/2qpmOmHuOFbBHoLDapNFW9rGbNO G+Z/4/3pMvNWqL7K7llAbVXR/xePaDC89yXro7sQLGelcq7w0bTvcdk5MVyd6q1C9mUa 2X1KTuEz91QL1VLaF1asQG8hQ1CGqVxCjckd4gLSnID2zmhzjOlolwpp9lFT8KjGKYAC 4PzkZZG1Ti0jqbDlo50lEQV/xJymEmonKFP/jjqgde96/Nyag5U+fgdM4pGnErcxvoAP F89g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=J4U91+KdS8PQE0p29V5wPuN9QbhEeMD+9Bl1l8h5hgU=; fh=FtirvJJ13L8gehLQ375a0lCTkAroffAi0JfnF40Fa78=; b=fYLgyUxTgbDCQDLt54Q17zpAxXN4brkHV/iXcmn/2ZVzLo/IjJqqBTjzKWZeS+nY3q euotPOgV0vypr7TaOWOSpTWQvNkhuWtVeHafkY24ngjGj9JWaAqQlhmg+WlJCXd0yeQc TedcUzNDMfHlw+Z2GKEViCsXyR1FrQp8L25YFJlRFrgr0XJgyqEh5KoNnWPU7qFXAOZd HmExBcHp3j/kwy1KxKZeTWxu/3rpxucesbsXYvLsgyD1qiN3+fp3nJ3ScjI6W8UXmNMr 8qQJLvffhJy8s6KP+CPMP6WmKxDDXCQZLdrOphUjwqNjkfyu8QGZiASngyhetljhfQiV BGpw== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43807-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43807-ouuuleilei=gmail.com@vger.kernel.org" Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id ck27-20020a056a02091b00b005cfbc9de943si6808083pgb.382.2024.01.29.18.39.20 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 18:39:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43807-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43807-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43807-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id CB154B24F60 for ; Tue, 30 Jan 2024 02:24:47 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8E7D2383A6; Tue, 30 Jan 2024 02:23:10 +0000 (UTC) Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A2A1B1A723; Tue, 30 Jan 2024 02:23:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581387; cv=none; b=pJSkntOQmbFHVuVcflDeKuKE8j55ebQcYnZ/tHZCCtpuMpgQEAdcTnMAONIIpN4kg7mBnt+oX//kw4IabXVgiv9s3IIOnMyNe1dxHFrzxBzoDmWwsdbXVuQwq11QOqQnZAukl2y6cpZjJRfP8/uWpWijaVfSE5bBCy8aMnD1eSg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581387; c=relaxed/simple; bh=h4mAM5munNDh64f9BT7eKjkJVG7xHUqubnznwxYlHRM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=O0JGNCXs9neLjqNVuFAiVDv3gVEl7wkmwSZPLcvFRf8tYR8EDbq9eK+uF8l+T5/ENNow7Rs/K3CE4wybEYRvloA9Ih9nwHdhWm2dKBjcUitYZT6+R25H8wqjcenx13WNdEVRU6t7tXdFd88PLM84YeMSJF4HUKniDamS//4sCDg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.93.142]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4TP88f573Kz4f3lwg; Tue, 30 Jan 2024 10:22:58 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id C94E11A0171; Tue, 30 Jan 2024 10:23:02 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgBHGBGBXbhlrAigCQ--.55484S6; Tue, 30 Jan 2024 10:23:02 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v4 02/14] md: don't ignore read-only array in md_check_recovery() Date: Tue, 30 Jan 2024 10:18:31 +0800 Message-Id: <20240130021843.3608859-3-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240130021843.3608859-1-yukuai1@huaweicloud.com> References: <20240130021843.3608859-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgBHGBGBXbhlrAigCQ--.55484S6 X-Coremail-Antispam: 1UD129KBjvJXoWxCF1xZw47XF4fCr17tFyUGFg_yoWrXw45pa yfKas8Cr4UtFy3urZrK3WDXayYvr10qrZ0yFy3u34fAw1rtr4Sk34ruFyUJFWDtFyFqw4Y vw4rJF43ZFy8Kw7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPj14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_Jryl82xGYIkIc2 x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJw A2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq3wAS 0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2 IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0 Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2kIc2 xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v2 6r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_GFv_WrylIxkGc2 Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_ Cr0_Gr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVWUJVW8Jw CI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWIevJa73UjIFyTuYvjfUOJPEUUUU U X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789481305773560257 X-GMAIL-MSGID: 1789481305773560257 From: Yu Kuai Usually if the array is not read-write, md_check_recovery() won't register new sync_thread in the first place. And if the array is read-write and sync_thread is registered, md_set_readonly() will unregister sync_thread before setting the array read-only. md/raid follow this behavior hence there is no problem. After commit f52f5c71f3d4 ("md: fix stopping sync thread"), following hang can be triggered by test shell/integrity-caching.sh: 1) array is read-only. dm-raid update super block: rs_update_sbs ro = mddev->ro mddev->ro = 0 -> set array read-write md_update_sb 2) register new sync thread concurrently. 3) dm-raid set array back to read-only: rs_update_sbs mddev->ro = ro 4) stop the array: raid_dtr md_stop stop_sync_thread set_bit(MD_RECOVERY_INTR, &mddev->recovery); md_wakeup_thread_directly(mddev->sync_thread); wait_event(..., !test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)) 5) sync thread done: md_do_sync set_bit(MD_RECOVERY_DONE, &mddev->recovery); md_wakeup_thread(mddev->thread); 6) daemon thread can't unregister sync thread: md_check_recovery if (!md_is_rdwr(mddev) && !test_bit(MD_RECOVERY_NEEDED, &mddev->recovery)) return; -> -> MD_RECOVERY_RUNNING can't be cleared, hence step 4 hang; The root cause is that dm-raid manipulate 'mddev->ro' by itself, however, dm-raid really should stop sync thread before setting the array read-only. Unfortunately, I need to read more code before I can refacter the handler of 'mddev->ro' in dm-raid, hence let's fix the problem the easy way for now to prevent dm-raid regression. Reported-by: Mikulas Patocka Closes: https://lore.kernel.org/all/9801e40-8ac7-e225-6a71-309dcf9dc9aa@redhat.com/ Fixes: ecbfb9f118bc ("dm raid: add raid level takeover support") Fixes: f52f5c71f3d4 ("md: fix stopping sync thread") Signed-off-by: Yu Kuai --- drivers/md/md.c | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 07b80278eaa5..6906d023f1d6 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -9445,6 +9445,20 @@ static void md_start_sync(struct work_struct *ws) sysfs_notify_dirent_safe(mddev->sysfs_action); } +static void unregister_sync_thread(struct mddev *mddev) +{ + if (!test_bit(MD_RECOVERY_DONE, &mddev->recovery)) { + /* resync/recovery still happening */ + clear_bit(MD_RECOVERY_NEEDED, &mddev->recovery); + return; + } + + if (WARN_ON_ONCE(!mddev->sync_thread)) + return; + + md_reap_sync_thread(mddev); +} + /* * This routine is regularly called by all per-raid-array threads to * deal with generic issues like resync and super-block update. @@ -9482,7 +9496,8 @@ void md_check_recovery(struct mddev *mddev) } if (!md_is_rdwr(mddev) && - !test_bit(MD_RECOVERY_NEEDED, &mddev->recovery)) + !test_bit(MD_RECOVERY_NEEDED, &mddev->recovery) && + !test_bit(MD_RECOVERY_DONE, &mddev->recovery)) return; if ( ! ( (mddev->sb_flags & ~ (1<recovery)) { - /* sync_work already queued. */ - clear_bit(MD_RECOVERY_NEEDED, &mddev->recovery); + unregister_sync_thread(mddev); goto unlock; } @@ -9568,16 +9582,7 @@ void md_check_recovery(struct mddev *mddev) * still set. */ if (test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)) { - if (!test_bit(MD_RECOVERY_DONE, &mddev->recovery)) { - /* resync/recovery still happening */ - clear_bit(MD_RECOVERY_NEEDED, &mddev->recovery); - goto unlock; - } - - if (WARN_ON_ONCE(!mddev->sync_thread)) - goto unlock; - - md_reap_sync_thread(mddev); + unregister_sync_thread(mddev); goto unlock; } From patchwork Tue Jan 30 02:18:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 193802 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp956967dyb; Mon, 29 Jan 2024 18:25:45 -0800 (PST) X-Google-Smtp-Source: AGHT+IGFPdSL0KYBxKTgTSgXXK2fXiXiRGTES9LBJ2VgnHhpcjLVNiPyqkGmQwgNYmS+Rx13SsSq X-Received: by 2002:a05:622a:1211:b0:42a:426b:3101 with SMTP id y17-20020a05622a121100b0042a426b3101mr6847438qtx.71.1706581544870; Mon, 29 Jan 2024 18:25:44 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706581544; cv=pass; d=google.com; s=arc-20160816; b=J4zFIwNnkUQed5vBzYc0VehSy7SE5SlvQHoK38tynvU9Jke/O36+BSWZrQ0NtnHMsi MWK8tO7FtvfKGjm4k8ZKXRgYV3TvwJnrOQfcaE5JgrAiciQDvei8Dzec1wzRELZRF/i6 +15ZmHkozWOmmtKSCUCohTyzT8QFRMcHp3z+sKQG+knokdZTiNPgMMUDTe8IfOzrMWmD 57ksHtbkhvR1favP6ugvTXuPL7jLEZAPzaMTPjEtCManheUYz7S3ZAKVmWlgonY1fe3D U0MkafScGsNcrewmLC+yUNycVjPJYFHXqUg/xKkEk6NqoYO+QbeGB9GPPFEQnirhCKgx 6UaQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=D0W001Odz43Z3JUWcykLDV1gM6F19antxKYIQwlWdgQ=; fh=FtirvJJ13L8gehLQ375a0lCTkAroffAi0JfnF40Fa78=; b=HyTzIta42WbuG/9WQ+ujraCQPmy/7v2ZdusHbXodtXuMxNGrIesGQ00boRoNwpLnNG 0GXgNrctQ06Z83EL6fQVZcbUQyWT7R1bl+zVXQnDs2hatV1XEGVvoaP2iJepPbTIvuu1 fMqhhZIYjo6O8+aMb6uFOuaeeIIXD38Ox+H4oV67XeBVrYb9dJMpLuTzeqJ4fqNbtrSh 0Y1YHqxRtNKJuldCrUzCZdrHJh+Heeo5axdtX09jg8wGVRLAzXcVW7DLcRwztuJsdDop kH6m7iOVYgM/nrWuVzTBv7S8MwNRAt0RUvytMMZEi5HyBinhH5ZPmeHDxhhfBxoCypTX jDvw== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43809-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43809-ouuuleilei=gmail.com@vger.kernel.org" Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id u12-20020a05622a198c00b0042ab75b006esi298093qtc.432.2024.01.29.18.25.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 18:25:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43809-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43809-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43809-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 942DE1C216B2 for ; Tue, 30 Jan 2024 02:25:44 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9BAAD53E1A; Tue, 30 Jan 2024 02:23:12 +0000 (UTC) Received: from dggsgout12.his.huawei.com (unknown [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E6CA5364A5; Tue, 30 Jan 2024 02:23:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581389; cv=none; b=dFyWB86/7ArjJbzCxxzVhu/D3jt3D2//4mMv3RspAxfb7fr1pG2lVMlBmVldKUIxuXIpXXmafGBmKdwi8jUc5QPUTAArU/iApPaciYhAfJ3CN3ByvNxK36wNKTthyo8S6SOJ6uGLyUTxIGd/hg0HVJ9eYp9F5f+Og2SIeT3S4rg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581389; c=relaxed/simple; bh=dZCpGLgs1JsjUcuAOwp41TG3RiXfam80H98EBcaCbRk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KXHG9b22vntqs74ToEkK/pOVuGdNJIKHdZAjGAt4RHWayXkl5bfPgdYiENJWQVcICghLKsunM0CQaFqswYXV1aetr0VEzK7q+v35rMAVfPLUB61PgNhqf/9qK0dPWAoVVjh8rEEaOR4BiBZduQuCJTpuWjkcqCicfqbKyyM9aH0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.93.142]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4TP88g6rn3z4f3lw2; Tue, 30 Jan 2024 10:22:59 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 14F341A016E; Tue, 30 Jan 2024 10:23:04 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgBHGBGBXbhlrAigCQ--.55484S8; Tue, 30 Jan 2024 10:23:03 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v4 04/14] md: don't register sync_thread for reshape directly Date: Tue, 30 Jan 2024 10:18:33 +0800 Message-Id: <20240130021843.3608859-5-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240130021843.3608859-1-yukuai1@huaweicloud.com> References: <20240130021843.3608859-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgBHGBGBXbhlrAigCQ--.55484S8 X-Coremail-Antispam: 1UD129KBjvJXoW3Xry3Kw4kuF1xAry3KryrCrg_yoW7KF1fpa yfKF9xJr48A343ZrWUta4DXFW5uw1jqrWqyry3W3s5A3ZYyrZ3JFy5uFyUJFWkAa4kta15 ta45tFWDZFyFgw7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPF14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E 14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_GFv_WrylIx kGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAF wI0_Cr0_Gr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVWUJV W8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWIevJa73UjIFyTuYvjfUOBTY UUUUU X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789480449588961994 X-GMAIL-MSGID: 1789480449588961994 From: Yu Kuai Currently, if reshape is interrupted, then reassemble the array will register sync_thread directly from pers->run(), in this case 'MD_RECOVERY_RUNNING' is set directly, however, there is no guarantee that md_do_sync() will be executed, hence stop_sync_thread() will hang because 'MD_RECOVERY_RUNNING' can't be cleared. Last patch make sure that md_do_sync() will set MD_RECOVERY_DONE, however, following hang can still be triggered by dm-raid test shell/lvconvert-raid-reshape.sh occasionally: [root@fedora ~]# cat /proc/1982/stack [<0>] stop_sync_thread+0x1ab/0x270 [md_mod] [<0>] md_frozen_sync_thread+0x5c/0xa0 [md_mod] [<0>] raid_presuspend+0x1e/0x70 [dm_raid] [<0>] dm_table_presuspend_targets+0x40/0xb0 [dm_mod] [<0>] __dm_destroy+0x2a5/0x310 [dm_mod] [<0>] dm_destroy+0x16/0x30 [dm_mod] [<0>] dev_remove+0x165/0x290 [dm_mod] [<0>] ctl_ioctl+0x4bb/0x7b0 [dm_mod] [<0>] dm_ctl_ioctl+0x11/0x20 [dm_mod] [<0>] vfs_ioctl+0x21/0x60 [<0>] __x64_sys_ioctl+0xb9/0xe0 [<0>] do_syscall_64+0xc6/0x230 [<0>] entry_SYSCALL_64_after_hwframe+0x6c/0x74 Meanwhile mddev->recovery is: MD_RECOVERY_RUNNING | MD_RECOVERY_INTR | MD_RECOVERY_RESHAPE | MD_RECOVERY_FROZEN Fix this problem by remove the code to register sync_thread directly from raid10 and raid5. And let md_check_recovery() to register sync_thread. Fixes: f67055780caa ("[PATCH] md: Checkpoint and allow restart of raid5 reshape") Fixes: f52f5c71f3d4 ("md: fix stopping sync thread") Signed-off-by: Yu Kuai --- drivers/md/md.c | 5 ++++- drivers/md/raid10.c | 16 ++-------------- drivers/md/raid5.c | 29 ++--------------------------- 3 files changed, 8 insertions(+), 42 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index c65dfd156090..6c5d0a372927 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -9372,6 +9372,7 @@ static void md_start_sync(struct work_struct *ws) struct mddev *mddev = container_of(ws, struct mddev, sync_work); int spares = 0; bool suspend = false; + char *name; if (md_spares_need_change(mddev)) suspend = true; @@ -9404,8 +9405,10 @@ static void md_start_sync(struct work_struct *ws) if (spares) md_bitmap_write_all(mddev->bitmap); + name = test_bit(MD_RECOVERY_RESHAPE, &mddev->recovery) ? + "reshape" : "resync"; rcu_assign_pointer(mddev->sync_thread, - md_register_thread(md_do_sync, mddev, "resync")); + md_register_thread(md_do_sync, mddev, name)); if (!mddev->sync_thread) { pr_warn("%s: could not start resync thread...\n", mdname(mddev)); diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index 7412066ea22c..a5f8419e2df1 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -4175,11 +4175,7 @@ static int raid10_run(struct mddev *mddev) clear_bit(MD_RECOVERY_SYNC, &mddev->recovery); clear_bit(MD_RECOVERY_CHECK, &mddev->recovery); set_bit(MD_RECOVERY_RESHAPE, &mddev->recovery); - set_bit(MD_RECOVERY_RUNNING, &mddev->recovery); - rcu_assign_pointer(mddev->sync_thread, - md_register_thread(md_do_sync, mddev, "reshape")); - if (!mddev->sync_thread) - goto out_free_conf; + set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); } return 0; @@ -4573,16 +4569,8 @@ static int raid10_start_reshape(struct mddev *mddev) clear_bit(MD_RECOVERY_CHECK, &mddev->recovery); clear_bit(MD_RECOVERY_DONE, &mddev->recovery); set_bit(MD_RECOVERY_RESHAPE, &mddev->recovery); - set_bit(MD_RECOVERY_RUNNING, &mddev->recovery); - - rcu_assign_pointer(mddev->sync_thread, - md_register_thread(md_do_sync, mddev, "reshape")); - if (!mddev->sync_thread) { - ret = -EAGAIN; - goto abort; - } + set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); conf->reshape_checkpoint = jiffies; - md_wakeup_thread(mddev->sync_thread); md_new_event(); return 0; diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 8497880135ee..6a7a32f7fb91 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -7936,11 +7936,7 @@ static int raid5_run(struct mddev *mddev) clear_bit(MD_RECOVERY_SYNC, &mddev->recovery); clear_bit(MD_RECOVERY_CHECK, &mddev->recovery); set_bit(MD_RECOVERY_RESHAPE, &mddev->recovery); - set_bit(MD_RECOVERY_RUNNING, &mddev->recovery); - rcu_assign_pointer(mddev->sync_thread, - md_register_thread(md_do_sync, mddev, "reshape")); - if (!mddev->sync_thread) - goto abort; + set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); } /* Ok, everything is just fine now */ @@ -8506,29 +8502,8 @@ static int raid5_start_reshape(struct mddev *mddev) clear_bit(MD_RECOVERY_CHECK, &mddev->recovery); clear_bit(MD_RECOVERY_DONE, &mddev->recovery); set_bit(MD_RECOVERY_RESHAPE, &mddev->recovery); - set_bit(MD_RECOVERY_RUNNING, &mddev->recovery); - rcu_assign_pointer(mddev->sync_thread, - md_register_thread(md_do_sync, mddev, "reshape")); - if (!mddev->sync_thread) { - mddev->recovery = 0; - spin_lock_irq(&conf->device_lock); - write_seqcount_begin(&conf->gen_lock); - mddev->raid_disks = conf->raid_disks = conf->previous_raid_disks; - mddev->new_chunk_sectors = - conf->chunk_sectors = conf->prev_chunk_sectors; - mddev->new_layout = conf->algorithm = conf->prev_algo; - rdev_for_each(rdev, mddev) - rdev->new_data_offset = rdev->data_offset; - smp_wmb(); - conf->generation --; - conf->reshape_progress = MaxSector; - mddev->reshape_position = MaxSector; - write_seqcount_end(&conf->gen_lock); - spin_unlock_irq(&conf->device_lock); - return -EAGAIN; - } + set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); conf->reshape_checkpoint = jiffies; - md_wakeup_thread(mddev->sync_thread); md_new_event(); return 0; } From patchwork Tue Jan 30 02:18:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 193801 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp956940dyb; Mon, 29 Jan 2024 18:25:40 -0800 (PST) X-Google-Smtp-Source: AGHT+IFwghP76bqg5mOzrK74FKrnK6psivBzuK6GrPPNDeigtVMMYmkZUzd0sytCh+XDwjVG/3Kn X-Received: by 2002:a17:906:1844:b0:a31:7f7b:275f with SMTP id w4-20020a170906184400b00a317f7b275fmr5143471eje.48.1706581540356; Mon, 29 Jan 2024 18:25:40 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706581540; cv=pass; d=google.com; s=arc-20160816; b=ly8HkcHZAzmjf9mPBO+X1Jm31InrEIy9Fhp75rfdfQYE3H82lpeUlpLGIefzH3hyOx NY3KaRB95PCxq6PqW598C0t+x8yzR2ehPaMv7fZTFwaNKmHPokkBgP3BtFlv/1GdXyZr tqxY+WZ6nuRR8uzNFh1XnWWvClmvZy/aeE1UF5ct2tkFsJ8So04ESf4rK+JRpWsPm80E DRAdABVcautGxgZc7aba5GuGjwmB7D8+djsEwCIFdY4QsJbYEvJYwTV36NOuh7dpDeNi QvDtzAmY96NpoSeCnyma8gZO8e73dvvqfB/MSYqFNlnEUqBDoVjd+U4kkpha/KdK6lAj 8jTw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=X3xSENRjDxfqfuZaMv+JkDLcA9tanKMO67aBb3kN6Zc=; fh=FtirvJJ13L8gehLQ375a0lCTkAroffAi0JfnF40Fa78=; b=WRRqjLcNOUUB7BKHgET58j36AhdfQtq0rr6/2/hp4w212NGSHw38h++JlB9Qfp1TiV FolQJmRwc0nz+rk73x7tkhD79veUcCPq+E1nKz66mQdzMDHPQshb3Ue2SueG6I951q1X EySq1LOz8ay7GzaHFROUpvV3VyCt9a9i2gAtGvk3xLX9adiq09QI91HIXEa9SvBltg8d A8f6VYmrpIBKj5ky68hbMwEol6Z7DZo2O4AXoiOWULp1kE8s/wnmp9KkH3tPlDZ3zb/5 W3Xri7EDPUirDMpj6kKDmJ8PHzSAMEiNgY25iUXodSXU5zZAdJPu+KC30AFT6NWBNcgf CjNQ== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43810-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43810-ouuuleilei=gmail.com@vger.kernel.org" Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id by6-20020a170906a2c600b00a3109b41f3asi3952739ejb.225.2024.01.29.18.25.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 18:25:40 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43810-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43810-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43810-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id CA1641F250CF for ; Tue, 30 Jan 2024 02:25:39 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 633FD53805; Tue, 30 Jan 2024 02:23:12 +0000 (UTC) Received: from dggsgout12.his.huawei.com (unknown [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 75FD937153; Tue, 30 Jan 2024 02:23:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581389; cv=none; b=P6aKMqZc7fXTu0qwCN7S/qEfL8mmy5BtGw9GY9AufQtEUaPFOMftX/xcNsgn/50Dn8q7wvNiv9BN5pFiqJb+/I93DoVrtmO67KDVsIr7JSivTeAf0vLJMvVxJkljdwb35ekpcRcLP/Qpig0bQxmQNpDK9Xx5XJhXSTEbCQnzlZU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581389; c=relaxed/simple; bh=YYyayD+v2iivP+R6gR4731n5JA1gzwDoJc52NdiV/do=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=YVQgc9JgiskI8Yb5iBqCEx20p7+fkUmQqFnKe1QGjHGb9lNlX3xZxPCzmsQzqWKOYIolloXj4L7RjCOmHCNHLLrH45ETeHj//qHY0otmHq4D7Gj/6D8GEz5fdsIOzRz/vLPKddyJROJD14PcLEo05g7ciXfJ+nR/gkXgzlDkBBk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.93.142]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4TP88h42hfz4f3lwW; Tue, 30 Jan 2024 10:23:00 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id A8FE01A016E; Tue, 30 Jan 2024 10:23:04 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgBHGBGBXbhlrAigCQ--.55484S9; Tue, 30 Jan 2024 10:23:04 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v4 05/14] md: export helpers to stop sync_thread Date: Tue, 30 Jan 2024 10:18:34 +0800 Message-Id: <20240130021843.3608859-6-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240130021843.3608859-1-yukuai1@huaweicloud.com> References: <20240130021843.3608859-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgBHGBGBXbhlrAigCQ--.55484S9 X-Coremail-Antispam: 1UD129KBjvJXoWxZryrJF1rCF1fuFyDGF1Utrb_yoW5CFWrpr WIqF95Ar4YyrZxZry7Ja4Duay5Zwn2gFyqyry3C3yfJ3Z3trsrGF1Yva4UAFWkGa4fXr1U ta15KFW3uFy8Kr7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPI14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E 14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_GFv_WrylIx kGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUCVW8JwCI42IY6xIIjxv20xvEc7CjxVAF wI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Jr 0_Gr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBIdaVFxhVjvjDU0xZFpf9x0JUQ SdkUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789480445179609552 X-GMAIL-MSGID: 1789480445179609552 From: Yu Kuai The new heleprs will be used in dm-raid in later patches to fix regressions and prevent calling md_reap_sync_thread() directly. Signed-off-by: Yu Kuai --- drivers/md/md.c | 33 +++++++++++++++++++++++++++++++-- drivers/md/md.h | 3 +++ 2 files changed, 34 insertions(+), 2 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 6c5d0a372927..6a76bd27e381 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -4915,16 +4915,45 @@ static void stop_sync_thread(struct mddev *mddev, bool locked, bool check_seq) mddev_lock_nointr(mddev); } +void md_idle_sync_thread(struct mddev *mddev) +{ + lockdep_assert_held(&mddev->reconfig_mutex); + + clear_bit(MD_RECOVERY_FROZEN, &mddev->recovery); + stop_sync_thread(mddev, true, true); +} +EXPORT_SYMBOL_GPL(md_idle_sync_thread); + +void md_frozen_sync_thread(struct mddev *mddev) +{ + lockdep_assert_held(&mddev->reconfig_mutex); + + set_bit(MD_RECOVERY_FROZEN, &mddev->recovery); + stop_sync_thread(mddev, true, false); +} +EXPORT_SYMBOL_GPL(md_frozen_sync_thread); + +void md_unfrozen_sync_thread(struct mddev *mddev) +{ + lockdep_assert_held(&mddev->reconfig_mutex); + + clear_bit(MD_RECOVERY_FROZEN, &mddev->recovery); + set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); + md_wakeup_thread(mddev->thread); + sysfs_notify_dirent_safe(mddev->sysfs_action); +} +EXPORT_SYMBOL_GPL(md_unfrozen_sync_thread); + static void idle_sync_thread(struct mddev *mddev) { mutex_lock(&mddev->sync_mutex); - clear_bit(MD_RECOVERY_FROZEN, &mddev->recovery); if (mddev_lock(mddev)) { mutex_unlock(&mddev->sync_mutex); return; } + clear_bit(MD_RECOVERY_FROZEN, &mddev->recovery); stop_sync_thread(mddev, false, true); mutex_unlock(&mddev->sync_mutex); } @@ -4932,13 +4961,13 @@ static void idle_sync_thread(struct mddev *mddev) static void frozen_sync_thread(struct mddev *mddev) { mutex_lock(&mddev->sync_mutex); - set_bit(MD_RECOVERY_FROZEN, &mddev->recovery); if (mddev_lock(mddev)) { mutex_unlock(&mddev->sync_mutex); return; } + set_bit(MD_RECOVERY_FROZEN, &mddev->recovery); stop_sync_thread(mddev, false, false); mutex_unlock(&mddev->sync_mutex); } diff --git a/drivers/md/md.h b/drivers/md/md.h index 8d881cc59799..437ab70ce79b 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -781,6 +781,9 @@ extern void md_rdev_clear(struct md_rdev *rdev); extern void md_handle_request(struct mddev *mddev, struct bio *bio); extern int mddev_suspend(struct mddev *mddev, bool interruptible); extern void mddev_resume(struct mddev *mddev); +extern void md_idle_sync_thread(struct mddev *mddev); +extern void md_frozen_sync_thread(struct mddev *mddev); +extern void md_unfrozen_sync_thread(struct mddev *mddev); extern void md_reload_sb(struct mddev *mddev, int raid_disk); extern void md_update_sb(struct mddev *mddev, int force); From patchwork Tue Jan 30 02:18:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 193812 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp961497dyb; Mon, 29 Jan 2024 18:41:19 -0800 (PST) X-Google-Smtp-Source: AGHT+IG3iLwZg6hVtAHTPBDxWX2Vc3yA1LYH5mFnGgsJVaq2YwpmNwo7UxnhG6hcTm8Y8aSfgD4U X-Received: by 2002:a5b:ed0:0:b0:dc3:5468:b385 with SMTP id a16-20020a5b0ed0000000b00dc35468b385mr3610684ybs.95.1706582478871; Mon, 29 Jan 2024 18:41:18 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706582478; cv=pass; d=google.com; s=arc-20160816; b=RP8GWT6ZzTO0f+r+jqyq3BINMRdHSJRhn8zE7EwK1pTJT7fa2Ki7I+K3hkdaCOZ6mx /hkST+Lu+4ph0TFHbKz0+Ktipu14G/HOnL6e+XzAAhVo0r5Qpl1+pK+DREEvb8537qkD rO5F8DtsbaKR2qxgOsKUL7CYLNgG8XkhhXreBZn4PkamTD5CkiBPR2HsZlIueHQM17Eb b7Bve8JIrzqjIxDX2jCfaj+f+zmsW9ltTI+oqXhwyipcx1CxSsfgpxp2GTOzkrr0HmPM 01/DT0FfFOlqPa1XznVXu/Bvsh+W/0y6PHutioBKikqJ74qg9Bdh+mm989WMh+e85Qah 5nwg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=kQ7iVhVb18spXoVAwrqbgq++EMzPS7foKEKx8QVjwOM=; fh=FtirvJJ13L8gehLQ375a0lCTkAroffAi0JfnF40Fa78=; b=RjRRVgIZ37MprPFIB+Tns0V9N0MCs48xmoq0Qapw9ITH/bPeD53L6WBWVCxuFhIP7h 1ZYMv//aDI2oDfc5dzGetOJDcK8DFwwrl1oEBtMZ9m5d5jZIecVJljo2pZIA3xB1rl1j Xsd2bNIDOG6f4NNKv5oaxX920vDczQ+Rrt5p8kPWwujE2+1w5btNnXg3k3K4TzJkjZjM i6qymtQ9EgcBAAC1V/iANRdg2VDgMBDsPYVClQLo+SN5r6G1mcqEB2V3+WlQ+foaQ1zA ZeQezTWP/RDqWJVgqVnp9uijDaxrXfq+jA2lQtHX7MaTio0nR+x2Y57Tk/M/wvoDM6tH 51mw== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43811-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43811-ouuuleilei=gmail.com@vger.kernel.org" Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id b34-20020a630c22000000b005ca4098bf66si6602128pgl.637.2024.01.29.18.41.18 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 18:41:18 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43811-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43811-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43811-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 7254AB214C3 for ; Tue, 30 Jan 2024 02:26:16 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9DBE9383A9; Tue, 30 Jan 2024 02:23:13 +0000 (UTC) Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2CA9F374EE; Tue, 30 Jan 2024 02:23:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581390; cv=none; b=FqY9Qu5CPyYiGFXWLRh8e/zbApRfdnpY0pAjesQPEPk2zgKA6I6/cy/TjoPsDWNx2nff5LV3Z+rCGCM0pZHJxykrHwjpPlB8gSfsbJfGSVwR6yg+eYNfC8/BpyFmKeTZ/DqlQbDQ3LyMJmyKaxBmkRpGdPdHwFSjPAHkmCA9Rjs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581390; c=relaxed/simple; bh=6BrWeORSG3spzobIm2H1STSTTDZ6J8PBa6b1+irB9U4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=gcA0vQOcWjtVmfDPhatRBIk2mGVm7JK9DER45NEGKpZHnwPCifDYMHbdfuL6NpHGlv0Jf3n1ftHdj9jhfq2om6zUxWQ8jqOScNUnR9/rOzELHqMIqixsnS16cvsC+baas52CVZEzeHRwzS83m3qU1O+qpQG4aC6oXqaBqGGl/Ts= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4TP88k5lLGz4f3kKT; Tue, 30 Jan 2024 10:23:02 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 503351A0199; Tue, 30 Jan 2024 10:23:05 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgBHGBGBXbhlrAigCQ--.55484S10; Tue, 30 Jan 2024 10:23:05 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v4 06/14] dm-raid: really frozen sync_thread during suspend Date: Tue, 30 Jan 2024 10:18:35 +0800 Message-Id: <20240130021843.3608859-7-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240130021843.3608859-1-yukuai1@huaweicloud.com> References: <20240130021843.3608859-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgBHGBGBXbhlrAigCQ--.55484S10 X-Coremail-Antispam: 1UD129KBjvJXoWxur1xXF4fWry8tryDtFW3trb_yoWrKw15pa yrtws0vw48JrW7Za9Fy3WkXFWYvwnIgrWUtr93WayrJ3WSkws3ury8Kw47ZFWDtFyxJa4Y yr4Dtw45uFWjgFJanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPI14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E 14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_GFv_WrylIx kGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUCVW8JwCI42IY6xIIjxv20xvEc7CjxVAF wI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Jr 0_Gr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBIdaVFxhVjvjDU0xZFpf9x0JUQ SdkUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789481429261319978 X-GMAIL-MSGID: 1789481429261319978 From: Yu Kuai 1) The flag MD_RECOVERY_FROZEN doesn't mean that sync thread is frozen, it only prevent new sync_thread to start, and it can't stop the running sync thread; 2) The flag MD_RECOVERY_FROZEN doesn't mean that writes are stopped, use it as condition for md_stop_writes() in raid_postsuspend() doesn't look correct. 3) raid_message can set/clear the flag MD_RECOVERY_FROZEN at anytime, and if MD_RECOVERY_FROZEN is cleared while the array is suspended, new sync_thread can start unexpected. Fix above problems by using the new helper to suspend the array during suspend, also disallow raid_message() to change sync_thread status during suspend. Note that after commit f52f5c71f3d4 ("md: fix stopping sync thread"), the test shell/lvconvert-raid-reshape.sh start to hang in stop_sync_thread(), and with previous fixes, the test won't hang there anymore, however, the test will still fail and complain that ext4 is corrupted. And with this patch, the test won't hang due to stop_sync_thread() or fail due to ext4 is corrupted anymore. However, there is still a deadlock related to dm-raid456 that will be fixed in following patches. Reported-by: Mikulas Patocka Closes: https://lore.kernel.org/all/e5e8afe2-e9a8-49a2-5ab0-958d4065c55e@redhat.com/ Fixes: 1af2048a3e87 ("dm raid: fix deadlock caused by premature md_stop_writes()") Fixes: 9dbd1aa3a81c ("dm raid: add reshaping support to the target") Fixes: f52f5c71f3d4 ("md: fix stopping sync thread") Signed-off-by: Yu Kuai --- drivers/md/dm-raid.c | 38 +++++++++++++++++++++++++++++--------- 1 file changed, 29 insertions(+), 9 deletions(-) diff --git a/drivers/md/dm-raid.c b/drivers/md/dm-raid.c index eb009d6bb03a..5ce3c6020b1b 100644 --- a/drivers/md/dm-raid.c +++ b/drivers/md/dm-raid.c @@ -3240,11 +3240,12 @@ static int raid_ctr(struct dm_target *ti, unsigned int argc, char **argv) rs->md.ro = 1; rs->md.in_sync = 1; - /* Keep array frozen until resume. */ - set_bit(MD_RECOVERY_FROZEN, &rs->md.recovery); - /* Has to be held on running the array */ mddev_suspend_and_lock_nointr(&rs->md); + + /* Keep array frozen until resume. */ + md_frozen_sync_thread(&rs->md); + r = md_run(&rs->md); rs->md.in_sync = 0; /* Assume already marked dirty */ if (r) { @@ -3722,6 +3723,9 @@ static int raid_message(struct dm_target *ti, unsigned int argc, char **argv, if (!mddev->pers || !mddev->pers->sync_request) return -EINVAL; + if (test_bit(RT_FLAG_RS_SUSPENDED, &rs->runtime_flags)) + return -EBUSY; + if (!strcasecmp(argv[0], "frozen")) set_bit(MD_RECOVERY_FROZEN, &mddev->recovery); else @@ -3791,15 +3795,31 @@ static void raid_io_hints(struct dm_target *ti, struct queue_limits *limits) blk_limits_io_opt(limits, chunk_size_bytes * mddev_data_stripes(rs)); } +static void raid_presuspend(struct dm_target *ti) +{ + struct raid_set *rs = ti->private; + + mddev_lock_nointr(&rs->md); + md_frozen_sync_thread(&rs->md); + mddev_unlock(&rs->md); +} + +static void raid_presuspend_undo(struct dm_target *ti) +{ + struct raid_set *rs = ti->private; + + mddev_lock_nointr(&rs->md); + md_unfrozen_sync_thread(&rs->md); + mddev_unlock(&rs->md); +} + static void raid_postsuspend(struct dm_target *ti) { struct raid_set *rs = ti->private; if (!test_and_set_bit(RT_FLAG_RS_SUSPENDED, &rs->runtime_flags)) { /* Writes have to be stopped before suspending to avoid deadlocks. */ - if (!test_bit(MD_RECOVERY_FROZEN, &rs->md.recovery)) - md_stop_writes(&rs->md); - + md_stop_writes(&rs->md); mddev_suspend(&rs->md, false); } } @@ -4012,8 +4032,6 @@ static int raid_preresume(struct dm_target *ti) } /* Check for any resize/reshape on @rs and adjust/initiate */ - /* Be prepared for mddev_resume() in raid_resume() */ - set_bit(MD_RECOVERY_FROZEN, &mddev->recovery); if (mddev->recovery_cp && mddev->recovery_cp < MaxSector) { set_bit(MD_RECOVERY_REQUESTED, &mddev->recovery); mddev->resync_min = mddev->recovery_cp; @@ -4056,9 +4074,9 @@ static void raid_resume(struct dm_target *ti) rs_set_capacity(rs); mddev_lock_nointr(mddev); - clear_bit(MD_RECOVERY_FROZEN, &mddev->recovery); mddev->ro = 0; mddev->in_sync = 0; + md_unfrozen_sync_thread(mddev); mddev_unlock_and_resume(mddev); } } @@ -4074,6 +4092,8 @@ static struct target_type raid_target = { .message = raid_message, .iterate_devices = raid_iterate_devices, .io_hints = raid_io_hints, + .presuspend = raid_presuspend, + .presuspend_undo = raid_presuspend_undo, .postsuspend = raid_postsuspend, .preresume = raid_preresume, .resume = raid_resume, From patchwork Tue Jan 30 02:18:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 193804 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp957950dyb; Mon, 29 Jan 2024 18:28:44 -0800 (PST) X-Google-Smtp-Source: AGHT+IE3JKJj7LBuJ0InKieu2vxczq3A5yEibKfaIswPi5mEV2saStGNN4bO2dmZmoJ3S3yqKBrP X-Received: by 2002:a05:6830:1253:b0:6dd:e1b0:443e with SMTP id s19-20020a056830125300b006dde1b0443emr7212793otp.5.1706581724701; Mon, 29 Jan 2024 18:28:44 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706581724; cv=pass; d=google.com; s=arc-20160816; b=kW2QOsC4omo676pSuxrEu3f3SfOSRaPKpY3/r+T9TXrRKU19auF/Hg13aZEVdRvYjg eNWZ/U1PeOstBTF1k5UvtR+T9NIpPEGGKh2RBJF8Ed7M7NOJDaxmtrRRON3nbKvznJdS irlWgLVhrFqszDZ3n6xg0SnmZ2q9X13ke1QUZRpTcanYhea5L7kxYi5VRKGTYp8+yK+1 PZ5sEF2ZJ6LKPs8CxevHcC+qXZLR9znP+ad3eoFPtWoxlM3hWjKe8xA+d50c5ycvJYK1 3hk4CflVx9XmpXzL2R5BAA8dbB6fvRcCjAh+e3n/Ko1dp/drIOLI4MVSUPcJUbCurlZg eNsg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=0h6sLTtSXWd4NNSNFaiFXDbIJAObc1SgGd1OcMYigCo=; fh=FtirvJJ13L8gehLQ375a0lCTkAroffAi0JfnF40Fa78=; b=z3YfAeQdsbTBEEP6/TfYHbro5JXmIsagxpIO0pJWTeiY0c3DcdIrBPxrJgADHeKIFr Xsvfo98QlJcTBnZnzxmx97qdX53H6NB15tQxDgtbYdgBP3eHhd81NEOK3xzdcXBwVJPE anLKce39uEnyFInxgRicnGczcxtLktefUX46XWeNXCVStO99+zEAPq3UHMyj6V8iOW96 zvHH7T8W+oZBQ1tbZhubPd0ZhXfaDs0Ps6bd7fwKSJqo161uP/DeZt1XSB2OK8DhcqOU YWpCYIhoCMQKzJ/Cg113UD+OjLz4jm6DvsLSAj5TgtfvsGbx9kasRjeSvmjG1nMnITRP fgWw== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43812-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43812-ouuuleilei=gmail.com@vger.kernel.org" Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id g191-20020a636bc8000000b005c5fe04e4eesi443364pgc.852.2024.01.29.18.28.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 18:28:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43812-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43812-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43812-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 34ED328B1D6 for ; Tue, 30 Jan 2024 02:26:14 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 991286F074; Tue, 30 Jan 2024 02:23:13 +0000 (UTC) Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B5EE7376F9; Tue, 30 Jan 2024 02:23:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581390; cv=none; b=oxKr8Utuu13Zk2lfx8zidpb07SjTmSVGGpuGqVTKynzzLl+N4kh181JrlFA60da43aB5XxJbjvWEoqyZH454NXz48KyjPFn9yxn2n9n9SjW3OKH+AGAgkQXdzbfC5q4YEpvhVkbNsCkw9KalSvSa5+va5x+ptAXPHBY4+/wvcq4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581390; c=relaxed/simple; bh=orTUf8OIgVO5EnJ8e56Og6bXoRyUdV4mYFLO+shF788=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KK9FEYLY5N/psPDfG5cGruptqJMmrLE863V6maW6hvcwqdtMTnOiPE4vUr0TW5XfiwZ8v/92T/XG3srs9HH+c/FVKZKZONiedglcdS/dCg0VUGJgPZNh/nsiD7rG8PhbyD9nwQ9nWKKsrwOcLn0D/8oajXJYWVvERejNvEPa+YU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4TP88g3JyZz4f3m6n; Tue, 30 Jan 2024 10:22:59 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id E45671A0199; Tue, 30 Jan 2024 10:23:05 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgBHGBGBXbhlrAigCQ--.55484S11; Tue, 30 Jan 2024 10:23:05 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v4 07/14] md/dm-raid: don't call md_reap_sync_thread() directly Date: Tue, 30 Jan 2024 10:18:36 +0800 Message-Id: <20240130021843.3608859-8-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240130021843.3608859-1-yukuai1@huaweicloud.com> References: <20240130021843.3608859-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgBHGBGBXbhlrAigCQ--.55484S11 X-Coremail-Antispam: 1UD129KBjvJXoW7AF1xAF18KFyDuFy5Ww48JFb_yoW8Kw4fpa yakas8Ar48JrW3ZFsrt3WDWayFv3ZFq3yqyFWfG3y3JF1fKF43WF1j9a17ZFWDJFWfta4U XFZ8tF45uF4YqFJanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPI14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E 14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_GFv_WrylIx kGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUCVW8JwCI42IY6xIIjxv20xvEc7CjxVAF wI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Jr 0_Gr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBIdaVFxhVjvjDU0xZFpf9x0JUQ SdkUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789480638518153016 X-GMAIL-MSGID: 1789480638518153016 From: Yu Kuai Currently md_reap_sync_thread() is called from raid_message() directly without holding 'reconfig_mutex', this is definitely unsafe because md_reap_sync_thread() can change many fields that is protected by 'reconfig_mutex'. However, hold 'reconfig_mutex' here is still problematic because this will cause deadlock, for example, commit 130443d60b1b ("md: refactor idle/frozen_sync_thread() to fix deadlock"). Fix this problem by using stop_sync_thread() to unregister sync_thread, like md/raid did. Fixes: be83651f0050 ("DM RAID: Add message/status support for changing sync action") Signed-off-by: Yu Kuai --- drivers/md/dm-raid.c | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/drivers/md/dm-raid.c b/drivers/md/dm-raid.c index 5ce3c6020b1b..6b6c011d9f69 100644 --- a/drivers/md/dm-raid.c +++ b/drivers/md/dm-raid.c @@ -3719,6 +3719,7 @@ static int raid_message(struct dm_target *ti, unsigned int argc, char **argv, { struct raid_set *rs = ti->private; struct mddev *mddev = &rs->md; + int ret = 0; if (!mddev->pers || !mddev->pers->sync_request) return -EINVAL; @@ -3726,17 +3727,24 @@ static int raid_message(struct dm_target *ti, unsigned int argc, char **argv, if (test_bit(RT_FLAG_RS_SUSPENDED, &rs->runtime_flags)) return -EBUSY; - if (!strcasecmp(argv[0], "frozen")) - set_bit(MD_RECOVERY_FROZEN, &mddev->recovery); - else - clear_bit(MD_RECOVERY_FROZEN, &mddev->recovery); + if (!strcasecmp(argv[0], "frozen")) { + ret = mddev_lock(mddev); + if (ret) + return ret; - if (!strcasecmp(argv[0], "idle") || !strcasecmp(argv[0], "frozen")) { - if (mddev->sync_thread) { - set_bit(MD_RECOVERY_INTR, &mddev->recovery); - md_reap_sync_thread(mddev); - } - } else if (decipher_sync_action(mddev, mddev->recovery) != st_idle) + md_frozen_sync_thread(mddev); + mddev_unlock(mddev); + } else if (!strcasecmp(argv[0], "idle")) { + ret = mddev_lock(mddev); + if (ret) + return ret; + + md_idle_sync_thread(mddev); + mddev_unlock(mddev); + } + + clear_bit(MD_RECOVERY_FROZEN, &mddev->recovery); + if (decipher_sync_action(mddev, mddev->recovery) != st_idle) return -EBUSY; else if (!strcasecmp(argv[0], "resync")) ; /* MD_RECOVERY_NEEDED set below */ From patchwork Tue Jan 30 02:18:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 193805 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp957998dyb; Mon, 29 Jan 2024 18:28:56 -0800 (PST) X-Google-Smtp-Source: AGHT+IHRT2TETzUDVIHY0LBIS1+kn49uExdOZcDTYrefXmMbQn4biUfav99pQLJHwY7DooFSXks1 X-Received: by 2002:a05:6358:5919:b0:178:75cb:18dc with SMTP id g25-20020a056358591900b0017875cb18dcmr2787163rwf.10.1706581736060; Mon, 29 Jan 2024 18:28:56 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706581736; cv=pass; d=google.com; s=arc-20160816; b=nD3eeMNI8Z3ARqRYNrKEN5RjN3TODzEStVosDg/lBlsyxpbLxXQIWXYhzjP/KWi24k jv23BiSKnBdt77K4tKADtXyKHc7ep9c03GMat0FnM1MT/6tE9R9ZNR+p5WavtMkl2eUe oSLqiXowF06ANgWrlVYhCbPuSdJ6p/UtF4PaCUqe+IStF+Qfo8Q1NxhX9cFD19AONvdr CzFj17gLHlgFFrbe3b56z7HJF912vckllk+4OQ/sBdjswc2tCvq1sipXzFvNvhaEx4RQ eSh7TQoH9otX8Ywy2hCdJviV8ofO0HgQ7RhzskscNuLpgByMfQ83lqdL0zA6RIz734mq 0V/g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=CXVSsE0dlAUeg6iRbCAj98YQq857rZIrgf0DFbaRC6E=; fh=FtirvJJ13L8gehLQ375a0lCTkAroffAi0JfnF40Fa78=; b=pKHhqB32+CzgnatkdaFoqqGYYwfkJIQ3zUwVoZQMhTObyPrKHICudohgWOfDC80kJU gp0FZH9iXeEtG+IXC1e+QKNjAaiU11rM7bCSA7zarsLJ7GkV05Q51IzzkuK54UAKpu3P Ss/n8ncwZT9kbTMcaRmK3le2k0g3iUOoT80ZBT2VAiRPzQzn13Ujy2bHcs1rqYGyEvVV wqB2Vz7ZCXanHl+uEaAUKBxoh2DAuohBDAqn0yBueKCiRTGzpGPe44dYJwyAZTOIDqiO XfsPLafYzMy8YUuxZUR0EHKVHq0G2QZnTh6U37BdFoiBQCLlC9dat+42Ng+/tfW7mS1j DOxA== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43813-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43813-ouuuleilei=gmail.com@vger.kernel.org" Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id f27-20020a63755b000000b005cdfb84720fsi4921624pgn.857.2024.01.29.18.28.55 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 18:28:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43813-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43813-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43813-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id CCBB728B580 for ; Tue, 30 Jan 2024 02:26:31 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 224C2158D9B; Tue, 30 Jan 2024 02:23:14 +0000 (UTC) Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A9B5381A0; Tue, 30 Jan 2024 02:23:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581391; cv=none; b=gCSj+vP7bNo+oY8bgK99MPp/VQwqS/6vultIq1ROfYBbBCauMakFXc4NL8n+E0CPBcteLTxiAvp6ivklq3lKCRdIqTkdMnWXJqmDoEgxmkbU1o4Nvt9QBbqUeZMfu7vzeW+KopPggISRyEqepyqkXcAbtpd0V1o9fWqY0g0UNR4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581391; c=relaxed/simple; bh=yeO1LGdlv/HUZdhkshDDNxzEa5t2WUEBZmRVtUU3LHw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KavS/Pi6lcX0IIcTiJMZ9F4KBw2CGOP+PywrR1Z6EsQr0tSZZpZNa3dihh41edvmItSdMaB+FDWDGKv+rXkooPYdgCgxhrabxGNRucj6V9g+mwi91rYYuIxJGgVfWQwoSAg2NryVgiLjeuD3UvygAEEQgxt41OSuBceexnC9aw0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4TP88h0c62z4f3m7f; Tue, 30 Jan 2024 10:23:00 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 899731A0199; Tue, 30 Jan 2024 10:23:06 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgBHGBGBXbhlrAigCQ--.55484S12; Tue, 30 Jan 2024 10:23:06 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v4 08/14] dm-raid: add a new helper prepare_suspend() in md_personality Date: Tue, 30 Jan 2024 10:18:37 +0800 Message-Id: <20240130021843.3608859-9-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240130021843.3608859-1-yukuai1@huaweicloud.com> References: <20240130021843.3608859-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgBHGBGBXbhlrAigCQ--.55484S12 X-Coremail-Antispam: 1UD129KBjvJXoW7Kr47Kr4xtrykGw13uF4fGrg_yoW8Wr48pa yIqay5Ar4UJayavw1DXr4kZa4aq3ZIgrWqyryfJayfZa42grn5W3WFqayDZrZ0yFy3CF13 Aa1Utw4kuFy09rDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPI14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E 14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_GFv_WrylIx kGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUCVW8JwCI42IY6xIIjxv20xvEc7CjxVAF wI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Jr 0_Gr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBIdaVFxhVjvjDU0xZFpf9x0JUQ SdkUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789480650750718683 X-GMAIL-MSGID: 1789480650750718683 From: Yu Kuai There are no functional changes for now, prepare to fix a deadlock for dm-raid456. Signed-off-by: Yu Kuai --- drivers/md/dm-raid.c | 10 +++++++--- drivers/md/md.h | 1 + 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers/md/dm-raid.c b/drivers/md/dm-raid.c index 6b6c011d9f69..16d3348abfed 100644 --- a/drivers/md/dm-raid.c +++ b/drivers/md/dm-raid.c @@ -3806,10 +3806,14 @@ static void raid_io_hints(struct dm_target *ti, struct queue_limits *limits) static void raid_presuspend(struct dm_target *ti) { struct raid_set *rs = ti->private; + struct mddev *mddev = &rs->md; - mddev_lock_nointr(&rs->md); - md_frozen_sync_thread(&rs->md); - mddev_unlock(&rs->md); + mddev_lock_nointr(mddev); + md_frozen_sync_thread(mddev); + mddev_unlock(mddev); + + if (mddev->pers && mddev->pers->prepare_suspend) + mddev->pers->prepare_suspend(mddev); } static void raid_presuspend_undo(struct dm_target *ti) diff --git a/drivers/md/md.h b/drivers/md/md.h index 437ab70ce79b..29b476ff3b9f 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -617,6 +617,7 @@ struct md_personality int (*start_reshape) (struct mddev *mddev); void (*finish_reshape) (struct mddev *mddev); void (*update_reshape_pos) (struct mddev *mddev); + void (*prepare_suspend) (struct mddev *mddev); /* quiesce suspends or resumes internal processing. * 1 - stop new actions and wait for action io to complete * 0 - return to normal behaviour From patchwork Tue Jan 30 02:18:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 193806 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp958066dyb; Mon, 29 Jan 2024 18:29:10 -0800 (PST) X-Google-Smtp-Source: AGHT+IFkHm9S0CScGuwibKc6mHv0swWk6Swai/O8sE0K/IXpBRuyvUQllRlkhKHlmYoWTLE97uGX X-Received: by 2002:a17:902:bd47:b0:1d7:5edd:afb with SMTP id b7-20020a170902bd4700b001d75edd0afbmr3565491plx.38.1706581750521; Mon, 29 Jan 2024 18:29:10 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706581750; cv=pass; d=google.com; s=arc-20160816; b=Vmnc++FzN9QLs2OXfUTucGqotmhcwAhez+ZTJUnwJ4MvSziB31ExafYvyyF/hiQE4n aX++XMTMoE+/Uor/ppBQrKvz1fgdQp21NZuOYRnYtqdFp5sjheMG8IzLxx/kj/Qu2kY/ q2xJN1N6EmR5Lzo8Jx9XZcJ1ALFI55ufd9vODuKP+WIe+AIXjtwZt5xNhyU1HBt0EZnc gaMyVsmqOFg2SZ/pQ18F2x+JB3YXUPz3PILWZslSXyrZGMKaebjoD2TjPfsuwZ4lwHfo EIIVfGyKYex0e3nuPsszXwuFb7PLdnTsqw1KisNy9kNHV8yl15xNO+NYwtbyvgQJSbee 6svA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=lHGrIVsDNKxSCOG2cn14hvyBCmQfTCudAexaAQ0XYAc=; fh=FtirvJJ13L8gehLQ375a0lCTkAroffAi0JfnF40Fa78=; b=oZyMbJGtbPHyY0dbqdSjV/T6qVs42WEBP00OvkTmz8Y8fT/Ct0TNMvC+2/BhI92nYE aQpvUOp1kB0ZwsjbsJgaqSfYalU9tbQ1wGoezAT7zUa1XEiz7MgZbMKRP7CjrcckraGN w2zEAXeyD6WIYo6IgOGpKi4Z4QiT/neiivdYWhf5vz5WQz21GD53dHRi3ia7fnndPBur bhYwKNeCaNOnEBCVX6X5DtljCJZJjio4qI2A/d83fS47fz1Ti7sJqzgqT4xxJw5eNXwE ZdzSCliZaaE4PYdutORYmXAidgWVmO3yjaLwKYrs4awQJGcjgCtL5ZF9yiEdFXI2Havl +ZYg== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43814-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43814-ouuuleilei=gmail.com@vger.kernel.org" Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id m2-20020a170902c44200b001d8eaf6626esi2087036plm.440.2024.01.29.18.29.10 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 18:29:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43814-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43814-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43814-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id E2A34285275 for ; Tue, 30 Jan 2024 02:26:53 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B0E6215A4B1; Tue, 30 Jan 2024 02:23:14 +0000 (UTC) Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D5B34381D4; Tue, 30 Jan 2024 02:23:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581392; cv=none; b=sk6fXb9jCNzTsS4AIEljBOCo39ytQrKtVGFnKWVmJzUYa8rdmmWjxbKjVw5VtynvJlNEbLuUJ//zVlVy1PZsZtfuMZlx0jQszqJxwx6THyxL+S9y5iUcIst5O9UkJV/trpsI8LcuEPr0Uyf3hRrfxwoeTR3lLosByblcys2PJk0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581392; c=relaxed/simple; bh=uMmvtP4Kk96xFStFUNR5RYqJMKlOONO/eRXGKcqJhr0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=oi9+wRC571OQsfs/n2B3L6jpq4FiGM1KY9dcv3dVP6erPTD2CZKdx2A1wnJVi3baDrMB30yc6drNOuxOiNBFf8S7oh83RTYYeKndOFzfrsBoDZFYkSkNZk3cc9kX0xp1bGpP0zOkAHLMZyuzWTdAIJsEPMrj8XPbyDK5gxEVJVI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4TP88l0Y9Jz4f3lwn; Tue, 30 Jan 2024 10:23:03 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 314AC1A0199; Tue, 30 Jan 2024 10:23:07 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgBHGBGBXbhlrAigCQ--.55484S13; Tue, 30 Jan 2024 10:23:06 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v4 09/14] md: export helper md_is_rdwr() Date: Tue, 30 Jan 2024 10:18:38 +0800 Message-Id: <20240130021843.3608859-10-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240130021843.3608859-1-yukuai1@huaweicloud.com> References: <20240130021843.3608859-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgBHGBGBXbhlrAigCQ--.55484S13 X-Coremail-Antispam: 1UD129KBjvJXoW7Kr47Kr4xtrykGw13uF4fGrg_yoW8XFyfpa 9agFy3ArW5ZF43X3yUJayDua45Z3WfKrWqkry3Z3ykXa43Jrs8CF4rGFWDJrWDWFyfAF1a ga15JF1UuF10ywUanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPI14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E 14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_GFv_WrylIx kGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUCVW8JwCI42IY6xIIjxv20xvEc7CjxVAF wI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Gr 0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBIdaVFxhVjvjDU0xZFpf9x0JUQ SdkUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789480665423073595 X-GMAIL-MSGID: 1789480665423073595 From: Yu Kuai There are no functional changes for now, prepare to fix a deadlock for dm-raid456. Signed-off-by: Yu Kuai --- drivers/md/md.c | 12 ------------ drivers/md/md.h | 12 ++++++++++++ 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 6a76bd27e381..b942cc126ce0 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -99,18 +99,6 @@ static void mddev_detach(struct mddev *mddev); static void export_rdev(struct md_rdev *rdev, struct mddev *mddev); static void md_wakeup_thread_directly(struct md_thread __rcu *thread); -enum md_ro_state { - MD_RDWR, - MD_RDONLY, - MD_AUTO_READ, - MD_MAX_STATE -}; - -static bool md_is_rdwr(struct mddev *mddev) -{ - return (mddev->ro == MD_RDWR); -} - /* * Default number of read corrections we'll attempt on an rdev * before ejecting it from the array. We divide the read error diff --git a/drivers/md/md.h b/drivers/md/md.h index 29b476ff3b9f..98da86d38ba8 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -558,6 +558,18 @@ enum recovery_flags { MD_RESYNCING_REMOTE, /* remote node is running resync thread */ }; +enum md_ro_state { + MD_RDWR, + MD_RDONLY, + MD_AUTO_READ, + MD_MAX_STATE +}; + +static bool md_is_rdwr(struct mddev *mddev) +{ + return (mddev->ro == MD_RDWR); +} + static inline int __must_check mddev_lock(struct mddev *mddev) { return mutex_lock_interruptible(&mddev->reconfig_mutex); From patchwork Tue Jan 30 02:18:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 193816 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp964653dyb; Mon, 29 Jan 2024 18:52:44 -0800 (PST) X-Google-Smtp-Source: AGHT+IE138v7ysX64BuNHX2uhHTKTBDwfNBTestXP+7SR+M2r4hNkpD8Vm8zr8RumCIUh5gSFVIt X-Received: by 2002:a17:907:a587:b0:a35:278:1c23 with SMTP id vs7-20020a170907a58700b00a3502781c23mr333841ejc.35.1706583164035; Mon, 29 Jan 2024 18:52:44 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706583164; cv=pass; d=google.com; s=arc-20160816; b=oWZvZ2aLK8aXb8Q7L9C8vYTcuunC0RnXX+GKjgyqtl8OXk1FoY3UxFKcBIVReMAynt yp4EpA+7Px391IX3KKxqXXLLll7CI2qhFgKME9hM9mzWXps1w3VAkt6//+U7ExPOzCqn zEyQGOFksCzyAxZU1XoLC7v60DZWYx13tYH2XoZ0S/AxIMuf7nCMO0uPA27teZeiK3UF ftsxesArC3JrDgzHDKis6RnyWUxORnZQUvdFyhqVZpQSyk8bqzMfftNawQI8SeJiTdoY QDBP2D2SnJfoueuQjQHdpenoPPL6sdGfNoDpU4ICn4nPZ3AqGdvulw15yEJo44yELXMW aTDg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=2fF99IaCNquBgHf7hcoZW47islH522Eb0h4ofbbUL7I=; fh=FtirvJJ13L8gehLQ375a0lCTkAroffAi0JfnF40Fa78=; b=MRILDMwLE3lsuIo3FKS0GqaDVCb2Q7QLA+hfG8pPqqqy48yHI09MV3zdcOLhjnul5p IThsYw9b9dkhc69m4H/ksr0OfmnhaFjXNlhXY9007K1S2QQETo8hbBA8PcBUw+oBsL4r iZ3ip/EZ5lyalPt9rI12fZuPNh8kDpGCvfB7mZt3H/jzHOsJudMO1ZTDuOfe4AAIJzoH MrY7Zg+RoSIvG4tc8I5kMUhgssXDvdnob7r7QB1c0v06V6nYiLWiCIblM1QKfrQ5B1A3 DZTuySmkYZygPXJeZwwwSIfrYNt1m+VdQZhDmxzWJVGjoW5S/rs/KjAnkVeGgC+yzYNk WLEQ== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43815-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43815-ouuuleilei=gmail.com@vger.kernel.org" Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id r14-20020a170906350e00b00a3605794828si518211eja.125.2024.01.29.18.52.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 18:52:44 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43815-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43815-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43815-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 021A01F27858 for ; Tue, 30 Jan 2024 02:27:09 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 11C7015AAB1; Tue, 30 Jan 2024 02:23:15 +0000 (UTC) Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C2A97364D2; Tue, 30 Jan 2024 02:23:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581392; cv=none; b=gFGZOjacjQDm2B7biD3RIdlZiS3JpT37ADfWqphHp43IgjTZF0n/HPzvH8ccd7uk2TRPLjhDAchtusiX7lVuEWjeL7swl26rISdzgIn8ZniMj6/THhr6h1geh5XYm2aedCzXFKT5WUWawkBCXzcJyGXN7lheDMbyjZ0S/8lqUx4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581392; c=relaxed/simple; bh=6YDQmrYqLiTMI1nBqXhhAgN5M0sQ1yDoPL9hqgOUqgk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=SgzKMzdo4I1daiBAK7NbIQCNCE15QtZSCQFZRmetZhEFEoB+1H4SyZlLw0hj3HmqtnIzmxy0KJ2MmCB3D8Pv+7NnypvkjnpZ4pwzYy9I7EpBXys+YpAiQbbHAGMKumj7H3j2Yvch2VDXFYn8j7zuVB4K2bb54d1R3txeUNEjcTM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4TP88j2RTTz4f3m6y; Tue, 30 Jan 2024 10:23:01 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id C90E31A01E9; Tue, 30 Jan 2024 10:23:07 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgBHGBGBXbhlrAigCQ--.55484S14; Tue, 30 Jan 2024 10:23:07 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v4 10/14] md: don't suspend the array for interrupted reshape Date: Tue, 30 Jan 2024 10:18:39 +0800 Message-Id: <20240130021843.3608859-11-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240130021843.3608859-1-yukuai1@huaweicloud.com> References: <20240130021843.3608859-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgBHGBGBXbhlrAigCQ--.55484S14 X-Coremail-Antispam: 1UD129KBjvJXoW7uF1fuFy3ZFWkArykCr17Jrb_yoW8ur13p3 yakF1ayrs8X39IvayUG3Z7KFyFk3s5trWYy3srG34UAw13Gr1fGr43Gr4q9FyY9ry3trs0 qr1Yq3s5GF1qkaDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPI14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E 14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_GFv_WrylIx kGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVW8JVW5JwCI42IY6xIIjxv20xvEc7CjxVAF wI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Gr 0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBIdaVFxhVjvjDU0xZFpf9x0JUQ SdkUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789482147841023690 X-GMAIL-MSGID: 1789482147841023690 From: Yu Kuai md_start_sync() will suspend the array if there are spares that can be added or removed from conf, however, if reshape is still in progress, this won't happen at all or data will be corrupted(remove_and_add_spares won't be called from md_choose_sync_action for reshape), hence there is no need to suspend the array if reshape is not done yet. Meanwhile, there is a potential deadlock for raid456: 1) reshape is interrupted; 2) set one of the disk WantReplacement, and add a new disk to the array, however, recovery won't start until the reshape is finished; 3) then issue an IO across reshpae position, this IO will wait for reshape to make progress; 4) continue to reshape, then md_start_sync() found there is a spare disk that can be added to conf, mddev_suspend() is called; Step 4 and step 3 is waiting for each other, deadlock triggered. Noted this problem is found by code review, and it's not reporduced yet. Fix this porblem by don't suspend the array for interrupted reshape, this is safe because conf won't be changed until reshape is done. Fixes: bc08041b32ab ("md: suspend array in md_start_sync() if array need reconfiguration") Signed-off-by: Yu Kuai --- drivers/md/md.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index b942cc126ce0..093abf3ce27b 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -9391,12 +9391,17 @@ static void md_start_sync(struct work_struct *ws) bool suspend = false; char *name; - if (md_spares_need_change(mddev)) + /* + * If reshape is still in progress, spares won't be added or removed + * from conf until reshape is done. + */ + if (mddev->reshape_position == MaxSector && + md_spares_need_change(mddev)) { suspend = true; + mddev_suspend(mddev, false); + } - suspend ? mddev_suspend_and_lock_nointr(mddev) : - mddev_lock_nointr(mddev); - + mddev_lock_nointr(mddev); if (!md_is_rdwr(mddev)) { /* * On a read-only array we can: From patchwork Tue Jan 30 02:18:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 193807 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp958667dyb; Mon, 29 Jan 2024 18:31:10 -0800 (PST) X-Google-Smtp-Source: AGHT+IE757NITg/Xdf2j5n025UnnscYZfh8Yrj13VaBJAeRkTrBxQ9ufAGrqN81jtZ+XMaMLnWZm X-Received: by 2002:a05:6358:7289:b0:175:d3e6:8fa0 with SMTP id w9-20020a056358728900b00175d3e68fa0mr4536266rwf.56.1706581870071; Mon, 29 Jan 2024 18:31:10 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706581870; cv=pass; d=google.com; s=arc-20160816; b=lYBlKCfYbNSOSkTgRzbI6V9S9NqXxTYAvKongdjB+W7MNUiDxmQTDAoP3jBBaocHka 8t6h0mluja+TmNmjhiCQYcz4UShRQmFg0P0eQB75oFhrCvM9YI9z9yyUiMTMAEPnnEEI ttrJ17DZMsHOtGf/t/O5WLLRlX/f1SD9sKlC1Dmbc3k4nrFLLCw2jcahRjUeFZwriEMW TVpGaF+2ihxRf9qOQhh5hwKuRnzubDZjfa02hA0Mw9qCEtIbwa1jYHN84ckcRZ92I4L7 FSbxO2hxlU6soiiAk/wp1ItFB4HJruqRY2vy18uoJnFXObLdJZx1BExvMxvUAX+d0BMA QYzg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=8cOrC8fJEWkvOq7Px5chaZHy82aZGYOqpWE+lwxGTiU=; fh=FtirvJJ13L8gehLQ375a0lCTkAroffAi0JfnF40Fa78=; b=nsOoU6ZOomKQu9i83V2w5BPXwFcnMNmex8d45FJ+U2wwwDfmfDyIfGSP8nimfoktsz Sp76fpYDwDGAyx6K/+9Iaj5eOupp8Rorbf7UCO0VJBrOpvmKv+DFtnUVEiI3jevSJ1SZ waUXO0+rsHALK7UuqpzDrOkio0pRDE5pZ/tFtQ4EC9XJSLf+uE4zj0vmbktB1k50IBXa 5dRdMl2z2sY6VfUHi/CvEwXq83OgFaErwKNai1kmFWTDMMAe0RZ7oOXNxozM1jgiBHCM 9w1TwRRZCyjMZ17JElsej1SxsJrz688r3JN082Ykc0jr0i35Btk8dSPNsHktC6YbPzOZ +Vlg== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43817-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43817-ouuuleilei=gmail.com@vger.kernel.org" Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id s2-20020a632c02000000b005d795f96f31si6456695pgs.152.2024.01.29.18.31.09 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 18:31:10 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43817-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43817-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43817-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id CA6BD285723 for ; Tue, 30 Jan 2024 02:27:50 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 970AB38F80; Tue, 30 Jan 2024 02:23:16 +0000 (UTC) Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BB4BF45006; Tue, 30 Jan 2024 02:23:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581394; cv=none; b=lA759xshUPp18Axgz6yUZvdpIzu3uYmIsh64fr13JFuv+qTTyrQWLjMjNVLbYVNz0smYpBrV5Kf3V/7ox2GGKHhRmTUBmw/dAnDHLOcEnHOMRlRcarud21hiyudmAiwnJPubxoNmTv79v0HDZkpujg+OZOhgOs+3KvNkeF3PXKM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581394; c=relaxed/simple; bh=Y2CyAULl8CDT1e354bpgbi56zPjrKUGKSSoo0xNiEag=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ZFAOdsgXvzeefK2FVJjnlOG4A3kWylK0EKCXizaRWRfp+92AzXlxMDmFKwbeqIkCX5syTJ0OVrfHBaQYRN1ZLatcWgt+MK/dRh3NQdpGX/HHcn+co4A0pmvirzF7SFLRYPnETaoQuoqlCFdB8IpM9mOEe/hYBLCxC9TXa+HDBws= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4TP88n6Xy6z4f3kKV; Tue, 30 Jan 2024 10:23:05 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 6D3C11A0232; Tue, 30 Jan 2024 10:23:08 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgBHGBGBXbhlrAigCQ--.55484S15; Tue, 30 Jan 2024 10:23:08 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v4 11/14] md/raid456: fix a deadlock for dm-raid456 while io concurrent with reshape Date: Tue, 30 Jan 2024 10:18:40 +0800 Message-Id: <20240130021843.3608859-12-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240130021843.3608859-1-yukuai1@huaweicloud.com> References: <20240130021843.3608859-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgBHGBGBXbhlrAigCQ--.55484S15 X-Coremail-Antispam: 1UD129KBjvJXoW3Gr13WF4xJry7XF4UXw13Jwb_yoW7uF1fpa 9Fkas8Zr40v34akrWDJF4kWa4F9Fs7Kry3ZF93Cw4fZ3W3Xr1kA3W5Jay5ur98AFy8J3y5 JFW5tFyDurs8KFDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPI14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E 14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_GFv_WrylIx kGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVW8JVW5JwCI42IY6xIIjxv20xvEc7CjxVAF wI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Gr 0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBIdaVFxhVjvjDU0xZFpf9x0JUQ SdkUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789480790504980258 X-GMAIL-MSGID: 1789480790504980258 From: Yu Kuai For raid456, if reshape is still in progress, then IO across reshape position will wait for reshape to make progress. However, for dm-raid, in following cases reshape will never make progress hence IO will hang: 1) the array is read-only; 2) MD_RECOVERY_WAIT is set; 3) MD_RECOVERY_FROZEN is set; After commit c467e97f079f ("md/raid6: use valid sector values to determine if an I/O should wait on the reshape") fix the problem that IO across reshape position doesn't wait for reshape, the dm-raid test shell/lvconvert-raid-reshape.sh start to hang: [root@fedora ~]# cat /proc/979/stack [<0>] wait_woken+0x7d/0x90 [<0>] raid5_make_request+0x929/0x1d70 [raid456] [<0>] md_handle_request+0xc2/0x3b0 [md_mod] [<0>] raid_map+0x2c/0x50 [dm_raid] [<0>] __map_bio+0x251/0x380 [dm_mod] [<0>] dm_submit_bio+0x1f0/0x760 [dm_mod] [<0>] __submit_bio+0xc2/0x1c0 [<0>] submit_bio_noacct_nocheck+0x17f/0x450 [<0>] submit_bio_noacct+0x2bc/0x780 [<0>] submit_bio+0x70/0xc0 [<0>] mpage_readahead+0x169/0x1f0 [<0>] blkdev_readahead+0x18/0x30 [<0>] read_pages+0x7c/0x3b0 [<0>] page_cache_ra_unbounded+0x1ab/0x280 [<0>] force_page_cache_ra+0x9e/0x130 [<0>] page_cache_sync_ra+0x3b/0x110 [<0>] filemap_get_pages+0x143/0xa30 [<0>] filemap_read+0xdc/0x4b0 [<0>] blkdev_read_iter+0x75/0x200 [<0>] vfs_read+0x272/0x460 [<0>] ksys_read+0x7a/0x170 [<0>] __x64_sys_read+0x1c/0x30 [<0>] do_syscall_64+0xc6/0x230 [<0>] entry_SYSCALL_64_after_hwframe+0x6c/0x74 This is because reshape can't make progress. For md/raid, the problem doesn't exist because register new sync_thread doesn't rely on the IO to be done any more: 1) If array is read-only, it can switch to read-write by ioctl/sysfs; 2) md/raid never set MD_RECOVERY_WAIT; 3) If MD_RECOVERY_FROZEN is set, mddev_suspend() doesn't hold 'reconfig_mutex', hence it can be cleared and reshape can continue by sysfs api 'sync_action'. However, I'm not sure yet how to avoid the problem in dm-raid yet. This patch detect the above 3 cases in dm_suspend(), and fail those IO directly. If user really meet the IO error, then it means they're reading the wrong data before c467e97f079f. And it's safe to read/write the array after reshape make progress successfully. Signed-off-by: Yu Kuai --- drivers/md/md.h | 2 +- drivers/md/raid5.c | 32 +++++++++++++++++++++++++++++++- 2 files changed, 32 insertions(+), 2 deletions(-) diff --git a/drivers/md/md.h b/drivers/md/md.h index 98da86d38ba8..8e81f9e2fb20 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -565,7 +565,7 @@ enum md_ro_state { MD_MAX_STATE }; -static bool md_is_rdwr(struct mddev *mddev) +static inline bool md_is_rdwr(struct mddev *mddev) { return (mddev->ro == MD_RDWR); } diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 6a7a32f7fb91..812d7ec64da5 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -5915,6 +5915,13 @@ static int add_all_stripe_bios(struct r5conf *conf, return ret; } +static bool reshape_disabled(struct mddev *mddev) +{ + return !md_is_rdwr(mddev) || + test_bit(MD_RECOVERY_WAIT, &mddev->recovery) || + test_bit(MD_RECOVERY_FROZEN, &mddev->recovery); +} + static enum stripe_result make_stripe_request(struct mddev *mddev, struct r5conf *conf, struct stripe_request_ctx *ctx, sector_t logical_sector, struct bio *bi) @@ -5946,7 +5953,8 @@ static enum stripe_result make_stripe_request(struct mddev *mddev, if (ahead_of_reshape(mddev, logical_sector, conf->reshape_safe)) { spin_unlock_irq(&conf->device_lock); - return STRIPE_SCHEDULE_AND_RETRY; + ret = STRIPE_SCHEDULE_AND_RETRY; + goto out; } } spin_unlock_irq(&conf->device_lock); @@ -6025,6 +6033,13 @@ static enum stripe_result make_stripe_request(struct mddev *mddev, out_release: raid5_release_stripe(sh); +out: + if (ret == STRIPE_SCHEDULE_AND_RETRY && !mddev->gendisk && + reshape_disabled(mddev)) { + bi->bi_status = BLK_STS_IOERR; + ret = STRIPE_FAIL; + pr_err("dm-raid456: io failed across reshape position while reshape can't make progress"); + } return ret; } @@ -8909,6 +8924,18 @@ static int raid5_start(struct mddev *mddev) return r5l_start(conf->log); } +/* + * This is only used for dm-raid456, caller already frozen sync_thread, hence + * if rehsape is still in progress, io that is waiting for reshape can never be + * done now, hence wake up and handle those IO. + */ +static void raid5_prepare_suspend(struct mddev *mddev) +{ + struct r5conf *conf = mddev->private; + + wake_up(&conf->wait_for_overlap); +} + static struct md_personality raid6_personality = { .name = "raid6", @@ -8932,6 +8959,7 @@ static struct md_personality raid6_personality = .quiesce = raid5_quiesce, .takeover = raid6_takeover, .change_consistency_policy = raid5_change_consistency_policy, + .prepare_suspend = raid5_prepare_suspend, }; static struct md_personality raid5_personality = { @@ -8956,6 +8984,7 @@ static struct md_personality raid5_personality = .quiesce = raid5_quiesce, .takeover = raid5_takeover, .change_consistency_policy = raid5_change_consistency_policy, + .prepare_suspend = raid5_prepare_suspend, }; static struct md_personality raid4_personality = @@ -8981,6 +9010,7 @@ static struct md_personality raid4_personality = .quiesce = raid5_quiesce, .takeover = raid4_takeover, .change_consistency_policy = raid5_change_consistency_policy, + .prepare_suspend = raid5_prepare_suspend, }; static int __init raid5_init(void) From patchwork Tue Jan 30 02:18:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 193813 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp961911dyb; Mon, 29 Jan 2024 18:42:55 -0800 (PST) X-Google-Smtp-Source: AGHT+IEC2P1YXCTCM/A6/puSBNsb6vr9FCzGUq7J4BakjD44KsNlfAlfJCALr7pevLnhC4OM0hJd X-Received: by 2002:a17:90a:6982:b0:28d:c2ec:8950 with SMTP id s2-20020a17090a698200b0028dc2ec8950mr440741pjj.2.1706582575767; Mon, 29 Jan 2024 18:42:55 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706582575; cv=pass; d=google.com; s=arc-20160816; b=hkcWskdGJAhQB0XfMOi9MF0HJ8JtLut17g+9lWXwD6nk6nO3tM6zns96Zxe8IeAgVD TlKPYzN5Sf0eLNFZ18p1nKyQPttXUMtiAho02Jf3eQVgZG9I7xqkU7U83UJkEDT3Wn7n u8b1fJSJxSrpQ6Tf3FsQHPDHrOSaBAI1oB43FEGwvxHjBsVkSC9tv5b20epx5TY80R82 IH3rsHv6CiiKMEIoIeTHhcL5iGFjd25Fuf3Pmnh1DMVyV3tyj9/4pUaw8HVKA9iM3eMd +R8krHQQu9jWtiJCEh/zEhcWraL6+x1ofiFCX58vCrVMYwlH1TQ4KjJei4NHrNZwpY3z lbwA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=Gwajdbjq2bg1uihxFCNEmqNIL/yeEurfElrgwydZMvg=; fh=FtirvJJ13L8gehLQ375a0lCTkAroffAi0JfnF40Fa78=; b=u3Q/Wm3KE/MSEGgtyhmDanHyCOe/Vq3vaDWAFw174XLEo13rrcwGElsqsBk4urRwyi olBKoQ4U/fHIK6D7JQ+iR8PfCrpPyBbM7mJpzXpVYucc9hlhJfunly5ZGtp3mK7/swTB MoFOkcLIKBmcLmyRn2OQDEDOGqKWWCzJP/PnntBVf4TVTxbfEtROWnV72EbPgHRWYfQa /hilBwLhEvoZ3MLecU16Q5xL1m+oehcqWC7oFqWWf45FlBZiLOb1vgATU78iFyd67gDS F7UEaM6jjSPJUoZ0tMbB3MU1zo2s+6jPGyn6HJy8k8hTS5d6IMufNQSTE043MWuDHRUo YCXA== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43816-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43816-ouuuleilei=gmail.com@vger.kernel.org" Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id z6-20020a170903018600b001d8ac7882b6si705458plg.561.2024.01.29.18.42.55 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 18:42:55 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43816-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43816-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43816-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id AE556B24DB7 for ; Tue, 30 Jan 2024 02:27:29 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9DF8615B0E4; Tue, 30 Jan 2024 02:23:15 +0000 (UTC) Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8C024C60E; Tue, 30 Jan 2024 02:23:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581393; cv=none; b=erjK2aNotzG6ZMkPcS903hm81ydFaE9U8DZjAHsTwplD7F3Xs9dzf+vB8D/L1oAW/NkU/lTnBzt9GQN+U28blhZuV91gEpkakulSVJc0SOTXlz+gtU1GzwVo7HpK5/YjrLYdo769g+UX6iOS1nZM7nKWnTv45VafdTa64mz0j3g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581393; c=relaxed/simple; bh=a7HDR2QZuFH8EpmOVAfdCFF1UeWbNCM4RUwmYnTBG/s=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=mzrOYxH5sMUW4Yw6BGrfR/xjcC5oM9z7HhU0sl3xgZ/NjnRe5jG3NyaiWHvB4hV962zn/CpKK4YBT7WVrkW/qEp85QVkydIjC1ehBTulZEyKqyE2oII/ibrZ3NH8r9rxNW4y0o/GaG2X1R/HWyYtm/wEVKpz8u1Wck/SBjwOCMU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4TP88k4D4Nz4f3m7G; Tue, 30 Jan 2024 10:23:02 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 11C081A027B; Tue, 30 Jan 2024 10:23:09 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgBHGBGBXbhlrAigCQ--.55484S16; Tue, 30 Jan 2024 10:23:08 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v4 12/14] dm-raid: fix lockdep waring in "pers->hot_add_disk" Date: Tue, 30 Jan 2024 10:18:41 +0800 Message-Id: <20240130021843.3608859-13-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240130021843.3608859-1-yukuai1@huaweicloud.com> References: <20240130021843.3608859-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgBHGBGBXbhlrAigCQ--.55484S16 X-Coremail-Antispam: 1UD129KBjvJXoW7Cr45Jw1UZF45JF13GFW5GFg_yoW8Jw4Up3 ZrK343Kw4DJr48Za1qvw1q9a45tan8K3ySy39xG395ZFy7ZrZI9ws8Ga1agFWDJFZaya98 AFW3J398Was5K3DanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPI14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E 14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_GFv_WrylIx kGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVW8JVW5JwCI42IY6xIIjxv20xvEc7CjxVAF wI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Gr 0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBIdaVFxhVjvjDU0xZFpf9x0JUQ SdkUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789481530895096410 X-GMAIL-MSGID: 1789481530895096410 From: Yu Kuai The lockdep assert is added by commit a448af25becf ("md/raid10: remove rcu protection to access rdev from conf") in print_conf(). And I didn't notice that dm-raid is calling "pers->hot_add_disk" without holding 'reconfig_mutex'. "pers->hot_add_disk" read and write many fields that is protected by 'reconfig_mutex', and raid_resume() already grab the lock in other contex. Hence fix this problem by protecting "pers->host_add_disk" with the lock. Fixes: 9092c02d9435 ("DM RAID: Add ability to restore transiently failed devices on resume") Fixes: a448af25becf ("md/raid10: remove rcu protection to access rdev from conf") Signed-off-by: Yu Kuai --- drivers/md/dm-raid.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/md/dm-raid.c b/drivers/md/dm-raid.c index 16d3348abfed..5f78cc19d6f3 100644 --- a/drivers/md/dm-raid.c +++ b/drivers/md/dm-raid.c @@ -4077,7 +4077,9 @@ static void raid_resume(struct dm_target *ti) * Take this opportunity to check whether any failed * devices are reachable again. */ + mddev_lock_nointr(mddev); attempt_restore_of_faulty_devices(rs); + mddev_unlock(mddev); } if (test_and_clear_bit(RT_FLAG_RS_SUSPENDED, &rs->runtime_flags)) { From patchwork Tue Jan 30 02:18:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 193803 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp957653dyb; Mon, 29 Jan 2024 18:27:46 -0800 (PST) X-Google-Smtp-Source: AGHT+IHnqF1LtHuLRXJ0nds4oMJhiDkrIVDY40lBatVtJmgoOwKfpg/AOFsyDhUXaIS51pczsWgY X-Received: by 2002:a05:6214:2681:b0:686:9641:4ea0 with SMTP id gm1-20020a056214268100b0068696414ea0mr8656319qvb.77.1706581666563; Mon, 29 Jan 2024 18:27:46 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706581666; cv=pass; d=google.com; s=arc-20160816; b=Q2wf9UKb7tfc+ZYZK2aYMb0sR5sFj+DPehqVwam7U8bgN8Fm2VWltGzF8eB6K8J6ku uoZMJT+HKGIUGrKUuQatcqTWfLLmC0Key/leIIfs05HxnMfBNVylY17rpr1fMjXpMCMV cMjTbv+aEkC71c/YIeVU83Osbqa7rlmOspGrZ/Y5vNiddArF1p0+lIMI6Yy/dyfxnuIO utIZlqHNaJRQWP26V1qk6kSRG1ZkJHQEo4q7azTlh94awllkXjvfLQlKCAka1GScN6kO WLaxx/na3PGn6ACIzfdvKHiuFGSpaUCKJ6vcKgbPXvfVkiccI9kBXcoUaUfwX/PEAGHB 5FNg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=5je7DXjlS71CV3FJEKNFh7pGpon6xEdnXDko1q92XM0=; fh=FtirvJJ13L8gehLQ375a0lCTkAroffAi0JfnF40Fa78=; b=x06+Eagt7dgsOUr/+qOHMQzsV8JUg2Qh9tptWAvCPjJyLXJXyh4+7iegmBKueIRYo6 FXBy0SfPCOGfFIAK2Xd8fN42W+uPHGXd4mPa2n4+6kzDoXB853x4nk7MoZCYrUgoVgJ4 0DBCujj1aOBU8T4UxFjzJ7Wdo69B4FmGlykBSyg8kq46+T6WYn7i9fxtWV5VIYDp2TTX vwOOchiXYar33OvC0nyhX33QSm5gfcVQUa44ZaQYmygeBqsvImqTA7B6lTbTKLDk+4HI YabigzNzMZQRmyIkeQHhevz7QFLaxexb0dYmbgvbgNQfwvcJFy8ndhnhozSCpJ4fE590 SCmQ== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43818-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43818-ouuuleilei=gmail.com@vger.kernel.org" Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id r6-20020a0562140c8600b006800eced41asi8935934qvr.614.2024.01.29.18.27.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 18:27:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43818-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43818-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43818-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 5AE761C24CCF for ; Tue, 30 Jan 2024 02:27:46 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5D88A15B110; Tue, 30 Jan 2024 02:23:16 +0000 (UTC) Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B501A53E3B; Tue, 30 Jan 2024 02:23:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581394; cv=none; b=e28vdG5VK5NTzg4wYD4v6qpm5bHwoRO4rVRlSlAcXlchQjEygwx65YyC6I2D16rkGDIj1vwK+/s6OM4F9iwfIFuPph5Xg1vW4oZ9kJVTWI2x9xXmI7oZuz3puKHy7ahQVt4df6Ra2iQ9S8FMyq0GWMrrtMMVI7YXd1ewjrSlEss= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581394; c=relaxed/simple; bh=BcYpGvuVymYjsYzO5oAZoUEk24DPF79BmnHTgMndg6g=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=X2SvmRzwslc2ZboWsx7VFY9MRtvzrMlEKsnGSZoZGaHqrZatDyBzKv/r7D6HVi4cdwR4wUQIzhyrclUZQ60c3bNrVGnzOcUR5fibwYfR5/9Xi/NNJXPOBY7Uu7sQiRVntm/ia94gSDbZPwraNXapFZPb54t1mZpmFPxVygrX9KU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4TP88q1CxKz4f3kKY; Tue, 30 Jan 2024 10:23:07 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id AB8691A0232; Tue, 30 Jan 2024 10:23:09 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgBHGBGBXbhlrAigCQ--.55484S17; Tue, 30 Jan 2024 10:23:09 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH RFC v4 13/14] dm: wait for IO completion before removing dm device Date: Tue, 30 Jan 2024 10:18:42 +0800 Message-Id: <20240130021843.3608859-14-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240130021843.3608859-1-yukuai1@huaweicloud.com> References: <20240130021843.3608859-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgBHGBGBXbhlrAigCQ--.55484S17 X-Coremail-Antispam: 1UD129KBjvJXoW7KFy3tr1rGFWxAw4UGw1rZwb_yoW8Gw4xpF 4aqay5KrW5XrWjkw4Ut3yUCF1FyFs3K3yxCrW29w1fu3W5Gr9xXF48tFW3JrWUAFy5Arsx ZF4Ut3s8Gw48JwUanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPI14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E 14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_GFv_WrylIx kGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVW8JVW5JwCI42IY6xIIjxv20xvEc7CjxVAF wI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Gr 0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBIdaVFxhVjvjDU0xZFpf9x0JUQ SdkUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789480577843841987 X-GMAIL-MSGID: 1789480577843841987 From: Yu Kuai __dm_destroy() guarantee that device openers is zero, and then only call 'presuspend' and 'postsuspend' for the target. For request-based dm, 'md->holders' will be grabbed for each rq and __dm_destroy() will wait for 'md->holders' to be zero. However, for bio-based device, __dm_destroy() doesn't wait for all bios to be done. Fix this problem by calling dm_wait_for_completion() to wail for all inflight IO to be done, like what dm_suspend() does. Signed-off-by: Yu Kuai --- drivers/md/dm.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 8dcabf84d866..2c0eae67d0f1 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -58,6 +58,7 @@ static DEFINE_IDR(_minor_idr); static DEFINE_SPINLOCK(_minor_lock); static void do_deferred_remove(struct work_struct *w); +static int dm_wait_for_completion(struct mapped_device *md, unsigned int task_state); static DECLARE_WORK(deferred_remove_work, do_deferred_remove); @@ -2495,6 +2496,8 @@ static void __dm_destroy(struct mapped_device *md, bool wait) if (!dm_suspended_md(md)) { dm_table_presuspend_targets(map); set_bit(DMF_SUSPENDED, &md->flags); + if (wait) + dm_wait_for_completion(md, TASK_UNINTERRUPTIBLE); set_bit(DMF_POST_SUSPENDING, &md->flags); dm_table_postsuspend_targets(map); } From patchwork Tue Jan 30 02:18:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 193808 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp958677dyb; Mon, 29 Jan 2024 18:31:11 -0800 (PST) X-Google-Smtp-Source: AGHT+IEiy+jsUeAi0kMH5p2zeZBOQmYnrBLaZ3xRjIJQC5bUwTSvsDMoqOq4pPUoG04RFow9Znvb X-Received: by 2002:a05:6a21:2d88:b0:19c:9abb:8a64 with SMTP id ty8-20020a056a212d8800b0019c9abb8a64mr7715327pzb.24.1706581871800; Mon, 29 Jan 2024 18:31:11 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706581871; cv=pass; d=google.com; s=arc-20160816; b=km+7YkW0eEq1Ah4sC6tU/4WEleld4zC+BPYQsZngjr/Ny6k1ztD6Fa8qrEHUsxIXTh Nq0JO4k+bH3UkaCi753NGDplqd3C1PBTBR/cC4LqUaesfQbshkGEoqHUcdSQ+T7z0z9F tc5/0SHxKAeP9OvUF/Gnb8XO1GbFhDvqW+tR+h+VCwj6ra+OGftVbd3C6AgD5qfo/VDi 3xBWBjHwH6kJERRw4i9RUFl2o/IiQxYf1fe65uye1kdGTYycrBM16FQArW9YdD8yv8HW cNyT57wyZyWZ3lCEBZ1ryvgHSgWN5UbRO4jAG8azoUeTDjZ1eRv38YYrCrdV7dHclxhS GsNg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=GazZQ1YqdEujEcu8USxkKlwQjkD9tfrhZuIV9KZmOWs=; fh=FtirvJJ13L8gehLQ375a0lCTkAroffAi0JfnF40Fa78=; b=u8pu8SQ5Y0kGNKLJVdw/RXVKteua+fsUQW8+YFW2QEnGaWDd8MXLcwucsae3oXvgcJ pyXeNEsekz1ZT1lu+ROVU9IiDyVReNWmuAG/HP9EVqnpUGGhaMAQPRw8AJWr6iMo9AdP ujt/LwLbHoF942ndRxsvJplehcB8kj6+rCTPUdYMov5LsR0sx1tPQOcPKi0lVoZtAvoH cyjHthlfUldxHbF6BSQw6mG1klg1v8u4Gmhi2BGd8HnS/hpViyr51onS3O51F+R2WHpf I94iysbrc/qmqj6s2oYlDNnlPtWThrtR4O+l1N42iJXgs6gqp/p+hfUrPhiA+zZjV93Q qTdg== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43819-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43819-ouuuleilei=gmail.com@vger.kernel.org" Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id gx11-20020a056a001e0b00b006dbd45663b4si6631189pfb.228.2024.01.29.18.31.11 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 18:31:11 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43819-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-43819-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43819-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 5CC32285FB7 for ; Tue, 30 Jan 2024 02:27:53 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CC71A15B11A; Tue, 30 Jan 2024 02:23:16 +0000 (UTC) Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0F9A554746; Tue, 30 Jan 2024 02:23:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581395; cv=none; b=oOw+P9H78tnNMRWBUPNvnTvUmRC62toQOfw7LtcE0jFloJvrHXTKADZ+VBY5pJkCKFBGuTmxXzVm0Dxd/d36uC7b0rKnY+ZD0Q+VtTSi3X25Nc3sZ1ASbDlwVXsQfKT/BM0eq4HBVDHxD7upZgM5Eyg+YqhooClBpdPSe7iqOAE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706581395; c=relaxed/simple; bh=lt3pNMtDHV/r0zFHslB52IvlGb8ZyJ3dbmeLd5B+QBE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=jzlccQ0Et0JEmlTBp64nQY0r2bUvFRbNPFt6i9yyG9vbR1oM2v9ioX98JP7lrMfg86kVM2zG6ofDqWyCNMdFVeqQq6mojA99H4Vlql5uD5flJWOXQjbXdWl4c7bMK4xFRADfRNrJxdbCBoyofMG93EN4d4txhGlqGY06HIh+V6g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.93.142]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4TP88p1PG5z4f3lwP; Tue, 30 Jan 2024 10:23:06 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 4D25E1A016E; Tue, 30 Jan 2024 10:23:10 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgBHGBGBXbhlrAigCQ--.55484S18; Tue, 30 Jan 2024 10:23:10 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v4 RFC 14/14] dm-raid: remove mddev_suspend/resume() Date: Tue, 30 Jan 2024 10:18:43 +0800 Message-Id: <20240130021843.3608859-15-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240130021843.3608859-1-yukuai1@huaweicloud.com> References: <20240130021843.3608859-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgBHGBGBXbhlrAigCQ--.55484S18 X-Coremail-Antispam: 1UD129KBjvJXoW7CF4fCw4fWryruryfZr1UZFb_yoW8Kw4rpw 4IgFWYyw1UJFZrXw4DA3Z2gFy5twn5KrWqkrZxW34fW3W3Gr13Wr18GayUXFWDKFWfAF1D Aa15tw4UuryIgrJanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPI14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E 14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_GFv_WrylIx kGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVW8JVW5JwCI42IY6xIIjxv20xvEc7CjxVAF wI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Gr 0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBIdaVFxhVjvjDU0xZFpf9x0JUQ SdkUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789480793018221917 X-GMAIL-MSGID: 1789480793018221917 From: Yu Kuai dm_suspend() already make sure that no new IO can be issued and will wait for all dispatched IO to be done. There is no need to call mddev_suspend() to make sure that again. Signed-off-by: Yu Kuai --- drivers/md/dm-raid.c | 8 +++----- drivers/md/md.c | 11 +++++++++++ 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/drivers/md/dm-raid.c b/drivers/md/dm-raid.c index 5f78cc19d6f3..ed8c28952b14 100644 --- a/drivers/md/dm-raid.c +++ b/drivers/md/dm-raid.c @@ -3241,7 +3241,7 @@ static int raid_ctr(struct dm_target *ti, unsigned int argc, char **argv) rs->md.in_sync = 1; /* Has to be held on running the array */ - mddev_suspend_and_lock_nointr(&rs->md); + mddev_lock_nointr(&rs->md); /* Keep array frozen until resume. */ md_frozen_sync_thread(&rs->md); @@ -3829,11 +3829,9 @@ static void raid_postsuspend(struct dm_target *ti) { struct raid_set *rs = ti->private; - if (!test_and_set_bit(RT_FLAG_RS_SUSPENDED, &rs->runtime_flags)) { + if (!test_and_set_bit(RT_FLAG_RS_SUSPENDED, &rs->runtime_flags)) /* Writes have to be stopped before suspending to avoid deadlocks. */ md_stop_writes(&rs->md); - mddev_suspend(&rs->md, false); - } } static void attempt_restore_of_faulty_devices(struct raid_set *rs) @@ -4091,7 +4089,7 @@ static void raid_resume(struct dm_target *ti) mddev->ro = 0; mddev->in_sync = 0; md_unfrozen_sync_thread(mddev); - mddev_unlock_and_resume(mddev); + mddev_unlock(mddev); } } diff --git a/drivers/md/md.c b/drivers/md/md.c index 093abf3ce27b..e3a56a958b47 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -437,6 +437,10 @@ int mddev_suspend(struct mddev *mddev, bool interruptible) { int err = 0; + /* Array is supended from dm_suspend() for dm-raid. */ + if (!mddev->gendisk) + return 0; + /* * hold reconfig_mutex to wait for normal io will deadlock, because * other context can't update super_block, and normal io can rely on @@ -488,6 +492,13 @@ EXPORT_SYMBOL_GPL(mddev_suspend); static void __mddev_resume(struct mddev *mddev, bool recovery_needed) { + /* + * Array is supended from dm_suspend() and resumed from dm_resume() for + * dm-raid. + */ + if (!mddev->gendisk) + return; + lockdep_assert_not_held(&mddev->reconfig_mutex); mutex_lock(&mddev->suspend_mutex);