From patchwork Thu Feb 1 09:25:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 195217 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2719:b0:106:209c:c626 with SMTP id hl25csp29430dyb; Thu, 1 Feb 2024 01:31:03 -0800 (PST) X-Google-Smtp-Source: AGHT+IFYP5gXKr0DlvaExxBhR7HXFCcecKF8+rtGM14OG4s5JVksOqfrTlfhCj4cGBcNqkg35kp0 X-Received: by 2002:a05:6402:1cb9:b0:55f:aed2:98eb with SMTP id cz25-20020a0564021cb900b0055faed298ebmr877070edb.40.1706779863545; Thu, 01 Feb 2024 01:31:03 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706779863; cv=pass; d=google.com; s=arc-20160816; b=kSnJ5TgiHFLglvVajlffrWagftZ7ZaWx99YNwTbTltz19R/65cke8rBmKzu9BeAbGc W/kNpVNofcE5tvM5u+7HPFRR7eNE4Dv+qv2QJ8xx+dcnDyqVg0TvHIo9BST9W1vmvzvl E1dPJiFNwF3IcpWR1IsRC4MyzM+9KUMsbejTlnqth3yFqZFFsPbk7Sykd4o7PKdvJlAn YFWOFHe6Omg8A9aQ4i+aowF+LKUDhk9pzEzkuh1+5GZcNhcExI69Af1XSN4vW11R3Lcu LsN+HtsluQrzx+M4e4P/PByXzTCMh8Nfjlv9x6632xqXxau0Jaxo3m0IMxa4zgKfeN6A v7lw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=dBaWySK1O6HNfOrqRqopNpWnbd2nAQFt9nqs3kj8Nq8=; fh=ys5ZFiOR4HW6mMVRzQqB0m6by8yvbuK9vDYMYQL2vk4=; b=e0ib/pUmPPyrvadKocOmZnXu/5En1bIKElBozTM8BZu+yh2GO8Aw3e/LF6mKQcQ6Y5 rkVb5F5X6Qpuq90dbtANLeig0P3uJdpnwxEUvnOQJP1QncLnnv5NJoUDR0THpj2hCAZ7 MtJTqAr14IKSbeKUHN4cEpCR12upIUezoJHB5E9RujbGDDrZIGhZPaESXjZuD5RQ+sMq 1wi1z6Q9+3hH8eTPjHHUI6uyCntZn7O7N250DW0OX3C3Xpe6SDDZa3ndpdEixZrhgHrZ DGDXCid7LDBeKn404pdLtzs2TLoR5oEfgxt4E4RLSuxPCr+kwcz/4u6hayRAtbegtcfR Ft4g==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47864-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47864-ouuuleilei=gmail.com@vger.kernel.org" X-Forwarded-Encrypted: i=1; AJvYcCWkpZuyEwEGsKreOY7Mvdhd0hxCSwSbM0L6aGDlRvgApmhqutooxnqeAJ2EI5UshX8GlP6lQAL2pq1CNDHvpnnllD5q2g== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id g15-20020a056402090f00b0055a748b0999si6486409edz.544.2024.02.01.01.31.03 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Feb 2024 01:31:03 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47864-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47864-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47864-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 2CFCC1F275DB for ; Thu, 1 Feb 2024 09:31:03 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D853715CD42; Thu, 1 Feb 2024 09:30:32 +0000 (UTC) Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F05E15B105; Thu, 1 Feb 2024 09:30:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779830; cv=none; b=bEEIjJCE816EvQ4nP0jxfD/AtoqUMOi4AMs4q2HNCenaZ7Q7WRMihnywTCSFFEkDCVLToYLIvQSem3Vk/DqxoCGKQ78tdvhbe2YRTwPduzYe8UCHV1ALXFYApP/N5tK7pkzYzXxeCtXXn+an+0ZkkLtj0IyJVo+fhqx91seX3fA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779830; c=relaxed/simple; bh=niaIFrprWvPwC2Lv0RN5Cl1tFCytXopzeriQpGSoAMg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=MpoSpwa8+pBknnpZirnoXmS4nUWBxljhhyFl370geFCfkrMyb339iRhwugW4VexslKOrYRKqHTj8AIHU6vB0uPsSVl7MUoc0/snWPMTfXLhsF/kh9cTeDPMQbvE1HZfZsdTQK5aF1VD95dKGo38+rYIuTLHrRNpUgJfmZkx+l74= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4TQYXs2rwzz4f3l6f; Thu, 1 Feb 2024 17:30:21 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 974E11A0283; Thu, 1 Feb 2024 17:30:25 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgCXaBGtZLtl8V6KCg--.33515S5; Thu, 01 Feb 2024 17:30:25 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, blazej.kucman@linux.intel.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v5 01/14] md: don't ignore suspended array in md_check_recovery() Date: Thu, 1 Feb 2024 17:25:46 +0800 Message-Id: <20240201092559.910982-2-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240201092559.910982-1-yukuai1@huaweicloud.com> References: <20240201092559.910982-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgCXaBGtZLtl8V6KCg--.33515S5 X-Coremail-Antispam: 1UD129KBjvJXoWxJryDAF43ArW7JrWrZr1xGrg_yoW8ZFy7pa yIkF1YyrWjyFZ7Aa4qka4UZa4rAr1jqrW5AFy3u34rCa4fKw43CrWYgFyDXFyqkFyxKrZY vw4rJa15uw18KF7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBE14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_Jr4l82xGYIkIc2 x26xkF7I0E14v26r4j6ryUM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_Ar0_tr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJw A2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq3wAS 0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2 IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0 Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2kIc2 xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v2 6r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_GFv_WrylIxkGc2 Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_ Gr0_Cr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMI IF0xvEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x0JU4T5dUUUUU = X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789688401974209136 X-GMAIL-MSGID: 1789688401974209136 From: Yu Kuai mddev_suspend() never stop sync_thread, hence it doesn't make sense to ignore suspended array in md_check_recovery(), which might cause sync_thread can't be unregistered. After commit f52f5c71f3d4 ("md: fix stopping sync thread"), following hang can be triggered by test shell/integrity-caching.sh: 1) suspend the array: raid_postsuspend mddev_suspend 2) stop the array: raid_dtr md_stop __md_stop_writes stop_sync_thread set_bit(MD_RECOVERY_INTR, &mddev->recovery); md_wakeup_thread_directly(mddev->sync_thread); wait_event(..., !test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)) 3) sync thread done: md_do_sync set_bit(MD_RECOVERY_DONE, &mddev->recovery); md_wakeup_thread(mddev->thread); 4) daemon thread can't unregister sync thread: md_check_recovery if (mddev->suspended) return; -> return directly md_read_sync_thread clear_bit(MD_RECOVERY_RUNNING, &mddev->recovery); -> MD_RECOVERY_RUNNING can't be cleared, hence step 2 hang; This problem is not just related to dm-raid, fix it by ignoring suspended array in md_check_recovery(). And follow up patches will improve dm-raid better to frozen sync thread during suspend. Reported-by: Mikulas Patocka Closes: https://lore.kernel.org/all/8fb335e-6d2c-dbb5-d7-ded8db5145a@redhat.com/ Fixes: 68866e425be2 ("MD: no sync IO while suspended") Fixes: f52f5c71f3d4 ("md: fix stopping sync thread") Signed-off-by: Yu Kuai --- drivers/md/md.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 2266358d8074..07b80278eaa5 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -9469,9 +9469,6 @@ static void md_start_sync(struct work_struct *ws) */ void md_check_recovery(struct mddev *mddev) { - if (READ_ONCE(mddev->suspended)) - return; - if (mddev->bitmap) md_bitmap_daemon_work(mddev); From patchwork Thu Feb 1 09:25:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 195218 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2719:b0:106:209c:c626 with SMTP id hl25csp29958dyb; Thu, 1 Feb 2024 01:32:21 -0800 (PST) X-Google-Smtp-Source: AGHT+IEluprewXM2U43FLBzsKjU5cCDcC9eGbbdpyJwiPa6pugq0oUtZSURCEUyAeeSRsf6ieWfh X-Received: by 2002:aa7:d948:0:b0:55f:af58:21e6 with SMTP id l8-20020aa7d948000000b0055faf5821e6mr1007659eds.26.1706779941614; Thu, 01 Feb 2024 01:32:21 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706779941; cv=pass; d=google.com; s=arc-20160816; b=mjK9L2QUTwDBhP039M5hUhqTI7YGDuxKwJxMhYwMxLwD3c4GwUckKDODEbrLUbA/ED zHcEcLyGhPraSiCphCQetxkdngtAJoLf8Z0Crs43exr6woW/DqTf+6m+QwS2HZoXHXkf sFRY73+4Ugfra+FiyDT9WrG/84xbqMnDa8ovoYdh2WhocKrwzf4EXVEke433EExvcbcs CXMYI8+G4cO5DiaTqhdkH2OYu1sTQdKN+D6zE/L3qp1aIGo0L6RSTVUjOyKCQzceb3pc GYj4n/NT3qG/79tSz+pEWJHYKvzDrXQx7gNuOdxLyJt72pambscV+17FgkmhuQhXqv2D qcrw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=J4U91+KdS8PQE0p29V5wPuN9QbhEeMD+9Bl1l8h5hgU=; fh=rnmJwc6KVhYsf5QoL8m4tz3khfkhgvAbyfmdoYYao+w=; b=p7KQaRsz9wNaCjc4+ys+E7fSaE4PuklU5r4hCwSk7FamJgvbTwoK2mjtN44ZfUXvMz 4iVsiMrdMydkOvZZU6dCE80v9a69wHg+cKUlnlP1rBBtKbnSahzf73QzhX9N/VCDdbYG gOUSHZ58rTWB+15wDEYRjEgNmcgA42JcTx7oEJfs8tK/nrYr8IxtZ3IOkTEz7uDhgfsq HY8iUVWp2taf2m2RmK2ZtYLxdTqtiy/FiCmfa7UFVkcRKKUvfXf8nSNDeFQuUMrX6la3 LVrrvAh/0Nb1nd0tSgt9/3ShISQDndSalk8/tf1KGEhI0NAKxOW2BAyhFzRAkNHhYkEu FLLA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47866-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47866-ouuuleilei=gmail.com@vger.kernel.org" X-Forwarded-Encrypted: i=1; AJvYcCX1sz+TwL7OrC3/XuzWqLzATFrgt8yZcAzD9WHEAEqZL+O4W+VuYzlGZUqe/fqjHPdm6boqs+GPaSCc9xQ5R90tV6j1wA== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id b19-20020a0564021f1300b0055f179146easi3530742edb.651.2024.02.01.01.32.21 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Feb 2024 01:32:21 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47866-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47866-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47866-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id E3F851F22003 for ; Thu, 1 Feb 2024 09:32:20 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9A1BB15DBDD; Thu, 1 Feb 2024 09:30:35 +0000 (UTC) Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9592915AAB9; Thu, 1 Feb 2024 09:30:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779832; cv=none; b=i6RXa6LFcy1OzG9BYElybDzAe6ybDjBFhoFv1Cpl7qHYZc7uknfuBB1hU1vRCosmdfEEXbSw+1t3ahVcDHbAmYEvhYFwzJyfnBHk2c5e98HnqjHgQ7gWkruKgdIrEQVV7GRioyTXFNS44RrX/+x2aNip+Sm8k4HmT9FJiCmVP20= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779832; c=relaxed/simple; bh=h4mAM5munNDh64f9BT7eKjkJVG7xHUqubnznwxYlHRM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=YN3mow2QnGKulmWShuvi2DwD8K52V5gKvZg5UlkTnrLK4I0PZLh+Ucy5GVpMoKVzfCO/Ph3IxyopvrDt/nb1Z9BAlWHta9koobG6rvSLyCtmMEwT1aOuOpnkY1ARBgWD+LOf4UZ+rSIyZH1uPXJghG4ejnpfGKFMkWufYY5iRzY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4TQYXv60ztz4f3k6S; Thu, 1 Feb 2024 17:30:23 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 6C9221A038B; Thu, 1 Feb 2024 17:30:26 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgCXaBGtZLtl8V6KCg--.33515S6; Thu, 01 Feb 2024 17:30:26 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, blazej.kucman@linux.intel.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v5 02/14] md: don't ignore read-only array in md_check_recovery() Date: Thu, 1 Feb 2024 17:25:47 +0800 Message-Id: <20240201092559.910982-3-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240201092559.910982-1-yukuai1@huaweicloud.com> References: <20240201092559.910982-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgCXaBGtZLtl8V6KCg--.33515S6 X-Coremail-Antispam: 1UD129KBjvJXoWxCF1xZw47XF4fCr17tFyUGFg_yoWrXw45pa yfKas8Cr4UtFy3urZrK3WDXayYvr10qrZ0yFy3u34fAw1rtr4Sk34ruFyUJFWDtFyFqw4Y vw4rJF43ZFy8Kw7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPj14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_Jryl82xGYIkIc2 x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJw A2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq3wAS 0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2 IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0 Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2kIc2 xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v2 6r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_GFv_WrylIxkGc2 Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_ Cr0_Gr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVWUJVW8Jw CI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWIevJa73UjIFyTuYvjfUOJPEUUUU U X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789688484249987020 X-GMAIL-MSGID: 1789688484249987020 From: Yu Kuai Usually if the array is not read-write, md_check_recovery() won't register new sync_thread in the first place. And if the array is read-write and sync_thread is registered, md_set_readonly() will unregister sync_thread before setting the array read-only. md/raid follow this behavior hence there is no problem. After commit f52f5c71f3d4 ("md: fix stopping sync thread"), following hang can be triggered by test shell/integrity-caching.sh: 1) array is read-only. dm-raid update super block: rs_update_sbs ro = mddev->ro mddev->ro = 0 -> set array read-write md_update_sb 2) register new sync thread concurrently. 3) dm-raid set array back to read-only: rs_update_sbs mddev->ro = ro 4) stop the array: raid_dtr md_stop stop_sync_thread set_bit(MD_RECOVERY_INTR, &mddev->recovery); md_wakeup_thread_directly(mddev->sync_thread); wait_event(..., !test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)) 5) sync thread done: md_do_sync set_bit(MD_RECOVERY_DONE, &mddev->recovery); md_wakeup_thread(mddev->thread); 6) daemon thread can't unregister sync thread: md_check_recovery if (!md_is_rdwr(mddev) && !test_bit(MD_RECOVERY_NEEDED, &mddev->recovery)) return; -> -> MD_RECOVERY_RUNNING can't be cleared, hence step 4 hang; The root cause is that dm-raid manipulate 'mddev->ro' by itself, however, dm-raid really should stop sync thread before setting the array read-only. Unfortunately, I need to read more code before I can refacter the handler of 'mddev->ro' in dm-raid, hence let's fix the problem the easy way for now to prevent dm-raid regression. Reported-by: Mikulas Patocka Closes: https://lore.kernel.org/all/9801e40-8ac7-e225-6a71-309dcf9dc9aa@redhat.com/ Fixes: ecbfb9f118bc ("dm raid: add raid level takeover support") Fixes: f52f5c71f3d4 ("md: fix stopping sync thread") Signed-off-by: Yu Kuai --- drivers/md/md.c | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 07b80278eaa5..6906d023f1d6 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -9445,6 +9445,20 @@ static void md_start_sync(struct work_struct *ws) sysfs_notify_dirent_safe(mddev->sysfs_action); } +static void unregister_sync_thread(struct mddev *mddev) +{ + if (!test_bit(MD_RECOVERY_DONE, &mddev->recovery)) { + /* resync/recovery still happening */ + clear_bit(MD_RECOVERY_NEEDED, &mddev->recovery); + return; + } + + if (WARN_ON_ONCE(!mddev->sync_thread)) + return; + + md_reap_sync_thread(mddev); +} + /* * This routine is regularly called by all per-raid-array threads to * deal with generic issues like resync and super-block update. @@ -9482,7 +9496,8 @@ void md_check_recovery(struct mddev *mddev) } if (!md_is_rdwr(mddev) && - !test_bit(MD_RECOVERY_NEEDED, &mddev->recovery)) + !test_bit(MD_RECOVERY_NEEDED, &mddev->recovery) && + !test_bit(MD_RECOVERY_DONE, &mddev->recovery)) return; if ( ! ( (mddev->sb_flags & ~ (1<recovery)) { - /* sync_work already queued. */ - clear_bit(MD_RECOVERY_NEEDED, &mddev->recovery); + unregister_sync_thread(mddev); goto unlock; } @@ -9568,16 +9582,7 @@ void md_check_recovery(struct mddev *mddev) * still set. */ if (test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)) { - if (!test_bit(MD_RECOVERY_DONE, &mddev->recovery)) { - /* resync/recovery still happening */ - clear_bit(MD_RECOVERY_NEEDED, &mddev->recovery); - goto unlock; - } - - if (WARN_ON_ONCE(!mddev->sync_thread)) - goto unlock; - - md_reap_sync_thread(mddev); + unregister_sync_thread(mddev); goto unlock; } From patchwork Thu Feb 1 09:25:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 195221 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2719:b0:106:209c:c626 with SMTP id hl25csp30218dyb; Thu, 1 Feb 2024 01:32:57 -0800 (PST) X-Google-Smtp-Source: AGHT+IHR8GQ9z/0YET6aQ1He8H1u79iJc90/r4F1tcoLc081h0aCD4fZtTWqB0VYol/wmo8uM/Fo X-Received: by 2002:a05:6402:64f:b0:55e:d46b:7d82 with SMTP id u15-20020a056402064f00b0055ed46b7d82mr3083138edx.28.1706779976818; Thu, 01 Feb 2024 01:32:56 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706779976; cv=pass; d=google.com; s=arc-20160816; b=RKYNFyUQNgRdR2v3gK8PRuyeH50mJhICSKc/2WKq4o+CcrFb7xdp7094P38hW7/n8s G+WmSVsIbKoxM+WB14HAkAw2mBcNsJ0G00c1n+BSTvTiorYfRdmC4EgP4s7BZWxMUIdA s00YUcQj7r7hyNxzhpajgKpjMn/wKhFDsHagRDZIJG8DCh7OkZV9LZL1LmeJPsdkGqOV kr+0pHjU7O5GWqJuT5nhon99jQtf1zpj146S2ZhqRarYCNI5PSHPwLaqM25WlWvgZWIx YzceQuAxGUkHpaXOdCP2tNSj3UY7uH+WWEyBC/Yf4xy4ZBlXmBk5GZhJrj0yVx3wXTlu LodA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=D0W001Odz43Z3JUWcykLDV1gM6F19antxKYIQwlWdgQ=; fh=Uuyp+DVOK2bTfCK2sE0rW4sqNa9+N9q/mjiZ0ouAmAs=; b=c9EvbeExBYIRNMIja9xvyfJlqnwxy7pF4Q3sgVVhFWN/95XV4ztbwaITqcecjbXUbR aCKARjWrhiKr3dGeD8jZNolXJuBijL9/isMbNRpwojy8xeOKWzOB5YM5mOoX7vTi5M/H FL0ltlaz9K4gFj0j/6gEJQXuOT2Tr4oGRb+Kntgjo7k8O2wTNoHbf0v/9zBKddSOr3XO qvtRCjGrpRt+rK5dueS3f8GVFtRmMfnZGqJUQqqJHf+cv1yG68oRhkzp/SXX5NTSGyCX rt+qoD6V6lbrWYV6SadSg13BnvXml8nz6w1SENh4zDdwL+7id4Cjt9amqYicSj6F7C0V 8cjg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47869-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47869-ouuuleilei=gmail.com@vger.kernel.org" X-Forwarded-Encrypted: i=1; AJvYcCVPpViuLu4+IcgLueAdPZL/60DYFgVUXsj+Gvc4Aa4jpRoXO6c6JebTbmNmtf3sbKMEDtGRWD0Zcx/OeicLfCv/zAcUGg== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id f28-20020a50a6dc000000b0055ef2c893besi4313807edc.326.2024.02.01.01.32.56 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Feb 2024 01:32:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47869-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47869-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47869-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 3C2FB1F23A79 for ; Thu, 1 Feb 2024 09:32:56 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C4BF215F331; Thu, 1 Feb 2024 09:30:36 +0000 (UTC) Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 97A1515B971; Thu, 1 Feb 2024 09:30:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779833; cv=none; b=Gj4b7rRARc1DDtgPPgg5qfSvGNB9nsnd/aa/iNnaRiMaXUYju87ljRoUpxfs71FY6kD55u9p6bSiQa8CD1AME8g464S8cD1R7iwVUQvECs3ZpEipxtRzKAc7cmQLSsKQJf9yzuMwA6ZFUL1qbqgVk5ytoHCCsZJi0E4VMDLMTYA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779833; c=relaxed/simple; bh=dZCpGLgs1JsjUcuAOwp41TG3RiXfam80H98EBcaCbRk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=mfea+NlYlnnKesC84Bxk5Tdl6GxRZLwTqk1pOVQwZupVbKM1gUiJyegLB456HeUyUOl2xlMG34NYZFR3FK5Mb872571jgCydJVDpV0pOlmzW+79Jw9bvDHiRCCbzxjyUpG25WrxtEjor9qz00OsQa3g0Lv2dyn204OM+TDiXyXE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4TQYXs2qZnz4f3mHk; Thu, 1 Feb 2024 17:30:21 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id E535C1A0232; Thu, 1 Feb 2024 17:30:27 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgCXaBGtZLtl8V6KCg--.33515S8; Thu, 01 Feb 2024 17:30:27 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, blazej.kucman@linux.intel.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v5 04/14] md: don't register sync_thread for reshape directly Date: Thu, 1 Feb 2024 17:25:49 +0800 Message-Id: <20240201092559.910982-5-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240201092559.910982-1-yukuai1@huaweicloud.com> References: <20240201092559.910982-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgCXaBGtZLtl8V6KCg--.33515S8 X-Coremail-Antispam: 1UD129KBjvJXoW3Xry3Kw4kuF1xAry3KryrCrg_yoW7KF1fpa yfKF9xJr48A343ZrWUta4DXFW5uw1jqrWqyry3W3s5A3ZYyrZ3JFy5uFyUJFWkAa4kta15 ta45tFWDZFyFgw7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPF14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr1j6r xdM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0D M2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjx v20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1l F7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4IIrI8v6xkF7I0E8cxan2 IY04v7MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAF wI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVW8ZVWrXwCIc4 0Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r1j6r1xMIIF0xvE2Ix0cI8IcVCY1x0267AK xVWxJVW8Jr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r 4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Gr1j6F4UJbIYCTnIWIevJa73UjIFyTuYvjfUOBTY UUUUU X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789688521228187256 X-GMAIL-MSGID: 1789688521228187256 From: Yu Kuai Currently, if reshape is interrupted, then reassemble the array will register sync_thread directly from pers->run(), in this case 'MD_RECOVERY_RUNNING' is set directly, however, there is no guarantee that md_do_sync() will be executed, hence stop_sync_thread() will hang because 'MD_RECOVERY_RUNNING' can't be cleared. Last patch make sure that md_do_sync() will set MD_RECOVERY_DONE, however, following hang can still be triggered by dm-raid test shell/lvconvert-raid-reshape.sh occasionally: [root@fedora ~]# cat /proc/1982/stack [<0>] stop_sync_thread+0x1ab/0x270 [md_mod] [<0>] md_frozen_sync_thread+0x5c/0xa0 [md_mod] [<0>] raid_presuspend+0x1e/0x70 [dm_raid] [<0>] dm_table_presuspend_targets+0x40/0xb0 [dm_mod] [<0>] __dm_destroy+0x2a5/0x310 [dm_mod] [<0>] dm_destroy+0x16/0x30 [dm_mod] [<0>] dev_remove+0x165/0x290 [dm_mod] [<0>] ctl_ioctl+0x4bb/0x7b0 [dm_mod] [<0>] dm_ctl_ioctl+0x11/0x20 [dm_mod] [<0>] vfs_ioctl+0x21/0x60 [<0>] __x64_sys_ioctl+0xb9/0xe0 [<0>] do_syscall_64+0xc6/0x230 [<0>] entry_SYSCALL_64_after_hwframe+0x6c/0x74 Meanwhile mddev->recovery is: MD_RECOVERY_RUNNING | MD_RECOVERY_INTR | MD_RECOVERY_RESHAPE | MD_RECOVERY_FROZEN Fix this problem by remove the code to register sync_thread directly from raid10 and raid5. And let md_check_recovery() to register sync_thread. Fixes: f67055780caa ("[PATCH] md: Checkpoint and allow restart of raid5 reshape") Fixes: f52f5c71f3d4 ("md: fix stopping sync thread") Signed-off-by: Yu Kuai --- drivers/md/md.c | 5 ++++- drivers/md/raid10.c | 16 ++-------------- drivers/md/raid5.c | 29 ++--------------------------- 3 files changed, 8 insertions(+), 42 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index c65dfd156090..6c5d0a372927 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -9372,6 +9372,7 @@ static void md_start_sync(struct work_struct *ws) struct mddev *mddev = container_of(ws, struct mddev, sync_work); int spares = 0; bool suspend = false; + char *name; if (md_spares_need_change(mddev)) suspend = true; @@ -9404,8 +9405,10 @@ static void md_start_sync(struct work_struct *ws) if (spares) md_bitmap_write_all(mddev->bitmap); + name = test_bit(MD_RECOVERY_RESHAPE, &mddev->recovery) ? + "reshape" : "resync"; rcu_assign_pointer(mddev->sync_thread, - md_register_thread(md_do_sync, mddev, "resync")); + md_register_thread(md_do_sync, mddev, name)); if (!mddev->sync_thread) { pr_warn("%s: could not start resync thread...\n", mdname(mddev)); diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index 7412066ea22c..a5f8419e2df1 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -4175,11 +4175,7 @@ static int raid10_run(struct mddev *mddev) clear_bit(MD_RECOVERY_SYNC, &mddev->recovery); clear_bit(MD_RECOVERY_CHECK, &mddev->recovery); set_bit(MD_RECOVERY_RESHAPE, &mddev->recovery); - set_bit(MD_RECOVERY_RUNNING, &mddev->recovery); - rcu_assign_pointer(mddev->sync_thread, - md_register_thread(md_do_sync, mddev, "reshape")); - if (!mddev->sync_thread) - goto out_free_conf; + set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); } return 0; @@ -4573,16 +4569,8 @@ static int raid10_start_reshape(struct mddev *mddev) clear_bit(MD_RECOVERY_CHECK, &mddev->recovery); clear_bit(MD_RECOVERY_DONE, &mddev->recovery); set_bit(MD_RECOVERY_RESHAPE, &mddev->recovery); - set_bit(MD_RECOVERY_RUNNING, &mddev->recovery); - - rcu_assign_pointer(mddev->sync_thread, - md_register_thread(md_do_sync, mddev, "reshape")); - if (!mddev->sync_thread) { - ret = -EAGAIN; - goto abort; - } + set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); conf->reshape_checkpoint = jiffies; - md_wakeup_thread(mddev->sync_thread); md_new_event(); return 0; diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 8497880135ee..6a7a32f7fb91 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -7936,11 +7936,7 @@ static int raid5_run(struct mddev *mddev) clear_bit(MD_RECOVERY_SYNC, &mddev->recovery); clear_bit(MD_RECOVERY_CHECK, &mddev->recovery); set_bit(MD_RECOVERY_RESHAPE, &mddev->recovery); - set_bit(MD_RECOVERY_RUNNING, &mddev->recovery); - rcu_assign_pointer(mddev->sync_thread, - md_register_thread(md_do_sync, mddev, "reshape")); - if (!mddev->sync_thread) - goto abort; + set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); } /* Ok, everything is just fine now */ @@ -8506,29 +8502,8 @@ static int raid5_start_reshape(struct mddev *mddev) clear_bit(MD_RECOVERY_CHECK, &mddev->recovery); clear_bit(MD_RECOVERY_DONE, &mddev->recovery); set_bit(MD_RECOVERY_RESHAPE, &mddev->recovery); - set_bit(MD_RECOVERY_RUNNING, &mddev->recovery); - rcu_assign_pointer(mddev->sync_thread, - md_register_thread(md_do_sync, mddev, "reshape")); - if (!mddev->sync_thread) { - mddev->recovery = 0; - spin_lock_irq(&conf->device_lock); - write_seqcount_begin(&conf->gen_lock); - mddev->raid_disks = conf->raid_disks = conf->previous_raid_disks; - mddev->new_chunk_sectors = - conf->chunk_sectors = conf->prev_chunk_sectors; - mddev->new_layout = conf->algorithm = conf->prev_algo; - rdev_for_each(rdev, mddev) - rdev->new_data_offset = rdev->data_offset; - smp_wmb(); - conf->generation --; - conf->reshape_progress = MaxSector; - mddev->reshape_position = MaxSector; - write_seqcount_end(&conf->gen_lock); - spin_unlock_irq(&conf->device_lock); - return -EAGAIN; - } + set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); conf->reshape_checkpoint = jiffies; - md_wakeup_thread(mddev->sync_thread); md_new_event(); return 0; } From patchwork Thu Feb 1 09:25:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 195219 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2719:b0:106:209c:c626 with SMTP id hl25csp30078dyb; Thu, 1 Feb 2024 01:32:39 -0800 (PST) X-Google-Smtp-Source: AGHT+IEjSaUNKq0u56lIUWo679qIi0yIF27QevH30TIz4HHFrs6Ke6VmOaMCPJn5ogf8rjmWPntT X-Received: by 2002:a17:906:9c88:b0:a36:3152:f55f with SMTP id fj8-20020a1709069c8800b00a363152f55fmr6754967ejc.33.1706779959325; Thu, 01 Feb 2024 01:32:39 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706779959; cv=pass; d=google.com; s=arc-20160816; b=QVNWgKwEEWHImuB2qzngVzvbPYN0Nto0cPCrzb+4HwWUXlXXLlo9wHFYHjFI2I4bDL FJ5z9otgRafkkRxsXBwDlivCaRP72V++epb+3L0D/YLUGwqO8altVXgq3OJHWmsQ1QRX 49ttqJwtWWVfKWQS8CHx5n3Q1QB1ZnBFN1FZF8kZ/JhRM1Dp/PTEe3VvxFNnLTkefRVI 4Mj6dRkrO3TTQ1DknvPgEwS34kpz/ltpAmIQdz9dUr0zcjKLE4wM/ntN+nsWzoJqyINp 2P4NxM4cpcCzg10dm6ylqUBm9LjzK0wffiyl2u5/Mz8mZI0TiMwCTqMReH9QanEi46mX 05RQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=y959pjDZ7J8BFqaQCMjqyW0JW84A6rEyA3lltCKk3rY=; fh=yxU3U1G5P4WRHGJR2/fbLH54JYb4sc6LVMAk7ls2i5k=; b=dD2hRQSYTvbNyPjupdrP2zPX2NhlCo1YdatAMxgdrSyyr3eOnJPIP/+tOBSN6voTRt JHAlGoJkxg7aRixpzowtIVgHb2XrOHdT9V2L+NKr16EmTztWZbUmvZlXJUuOtzT8IXMn TlPezAuv5L+rPxyGdfydXL/o7P29cjQTl3h+lyo4RHA0zwgBF6AkrH8KtrhfC4TO/OmL Kj0mKmGEei4p9aRNfz139QNhmi305J/RFtDMkYnRF5Q27qJQYtaMzdqCV3l0cXOSXFtb In4diXDx1WQd6If+jkMXj2Q58aU3BLTzTVBq6uD6l+wzZYXQQV3AOMB/g1FijKhUJE1Q 6dOw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47868-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47868-ouuuleilei=gmail.com@vger.kernel.org" X-Forwarded-Encrypted: i=1; AJvYcCWCLctzvIWIakXNNavPiiabhBFEhJ8k2f0CrtmW9Z1dUQp0to9+xQvLQMbDNzJ6ovLVQRd7EZRvTJh+lTpONbEF1SE6Dw== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id w24-20020a1709067c9800b00a349fd53d0bsi6313894ejo.66.2024.02.01.01.32.39 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Feb 2024 01:32:39 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47868-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47868-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47868-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id E85AC1F229E4 for ; Thu, 1 Feb 2024 09:32:38 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2821815F31A; Thu, 1 Feb 2024 09:30:36 +0000 (UTC) Received: from dggsgout12.his.huawei.com (unknown [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B27A215B975; Thu, 1 Feb 2024 09:30:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779833; cv=none; b=HaVF47Yr/yGFfsyddbtBrP+ICCTmu+ZymH4hmgxlPLBf1AjHiJuClyb3OtuyjskWPkex+gUZps/PcrKvmiVNKGEZxSMtd61Cq+HQgb91IM7XStMbWRxIEguydEEhmYZKzWctpTYegx2hGFn8z/2mz3iyn2VDySvqewEteGs0gd8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779833; c=relaxed/simple; bh=W6rufQZZ+hnLEGX0/LnkFyb5yoGyGO/17y5HhbzHXQ8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=c5JEHjW3OyqPkLvxLRFnJFCl4I71n/NwsadaZeZjUwqnapD4uN41gQkEe+5Tcj74doe3tXZVnoIzmmizwLWNBZIDXrvGH6icTI1PlbIAHrTSPFyPHesOoWocGsPvoTHDr4oGsAjZ+AhA2Dp1Ap9FFrbkm5/ghQ3GkP8fK1wbXpo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4TQYXw3LNSz4f3l6h; Thu, 1 Feb 2024 17:30:24 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id A6FFF1A0199; Thu, 1 Feb 2024 17:30:28 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgCXaBGtZLtl8V6KCg--.33515S9; Thu, 01 Feb 2024 17:30:28 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, blazej.kucman@linux.intel.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v5 05/14] md: don't suspend the array for interrupted reshape Date: Thu, 1 Feb 2024 17:25:50 +0800 Message-Id: <20240201092559.910982-6-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240201092559.910982-1-yukuai1@huaweicloud.com> References: <20240201092559.910982-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgCXaBGtZLtl8V6KCg--.33515S9 X-Coremail-Antispam: 1UD129KBjvJXoW7uF1fuFy3ZFWkArykCr17Jrb_yoW8ur13p3 y3tF1ayrs8X39ava1UG3s2gFyYk3s5trWYy3srJ34UAw13Gr1fGr43Gr4qvFyY9ry3trs0 qr1Yq34rGF1qkaDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUP214x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr1j6r xdM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0D M2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjx v20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1l F7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4IIrI8v6xkF7I0E8cxan2 IY04v7MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAF wI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVW8ZVWrXwCIc4 0Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r1I6r4UMIIF0xvE2Ix0cI8IcVCY1x0267AK xVW8Jr0_Cr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVWUJV W8JwCI42IY6I8E87Iv6xkF7I0E14v26r4UJVWxJrUvcSsGvfC2KfnxnUUI43ZEXa7VUbmZ X7UUUUU== X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789688502603835195 X-GMAIL-MSGID: 1789688502603835195 From: Yu Kuai md_start_sync() will suspend the array if there are spares that can be added or removed from conf, however, if reshape is still in progress, this won't happen at all or data will be corrupted(remove_and_add_spares won't be called from md_choose_sync_action for reshape), hence there is no need to suspend the array if reshape is not done yet. Meanwhile, there is a potential deadlock for raid456: 1) reshape is interrupted; 2) set one of the disk WantReplacement, and add a new disk to the array, however, recovery won't start until the reshape is finished; 3) then issue an IO across reshpae position, this IO will wait for reshape to make progress; 4) continue to reshape, then md_start_sync() found there is a spare disk that can be added to conf, mddev_suspend() is called; Step 4 and step 3 is waiting for each other, deadlock triggered. Noted this problem is found by code review, and it's not reporduced yet. Fix this porblem by don't suspend the array for interrupted reshape, this is safe because conf won't be changed until reshape is done. Fixes: bc08041b32ab ("md: suspend array in md_start_sync() if array need reconfiguration") Signed-off-by: Yu Kuai --- drivers/md/md.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 6c5d0a372927..85fde05c37dd 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -9374,12 +9374,17 @@ static void md_start_sync(struct work_struct *ws) bool suspend = false; char *name; - if (md_spares_need_change(mddev)) + /* + * If reshape is still in progress, spares won't be added or removed + * from conf until reshape is done. + */ + if (mddev->reshape_position == MaxSector && + md_spares_need_change(mddev)) { suspend = true; + mddev_suspend(mddev, false); + } - suspend ? mddev_suspend_and_lock_nointr(mddev) : - mddev_lock_nointr(mddev); - + mddev_lock_nointr(mddev); if (!md_is_rdwr(mddev)) { /* * On a read-only array we can: From patchwork Thu Feb 1 09:25:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 195220 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2719:b0:106:209c:c626 with SMTP id hl25csp30210dyb; Thu, 1 Feb 2024 01:32:56 -0800 (PST) X-Google-Smtp-Source: AGHT+IHcFgtkuMpKpun/BExdPHs+REqgZD9X5sFlvHB1p+lkDVWC8vAEVGolAFdLkOMLsy6AF+r3 X-Received: by 2002:a05:6808:424d:b0:3bd:e980:fc58 with SMTP id dp13-20020a056808424d00b003bde980fc58mr1519905oib.52.1706779976145; Thu, 01 Feb 2024 01:32:56 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706779976; cv=pass; d=google.com; s=arc-20160816; b=X9uwo+af1xUeR+eJ6dt5whvyOjre4HxbM1T9jnkAuWNE9vRR4nmzr4CzQyTfrZc3uI CBfsnDWvZ7dPh4Qym622PCR2nteeHlIhWD4dKEH2H5t+4t7n+APT3uYdh9YtJQXFUPZ6 +vcvn+OUd72diHX5t6sA42MwHUjKnWhVW4+qyiADRf8EO5t3RzenLrNXIJ4cCqiq6S27 b5KR0OgmmlH88Y2cWgNwF7Dzq4pEiXaxpPoH7ApRHiNhyYjr0K8S+zlQTP3AWtyJ/NfD Ga6RfWZG3lm9QmCZQRO/+Ulr8IkriQwTi5VoL6wO4Eh3FFqW05cffN6o8knHX59obhhQ nBLQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=2c7chQ78HGOCdj1Eb70sTGCt/X3LxWmlJXF0/NePjQ4=; fh=H91B07heJB58d9sgtcRt4cj8P1RhWCLTu38CLfL/q90=; b=zLvOKE+ME4KJH5rMrKkEKH1KXOoOhPB8Tg/AgP/TlUNaPDtYAhmaXVnn7F18AgLekM gq7KTv3f3jfVw4ejiXwXLrX6U/hS7stbs4IIKhXQ4pGpsAQqBJ2NBP0QwLlixX80ACwt fVjpMalgAWRhZeDqFiNfUZYgNkmjrO7nNYiywkHZd7K2vs7Algkyz84bdr51zUsyT40b WFSt1ny+TvogKYu+vY5fA14a15sA1b3Ez6r9Rzg1LMbUIEjcPfJOk0WcBY5yndoNgKNa BVutATbyHez/+6emI3y7pZPAQDpKCrZMEq6MRMUBznqd3oA3yqPgvXgc3ATrBpwEjzA1 Fvcw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47870-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47870-ouuuleilei=gmail.com@vger.kernel.org" X-Forwarded-Encrypted: i=1; AJvYcCV6zXKwKBP3VqXeUYXBCHmFOPvOWkSoo6SkurwnVgbPuT4VR+WvDs/3v2FVBkZlz49anDIbxuEc7PWAsjZN68B1q3GqVg== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id p9-20020a05622a00c900b0042bf3484953si937834qtw.552.2024.02.01.01.32.56 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Feb 2024 01:32:56 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47870-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47870-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47870-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id E87291C27F39 for ; Thu, 1 Feb 2024 09:32:55 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BE82115F330; Thu, 1 Feb 2024 09:30:36 +0000 (UTC) Received: from dggsgout12.his.huawei.com (unknown [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6941215B990; Thu, 1 Feb 2024 09:30:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779834; cv=none; b=lEpLZ8jXEOpHJRBuHp1DSQhgRlqHuJzSZaOR2PoWIH/mam7h59ihptqksKR1AqNetjK7OK154hPbJgvV79AUl3IENvM7ZcG2KKOoGhaUezHJMfwJuIqVB6E+wUYkI8MWtMUOX2548aPAckbNjNs3SmXerz312in52YN2VDxHdgM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779834; c=relaxed/simple; bh=P78HzqbyoW17tfVjgGA+69KAblnYGxtOVqNCc1oclCo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tykn6fFwNjbVGh4uPTvZF1t7l3WNDCwTek1JD/+QsQj7BI0oOWPh32j9wKZGUlTadiOcFExJ8kl8T86bMv2z248PGLFeAcIyzp0FMs+Y+KDEhRjJNZ+13ACVDQCMRfM8pDVl8EV5mAPUGuoIYPHWjFpwwiCY/dP3wmglruoQn68= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4TQYXx1zxMz4f3l7R; Thu, 1 Feb 2024 17:30:25 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 7855C1A027B; Thu, 1 Feb 2024 17:30:29 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgCXaBGtZLtl8V6KCg--.33515S10; Thu, 01 Feb 2024 17:30:29 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, blazej.kucman@linux.intel.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v5 06/14] md: fix missing release of 'active_io' for flush Date: Thu, 1 Feb 2024 17:25:51 +0800 Message-Id: <20240201092559.910982-7-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240201092559.910982-1-yukuai1@huaweicloud.com> References: <20240201092559.910982-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgCXaBGtZLtl8V6KCg--.33515S10 X-Coremail-Antispam: 1UD129KBjvJXoW7Ww1UCw1Utw1xZF4kXr1fZwb_yoW8WF4fp3 y8tas8tr4UGF98t398JF48WF95W3WUtay09FZxAw13AFnxWFs8K39agrW8Xr95Gr1fJ398 ur4jgr1DWa4UWFUanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUP214x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr1j6r xdM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0D M2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjx v20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1l F7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4IIrI8v6xkF7I0E8cxan2 IY04v7MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAF wI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVW8ZVWrXwCIc4 0Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r1I6r4UMIIF0xvE2Ix0cI8IcVCY1x0267AK xVW8Jr0_Cr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVWUJV W8JwCI42IY6I8E87Iv6xkF7I0E14v26r4UJVWxJrUvcSsGvfC2KfnxnUUI43ZEXa7VUbmZ X7UUUUU== X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789688520276591481 X-GMAIL-MSGID: 1789688520276591481 From: Yu Kuai submit_flushes atomic_set(&mddev->flush_pending, 1); rdev_for_each_rcu(rdev, mddev) atomic_inc(&mddev->flush_pending); bi->bi_end_io = md_end_flush submit_bio(bi); /* flush io is done first */ md_end_flush if (atomic_dec_and_test(&mddev->flush_pending)) percpu_ref_put(&mddev->active_io) -> active_io is not released if (atomic_dec_and_test(&mddev->flush_pending)) -> missing release of active_io For consequence, mddev_suspend() will wait for 'active_io' to be zero forever. Fix this problem by releasing 'active_io' in submit_flushes() if 'flush_pending' is decreased to zero. Fixes: fa2bbff7b0b4 ("md: synchronize flush io with array reconfiguration") Reported-by: Blazej Kucman Closes: https://lore.kernel.org/lkml/20240130172524.0000417b@linux.intel.com/ Signed-off-by: Yu Kuai --- drivers/md/md.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 85fde05c37dd..9e41a9aaba8b 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -579,8 +579,12 @@ static void submit_flushes(struct work_struct *ws) rcu_read_lock(); } rcu_read_unlock(); - if (atomic_dec_and_test(&mddev->flush_pending)) + if (atomic_dec_and_test(&mddev->flush_pending)) { + /* The pair is percpu_ref_get() from md_flush_request() */ + percpu_ref_put(&mddev->active_io); + queue_work(md_wq, &mddev->flush_work); + } } static void md_submit_flush_data(struct work_struct *ws) From patchwork Thu Feb 1 09:25:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 195222 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2719:b0:106:209c:c626 with SMTP id hl25csp30535dyb; Thu, 1 Feb 2024 01:33:40 -0800 (PST) X-Google-Smtp-Source: AGHT+IEppRaMOi0GZSywjVBWSbqBlO/D8fI2p6ojUj5iqnsQhRdewhr8WwnhMS5DPUIWm9ezWIYV X-Received: by 2002:a05:620a:450f:b0:783:915c:1fc4 with SMTP id t15-20020a05620a450f00b00783915c1fc4mr2200359qkp.20.1706780020667; Thu, 01 Feb 2024 01:33:40 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706780020; cv=pass; d=google.com; s=arc-20160816; b=A8Tjqeecp6Bb0GGri2US3MgE9P1pKqeKzZve5vCRAB0s+DcV5uNDMA4kuViMdLV/8d M/GyG2LVmz128jVGzy6NsBog0KsJhfgydOcVuItNQPCOHaSnS/Zsh8q4Zed0yIII1r0M 8so1ZKPmuJCZ7n29qiGWxTnJSqz2Pol9eU/olkGFRCn3xmIzA2sC+2La4Nnl6QKZ3w37 ko0q0bCZVaoAVN4WngNmb2EEsns7Nq09QZsqvoxFWRQqXUCLET1utlGQpz0e49+BP6sD s9nUUn9SYg65sPZbWsgx8k7SqkZLFFCMGDlf31iPhTbWwyG8BG3c9wdUDMJAbZz32V7L 8Odg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=b3eJ83DrviuTqQQ68RGTsQGR0o7HCSRQU/aKwxPS4Cw=; fh=DTliBXooeebxRJ74TDpr0yJW1IdUrMVl4+4KQuEojLQ=; b=GIuYS18EO0TokSI06gbtWFRwIcIWNqJBqhX+ewcwbt/+WflkVBtYO0LguELppfK2A8 V8/dtAI67Fgfr/y1ve6GF33e9MzhpdzwwjvgGGPhntirCjv09wQj4IT0W48WURJbKljX rrrihk4/aUydvw31eKGLkXLcvGJkgq000TBCgC5HhY6g/1CrOO3DppZ3Oq8kne0U0fOW NLfspqauwJLaOfRpRncU8bzOJe0mmUP2kFXoGOYEZBYUL0IvTM20/i17WL954rOcU54t Vqt81ah6heW4LLyorwuoXGxXZh/Te1RiiENT+aKLlCxa5Pz8aORxgGUCSxPojZqO1hyA qLkA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47871-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47871-ouuuleilei=gmail.com@vger.kernel.org" X-Forwarded-Encrypted: i=1; AJvYcCXGWxvbLwRFqUwFFFactyMEYibpM4BUb6tPDt/8YkmZu1ChKt4BSQ2lu6VEGn9kJ7w8YV6BDALD5YWRpzo6UrlPcrkbHA== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id vr15-20020a05620a55af00b00783e118ed11si12015191qkn.482.2024.02.01.01.33.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Feb 2024 01:33:40 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47871-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47871-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47871-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 710CD1C27D08 for ; Thu, 1 Feb 2024 09:33:40 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 524E4161B46; Thu, 1 Feb 2024 09:30:38 +0000 (UTC) Received: from dggsgout12.his.huawei.com (unknown [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 55D6115B965; Thu, 1 Feb 2024 09:30:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779835; cv=none; b=Fcne0xVoYDYygn5n1eS8uMH+5/6U3+qnZPY/XYXzF60XSDBmPFTb8dPb/8FFhuxnTf4CzLvvuPdu/yzt9KueKV/uV9vSUfGkUkrHckl0eJxlCCcmrcA+VFgGyWYrTc7iQjaX1ZOpe+Svmeq+OJV0xCB++jb+S0lJ49Bqte3Twf8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779835; c=relaxed/simple; bh=NZ1TzNGo0Z0m3MqtenTynFlNQ6IJhqAzTBTv1Jw/zxo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=R0c3ai94OwiLxCbTzAwdC30R6LnCbNloBy/YqfSgsboc3BDaCJE0XyWFdskKJL115fx1IdcfnwvsevC49eafOkb0CIKHhwgWXuoz+xAwQRGg4QT9FUbLyiMfAEQYqdMow03iH9vXUA+xL+2UF5I9pBeu+0W1wilCji4vVAB3BAw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4TQYXy1qNxz4f3l7V; Thu, 1 Feb 2024 17:30:26 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 731E01A038B; Thu, 1 Feb 2024 17:30:30 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgCXaBGtZLtl8V6KCg--.33515S11; Thu, 01 Feb 2024 17:30:30 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, blazej.kucman@linux.intel.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v5 07/14] md: export helpers to stop sync_thread Date: Thu, 1 Feb 2024 17:25:52 +0800 Message-Id: <20240201092559.910982-8-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240201092559.910982-1-yukuai1@huaweicloud.com> References: <20240201092559.910982-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgCXaBGtZLtl8V6KCg--.33515S11 X-Coremail-Antispam: 1UD129KBjvJXoWxZryrJF1rCF1fuFyDGF1Utrb_yoW5CFWrpr WIqF95Ar4YyrZxZry7Ja4DuayYqwn2gFyqyry3C3yfJ3WftrsrGF1Yv3WUAFWkGa4fXr1U ta15KFW3uFy8Kr7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUP214x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr1j6r xdM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0D M2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjx v20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1l F7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4IIrI8v6xkF7I0E8cxan2 IY04v7MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAF wI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVW8ZVWrXwCIc4 0Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r1I6r4UMIIF0xvE2Ix0cI8IcVCY1x0267AK xVW8Jr0_Cr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVWUJV W8JwCI42IY6I8E87Iv6xkF7I0E14v26r4UJVWxJrUvcSsGvfC2KfnxnUUI43ZEXa7VUbmZ X7UUUUU== X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789688566970417638 X-GMAIL-MSGID: 1789688566970417638 From: Yu Kuai The new heleprs will be used in dm-raid in later patches to fix regressions and prevent calling md_reap_sync_thread() directly. Signed-off-by: Yu Kuai --- drivers/md/md.c | 33 +++++++++++++++++++++++++++++++-- drivers/md/md.h | 3 +++ 2 files changed, 34 insertions(+), 2 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 9e41a9aaba8b..19f16a77cf9c 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -4919,16 +4919,45 @@ static void stop_sync_thread(struct mddev *mddev, bool locked, bool check_seq) mddev_lock_nointr(mddev); } +void md_idle_sync_thread(struct mddev *mddev) +{ + lockdep_assert_held(&mddev->reconfig_mutex); + + clear_bit(MD_RECOVERY_FROZEN, &mddev->recovery); + stop_sync_thread(mddev, true, true); +} +EXPORT_SYMBOL_GPL(md_idle_sync_thread); + +void md_frozen_sync_thread(struct mddev *mddev) +{ + lockdep_assert_held(&mddev->reconfig_mutex); + + set_bit(MD_RECOVERY_FROZEN, &mddev->recovery); + stop_sync_thread(mddev, true, false); +} +EXPORT_SYMBOL_GPL(md_frozen_sync_thread); + +void md_unfrozen_sync_thread(struct mddev *mddev) +{ + lockdep_assert_held(&mddev->reconfig_mutex); + + clear_bit(MD_RECOVERY_FROZEN, &mddev->recovery); + set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); + md_wakeup_thread(mddev->thread); + sysfs_notify_dirent_safe(mddev->sysfs_action); +} +EXPORT_SYMBOL_GPL(md_unfrozen_sync_thread); + static void idle_sync_thread(struct mddev *mddev) { mutex_lock(&mddev->sync_mutex); - clear_bit(MD_RECOVERY_FROZEN, &mddev->recovery); if (mddev_lock(mddev)) { mutex_unlock(&mddev->sync_mutex); return; } + clear_bit(MD_RECOVERY_FROZEN, &mddev->recovery); stop_sync_thread(mddev, false, true); mutex_unlock(&mddev->sync_mutex); } @@ -4936,13 +4965,13 @@ static void idle_sync_thread(struct mddev *mddev) static void frozen_sync_thread(struct mddev *mddev) { mutex_lock(&mddev->sync_mutex); - set_bit(MD_RECOVERY_FROZEN, &mddev->recovery); if (mddev_lock(mddev)) { mutex_unlock(&mddev->sync_mutex); return; } + set_bit(MD_RECOVERY_FROZEN, &mddev->recovery); stop_sync_thread(mddev, false, false); mutex_unlock(&mddev->sync_mutex); } diff --git a/drivers/md/md.h b/drivers/md/md.h index 8d881cc59799..437ab70ce79b 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -781,6 +781,9 @@ extern void md_rdev_clear(struct md_rdev *rdev); extern void md_handle_request(struct mddev *mddev, struct bio *bio); extern int mddev_suspend(struct mddev *mddev, bool interruptible); extern void mddev_resume(struct mddev *mddev); +extern void md_idle_sync_thread(struct mddev *mddev); +extern void md_frozen_sync_thread(struct mddev *mddev); +extern void md_unfrozen_sync_thread(struct mddev *mddev); extern void md_reload_sb(struct mddev *mddev, int raid_disk); extern void md_update_sb(struct mddev *mddev, int force); From patchwork Thu Feb 1 09:25:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 195223 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2719:b0:106:209c:c626 with SMTP id hl25csp30602dyb; Thu, 1 Feb 2024 01:33:49 -0800 (PST) X-Google-Smtp-Source: AGHT+IFO5ZGF331Bjj8yCHBMiZ5H3cGwZ/fralVLdL/k6qKNLCPnXDrRiwFu/mT6U8UefM4ZYF6I X-Received: by 2002:aa7:df1a:0:b0:55f:3d24:e64d with SMTP id c26-20020aa7df1a000000b0055f3d24e64dmr1152251edy.3.1706780029524; Thu, 01 Feb 2024 01:33:49 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706780029; cv=pass; d=google.com; s=arc-20160816; b=b60D1auQgBFREQjBd8c1bW9loN/oGQUVuOOE+neA7zaABrRG6OwWDjIoKlvscJHxC4 OccX0uj21PcEBBwxw/VC7rw95SXqeJhJggz53WqjO/dnngwrTmel55mz6y0Ti/6Cq46x 9Hmwphkai/YBNDiKeWHoDjRDXIhhGi+XJwYszyWN+Y9/yLLRxFrnNH8cBijs8j0BOAer 9WxuqkNKnvUihFZNKYwEcpsHT5w3JwBlfxkgmk4ia30xbtCuLnFHM8dwTQC7djYeGHHN zs4PXSwmfrBcjA8Bl6SllEODVOvy5DfqMKhr0Gv6/Qu5QKEWNXdJQJv1H2aSy5CK+JlF vpwA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=t/Rfjl2seGG+IetcoK+NDdWeNbaN7vNh3uonOlKoaDA=; fh=AfgDUp6EEcEVbV7MNLN8kX6dqh71F4C9TcPvh6KsvOU=; b=TnxIfEtX6oTQ6yg/UR/zOtOvTi3ymoruR314Q1udTPBrxh/SDNNslHjyGWpHux83DI 8pFJOy46e0YdDatX4WYfh7fZ7QU4n2YMgbz21ebgcv3CGRwGdZdQTVwPNYgqpWuNu2Zt uiXdWp5YEKHcS4VTjlGkynwM2nV0GynhT9OVVYTvdNzVu1pG+IiB69P2jpwlyDbTIeGO LzY95aLyyONr0bHCdA3IREfmaQ+GWjQLzEtvphA+1E+LquhXr7x4NWfJl+fsX6DgIfda neLJXG+G4Vk/8/S9mVAydmKT1LatWE8YW9ME08B1YU6TjIbnLZdbPOJ+gTv2CjckGwFV kgjg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47872-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47872-ouuuleilei=gmail.com@vger.kernel.org" X-Forwarded-Encrypted: i=1; AJvYcCU4Aa8UP2AvRvISVWNiEdTh7BjEWUjTpnQU6V4AsuJXryxMt48anZOJ9QBb/vzuM8yKSroQ8ToMmSWBrJFhFAOn15XFYQ== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id b19-20020a0564021f1300b0055f179146easi3530742edb.651.2024.02.01.01.33.49 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Feb 2024 01:33:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47872-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47872-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47872-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 25C0E1F25A0C for ; Thu, 1 Feb 2024 09:33:49 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D3583161B56; Thu, 1 Feb 2024 09:30:38 +0000 (UTC) Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 62FF815D5BE; Thu, 1 Feb 2024 09:30:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779836; cv=none; b=QDU2qdDSimbgKemqBhypvU1lmp/GQElPfWmA7F8cit9un8moXnX1jlDwCeygc+F/Zd/sONb+nd0S9OO0p3r9zLzoPKt7ooraGtkv04yCpxho2NWY9gsQs3rn6Y5KMnI5+gap493JiNUVrT2I/CiWeL/YTj/RcAAKsrYi7w+osCM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779836; c=relaxed/simple; bh=vPUwDnj2r3EpC4pjhrctEkLZPrJIzYpfNoWsG8fF+D8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=eLtXtk4pjPjbd6OjfrCiqEf8mZ1V33QElLft0qtcwDrFUPVN7jxaaFxVRYnIutWnzA0EkQGdg9koo17U3PxecBlKq9xLyckRKEHASGlc3EYALGXEasDX0MdSawu5nrP/493bpA3Tat1Ud2ZcOWFmVxr7QE28LgJFMADjPOM7XWw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4TQYY04MC4z4f3kk9; Thu, 1 Feb 2024 17:30:28 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 32A511A0199; Thu, 1 Feb 2024 17:30:31 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgCXaBGtZLtl8V6KCg--.33515S12; Thu, 01 Feb 2024 17:30:30 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, blazej.kucman@linux.intel.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v5 08/14] md: export helper md_is_rdwr() Date: Thu, 1 Feb 2024 17:25:53 +0800 Message-Id: <20240201092559.910982-9-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240201092559.910982-1-yukuai1@huaweicloud.com> References: <20240201092559.910982-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgCXaBGtZLtl8V6KCg--.33515S12 X-Coremail-Antispam: 1UD129KBjvJXoW7Kr47Kr4xtrykGw13uF4fGrg_yoW8XFWrpa 9agFy3Ar4UZF43X34UJayDua45Z3WfKrWqkry3u393Za43XrZ8CF1rGFWDKrWkWFyfAF1a qa15JF1UuF1vkw7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUP214x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr1j6r xdM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0D M2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjx v20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1l F7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4IIrI8v6xkF7I0E8cxan2 IY04v7MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAF wI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVW8ZVWrXwCIc4 0Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r1I6r4UMIIF0xvE2Ix0cI8IcVCY1x0267AK xVW8Jr0_Cr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVW8JV WxJwCI42IY6I8E87Iv6xkF7I0E14v26r4UJVWxJrUvcSsGvfC2KfnxnUUI43ZEXa7VUbmZ X7UUUUU== X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789688576382895780 X-GMAIL-MSGID: 1789688576382895780 From: Yu Kuai There are no functional changes for now, prepare to fix a deadlock for dm-raid456. Signed-off-by: Yu Kuai --- drivers/md/md.c | 12 ------------ drivers/md/md.h | 12 ++++++++++++ 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 19f16a77cf9c..2c245341148a 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -99,18 +99,6 @@ static void mddev_detach(struct mddev *mddev); static void export_rdev(struct md_rdev *rdev, struct mddev *mddev); static void md_wakeup_thread_directly(struct md_thread __rcu *thread); -enum md_ro_state { - MD_RDWR, - MD_RDONLY, - MD_AUTO_READ, - MD_MAX_STATE -}; - -static bool md_is_rdwr(struct mddev *mddev) -{ - return (mddev->ro == MD_RDWR); -} - /* * Default number of read corrections we'll attempt on an rdev * before ejecting it from the array. We divide the read error diff --git a/drivers/md/md.h b/drivers/md/md.h index 437ab70ce79b..684cc866402a 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -558,6 +558,18 @@ enum recovery_flags { MD_RESYNCING_REMOTE, /* remote node is running resync thread */ }; +enum md_ro_state { + MD_RDWR, + MD_RDONLY, + MD_AUTO_READ, + MD_MAX_STATE +}; + +static inline bool md_is_rdwr(struct mddev *mddev) +{ + return (mddev->ro == MD_RDWR); +} + static inline int __must_check mddev_lock(struct mddev *mddev) { return mutex_lock_interruptible(&mddev->reconfig_mutex); From patchwork Thu Feb 1 09:25:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 195225 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2719:b0:106:209c:c626 with SMTP id hl25csp30774dyb; Thu, 1 Feb 2024 01:34:16 -0800 (PST) X-Google-Smtp-Source: AGHT+IHaRjjtdpQpVQEKgZ2TSVMYonWMmUZk8/LkoTlStgk8estwSfexzaognAHgQ4+hQeHJIxPt X-Received: by 2002:a17:906:f296:b0:a30:e4d8:2e46 with SMTP id gu22-20020a170906f29600b00a30e4d82e46mr3065610ejb.20.1706780056510; Thu, 01 Feb 2024 01:34:16 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706780056; cv=pass; d=google.com; s=arc-20160816; b=SawQNInhBG8r3m5jFzl93914/YSP5HHZM5uGrmvMyXYIYLzc9AlYNSmXpAteCe2Zdp CDoffV08ZcN3xa8A4tPJdSdEWSXcUjH907eL9wUHsl9yvBgDiUvKA5JpsoO0EC+CjAN8 cbRDHdtkTuJOHc9PDiAqjzFL0jUhr7ncNAI50j8El0CbrQHw/Ww6m/zBvGIpuQ0FQbd9 D+80Nl4hpKE6v05KidugcGXf1J6Gafsa4tF4uN5EGEqbbkfxIkyoILjTbAKojq58j4zP DUivwMOdBzeupPxmcQtqkjSHoRoQtDwAwe+sldkGcTz2Ybv4+7Jrrjs3xND0IdlQVIMT w24Q== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=kQ7iVhVb18spXoVAwrqbgq++EMzPS7foKEKx8QVjwOM=; fh=YGDlfKXAQItOpUo8arSe0WCUsb2355D6MfGSLCDLkHw=; b=qf8hyX+dh+dm62fDgIiWe7fh7Vk/IvMacIQEKSGEqPMhg3CuilHvVlLdIakba/8ioN mzl1NDSikhzAgBussQ5cPpDHNeJYLfeRmKl/ag3x2+UuDe8g1NOun13KgYrdCQqY1YrC +nMza3jHDDOvThHStwSx6Brs6FrCa9JfMcWEc7Pii6EpbWKlCx2e0xfGbxaOyDgZik4Q VfW2qlJ7meHIahpTGHh7UC+PuNwY54GK9xFLxzZ3QmukTxvAiub+KhvnpA9AApD/Oeb0 xBpKvs+9TL0QLAt8Xq2Ua+KiVKbWbHCbClCdI/a4rCFisaIV/Y9zH+RSrLxfs98f6ulZ d8CQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47873-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47873-ouuuleilei=gmail.com@vger.kernel.org" X-Forwarded-Encrypted: i=1; AJvYcCXCQNhllztc95mrRqEewBFFlQk0vn2xmFHDs+CyNp9JJTiDpeeDy0Y3MorMzzk83kfvh+fLZSurxc/FwlRTWRuQNWOOxw== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id gq19-20020a170906e25300b00a366d4e7ba1si1622028ejb.510.2024.02.01.01.34.16 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Feb 2024 01:34:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47873-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47873-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47873-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id EA7A31F277A1 for ; Thu, 1 Feb 2024 09:34:15 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 007A5163A8A; Thu, 1 Feb 2024 09:30:39 +0000 (UTC) Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 49B6D15DBBF; Thu, 1 Feb 2024 09:30:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779837; cv=none; b=jztwQsjLR/vrwUFxlzTICGWZ1lmNuzFXA3yKDc/2do0lVIin9QPdDqxwpa5aSTFs4R0U2JlPBm/rKirirzt//y6DdkPouDqJZSvaBRYLoRBezfPuzcKlCQsvMT5ZRjeyqrCPZq7MU3B+SqHwy9EgVJfsHhtTtlPNg9CBKvFBvfA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779837; c=relaxed/simple; bh=6BrWeORSG3spzobIm2H1STSTTDZ6J8PBa6b1+irB9U4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=oEP+kRDjKJTkourRzzHkm99elpA7m2b1nPLMG2mKGYF/8bdYy9D1/7cWeiTXXiBTqLz96V/ed8qKhQPXiCkJBTmlyMuvWxLiFdjlPSO3OegLvvu0Tkq9vu8vz5gDnhNMI5TSEGvOejuWSPR55v6pMtUj3aDGj+0qSFxGxB5d8EE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.93.142]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4TQYY01NkSz4f3l7c; Thu, 1 Feb 2024 17:30:28 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 668C51A0175; Thu, 1 Feb 2024 17:30:32 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgCXaBGtZLtl8V6KCg--.33515S13; Thu, 01 Feb 2024 17:30:32 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, blazej.kucman@linux.intel.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v5 09/14] dm-raid: really frozen sync_thread during suspend Date: Thu, 1 Feb 2024 17:25:54 +0800 Message-Id: <20240201092559.910982-10-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240201092559.910982-1-yukuai1@huaweicloud.com> References: <20240201092559.910982-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgCXaBGtZLtl8V6KCg--.33515S13 X-Coremail-Antispam: 1UD129KBjvJXoWxur1xXF4fWry8tryDtFW3trb_yoWrKw15pa yrtws0vw48JrW7Za9Fy3WkXFWYvwnIgrWUtr93WayrJ3WSkws3ury8Kw47ZFWDtFyxJa4Y yr4Dtw45uFWjgFJanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUP214x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr1j6r xdM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0D M2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjx v20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1l F7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4IIrI8v6xkF7I0E8cxan2 IY04v7MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAF wI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVW8ZVWrXwCIc4 0Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r1I6r4UMIIF0xvE2Ix0cI8IcVCY1x0267AK xVW8Jr0_Cr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVW8JV WxJwCI42IY6I8E87Iv6xkF7I0E14v26r4UJVWxJrUvcSsGvfC2KfnxnUUI43ZEXa7VUbmZ X7UUUUU== X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789688604563616859 X-GMAIL-MSGID: 1789688604563616859 From: Yu Kuai 1) The flag MD_RECOVERY_FROZEN doesn't mean that sync thread is frozen, it only prevent new sync_thread to start, and it can't stop the running sync thread; 2) The flag MD_RECOVERY_FROZEN doesn't mean that writes are stopped, use it as condition for md_stop_writes() in raid_postsuspend() doesn't look correct. 3) raid_message can set/clear the flag MD_RECOVERY_FROZEN at anytime, and if MD_RECOVERY_FROZEN is cleared while the array is suspended, new sync_thread can start unexpected. Fix above problems by using the new helper to suspend the array during suspend, also disallow raid_message() to change sync_thread status during suspend. Note that after commit f52f5c71f3d4 ("md: fix stopping sync thread"), the test shell/lvconvert-raid-reshape.sh start to hang in stop_sync_thread(), and with previous fixes, the test won't hang there anymore, however, the test will still fail and complain that ext4 is corrupted. And with this patch, the test won't hang due to stop_sync_thread() or fail due to ext4 is corrupted anymore. However, there is still a deadlock related to dm-raid456 that will be fixed in following patches. Reported-by: Mikulas Patocka Closes: https://lore.kernel.org/all/e5e8afe2-e9a8-49a2-5ab0-958d4065c55e@redhat.com/ Fixes: 1af2048a3e87 ("dm raid: fix deadlock caused by premature md_stop_writes()") Fixes: 9dbd1aa3a81c ("dm raid: add reshaping support to the target") Fixes: f52f5c71f3d4 ("md: fix stopping sync thread") Signed-off-by: Yu Kuai --- drivers/md/dm-raid.c | 38 +++++++++++++++++++++++++++++--------- 1 file changed, 29 insertions(+), 9 deletions(-) diff --git a/drivers/md/dm-raid.c b/drivers/md/dm-raid.c index eb009d6bb03a..5ce3c6020b1b 100644 --- a/drivers/md/dm-raid.c +++ b/drivers/md/dm-raid.c @@ -3240,11 +3240,12 @@ static int raid_ctr(struct dm_target *ti, unsigned int argc, char **argv) rs->md.ro = 1; rs->md.in_sync = 1; - /* Keep array frozen until resume. */ - set_bit(MD_RECOVERY_FROZEN, &rs->md.recovery); - /* Has to be held on running the array */ mddev_suspend_and_lock_nointr(&rs->md); + + /* Keep array frozen until resume. */ + md_frozen_sync_thread(&rs->md); + r = md_run(&rs->md); rs->md.in_sync = 0; /* Assume already marked dirty */ if (r) { @@ -3722,6 +3723,9 @@ static int raid_message(struct dm_target *ti, unsigned int argc, char **argv, if (!mddev->pers || !mddev->pers->sync_request) return -EINVAL; + if (test_bit(RT_FLAG_RS_SUSPENDED, &rs->runtime_flags)) + return -EBUSY; + if (!strcasecmp(argv[0], "frozen")) set_bit(MD_RECOVERY_FROZEN, &mddev->recovery); else @@ -3791,15 +3795,31 @@ static void raid_io_hints(struct dm_target *ti, struct queue_limits *limits) blk_limits_io_opt(limits, chunk_size_bytes * mddev_data_stripes(rs)); } +static void raid_presuspend(struct dm_target *ti) +{ + struct raid_set *rs = ti->private; + + mddev_lock_nointr(&rs->md); + md_frozen_sync_thread(&rs->md); + mddev_unlock(&rs->md); +} + +static void raid_presuspend_undo(struct dm_target *ti) +{ + struct raid_set *rs = ti->private; + + mddev_lock_nointr(&rs->md); + md_unfrozen_sync_thread(&rs->md); + mddev_unlock(&rs->md); +} + static void raid_postsuspend(struct dm_target *ti) { struct raid_set *rs = ti->private; if (!test_and_set_bit(RT_FLAG_RS_SUSPENDED, &rs->runtime_flags)) { /* Writes have to be stopped before suspending to avoid deadlocks. */ - if (!test_bit(MD_RECOVERY_FROZEN, &rs->md.recovery)) - md_stop_writes(&rs->md); - + md_stop_writes(&rs->md); mddev_suspend(&rs->md, false); } } @@ -4012,8 +4032,6 @@ static int raid_preresume(struct dm_target *ti) } /* Check for any resize/reshape on @rs and adjust/initiate */ - /* Be prepared for mddev_resume() in raid_resume() */ - set_bit(MD_RECOVERY_FROZEN, &mddev->recovery); if (mddev->recovery_cp && mddev->recovery_cp < MaxSector) { set_bit(MD_RECOVERY_REQUESTED, &mddev->recovery); mddev->resync_min = mddev->recovery_cp; @@ -4056,9 +4074,9 @@ static void raid_resume(struct dm_target *ti) rs_set_capacity(rs); mddev_lock_nointr(mddev); - clear_bit(MD_RECOVERY_FROZEN, &mddev->recovery); mddev->ro = 0; mddev->in_sync = 0; + md_unfrozen_sync_thread(mddev); mddev_unlock_and_resume(mddev); } } @@ -4074,6 +4092,8 @@ static struct target_type raid_target = { .message = raid_message, .iterate_devices = raid_iterate_devices, .io_hints = raid_io_hints, + .presuspend = raid_presuspend, + .presuspend_undo = raid_presuspend_undo, .postsuspend = raid_postsuspend, .preresume = raid_preresume, .resume = raid_resume, From patchwork Thu Feb 1 09:25:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 195224 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2719:b0:106:209c:c626 with SMTP id hl25csp30765dyb; Thu, 1 Feb 2024 01:34:15 -0800 (PST) X-Google-Smtp-Source: AGHT+IEdRLmBh9iYrjsamChyAixhxaTUdkOSDgNbM0JiGhD17UK40Q0+g8A9AkYNh30c16cgf7dr X-Received: by 2002:a17:906:3d52:b0:a35:d69f:e4e2 with SMTP id q18-20020a1709063d5200b00a35d69fe4e2mr3002208ejf.30.1706780055556; Thu, 01 Feb 2024 01:34:15 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706780055; cv=pass; d=google.com; s=arc-20160816; b=VDJA6vTntW0pRPjS10u6/kN8zIkuUicdgS9MnavplqB8PO5JFrl6DmXtiXhtpCSgqD 1MTWifl9lhtqv4noztx4yiETpHB6V6xdxRc8iSJimaiIzcYNQJIqshQFZfOCfypAf/xI HlcDx+kyHMh5bUqx/2cdbbOPZfYPT0WTK819qJpD1AKjagOQ+inWLCVzMH1/JxKwOy1j GiN4V5GtbcgknV2pMEilijhoZe+0m7ZU77oO0Uj+cuhyOw+fsq8bfsWYTg6zih8KiNP4 bWsy+OJbAIm06Eh7lXHrnCZG0+QUXCkkO60+3Chj/pgCW+MTlf8O44CF8D8Yiv/29hbJ 130w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=0h6sLTtSXWd4NNSNFaiFXDbIJAObc1SgGd1OcMYigCo=; fh=eur1kFM2Dhm4janP2ic2m3Us4wQ7RR6bf+Xr5KV4Gdo=; b=P5PFamepDXW7rc6wUF2MxMxxzAEZb4n7kWnK3FX0uRKyQm9BHtITD5KNfEAgQmMMTZ ml5ZA06gX+yiKqOvat4yggC4zjz45x7iw+xsCA+T/pRurU2L5TmI3teQh/kv0B+Tbbq4 SG94hkx0rQTjoQrMrv4Wzcf7LZw5o9L6cKFtBwSVWTiRe0YOT63/jQG8Ip4uA80BY9J9 hBH812wWnx60/3uwDAcnthoF7k20b+9kgVKRZtNa78AFlty0KpCFVbBFzpBnZu0C/+by F1FX1gfFpxSMrrjiSv2sc/AVLXWkwxxuREQa8d9vkw1yIZtjGYnvEf293jFKdNB7828J XJhA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47874-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47874-ouuuleilei=gmail.com@vger.kernel.org" X-Forwarded-Encrypted: i=1; AJvYcCVEG+wuQcm8q3huL6xDjmJVTG093YItW2QmaB7+gIeYcG/2gvUkbtZ0FoFn4udmoGn/TJGxVK1LZuCoe/+5zUoGtfavcg== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id t13-20020a170906064d00b00a31898cdf7bsi6628075ejb.136.2024.02.01.01.34.15 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Feb 2024 01:34:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47874-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47874-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47874-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id D983B1F266FD for ; Thu, 1 Feb 2024 09:34:14 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id EC552163A87; Thu, 1 Feb 2024 09:30:39 +0000 (UTC) Received: from dggsgout12.his.huawei.com (unknown [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 010A815F30D; Thu, 1 Feb 2024 09:30:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779837; cv=none; b=Yqfu25Wv87L3iQFJovSnOoE/LE38pBmcFBQl+r0/LYcmGg+oTOvB95pB4LdhwTMOaPIulGT2j+F+oZRHK62QEul4wXone4tidUxls6qtskldh43IaqCZiZtmsvtzoyoqqWZw3tE9oFZFa0QeUXI8zcnw8EV6OijcersUpqr73U0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779837; c=relaxed/simple; bh=orTUf8OIgVO5EnJ8e56Og6bXoRyUdV4mYFLO+shF788=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=EHdzCpQLEF8onvf7f2R2WU7u97oj7Zzzedhaqbm/eOoGZaFZipdCLCCPllpKA3wphonzvvKq2zXi4hMZxsTeKi0TE9DsyROHGgGCiHDr4JyISgvOt8+k3KEP2dUc61aiLtyiTIRZo0cxf+/V8UBfe1og5epDghhB734dVopI5k0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4TQYY06qCgz4f3l7f; Thu, 1 Feb 2024 17:30:28 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 2A1781A0232; Thu, 1 Feb 2024 17:30:33 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgCXaBGtZLtl8V6KCg--.33515S14; Thu, 01 Feb 2024 17:30:32 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, blazej.kucman@linux.intel.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v5 10/14] md/dm-raid: don't call md_reap_sync_thread() directly Date: Thu, 1 Feb 2024 17:25:55 +0800 Message-Id: <20240201092559.910982-11-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240201092559.910982-1-yukuai1@huaweicloud.com> References: <20240201092559.910982-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgCXaBGtZLtl8V6KCg--.33515S14 X-Coremail-Antispam: 1UD129KBjvJXoW7AF1xAF18KFyDuFy5Ww48JFb_yoW8Kw4fpa yakas8Ar48JrW3ZFsrt3WDWayFv3ZFq3yqyFWfG3y3JF1fKF43WF1j9a17ZFWDJFWfta4U XFZ8tF45uF4YqFJanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUP214x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr1j6r xdM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0D M2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjx v20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1l F7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4IIrI8v6xkF7I0E8cxan2 IY04v7MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAF wI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVW8ZVWrXwCIc4 0Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r4j6ryUMIIF0xvE2Ix0cI8IcVCY1x0267AK xVW8Jr0_Cr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVW8JV WxJwCI42IY6I8E87Iv6xkF7I0E14v26r4UJVWxJrUvcSsGvfC2KfnxnUUI43ZEXa7VUbmZ X7UUUUU== X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789688603466514060 X-GMAIL-MSGID: 1789688603466514060 From: Yu Kuai Currently md_reap_sync_thread() is called from raid_message() directly without holding 'reconfig_mutex', this is definitely unsafe because md_reap_sync_thread() can change many fields that is protected by 'reconfig_mutex'. However, hold 'reconfig_mutex' here is still problematic because this will cause deadlock, for example, commit 130443d60b1b ("md: refactor idle/frozen_sync_thread() to fix deadlock"). Fix this problem by using stop_sync_thread() to unregister sync_thread, like md/raid did. Fixes: be83651f0050 ("DM RAID: Add message/status support for changing sync action") Signed-off-by: Yu Kuai --- drivers/md/dm-raid.c | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/drivers/md/dm-raid.c b/drivers/md/dm-raid.c index 5ce3c6020b1b..6b6c011d9f69 100644 --- a/drivers/md/dm-raid.c +++ b/drivers/md/dm-raid.c @@ -3719,6 +3719,7 @@ static int raid_message(struct dm_target *ti, unsigned int argc, char **argv, { struct raid_set *rs = ti->private; struct mddev *mddev = &rs->md; + int ret = 0; if (!mddev->pers || !mddev->pers->sync_request) return -EINVAL; @@ -3726,17 +3727,24 @@ static int raid_message(struct dm_target *ti, unsigned int argc, char **argv, if (test_bit(RT_FLAG_RS_SUSPENDED, &rs->runtime_flags)) return -EBUSY; - if (!strcasecmp(argv[0], "frozen")) - set_bit(MD_RECOVERY_FROZEN, &mddev->recovery); - else - clear_bit(MD_RECOVERY_FROZEN, &mddev->recovery); + if (!strcasecmp(argv[0], "frozen")) { + ret = mddev_lock(mddev); + if (ret) + return ret; - if (!strcasecmp(argv[0], "idle") || !strcasecmp(argv[0], "frozen")) { - if (mddev->sync_thread) { - set_bit(MD_RECOVERY_INTR, &mddev->recovery); - md_reap_sync_thread(mddev); - } - } else if (decipher_sync_action(mddev, mddev->recovery) != st_idle) + md_frozen_sync_thread(mddev); + mddev_unlock(mddev); + } else if (!strcasecmp(argv[0], "idle")) { + ret = mddev_lock(mddev); + if (ret) + return ret; + + md_idle_sync_thread(mddev); + mddev_unlock(mddev); + } + + clear_bit(MD_RECOVERY_FROZEN, &mddev->recovery); + if (decipher_sync_action(mddev, mddev->recovery) != st_idle) return -EBUSY; else if (!strcasecmp(argv[0], "resync")) ; /* MD_RECOVERY_NEEDED set below */ From patchwork Thu Feb 1 09:25:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 195226 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2719:b0:106:209c:c626 with SMTP id hl25csp30960dyb; Thu, 1 Feb 2024 01:34:40 -0800 (PST) X-Google-Smtp-Source: AGHT+IF6pajeuZ3BpUXZJPzYtqXgUGoN4KAjDBLxmWlk6oywlSMr+7GWKjWVluIF11lemqL2Cldq X-Received: by 2002:a17:90b:1052:b0:295:d2b5:699b with SMTP id gq18-20020a17090b105200b00295d2b5699bmr4282991pjb.9.1706780080800; Thu, 01 Feb 2024 01:34:40 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706780080; cv=pass; d=google.com; s=arc-20160816; b=xhXSP0L0d8ffCBUPlsKSFWFvlNs4qDFRU8sKjbiYJdmrAWgnyEGPuWJWTBYer2A4In onJ7TrxRVYeVJBeG1CHVJr2CvhJ+6u8j2Az1XKT7dSMLJiuGIEc61S4fDXRL/uTHiphn wkaxCNUXinc70lm8QiT+0AsbnqExC4BHyMpmxUZ9HUaGMno+BpTvRTfVQFeLBjo/wzR7 xsTjpSGUOBp72ZO2LqCFWGWWl2XEEwXCuhpEKXtfzZWepDHdI89qATeXC44l0g5XTEr/ Mzuxm2GzTCAJS3VskysL4oC1SxtbtUfnvyTdI/GtGwgaMN9XVwebP0vDchwW9pxW4Dqu nFVg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=YesPop8N4Nj1TL4Lwk2VerrSF3fxZ2OETwBiOABqBkU=; fh=IieR2OFhmSLEHpUsr659qE7y52NLE0XA7VruMDfsZN0=; b=0qwuG7qxfulID7rqg96Onxy4OJJd6pwND/tnWR/p0cQspVvJtouUEjNojZD3VZlfYL B+8tJzYl7gM0TW0u58ur18MQUCyIdAzbnvAYj7iYAzyrnfeRZS/fGW/uMDTe1mWgQY7g zTnPa43B50N4Tm80Ua21LeNB4CmltlBIS47zioL6KtHjR0VodwRB/+UKJPGhQzQo8/Hn VlxRsgfWhaUIYZBxbT7EVb3pfAyk6yItCHIiAftH6XXEt5v66CaRtgIPgYj2JLhmvqCJ gW1JUI14BA+09qc8bsSysSQxyi/wMsuUe0cx4tcpViG+K+XovRl1+xflpIL6VYjjgvC7 ttVA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47875-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47875-ouuuleilei=gmail.com@vger.kernel.org" X-Forwarded-Encrypted: i=1; AJvYcCWWTexQUovTlFxRk1MdEJSli18A9b4YjUBsG0296B/rpRrhWAtH/6hgCo/pxNZlFsdBorCTGAnE8ufPpf8jhwRuVPUXFA== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id z10-20020a17090acb0a00b0029065135790si3310336pjt.166.2024.02.01.01.34.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Feb 2024 01:34:40 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47875-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47875-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47875-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id AA9D3284972 for ; Thu, 1 Feb 2024 09:34:37 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C33CD163AB5; Thu, 1 Feb 2024 09:30:40 +0000 (UTC) Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 39F5616086A; Thu, 1 Feb 2024 09:30:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779839; cv=none; b=IEVDQEiRHJG4FWFCTIFTA9Ia/c0OK9gZVQd91l4HPgqUtJv493pXcp9TQS6IjnKB1iZHRIhzM8W/soUlBS9XjXapTJnbcbS8xuVnCkCoOZirppq4a4FySXGUuDlqMu0V0hohM7burxxJALZYB+1ft6Lc+ne76nsCQOfdUYHisII= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779839; c=relaxed/simple; bh=y1m/wjQCy2MHnV2sXFZWtN+UQNGVqcl5T4H2E6C4AX0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cnMbKemnrT/EH5uI+RrCNJI+9LVjZ8GG4OuhWDZOU+szXWkK+FYqzaTN/LABkj0AA0ecxrhKB0CRFdO2bRxsZXLiNPcPn8N4LF3yFNqthY+20QxQ6DUhsW86y/agxF5RNEWT66/7Pr9+jdquqCvPimkgEQSy7yTWGDZBv4JOD2Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4TQYXz4GXNz4f3mJ6; Thu, 1 Feb 2024 17:30:27 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 22D6E1A0232; Thu, 1 Feb 2024 17:30:34 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgCXaBGtZLtl8V6KCg--.33515S15; Thu, 01 Feb 2024 17:30:33 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, blazej.kucman@linux.intel.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v5 11/14] dm-raid: add a new helper prepare_suspend() in md_personality Date: Thu, 1 Feb 2024 17:25:56 +0800 Message-Id: <20240201092559.910982-12-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240201092559.910982-1-yukuai1@huaweicloud.com> References: <20240201092559.910982-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgCXaBGtZLtl8V6KCg--.33515S15 X-Coremail-Antispam: 1UD129KBjvJXoW7Kr47Kr4xtrykGw13uF4fGrg_yoW8Wr48pa yIqay5Ar4UJayaqw1DXr4kZa4aq3ZIgrWqkry3JayfZa42gr1rW3WFgayDZrZ8JFyakFnx Aa1Utw4kuFy0937anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUP214x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr1j6r xdM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0D M2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjx v20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1l F7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4IIrI8v6xkF7I0E8cxan2 IY04v7MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAF wI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVW8ZVWrXwCIc4 0Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r4j6ryUMIIF0xvE2Ix0cI8IcVCY1x0267AK xVW8Jr0_Cr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVW8JV WxJwCI42IY6I8E87Iv6xkF7I0E14v26r4UJVWxJrUvcSsGvfC2KfnxnUUI43ZEXa7VUbmZ X7UUUUU== X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789688629898779166 X-GMAIL-MSGID: 1789688629898779166 From: Yu Kuai There are no functional changes for now, prepare to fix a deadlock for dm-raid456. Signed-off-by: Yu Kuai --- drivers/md/dm-raid.c | 10 +++++++--- drivers/md/md.h | 1 + 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers/md/dm-raid.c b/drivers/md/dm-raid.c index 6b6c011d9f69..16d3348abfed 100644 --- a/drivers/md/dm-raid.c +++ b/drivers/md/dm-raid.c @@ -3806,10 +3806,14 @@ static void raid_io_hints(struct dm_target *ti, struct queue_limits *limits) static void raid_presuspend(struct dm_target *ti) { struct raid_set *rs = ti->private; + struct mddev *mddev = &rs->md; - mddev_lock_nointr(&rs->md); - md_frozen_sync_thread(&rs->md); - mddev_unlock(&rs->md); + mddev_lock_nointr(mddev); + md_frozen_sync_thread(mddev); + mddev_unlock(mddev); + + if (mddev->pers && mddev->pers->prepare_suspend) + mddev->pers->prepare_suspend(mddev); } static void raid_presuspend_undo(struct dm_target *ti) diff --git a/drivers/md/md.h b/drivers/md/md.h index 684cc866402a..8e81f9e2fb20 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -629,6 +629,7 @@ struct md_personality int (*start_reshape) (struct mddev *mddev); void (*finish_reshape) (struct mddev *mddev); void (*update_reshape_pos) (struct mddev *mddev); + void (*prepare_suspend) (struct mddev *mddev); /* quiesce suspends or resumes internal processing. * 1 - stop new actions and wait for action io to complete * 0 - return to normal behaviour From patchwork Thu Feb 1 09:25:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 195228 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2719:b0:106:209c:c626 with SMTP id hl25csp31107dyb; Thu, 1 Feb 2024 01:35:03 -0800 (PST) X-Google-Smtp-Source: AGHT+IE6+Op4gFzqOSw3uUE95I4Kqarzctg6bnYdfdLK1NAG/OAJ+C7aeM7TFLfsn9sSDxaLgrF/ X-Received: by 2002:a17:906:19d3:b0:a35:3ce3:c48c with SMTP id h19-20020a17090619d300b00a353ce3c48cmr1444714ejd.23.1706780102769; Thu, 01 Feb 2024 01:35:02 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706780102; cv=pass; d=google.com; s=arc-20160816; b=KWYqqLYobIQkLN1YRFQJ7fqDCRTkgbjqLGhd7rmu4+nnmyYwRFVDgnVkGDZNeH/q6Z pzrQ8brNe0LG8Ov1ZaWNb8USsJA0PTQmxTftWtJmz1aLUy18kbKm9qkIiyC+6P4lE8lj ZTkHSB/U9p1GcMbyXb9S4YFOqyyWmiFgVR30PWZFJ7Qt9fSNbzigXn+Bg0uc37PhO6BQ eBmNGddxIefn56DYzXH8/LwEp2sxwDqEr1vZblgxtNH2vcNkDy5qXjzNNOc1c8HDIs/e B9CYsUfw3UCWKwfeP+DtqwYVkEKcvt9rmQHtczETWCIVGyZtdedCYhTZpt3gkaVAfCmn 6nvQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=A3rA7nTmdKlYzPbxLgfdmc9uaGtpYRTcesOQSYJb5Es=; fh=0/ZXGOG/pZGiw+6C3zylFbQbyMyUlVUmaQEjKerYaBU=; b=uDKUluORfjt/IEx3VjVMN2+bQwa5K1t/FopqrnklYjsVwMs8YWQbn5hWKuJkXDP1WT YWNcOC+B0PyrF77mGt/I+iRdt06z1scNdQebqRM5BMfC9p5t1sRDWOlSkvmWK6e52Per l/zeVZksG0l/TeqlUN0ynLr0r8v0WMY3eJ2kueM43SPMMUqzoSNyAYsyLhcPPBPXC/O5 wi8RcquffPYMWMGLYLTGe6kIslqWlDgjjb2VWFYbUEI823QhNlDchdy4P8EnUdO52Xmc 0aXjM8jxdNldvORqH2/z+3X8b8dt228E8Hs44EBoe6GDsueBjpkUvqBjXNOe+IbrrBKe flDw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47876-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47876-ouuuleilei=gmail.com@vger.kernel.org" X-Forwarded-Encrypted: i=1; AJvYcCWUujiEJ4ZlyAuisvBlellGrx4QUM1px9/ptHMJr5e517XrKyUqiB0/1LIYwVuxKWuSx5YlYX3j3vBDBLD/de5ywbhO+Q== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id gq19-20020a170906e25300b00a366d4e7ba1si1623887ejb.510.2024.02.01.01.35.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Feb 2024 01:35:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47876-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47876-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47876-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 3BE731F28653 for ; Thu, 1 Feb 2024 09:35:02 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 746761649C8; Thu, 1 Feb 2024 09:30:42 +0000 (UTC) Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E5BBB160889; Thu, 1 Feb 2024 09:30:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779840; cv=none; b=e1cQs2DSj9LyitUMZkwlelm/EMjXBPeplRqPp3tDHcDHh4P0eGCIvs4r+4CX5DJH6wykJ/cyizmtp55KIAckEXVeNH1vzV2A+QIUc/yTnID1x6aXbSqeQ92nTmLvPP/N0bwnRMpKcwDYi4dxpAkGrqQFJaxtnJWzh8Xo7iNT3z0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779840; c=relaxed/simple; bh=FUHYpeJoejFasqU+NgXG4b97PBqfEWU8P6POrdD71SQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=GtS7HOf+2iuy8ti8nKuVJnG93x41O3iktF4E0vgUt0eAHL96M64qfBMu1+OcHxprGX81LnUx6KPjyHYwFkAB+0o6mb6Em8VkVRi36AQ+62Ntxz1ed4pjXgK0Y13PgmyvC/TkLniLOV0F3tsoPxRdWTeI957wWC+sRFJZtnQw/tc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4TQYY04Mt7z4f3lfZ; Thu, 1 Feb 2024 17:30:28 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 266F51A0232; Thu, 1 Feb 2024 17:30:35 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgCXaBGtZLtl8V6KCg--.33515S16; Thu, 01 Feb 2024 17:30:34 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, blazej.kucman@linux.intel.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v5 12/14] md/raid456: fix a deadlock for dm-raid456 while io concurrent with reshape Date: Thu, 1 Feb 2024 17:25:57 +0800 Message-Id: <20240201092559.910982-13-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240201092559.910982-1-yukuai1@huaweicloud.com> References: <20240201092559.910982-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgCXaBGtZLtl8V6KCg--.33515S16 X-Coremail-Antispam: 1UD129KBjvJXoW3Gr13WF4xJry7XF4UXw13Jwb_yoW7AF1xpa 9Fkas0vr40q34akrWDtF4kWa4F9FZ7KrW3ZF93Cw4fA3W7Xr1vy3W5Ja45ury5Ary8J3y5 JFW5tFyDWrs8Ka7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUP214x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr1j6r xdM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0D M2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjx v20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1l F7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4IIrI8v6xkF7I0E8cxan2 IY04v7MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAF wI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVW8ZVWrXwCIc4 0Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r4j6ryUMIIF0xvE2Ix0cI8IcVCY1x0267AK xVW8Jr0_Cr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVW8JV WxJwCI42IY6I8E87Iv6xkF7I0E14v26r4UJVWxJrUvcSsGvfC2KfnxnUUI43ZEXa7VUbmZ X7UUUUU== X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789688653237365969 X-GMAIL-MSGID: 1789688653237365969 From: Yu Kuai For raid456, if reshape is still in progress, then IO across reshape position will wait for reshape to make progress. However, for dm-raid, in following cases reshape will never make progress hence IO will hang: 1) the array is read-only; 2) MD_RECOVERY_WAIT is set; 3) MD_RECOVERY_FROZEN is set; After commit c467e97f079f ("md/raid6: use valid sector values to determine if an I/O should wait on the reshape") fix the problem that IO across reshape position doesn't wait for reshape, the dm-raid test shell/lvconvert-raid-reshape.sh start to hang: [root@fedora ~]# cat /proc/979/stack [<0>] wait_woken+0x7d/0x90 [<0>] raid5_make_request+0x929/0x1d70 [raid456] [<0>] md_handle_request+0xc2/0x3b0 [md_mod] [<0>] raid_map+0x2c/0x50 [dm_raid] [<0>] __map_bio+0x251/0x380 [dm_mod] [<0>] dm_submit_bio+0x1f0/0x760 [dm_mod] [<0>] __submit_bio+0xc2/0x1c0 [<0>] submit_bio_noacct_nocheck+0x17f/0x450 [<0>] submit_bio_noacct+0x2bc/0x780 [<0>] submit_bio+0x70/0xc0 [<0>] mpage_readahead+0x169/0x1f0 [<0>] blkdev_readahead+0x18/0x30 [<0>] read_pages+0x7c/0x3b0 [<0>] page_cache_ra_unbounded+0x1ab/0x280 [<0>] force_page_cache_ra+0x9e/0x130 [<0>] page_cache_sync_ra+0x3b/0x110 [<0>] filemap_get_pages+0x143/0xa30 [<0>] filemap_read+0xdc/0x4b0 [<0>] blkdev_read_iter+0x75/0x200 [<0>] vfs_read+0x272/0x460 [<0>] ksys_read+0x7a/0x170 [<0>] __x64_sys_read+0x1c/0x30 [<0>] do_syscall_64+0xc6/0x230 [<0>] entry_SYSCALL_64_after_hwframe+0x6c/0x74 This is because reshape can't make progress. For md/raid, the problem doesn't exist because register new sync_thread doesn't rely on the IO to be done any more: 1) If array is read-only, it can switch to read-write by ioctl/sysfs; 2) md/raid never set MD_RECOVERY_WAIT; 3) If MD_RECOVERY_FROZEN is set, mddev_suspend() doesn't hold 'reconfig_mutex', hence it can be cleared and reshape can continue by sysfs api 'sync_action'. However, I'm not sure yet how to avoid the problem in dm-raid yet. This patch detect the above 3 cases in dm_suspend(), and fail those IO directly. If user really meet the IO error, then it means they're reading the wrong data before c467e97f079f. And it's safe to read/write the array after reshape make progress successfully. Signed-off-by: Yu Kuai --- drivers/md/raid5.c | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 6a7a32f7fb91..812d7ec64da5 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -5915,6 +5915,13 @@ static int add_all_stripe_bios(struct r5conf *conf, return ret; } +static bool reshape_disabled(struct mddev *mddev) +{ + return !md_is_rdwr(mddev) || + test_bit(MD_RECOVERY_WAIT, &mddev->recovery) || + test_bit(MD_RECOVERY_FROZEN, &mddev->recovery); +} + static enum stripe_result make_stripe_request(struct mddev *mddev, struct r5conf *conf, struct stripe_request_ctx *ctx, sector_t logical_sector, struct bio *bi) @@ -5946,7 +5953,8 @@ static enum stripe_result make_stripe_request(struct mddev *mddev, if (ahead_of_reshape(mddev, logical_sector, conf->reshape_safe)) { spin_unlock_irq(&conf->device_lock); - return STRIPE_SCHEDULE_AND_RETRY; + ret = STRIPE_SCHEDULE_AND_RETRY; + goto out; } } spin_unlock_irq(&conf->device_lock); @@ -6025,6 +6033,13 @@ static enum stripe_result make_stripe_request(struct mddev *mddev, out_release: raid5_release_stripe(sh); +out: + if (ret == STRIPE_SCHEDULE_AND_RETRY && !mddev->gendisk && + reshape_disabled(mddev)) { + bi->bi_status = BLK_STS_IOERR; + ret = STRIPE_FAIL; + pr_err("dm-raid456: io failed across reshape position while reshape can't make progress"); + } return ret; } @@ -8909,6 +8924,18 @@ static int raid5_start(struct mddev *mddev) return r5l_start(conf->log); } +/* + * This is only used for dm-raid456, caller already frozen sync_thread, hence + * if rehsape is still in progress, io that is waiting for reshape can never be + * done now, hence wake up and handle those IO. + */ +static void raid5_prepare_suspend(struct mddev *mddev) +{ + struct r5conf *conf = mddev->private; + + wake_up(&conf->wait_for_overlap); +} + static struct md_personality raid6_personality = { .name = "raid6", @@ -8932,6 +8959,7 @@ static struct md_personality raid6_personality = .quiesce = raid5_quiesce, .takeover = raid6_takeover, .change_consistency_policy = raid5_change_consistency_policy, + .prepare_suspend = raid5_prepare_suspend, }; static struct md_personality raid5_personality = { @@ -8956,6 +8984,7 @@ static struct md_personality raid5_personality = .quiesce = raid5_quiesce, .takeover = raid5_takeover, .change_consistency_policy = raid5_change_consistency_policy, + .prepare_suspend = raid5_prepare_suspend, }; static struct md_personality raid4_personality = @@ -8981,6 +9010,7 @@ static struct md_personality raid4_personality = .quiesce = raid5_quiesce, .takeover = raid4_takeover, .change_consistency_policy = raid5_change_consistency_policy, + .prepare_suspend = raid5_prepare_suspend, }; static int __init raid5_init(void) From patchwork Thu Feb 1 09:25:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 195227 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2719:b0:106:209c:c626 with SMTP id hl25csp31080dyb; Thu, 1 Feb 2024 01:34:58 -0800 (PST) X-Google-Smtp-Source: AGHT+IHnvGVRIUE716A5Ln9LuPhL34w4gl3S9zOyN9vEZs6XcnXgwWLIreF3vbWNXtG3f/0iN1Kv X-Received: by 2002:a17:906:5944:b0:a35:a017:fb37 with SMTP id g4-20020a170906594400b00a35a017fb37mr3371070ejr.44.1706780098710; Thu, 01 Feb 2024 01:34:58 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706780098; cv=pass; d=google.com; s=arc-20160816; b=GUJ81V0seyVZ6Jcr5Np/tfmNcnOl4pFIApb56ctdg/ZIHDa22+CIZhigjlJa3serix E1fd0iA9fBXUzCbNo+6hUD/bE3NeFIMKE8Pbp4Na6Gg991AlgiP5nKYMxJIUSjqU4ZOg Itjh7AbAcSL8PcCkTJh0UI10SuLBC9qpR0Yiz6JbJ2gfZWON3dDildziiDcxQ8gOqI8f U8h8Qzrit46iR62hPksH0DjAPwwia1ORdj4WYdU3pyl4qpyZnsIchCQFJEgP+8EIOeU8 mI75ngQ3+Bzehug2Z8OjUaaykspnJ2ATebxpP1tiV2Wk3FFK7Ig82JRy6NjHlY7QmClv RLPg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=Gwajdbjq2bg1uihxFCNEmqNIL/yeEurfElrgwydZMvg=; fh=4kR0Wf2kEToqvXyrNqaMM9qup7Mym0KV7fdA7htr+BY=; b=ZIbkrrBjx8r/hh6c2D8Qm1fiNFG9QQPFC8hMEas3tO7Vo/UoaY1x9D44XEPaF+/Lhm XtAJKhvb7VFXSCXHJ/tVYIhkkx7cLdzvoBJoQiuzkAErSU/6+u0/l4HrROEQw+Ro8jse y7C0SjNH+VM+1GKWVTQv2WnlIhbHTjGOXXho3X4x9h99OTzkSytmQIAS1G0pcAqbEf2t 5hh37G07errFENaWj9pgFQzeCMhL4quHlvH+QVnbYRGBSu5Tmnpuj5XhdMweWbBRlKVf GPm0sgeF6UQq2bOhwwu5HSg/+72t6uyli7SDYLrrYGoDQF4P1aumYf5nyofEdsrFEZqK 4upQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47877-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47877-ouuuleilei=gmail.com@vger.kernel.org" X-Forwarded-Encrypted: i=1; AJvYcCWXB0hcSLag9uNzsKwOsgdAs7dCxDmsJARt6afFI2bl4gmvrgqbFAYUOrP4N2Jy82XNNdBUsz/YdFKJwdALxIjM7IHcqA== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id l14-20020a170906644e00b00a33923b5b5fsi6574117ejn.134.2024.02.01.01.34.58 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Feb 2024 01:34:58 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47877-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47877-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47877-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 257961F283C4 for ; Thu, 1 Feb 2024 09:34:58 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 18BDE1649BF; Thu, 1 Feb 2024 09:30:42 +0000 (UTC) Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D2FAE161B54; Thu, 1 Feb 2024 09:30:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779840; cv=none; b=FbP65hqUB4Visegpvmejemq/v3vcrUBq+Xk3u2d5P3ODutVi5mZVABFihWqN6vKZsjtvBdR7qHWZI0uCvzmmezvsYBMMiWbmPvAB8Y8j6Klhsg2saKfjjJa/sq22Syasq6MeL763Ua/4gUU2ikiuUlUqkmpv2wGHXfBL3XZDUaU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779840; c=relaxed/simple; bh=a7HDR2QZuFH8EpmOVAfdCFF1UeWbNCM4RUwmYnTBG/s=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QpewRAZUTgPPylKm9LR9evyfAqnmULklbZQhHZfb7C5ITveBa8B2F8y0d7ZuYndo7ulart+Tu/2sJuwoVBwom3ayB1grFdDz+HcpNkOwEyoQWuNMj4vyLZpIsvfO4BR0NWabJVSXo4sx5QLY6XSpQLV6nXN7yfqXMfFqLtfWSIE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4TQYY528Mxz4f3k6V; Thu, 1 Feb 2024 17:30:33 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id DB30F1A0232; Thu, 1 Feb 2024 17:30:35 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgCXaBGtZLtl8V6KCg--.33515S17; Thu, 01 Feb 2024 17:30:35 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, blazej.kucman@linux.intel.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v5 13/14] dm-raid: fix lockdep waring in "pers->hot_add_disk" Date: Thu, 1 Feb 2024 17:25:58 +0800 Message-Id: <20240201092559.910982-14-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240201092559.910982-1-yukuai1@huaweicloud.com> References: <20240201092559.910982-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgCXaBGtZLtl8V6KCg--.33515S17 X-Coremail-Antispam: 1UD129KBjvJXoW7Cr45Jw1UZF45JF13GFW5GFg_yoW8Jw4Up3 ZrK343Kw4DJr48Za1qvw1q9a45tan8K3ySy39xG395ZFy7ZrZI9ws8Ga1agFWDJFZaya98 AFW3J398Was5K3DanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUP214x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr1j6r xdM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0D M2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjx v20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1l F7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4IIrI8v6xkF7I0E8cxan2 IY04v7MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAF wI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVW8ZVWrXwCIc4 0Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r4j6ryUMIIF0xvE2Ix0cI8IcVCY1x0267AK xVW8Jr0_Cr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVW8JV WxJwCI42IY6I8E87Iv6xkF7I0E14v26r4UJVWxJrUvcSsGvfC2KfnxnUUI43ZEXa7VUbmZ X7UUUUU== X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789688649075583385 X-GMAIL-MSGID: 1789688649075583385 From: Yu Kuai The lockdep assert is added by commit a448af25becf ("md/raid10: remove rcu protection to access rdev from conf") in print_conf(). And I didn't notice that dm-raid is calling "pers->hot_add_disk" without holding 'reconfig_mutex'. "pers->hot_add_disk" read and write many fields that is protected by 'reconfig_mutex', and raid_resume() already grab the lock in other contex. Hence fix this problem by protecting "pers->host_add_disk" with the lock. Fixes: 9092c02d9435 ("DM RAID: Add ability to restore transiently failed devices on resume") Fixes: a448af25becf ("md/raid10: remove rcu protection to access rdev from conf") Signed-off-by: Yu Kuai --- drivers/md/dm-raid.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/md/dm-raid.c b/drivers/md/dm-raid.c index 16d3348abfed..5f78cc19d6f3 100644 --- a/drivers/md/dm-raid.c +++ b/drivers/md/dm-raid.c @@ -4077,7 +4077,9 @@ static void raid_resume(struct dm_target *ti) * Take this opportunity to check whether any failed * devices are reachable again. */ + mddev_lock_nointr(mddev); attempt_restore_of_faulty_devices(rs); + mddev_unlock(mddev); } if (test_and_clear_bit(RT_FLAG_RS_SUSPENDED, &rs->runtime_flags)) { From patchwork Thu Feb 1 09:25:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 195229 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2719:b0:106:209c:c626 with SMTP id hl25csp31234dyb; Thu, 1 Feb 2024 01:35:19 -0800 (PST) X-Google-Smtp-Source: AGHT+IFE8kXIl6WSndqvFvAdxje8v6XNeoX6NMIhEy6xdw+CC6lIhZkEUlTiRbwsLDhssIE6heqJ X-Received: by 2002:a17:90a:db97:b0:296:1cd7:4741 with SMTP id h23-20020a17090adb9700b002961cd74741mr635924pjv.4.1706780119315; Thu, 01 Feb 2024 01:35:19 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706780119; cv=pass; d=google.com; s=arc-20160816; b=q5VDR75q7TOi8gmhTTQCZzA53LPcmZIfxx9x3r6XMHE3FZmpOX5anLXP8cWh03A47p 0zZv3FE+dwO4MnubJmYD5HtR9p4xiDsAQ3mpxHMdiFRDAr+dmUlMnmrRsfcNx0PVdCjU 5QE0aKaJMdy2pRyNOM4VfCBKBIzw6nBfcDYmXCS7L/RwMrYgZBqyYVxMT61kUn56Ig+r Cz8OSmgoeZ9pbdVIIl1dnQ7ZhFu29gYJfhOM/cdkY9CV+A8mFmlHP/FJl2tQWf5S6pUe yABFu+rc9nRXad9ve0SCrKzyfKbO3BhFoeo4sSYcGKFvp6LAFHtQh2OuLxOkj6Dw861I lbMA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=wYzSQ7LkYaYosgM/AJHsamGhGmV1cPVUDvzL88OlU3Y=; fh=h3hROpDKOMyVv6DocEJuE0kpaN5k4M3gUl5zG1x+LSU=; b=pTUSRxwgiwRqpkyME+2Nxh+9GIWtiQINkPEWBYwHVDrVSmuXoR+YpS/Y0FQThLeaj/ V7xu/ZT/2ab06VHN3xFqn7a3s0fEcVIweutzsV5SxW9YTYge8gXpj6dkxfhtjDOfbDWh Nlt8s2GdGYnuFyjGT6uaU+KDs4FNgF/UldGtW0bB1hNV3mxRLlqxMr/amiE6/hwB8844 y+l/5QZIu1Wx38b4ix5q+cjtUC+BrQ6Il6vzzOn3KBpURUlOkZMONaH49F12Ll+Qw3xq c3OBDWdFD27IlZTWE33sUQcpIG7OmnN97kOtq9KFhAc5q+RD0DCRCtOu5S5CW7p/yFUz 5NHA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47878-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47878-ouuuleilei=gmail.com@vger.kernel.org" X-Forwarded-Encrypted: i=1; AJvYcCXU2F/bkV3fzaELPmSQG4G2EBtCsO7J0a9nmwEazOtgnUoIYce8WFmaVWkZ2zgQmgVqufT3snYvNW5dVDdTIoCL8XKNGw== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id y8-20020a17090aa40800b00290b369b456si3195697pjp.155.2024.02.01.01.35.19 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Feb 2024 01:35:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-47878-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=huaweicloud.com); spf=pass (google.com: domain of linux-kernel+bounces-47878-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-47878-ouuuleilei=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 1BE37282BF3 for ; Thu, 1 Feb 2024 09:35:19 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8CEAF4D9E3; Thu, 1 Feb 2024 09:30:44 +0000 (UTC) Received: from dggsgout12.his.huawei.com (unknown [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 646991649AD; Thu, 1 Feb 2024 09:30:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779843; cv=none; b=cS+r96GcXzHOFxwYtdZ9Q3udmSso4vJfLoDCt55Dlsfn71D3aU5tsRQhwydacCK4GW5YxzJidbF0AJJ5H4UQh0E6cTD83UPYZiA2Gd3ZLS6BG6brFQZUI6oEyqFYKXxiqAuppKEDYFT3Gctr7+yaiTh46iDa2s87xqnEsNHKxkc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706779843; c=relaxed/simple; bh=Pw8nxEepj3SW3fuzUXlVu8OndBo1SbgV3fSER76rjfk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ivz8vl+5acLGcqxweoSlnOR39eiPstaE/3RqCXdu0G6/fcY7Ty67Npx/jJFQI/7TwUyYzZkaW0r0S2ppdLG0LD+0gfQ7XBXQiZsIfluxfGbJj6m6LLJfT2eXrNg46/blzg6v7K5AuO0aM1XqRBonuD8vuBKrNK8dqGUjjAaMXS0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4TQYY42zs3z4f3l7p; Thu, 1 Feb 2024 17:30:32 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 9DD691A0390; Thu, 1 Feb 2024 17:30:36 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP1 (Coremail) with SMTP id cCh0CgCXaBGtZLtl8V6KCg--.33515S18; Thu, 01 Feb 2024 17:30:36 +0800 (CST) From: Yu Kuai To: mpatocka@redhat.com, heinzm@redhat.com, xni@redhat.com, blazej.kucman@linux.intel.com, agk@redhat.com, snitzer@kernel.org, dm-devel@lists.linux.dev, song@kernel.org, yukuai3@huawei.com, jbrassow@f14.redhat.com, neilb@suse.de, shli@fb.com, akpm@osdl.org Cc: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v5 14/14] dm-raid: remove mddev_suspend/resume() Date: Thu, 1 Feb 2024 17:25:59 +0800 Message-Id: <20240201092559.910982-15-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240201092559.910982-1-yukuai1@huaweicloud.com> References: <20240201092559.910982-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: cCh0CgCXaBGtZLtl8V6KCg--.33515S18 X-Coremail-Antispam: 1UD129KBjvJXoWxCF4UXw48Cw4kXrW8Cw4DJwb_yoW5GrWDpw 4IqFWayw4UtFZrXwsrA3WvgFy5twn5KrWjkrZxW34fWa43Gr13Wr18Gay5XFWDKFWfJF1D Aa1Utw48uryIgrJanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUP214x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Cr1j6r xdM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0D M2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjx v20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1l F7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4IIrI8v6xkF7I0E8cxan2 IY04v7MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAF wI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVW8ZVWrXwCIc4 0Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r4j6ryUMIIF0xvE2Ix0cI8IcVCY1x0267AK xVW8Jr0_Cr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVW8JV WxJwCI42IY6I8E87Iv6xkF7I0E14v26r4UJVWxJrUvcSsGvfC2KfnxnUUI43ZEXa7VUbmZ X7UUUUU== X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789688670372551576 X-GMAIL-MSGID: 1789688670372551576 From: Yu Kuai dm layer will make sure that no new IO can be issued and will wait for all dispatched IO to be done during suspend or before removing the device. Hence there is no need to call mddev_suspend/resume() again. BTW, mddev_suspend/resume() can't gurantee that there are no sync IO, and previous patch make sure that presuspend will stop sync thread. Signed-off-by: Yu Kuai --- drivers/md/dm-raid.c | 8 +++----- drivers/md/md.c | 11 +++++++++++ 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/drivers/md/dm-raid.c b/drivers/md/dm-raid.c index 5f78cc19d6f3..ed8c28952b14 100644 --- a/drivers/md/dm-raid.c +++ b/drivers/md/dm-raid.c @@ -3241,7 +3241,7 @@ static int raid_ctr(struct dm_target *ti, unsigned int argc, char **argv) rs->md.in_sync = 1; /* Has to be held on running the array */ - mddev_suspend_and_lock_nointr(&rs->md); + mddev_lock_nointr(&rs->md); /* Keep array frozen until resume. */ md_frozen_sync_thread(&rs->md); @@ -3829,11 +3829,9 @@ static void raid_postsuspend(struct dm_target *ti) { struct raid_set *rs = ti->private; - if (!test_and_set_bit(RT_FLAG_RS_SUSPENDED, &rs->runtime_flags)) { + if (!test_and_set_bit(RT_FLAG_RS_SUSPENDED, &rs->runtime_flags)) /* Writes have to be stopped before suspending to avoid deadlocks. */ md_stop_writes(&rs->md); - mddev_suspend(&rs->md, false); - } } static void attempt_restore_of_faulty_devices(struct raid_set *rs) @@ -4091,7 +4089,7 @@ static void raid_resume(struct dm_target *ti) mddev->ro = 0; mddev->in_sync = 0; md_unfrozen_sync_thread(mddev); - mddev_unlock_and_resume(mddev); + mddev_unlock(mddev); } } diff --git a/drivers/md/md.c b/drivers/md/md.c index 2c245341148a..a8db84c200fe 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -437,6 +437,10 @@ int mddev_suspend(struct mddev *mddev, bool interruptible) { int err = 0; + /* Array is supended from dm_suspend() for dm-raid. */ + if (!mddev->gendisk) + return 0; + /* * hold reconfig_mutex to wait for normal io will deadlock, because * other context can't update super_block, and normal io can rely on @@ -488,6 +492,13 @@ EXPORT_SYMBOL_GPL(mddev_suspend); static void __mddev_resume(struct mddev *mddev, bool recovery_needed) { + /* + * Array is supended from dm_suspend() and resumed from dm_resume() for + * dm-raid. + */ + if (!mddev->gendisk) + return; + lockdep_assert_not_held(&mddev->reconfig_mutex); mutex_lock(&mddev->suspend_mutex);