From patchwork Fri May 12 01:56:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 92913 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp4800703vqo; Thu, 11 May 2023 19:22:24 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5xemYTivegachGuUOCc1HqYvwWnS1RPjJNALOIa3mJRpxI+xeCm/D/WG6kTx34kIj3OldD X-Received: by 2002:a05:6a20:1604:b0:103:6f15:5b59 with SMTP id l4-20020a056a20160400b001036f155b59mr9004311pzj.61.1683858144427; Thu, 11 May 2023 19:22:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683858144; cv=none; d=google.com; s=arc-20160816; b=M5U81mtHMQF+DBFfYKoobYtroI6ia+ZswYpRgpGJOJhB0vN2UfgjMYcaBf1ICSUnAK tb406wv+dgIVudE+rF/L6bP2AcgK86wmtG5IdiA2LXEv021Hc7F4LQNPKPGehnhBhkOT AezovB9QCT364B1aWDRq6ABTY3fCSOR54DT8Gn248+j3kI8iOqc3eIv5V5fDBj77w+j9 cBTMkDStZAPzzRIdbJf35lMwXAtALqGD+Oz4bElDSUWfRRy/6Ejdx3ee4Vjy9WVMWNab bVtNjWmTLLXMvD7IEcEFpxwjdUynaUf4yUgxefOX3fDQZiOGqroNNiwp2Bl7G/7HMSyb 1hGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=3PIS1ybc9+8evTaOhifebTLZrf+xYNjxIdLG9sIWTi0=; b=fjfAK27H8ysRMS1JQxrNzUzlIcd+9iizeQp4m+aqbTc8KCHGYi828wmN9w+cdtt3wY oUcG0gk7/JaYkCN39uGLti5KjZX9MJF42Qdqq4ACsrw8BUhq+LsHzMtGDjtY4fB6Fh4q b7A1tvT7CPZbTzhUN2VW3MIRqe5Y/fqKuiOCU9YTpPFI+DS7Muu2xN1YEsl5eX275nWj lZiO5VL/GeYjc9A81762iPkX8KjXAnYObxIJ+xtfk6j0ipIFQ4P6C1l/QJeWIVXJz6rp uVZc/DQeoctyFWNHAgj3v9NGE4G5zViNAV7G33eNkeUqhoyUJRdv2hQozTOlOtYT1QEZ g8Zw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id c22-20020aa79536000000b00643994622e4si8866656pfp.98.2023.05.11.19.22.11; Thu, 11 May 2023 19:22:24 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239826AbjELB7S (ORCPT + 99 others); Thu, 11 May 2023 21:59:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48992 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239679AbjELB66 (ORCPT ); Thu, 11 May 2023 21:58:58 -0400 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 38A0959F1; Thu, 11 May 2023 18:58:55 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.153]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4QHX4B6t5fz4f3lXk; Fri, 12 May 2023 09:58:50 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgBnHbFYnV1kbjQOJQ--.16243S5; Fri, 12 May 2023 09:58:50 +0800 (CST) From: Yu Kuai To: logang@deltatee.com, reddunur@online.de, jovetoo@gmail.com, dgilmour76@gmail.com, song@kernel.org Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai3@huawei.com, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH -next 1/5] md/raid5: don't allow replacement while reshape is not done Date: Fri, 12 May 2023 09:56:06 +0800 Message-Id: <20230512015610.821290-2-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230512015610.821290-1-yukuai1@huaweicloud.com> References: <20230512015610.821290-1-yukuai1@huaweicloud.com> MIME-Version: 1.0 X-CM-TRANSID: gCh0CgBnHbFYnV1kbjQOJQ--.16243S5 X-Coremail-Antispam: 1UD129KBjvJXoW7uF4DZryrKFWkuFyUGrWDXFb_yoW8Gry3pa nIkF1DXr4UZws8ta1DJa48CFyS9Fs2grZ8KrW3u34Yv3Wjqr4kA3y8KryUCr4DZF4Y93ya qw47trWDA34Iy37anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBE14x267AKxVW5JVWrJwAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_Jr4l82xGYIkIc2 x26xkF7I0E14v26r4j6ryUM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_Ar0_tr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJw A2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq3wAS 0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2 IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0 Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2kIc2 xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v2 6r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_Jw0_GFylIxkGc2 Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_ Gr0_Cr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMI IF0xvEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x0JU2_M3UUUUU = X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765653237681769444?= X-GMAIL-MSGID: =?utf-8?q?1765653237681769444?= From: Yu Kuai Set rdev replacement has but not only two conditions: 1) MD_RECOVERY_RUNNING is not set; 2) rdev nr_pending is 0; If reshape is interrupted(for example, echo frozen to sync_action), then rdev replacement can be set. It's safe because reshape is always prior to resync in md_check_recovery(). However, if system reboots, then kernel will complain cannot handle concurrent replacement and reshape and this array is not able to assemble anymore. Fix this problem by don't allow replacement until reshape is done. Reported-by: Peter Neuwirth Link: https://lore.kernel.org/linux-raid/e2f96772-bfbc-f43b-6da1-f520e5164536@online.de/ Signed-off-by: Yu Kuai --- drivers/md/raid5.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index a58507a4345d..bd3b535c0739 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -8378,6 +8378,7 @@ static int raid5_add_disk(struct mddev *mddev, struct md_rdev *rdev) p = conf->disks + disk; tmp = rdev_mdlock_deref(mddev, p->rdev); if (test_bit(WantReplacement, &tmp->flags) && + mddev->reshape_position == MaxSector && p->replacement == NULL) { clear_bit(In_sync, &rdev->flags); set_bit(Replacement, &rdev->flags); From patchwork Fri May 12 01:56:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 92911 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp4792159vqo; Thu, 11 May 2023 19:00:20 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7nv8JeYaHXUt+ZNgKpzdoiaPZFKBHEU/gOALQKf7CwSo+WvtjLlvrfsPNCR7HLI+nJ6e4Q X-Received: by 2002:a17:902:cec8:b0:1ac:5c53:2c1b with SMTP id d8-20020a170902cec800b001ac5c532c1bmr21885053plg.66.1683856820164; Thu, 11 May 2023 19:00:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683856820; cv=none; d=google.com; s=arc-20160816; b=hXW3jWjIYVkWcwdRrJesOZQjURPN2VE1pPRXURQ8KpBvRo0o7AKW1eF78LS++TU4Ut xQodvEjRFYON7zKXqfy1Onp7tL9FGRe0mO5g40oAFTxLOFzz+tdI9BjcmLjkHvDlVg55 vNXD6qCTjsYgcYDgZTkV5OXK0ykzftTkbY2yaoJ41AXYnSE2PmqgR1yuApdr1we8LhbI MdRdXo3NHIzJBeBhHdM6aWifBYRHTJRdgNeTyfa6kjH8781dPtAKSHmhfIAoaHmLe5Nt Qo4FTwXq22iU0Hug6R3RZ48qTE5bMMqLEGa08qAxAcEG9P9FXrYM5MMZy/xNHdqhWQJX 0hEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=fO7G4cATXQzxiVqo40wqLPPOM3Uq+4ki3qWYP/L0usg=; b=Pat+GzqSX+ZLkyfxVprQU+G/S+iZD/C6ntpO8fD4i8/v7PUSU1oekDxC4PaXxmYmNC rafr2aSfDzXSX+tNiohIbTArTOt9aRIxwu2sw+lLHNl8O/L7WLh0Ueum2kG5OgH/JGMv 8WGKFgw12sAREgNOVu9Clkj7g5uIP8pc+NFxadcEhlG/Y9sXPbO5XD9ws1UfP4sEnFzy AzgAsDnCPXUaAvcA7IWUF48ixWgO/AGa7mq6tcGHAXy6FP/xdBF6Hx3CCnscSHkqjsnA f4mB3Qe1pnEMR1izz5fSTf+pbT/dE5vQ8saDrM2HwN9dUFKtZDlTw/60MnS9t44W+1+0 +74w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l2-20020a170902ec0200b001a242dbbd27si7814346pld.513.2023.05.11.19.00.05; Thu, 11 May 2023 19:00:20 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239816AbjELB7O (ORCPT + 99 others); Thu, 11 May 2023 21:59:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48998 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239650AbjELB66 (ORCPT ); Thu, 11 May 2023 21:58:58 -0400 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4141755B1; Thu, 11 May 2023 18:58:55 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.153]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4QHX4C3z8Qz4f3vdw; Fri, 12 May 2023 09:58:51 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgBnHbFYnV1kbjQOJQ--.16243S6; Fri, 12 May 2023 09:58:52 +0800 (CST) From: Yu Kuai To: logang@deltatee.com, reddunur@online.de, jovetoo@gmail.com, dgilmour76@gmail.com, song@kernel.org Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai3@huawei.com, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH -next 2/5] md: fix data corruption for raid456 when reshape restart while grow up Date: Fri, 12 May 2023 09:56:07 +0800 Message-Id: <20230512015610.821290-3-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230512015610.821290-1-yukuai1@huaweicloud.com> References: <20230512015610.821290-1-yukuai1@huaweicloud.com> MIME-Version: 1.0 X-CM-TRANSID: gCh0CgBnHbFYnV1kbjQOJQ--.16243S6 X-Coremail-Antispam: 1UD129KBjvJXoW7Cw1fKryftrW5tFWDKw4kJFb_yoW8Wryrpa yfKFn8Zr1UAw13Jw4UG3WDZa45Wry0qrW5Kas7u345AF17tr1xua1YgFZrXryDAa4vkrWY qw4UtFWUCFy09aDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBE14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_Jryl82xGYIkIc2 x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJw A2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq3wAS 0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2 IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0 Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2kIc2 xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v2 6r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_Jw0_GFylIxkGc2 Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_ Gr0_Cr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMI IF0xvEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x0JUHbyAUUUUU = X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765651848799639044?= X-GMAIL-MSGID: =?utf-8?q?1765651848799639044?= From: Yu Kuai Currently, if reshape is interrupted, echo "reshape" to sync_action will restart reshape from scratch, for example: echo frozen > sync_action echo reshape > sync_action This will corrupt data before reshape_position if the array is growing, fix the problem by continue reshape from reshape_position. Reported-by: Peter Neuwirth Link: https://lore.kernel.org/linux-raid/e2f96772-bfbc-f43b-6da1-f520e5164536@online.de/ Signed-off-by: Yu Kuai --- drivers/md/md.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index e1bc223d58b3..cf60d4b5356d 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -4827,11 +4827,21 @@ action_store(struct mddev *mddev, const char *page, size_t len) return -EINVAL; err = mddev_lock(mddev); if (!err) { - if (test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)) + if (test_bit(MD_RECOVERY_RUNNING, &mddev->recovery)) { err = -EBUSY; - else { + } else if (mddev->reshape_position == MaxSector || + mddev->pers->check_reshape == NULL || + mddev->pers->check_reshape(mddev)) { clear_bit(MD_RECOVERY_FROZEN, &mddev->recovery); err = mddev->pers->start_reshape(mddev); + } else { + /* + * If reshape is still in progress, and + * md_check_recovery() can continue to reshape, + * don't restart reshape because data can be + * corrupted for raid456. + */ + clear_bit(MD_RECOVERY_FROZEN, &mddev->recovery); } mddev_unlock(mddev); } From patchwork Fri May 12 01:56:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 92910 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp4791982vqo; Thu, 11 May 2023 19:00:00 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ77m77F4Y2Ow3xt7fTHDZQEIQUUFqXw0nwsAaE7MMMN9V9JWb3laVkJtUgaS1FSqdjrgfDE X-Received: by 2002:a17:903:249:b0:1a6:c67f:4ed with SMTP id j9-20020a170903024900b001a6c67f04edmr29488406plh.9.1683856799938; Thu, 11 May 2023 18:59:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683856799; cv=none; d=google.com; s=arc-20160816; b=SAd5kwvtUv0EKRxDPGCzDV0GDm7sb+rMA3hyDzxvMGerYHOUzqrronUUxeHk/sQ3mz YamJjXBhE8TU8ujprxQbrpoy9AhuYrhm+NCEYgezGqId713ocYayxJzYG0fHW6RJqYEN VEiJl6wXb4/Mtaa7M7fHdqcU0RFC1iJDhKe+nX93xs1eW1uZJSFsHqo2kKRSZJsROnRM ++WMgplZCLe1PzUj1ERWHd4xuZpSpNhcfUo5qIZXSWf7/rlsJs3v8ZobRB0UiKyMbF2h SVBGeL08EHTNYGsKqK9ZXgupoit9Ejgs30kKlxoXWGAk1P22DCNNXWhflOdzXq5SpezB o3dw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=UGtNbYniB5kITe9i1mR+JoymqEdOSPFtZu78CE6HIXY=; b=jkMkluZnQDwguOvAQSE0psWhYzV+4AkWE3XOmwV2vlhmpMIzEmzGhEf5+F1fB4QRvd mCwpffxKz5CD2GIgO9YvrI+ArpxIND0FUgxBszq9K7pKkbxLov86W4232cTbI/iCzSJq 34jmmskWDXdi9rj9oDb4YwOQNokbyIwdnj8VKeGjbC+OjQsNw7VT3ncQNm5NZoNU1yeA Puk0N0vhU5Sp/INt9JG07Vx+Iz9Zzg1Ezx9MUGHXdpK9vr0A66sFNAzQmtCWsKKU7ui8 auoot22/rMg4PvSyYDgQecpsGr460AcbR8mLbjBmDUPxpfcp/0l+Le67sRqvGl1NWZAs r47w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ja4-20020a170902efc400b001aaec5cffb2si7579072plb.280.2023.05.11.18.59.45; Thu, 11 May 2023 18:59:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239609AbjELB7L (ORCPT + 99 others); Thu, 11 May 2023 21:59:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49010 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239657AbjELB66 (ORCPT ); Thu, 11 May 2023 21:58:58 -0400 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 99FED5B8B; Thu, 11 May 2023 18:58:55 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.153]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4QHX4C62H7z4f3lY0; Fri, 12 May 2023 09:58:51 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgBnHbFYnV1kbjQOJQ--.16243S7; Fri, 12 May 2023 09:58:53 +0800 (CST) From: Yu Kuai To: logang@deltatee.com, reddunur@online.de, jovetoo@gmail.com, dgilmour76@gmail.com, song@kernel.org Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai3@huawei.com, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH -next 3/5] md: export md_is_rdwr() and is_md_suspended() Date: Fri, 12 May 2023 09:56:08 +0800 Message-Id: <20230512015610.821290-4-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230512015610.821290-1-yukuai1@huaweicloud.com> References: <20230512015610.821290-1-yukuai1@huaweicloud.com> MIME-Version: 1.0 X-CM-TRANSID: gCh0CgBnHbFYnV1kbjQOJQ--.16243S7 X-Coremail-Antispam: 1UD129KBjvJXoW7WFyxAFW8Cr47Wr43CrWruFg_yoW8uF1rpa ySqFyfZr4DZF43Gw4UJ3yDua45Zr1xKFW2yry7Cw4rZa43Xr1DCF1rWFWUXrykGFyfCr12 qa1UJFy8Cr40yrUanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBE14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JrWl82xGYIkIc2 x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJw A2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq3wAS 0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2 IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0 Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2kIc2 xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v2 6r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_Jw0_GFylIxkGc2 Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_ Gr0_Cr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMI IF0xvEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x0JUd8n5UUUUU = X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765651828112505090?= X-GMAIL-MSGID: =?utf-8?q?1765651828112505090?= From: Yu Kuai The two apis will be used later to fix a deadlock in raid456, there are no functional changes. Signed-off-by: Yu Kuai --- drivers/md/md.c | 16 ---------------- drivers/md/md.h | 17 +++++++++++++++++ 2 files changed, 17 insertions(+), 16 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index cf60d4b5356d..5db26b7e7314 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -90,18 +90,6 @@ static void mddev_detach(struct mddev *mddev); static void md_wakeup_thread_directly(struct md_thread __rcu *thread); static void export_rdev(struct md_rdev *rdev); -enum md_ro_state { - MD_RDWR, - MD_RDONLY, - MD_AUTO_READ, - MD_MAX_STATE -}; - -static bool md_is_rdwr(struct mddev *mddev) -{ - return (mddev->ro == MD_RDWR); -} - /* * Default number of read corrections we'll attempt on an rdev * before ejecting it from the array. We divide the read error @@ -377,10 +365,6 @@ EXPORT_SYMBOL_GPL(md_new_event); static LIST_HEAD(all_mddevs); static DEFINE_SPINLOCK(all_mddevs_lock); -static bool is_md_suspended(struct mddev *mddev) -{ - return percpu_ref_is_dying(&mddev->active_io); -} /* Rather than calling directly into the personality make_request function, * IO requests come here first so that we can check if the device is * being suspended pending a reconfiguration. diff --git a/drivers/md/md.h b/drivers/md/md.h index a228e78e7f3a..7827f27c1406 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -563,6 +563,23 @@ enum recovery_flags { MD_RESYNCING_REMOTE, /* remote node is running resync thread */ }; +enum md_ro_state { + MD_RDWR, + MD_RDONLY, + MD_AUTO_READ, + MD_MAX_STATE +}; + +static inline bool md_is_rdwr(struct mddev *mddev) +{ + return (mddev->ro == MD_RDWR); +} + +static inline bool is_md_suspended(struct mddev *mddev) +{ + return percpu_ref_is_dying(&mddev->active_io); +} + static inline int __must_check mddev_lock(struct mddev *mddev) { return mutex_lock_interruptible(&mddev->reconfig_mutex); From patchwork Fri May 12 01:56:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 92915 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp4800890vqo; Thu, 11 May 2023 19:22:57 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4dDnsr0kY1l9kF7GLENrOVSUcRTTj2inquulk7ceyX3zHooK5RbvbzDcflTAE4uGQOcwR7 X-Received: by 2002:a17:903:191:b0:1ac:aaf6:ee48 with SMTP id z17-20020a170903019100b001acaaf6ee48mr12476283plg.67.1683858177059; Thu, 11 May 2023 19:22:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683858177; cv=none; d=google.com; s=arc-20160816; b=XsFRy7HXRgSq2/fHowRGO9ekjrHDbKZPTACWTeGe1EPHhAcZZkaUx3CUsuFpy0niZH 7Uyz+X3XkHG0eM+iMgfXd+CME1Lp4JOQYXqJ0BLJcZw+3aotRMP23veq3fFDBEoVcyt4 eWhxnyWDEdBA1VFlmxCLFCHievY+5kRYoxQgfZ2BEjSQp6FwlcO+dlYURMKkB7r4UDVH piAt0Z/gfDNf77unC9tVPaNKGY2lHoNz/VSK1U6JXOnnaZmTY8n0unubhMAwv16tL9Aq qk1kZJnghRCRpbs/34pm81/l6kdaZs9vBfnV4LA56zA9o1iJRpr1uiZOBDUfPUOjbWNB khog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=zUCO/5rul2myv8xrHHxcV4JCSzCiPTT8QGmvzXfL1Bw=; b=XnhZbpDLIB0yJpCINp2ccnvyFe5+4NXAMCVrqFRgar/J2ygypPI3s99CnIvSh773i0 7emz50frtha31HRC1nTro6Zpzxk0/FcBnJp9kDtMVWdmm09GeFu57nIte01LTSgourFj iZhZiCA5gpS7RIuLyytyZAeYeMtEzVxXUluJeQiOhJN4+nYmbKvvesxerjXmLG89Gnp9 rh8xDMwT2kGKsppWAKfFEuRbxnI41CCZ0Eqv21IcIC3dZUonfqcFOAH94qPML24c6X6+ sW9E3+9U0lYkGeJGnogX9bEfVVJAg1YUh2Myj0/Qthp3XF6pqs7OmwYze12TtRhmxyCj gAUA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 10-20020a170902c14a00b001ac2a50ef09si7795378plj.256.2023.05.11.19.22.44; Thu, 11 May 2023 19:22:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239831AbjELB7U (ORCPT + 99 others); Thu, 11 May 2023 21:59:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49220 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239723AbjELB66 (ORCPT ); Thu, 11 May 2023 21:58:58 -0400 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 16AED5BA5; Thu, 11 May 2023 18:58:55 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.153]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4QHX4D34Fhz4f3v4f; Fri, 12 May 2023 09:58:52 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgBnHbFYnV1kbjQOJQ--.16243S8; Fri, 12 May 2023 09:58:53 +0800 (CST) From: Yu Kuai To: logang@deltatee.com, reddunur@online.de, jovetoo@gmail.com, dgilmour76@gmail.com, song@kernel.org Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai3@huawei.com, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH -next 4/5] md: add a new api prepare_suspend() in md_personality Date: Fri, 12 May 2023 09:56:09 +0800 Message-Id: <20230512015610.821290-5-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230512015610.821290-1-yukuai1@huaweicloud.com> References: <20230512015610.821290-1-yukuai1@huaweicloud.com> MIME-Version: 1.0 X-CM-TRANSID: gCh0CgBnHbFYnV1kbjQOJQ--.16243S8 X-Coremail-Antispam: 1UD129KBjvJXoW7Kr47Kr4xtr4xAFWDXF47Jwb_yoW8GFWDpa yxtF98Ar1UArZIkw4UX3Wkua43Xw1DKrWkKFW3Z3yrZa43WF1rGF1FgayDZr1DCa4Skrsx Zw1UJ3yDuF109rDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPF14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E 14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_Jw0_GFylIx kGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAF wI0_Cr0_Gr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVWUJV W8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWIevJa73UjIFyTuYvjfUOBTY UUUUU X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765653271835575941?= X-GMAIL-MSGID: =?utf-8?q?1765653271835575941?= From: Yu Kuai There are no functional changes, the new api will be used later to do special handling for raid456 in md_suspend(). Signed-off-by: Yu Kuai --- drivers/md/md.c | 4 ++++ drivers/md/md.h | 1 + 2 files changed, 5 insertions(+) diff --git a/drivers/md/md.c b/drivers/md/md.c index 5db26b7e7314..1ef81421e131 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -467,6 +467,10 @@ void mddev_suspend(struct mddev *mddev) wake_up(&mddev->sb_wait); set_bit(MD_ALLOW_SB_UPDATE, &mddev->flags); percpu_ref_kill(&mddev->active_io); + + if (mddev->pers->prepare_suspend) + mddev->pers->prepare_suspend(mddev); + wait_event(mddev->sb_wait, percpu_ref_is_zero(&mddev->active_io)); mddev->pers->quiesce(mddev, 1); clear_bit_unlock(MD_ALLOW_SB_UPDATE, &mddev->flags); diff --git a/drivers/md/md.h b/drivers/md/md.h index 7827f27c1406..15d82517572e 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -639,6 +639,7 @@ struct md_personality int (*start_reshape) (struct mddev *mddev); void (*finish_reshape) (struct mddev *mddev); void (*update_reshape_pos) (struct mddev *mddev); + void (*prepare_suspend) (struct mddev *mddev); /* quiesce suspends or resumes internal processing. * 1 - stop new actions and wait for action io to complete * 0 - return to normal behaviour From patchwork Fri May 12 01:56:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 92917 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp4800983vqo; Thu, 11 May 2023 19:23:09 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ61w7pFjE9KS64AoZLOwoZaTKxS77g+1OhYpAAonfuzv9BCSX3dUT/9HTm6Bigk8/VZY3i0 X-Received: by 2002:a17:903:191:b0:1ac:750e:33f4 with SMTP id z17-20020a170903019100b001ac750e33f4mr21074482plg.32.1683858188677; Thu, 11 May 2023 19:23:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683858188; cv=none; d=google.com; s=arc-20160816; b=Osintmva1W3M6tBkwFyMWd9RpQKA4JwtgrNISQ86OVyiWS4UJ/r7mQUKKmsAFwqgG+ cuPEqTDjvfQ5wwMTPCunfhB5LeruyUNjqiDNM/nmJaHPzNH7PN4ymQORYHd/OXhojD/l 2QGzeONl4Fsqzo7HjEdQ9IbktAUUGH+QtO9qN5KC0kc4PKwc9G7q69r/IfAB344czYcP mU7+otbU9Yqg7IEL60zP48kPhOoubrJVHwZPqeiSWOyaZ96N7Fxb4WCvPidKHVJAvTC1 jNVgKBxS8YD3ilaNJRnmfSTrgjQroVY795sP899Xx56uWmAflS3qcBpK4XCF4JDz48xu ceUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=8hOVqkhsrxIjugPdh17pos3BfufJIUDgVIIdRYY5YnE=; b=f8a0WZWnHcNJg+f4jQjl77UqNmxL/pTsMCPetj847hJr1DUJc54SqJ0UKWrPrfxSxn niTDgHkVAvfXbRMLcH7nZse+HLfVWW2/FTQgIrKWugak+OrHGC8h1WvrDG2CiZBMrkhR mmgxQzzkt6ILA6HJ7c93pEjLz/YMPql8mJys3mmNy66hgSisfDO5JEEdL5nsl9mMhucI AGQAeqSVP7mZ/zrP91YYWNA7zczlfYly1PHGHifgoSHMitYnsGuQLGDkCM1t+a3g67HM ykRdDEfQ5NzS6TT/zPf5crhb5seCDJ694daGIedq3H8EB9WaMxKSEDqpLQC0XR0NYpZF AQLg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ja4-20020a170902efc400b001aaec5cffb2si7623058plb.280.2023.05.11.19.22.56; Thu, 11 May 2023 19:23:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239838AbjELB7W (ORCPT + 99 others); Thu, 11 May 2023 21:59:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49038 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239750AbjELB67 (ORCPT ); Thu, 11 May 2023 21:58:59 -0400 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 87FAD6199; Thu, 11 May 2023 18:58:56 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.153]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4QHX4D1WCHz4f3mJf; Fri, 12 May 2023 09:58:52 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgBnHbFYnV1kbjQOJQ--.16243S9; Fri, 12 May 2023 09:58:53 +0800 (CST) From: Yu Kuai To: logang@deltatee.com, reddunur@online.de, jovetoo@gmail.com, dgilmour76@gmail.com, song@kernel.org Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai3@huawei.com, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH -next 5/5] md/raid5: fix a deadlock in the case that reshape is interrupted Date: Fri, 12 May 2023 09:56:10 +0800 Message-Id: <20230512015610.821290-6-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230512015610.821290-1-yukuai1@huaweicloud.com> References: <20230512015610.821290-1-yukuai1@huaweicloud.com> MIME-Version: 1.0 X-CM-TRANSID: gCh0CgBnHbFYnV1kbjQOJQ--.16243S9 X-Coremail-Antispam: 1UD129KBjvJXoW3Gr4rCryxtFWxuw4DAr4UXFb_yoW7Kw15pa 92kFn0vr4jqF9xKFWDta1kWa4F9397KrW3ta9xJw18Aw43Jr48A3W3WF45XF1UAa48J3yY qF1rt34Dur4jga7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPF14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E 14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_Jw0_GFylIx kGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUCVW8JwCI42IY6xIIjxv20xvEc7CjxVAF wI0_Cr0_Gr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVWUJV W8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWIevJa73UjIFyTuYvjfUOBTY UUUUU X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ X-CFilter-Loop: Reflected X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765653284023106460?= X-GMAIL-MSGID: =?utf-8?q?1765653284023106460?= From: Yu Kuai If reshape is in progress and io across reshape_position is issued, such io will wait for reshape to make progress(see details in the case that make_stripe_request() return STRIPE_SCHEDULE_AND_RETRY). It has been reported several times that if system reboot while growing raid5 to raid6, array assemble will hang infinitely([1, 2]). This is because following deadlock is triggered: 1) a normal io is waiting for reshape to progress, this io can be from system-udevd or mdadm. 2) while assemble, mdadm tries to suspend the array, hence 'reconfig_mutex' is held and mddev_suspend() must wait for normal io to be done. 3) daemon thread can't start reshape because 'reconfig_mutex' can't be held. 1) and 3) is unbreakable because they're foundation design. In order to break 2), following is possible solutions that I can think of: a) Let mddev_suspend() fail is not a good option, because this will break many scenarios since mddev_suspend() doesn't fail before. b) Fail the io that is waiting for reshape to make progress from mddev_suspend(). c) Return false for the io that is waiting for reshape to make progress from raid5_make_request(), and these io will wait for suspend to be done in md_handle_request(), where 'active_io' is not grabbed. c) sounds better than b), however, b) is used because it's easy and straightforward, and it's verified that mdadm can assemble in this case. On the other hand, c) breaks the logic that mddev_suspend() will wait for submitted io to be completely handled. Fix the problem by checking reshape in mddev_suspend(), if reshape can't make progress and there are still some io waiting for reshape, fail those io. Reported-by: Jove [1] https://lore.kernel.org/all/CAFig2csUV2QiomUhj_t3dPOgV300dbQ6XtM9ygKPdXJFSH__Nw@mail.gmail.com/ Reported-by: David Gilmour [2] https://lore.kernel.org/all/CAO2ABipzbw6QL5eNa44CQHjiVa-LTvS696Mh9QaTw+qsUKFUCw@mail.gmail.com/ Signed-off-by: Yu Kuai --- drivers/md/md.c | 1 + drivers/md/raid5.c | 44 +++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 44 insertions(+), 1 deletion(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 1ef81421e131..e2bef5bb00f2 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -9133,6 +9133,7 @@ void md_do_sync(struct md_thread *thread) spin_unlock(&mddev->lock); wake_up(&resync_wait); + wake_up(&mddev->sb_wait); md_wakeup_thread(mddev->thread); return; } diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index bd3b535c0739..7e2bbcfef325 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -5966,6 +5966,19 @@ static int add_all_stripe_bios(struct r5conf *conf, return ret; } +static bool reshape_inprogress(struct mddev *mddev) +{ + return test_bit(MD_RECOVERY_RESHAPE, &mddev->recovery) && + test_bit(MD_RECOVERY_RUNNING, &mddev->recovery) && + !test_bit(MD_RECOVERY_DONE, &mddev->recovery) && + !test_bit(MD_RECOVERY_INTR, &mddev->recovery); +} + +static bool reshape_disabled(struct mddev *mddev) +{ + return is_md_suspended(mddev) || !md_is_rdwr(mddev); +} + static enum stripe_result make_stripe_request(struct mddev *mddev, struct r5conf *conf, struct stripe_request_ctx *ctx, sector_t logical_sector, struct bio *bi) @@ -5997,7 +6010,8 @@ static enum stripe_result make_stripe_request(struct mddev *mddev, if (ahead_of_reshape(mddev, logical_sector, conf->reshape_safe)) { spin_unlock_irq(&conf->device_lock); - return STRIPE_SCHEDULE_AND_RETRY; + ret = STRIPE_SCHEDULE_AND_RETRY; + goto out; } } spin_unlock_irq(&conf->device_lock); @@ -6076,6 +6090,15 @@ static enum stripe_result make_stripe_request(struct mddev *mddev, out_release: raid5_release_stripe(sh); +out: + if (ret == STRIPE_SCHEDULE_AND_RETRY && !reshape_inprogress(mddev) && + reshape_disabled(mddev)) { + bi->bi_status = BLK_STS_IOERR; + ret = STRIPE_FAIL; + pr_err("md/raid456:%s: io failed across reshape position while reshape can't make progress.\n", + mdname(mddev)); + } + return ret; } @@ -9045,6 +9068,22 @@ static int raid5_start(struct mddev *mddev) return r5l_start(conf->log); } +static void raid5_prepare_suspend(struct mddev *mddev) +{ + struct r5conf *conf = mddev->private; + + wait_event(mddev->sb_wait, !reshape_inprogress(mddev) || + percpu_ref_is_zero(&mddev->active_io)); + if (percpu_ref_is_zero(&mddev->active_io)) + return; + + /* + * Reshape is not in progress, and array is suspended, io that is + * waiting for reshpape can never be done. + */ + wake_up(&conf->wait_for_overlap); +} + static struct md_personality raid6_personality = { .name = "raid6", @@ -9065,6 +9104,7 @@ static struct md_personality raid6_personality = .check_reshape = raid6_check_reshape, .start_reshape = raid5_start_reshape, .finish_reshape = raid5_finish_reshape, + .prepare_suspend = raid5_prepare_suspend, .quiesce = raid5_quiesce, .takeover = raid6_takeover, .change_consistency_policy = raid5_change_consistency_policy, @@ -9089,6 +9129,7 @@ static struct md_personality raid5_personality = .check_reshape = raid5_check_reshape, .start_reshape = raid5_start_reshape, .finish_reshape = raid5_finish_reshape, + .prepare_suspend = raid5_prepare_suspend, .quiesce = raid5_quiesce, .takeover = raid5_takeover, .change_consistency_policy = raid5_change_consistency_policy, @@ -9114,6 +9155,7 @@ static struct md_personality raid4_personality = .check_reshape = raid5_check_reshape, .start_reshape = raid5_start_reshape, .finish_reshape = raid5_finish_reshape, + .prepare_suspend = raid5_prepare_suspend, .quiesce = raid5_quiesce, .takeover = raid4_takeover, .change_consistency_policy = raid5_change_consistency_policy,