From patchwork Wed Nov 9 13:34:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christoph_B=C3=B6hmwalder?= X-Patchwork-Id: 17545 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp338751wru; Wed, 9 Nov 2022 05:40:31 -0800 (PST) X-Google-Smtp-Source: AA0mqf7gzG9PPTxB+I/UmkDTv761goZxtHsXizwqADnUqp8hyaM1hoAk7lqwsJt4CZriazBmRppn X-Received: by 2002:a17:906:43c7:b0:7ae:60eb:3f02 with SMTP id j7-20020a17090643c700b007ae60eb3f02mr16262579ejn.712.1668001231766; Wed, 09 Nov 2022 05:40:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668001231; cv=none; d=google.com; s=arc-20160816; b=rMx45xCoDtrLAM5Fib/1aIBe7LZVy5tRJYd7RSJ3NR9Niq/zm13lN2eK3ZdVhYL3di A8YxUZavRG9d4lEcyVcjjGayyU7XhvALvQS/4Ow1hJzQjZIyOdLKffRiSRDbVkpzJirY TFhp0g5ubfCWGVymqFhu6pXf+UcWiyCiNUMr8IPSt5H9mlyAn++2gu+gx66zQEr/Lzao LffHTiA734kiqoiSF5yVOFTQDcj0Tu0Pdos6NE0EbnoZIGWUA0UaieKIQ1zNp8YFUj2o 8wg+Y3h6Kb3Eb+ANTckQUKE72s+n1BFNzwDoFqHh1y0g/QNEHdr8abr2b/BYAMivcRNX pQGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=fMFIifcDEWp2tuYJ6xczHhtO2meaPJaEEnuy7NTfdVA=; b=s5DbCykz5zhvh08JfLaisYMDFuqgblcL2tgw6RCfnSZxbRxNSe1HjU+WkN0PHDXVs3 C3sBcHmfotbxxhz13WMFRqnyKJTFYjDSoj0RoWps8eNqNa7oImyBVi1Jsz+OIG1caj2Z imHD0+v0445PAt21zm44hdTwGij+t+hgEHp6Mk6aaMRnC8DFZ5KeDjZz/bm9jihWf75v prMxuV4Cfvb7pEx279JMSBq4jL1cPsG9GnEmWrbdeyziAzsssZ49CcZVLJw3mcfLw424 wbhYXOBRHUfwF++2WiF0gswXrLAj5vQth7PA+PfTD8AUx0GZzbTSBDqCqo8H/PRbKXSY 8BIg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linbit-com.20210112.gappssmtp.com header.s=20210112 header.b=6kEX59yO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id sh32-20020a1709076ea000b0078a30f54c57si16955275ejc.32.2022.11.09.05.40.07; Wed, 09 Nov 2022 05:40:31 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linbit-com.20210112.gappssmtp.com header.s=20210112 header.b=6kEX59yO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230194AbiKINfR (ORCPT + 99 others); Wed, 9 Nov 2022 08:35:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57740 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229915AbiKINfL (ORCPT ); Wed, 9 Nov 2022 08:35:11 -0500 Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DB2642B605 for ; Wed, 9 Nov 2022 05:35:09 -0800 (PST) Received: by mail-ed1-x536.google.com with SMTP id a67so27223788edf.12 for ; Wed, 09 Nov 2022 05:35:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linbit-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fMFIifcDEWp2tuYJ6xczHhtO2meaPJaEEnuy7NTfdVA=; b=6kEX59yOiWAA2z//lcDmb/ZWXfyTfEGiA7WYWTIEjzQJKuUX13VM+dNzv/hxsTZ+pC lk5yde0+fVgTDH8fbD8AG0mPOG1iA74UPHFMuiDIIQ8a4bEJf+KCTobE5zSuxlPUaHtH DzK3gsDExUAAzzRyDvN7BNwaKspgSNvPVAoS8WUyzp2EtiWV09ftAj5IrcYi1GfqtdF4 XRVOYeOanOl5krqDGBZN0dmyNKJCCgMLrO4qBWb6KZzLNhD71qyMlvugp4RrYKqDidUU fUNkIuJomMJo0c685105vF+bt1yqwNgAb+eUH/KBZGvWC5mbHB7APfBYiXC6H6eg5WbX vesA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fMFIifcDEWp2tuYJ6xczHhtO2meaPJaEEnuy7NTfdVA=; b=ccnwleUCXUCcFQO38h2OM5go/x4NBJXxM+ySirT8UZqfHXLto/D1SjDbz2jKTfKWwR DIYLD2/EZNexkLBtkWLKd2niP31EBfstGPTjOJoP+VjSSRcmovH27wGDMH1pZ/9s+epu qlfT7ELsTHlgqrVY9sRTTHta0LWUGaoFRugkf2fuYFAGpHxvoN9+2FS61EiEHAhUiZjs qQ1XAv8vHKlVmkoP8aqfa9tDxLp4WwumwMW45lV80LDTWwW92G7UoECdfEoV3IRvjUPb FV7ux1Ra4DZZaJ/4jwAHN4WTwdRsWKkAXCpgoDI2m5DFg9ailGxCapLGGk18NPyshWQF JmVg== X-Gm-Message-State: ACrzQf1t+YtoANKvRH1+ruQkJ+bMpvjaRArIwbC3f+vuhEge0vA2JE5Q SkfzzaI9HgiS0/OcJYnrqSkn5w== X-Received: by 2002:aa7:d744:0:b0:45c:e353:e891 with SMTP id a4-20020aa7d744000000b0045ce353e891mr61337607eds.36.1668000908425; Wed, 09 Nov 2022 05:35:08 -0800 (PST) Received: from localhost.localdomain (h082218028181.host.wavenet.at. [82.218.28.181]) by smtp.gmail.com with ESMTPSA id lx8-20020a170906af0800b00782fbb7f5f7sm5867463ejb.113.2022.11.09.05.35.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Nov 2022 05:35:08 -0800 (PST) From: =?utf-8?q?Christoph_B=C3=B6hmwalder?= To: Jens Axboe Cc: drbd-dev@lists.linbit.com, linux-kernel@vger.kernel.org, Lars Ellenberg , Philipp Reisner , linux-block@vger.kernel.org, =?utf-8?q?Christoph_B=C3=B6hmwalder?= , Joel Colledge Subject: [PATCH 1/3] drbd: use blk_queue_max_discard_sectors helper Date: Wed, 9 Nov 2022 14:34:51 +0100 Message-Id: <20221109133453.51652-2-christoph.boehmwalder@linbit.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221109133453.51652-1-christoph.boehmwalder@linbit.com> References: <20221109133453.51652-1-christoph.boehmwalder@linbit.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749026059530192527?= X-GMAIL-MSGID: =?utf-8?q?1749026059530192527?= We currently only set q->limits.max_discard_sectors, but that is not enough. Another field, max_hw_discard_sectors, was introduced in commit 0034af036554 ("block: make /sys/block//queue/discard_max_bytes writeable"). The difference is that max_discard_sectors can be changed from user space via sysfs, while max_hw_discard_sectors is the "hardware" upper limit. So use this helper, which sets both. This is also a fixup for commit 998e9cbcd615 ("drbd: cleanup decide_on_discard_support"): if discards are not supported, that does not necessarily mean we also want to disable write_zeroes. Fixes: 998e9cbcd615 ("drbd: cleanup decide_on_discard_support") Reviewed-by: Joel Colledge Signed-off-by: Christoph Böhmwalder Reviewed-by: Chaitanya Kulkarni --- drivers/block/drbd/drbd_nl.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c index 864c98e74875..249eba7d21c2 100644 --- a/drivers/block/drbd/drbd_nl.c +++ b/drivers/block/drbd/drbd_nl.c @@ -1210,6 +1210,7 @@ static void decide_on_discard_support(struct drbd_device *device, struct drbd_connection *connection = first_peer_device(device)->connection; struct request_queue *q = device->rq_queue; + unsigned int max_discard_sectors; if (bdev && !bdev_max_discard_sectors(bdev->backing_bdev)) goto not_supported; @@ -1230,15 +1231,14 @@ static void decide_on_discard_support(struct drbd_device *device, * topology on all peers. */ blk_queue_discard_granularity(q, 512); - q->limits.max_discard_sectors = drbd_max_discard_sectors(connection); - q->limits.max_write_zeroes_sectors = - drbd_max_discard_sectors(connection); + max_discard_sectors = drbd_max_discard_sectors(connection); + blk_queue_max_discard_sectors(q, max_discard_sectors); + blk_queue_max_write_zeroes_sectors(q, max_discard_sectors); return; not_supported: blk_queue_discard_granularity(q, 0); - q->limits.max_discard_sectors = 0; - q->limits.max_write_zeroes_sectors = 0; + blk_queue_max_discard_sectors(q, 0); } static void fixup_write_zeroes(struct drbd_device *device, struct request_queue *q) From patchwork Wed Nov 9 13:34:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christoph_B=C3=B6hmwalder?= X-Patchwork-Id: 17547 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp339803wru; Wed, 9 Nov 2022 05:42:58 -0800 (PST) X-Google-Smtp-Source: AMsMyM5f8AA4H2kcHfm8Ck90TcryRX6hl5MEwLrh2ksnw5MSp2x1CUgb9Ucfd3mRFjMfstDxY2t7 X-Received: by 2002:a05:6402:4006:b0:461:f4e2:82e0 with SMTP id d6-20020a056402400600b00461f4e282e0mr61645758eda.68.1668001378123; Wed, 09 Nov 2022 05:42:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668001378; cv=none; d=google.com; s=arc-20160816; b=q7cSsrne35OU1+6/ACl+QfJAh5xe9cuKoK+Hfs4KTotFFWYLqSeyCLAxbO0LecGdg4 XgNsIMbiCAC0EjOxJEUCWJo2BeMiwHGoAdjSZl4rmq2SSofB+C2t8+UYoW973Gp/Kn7v LaDeGVMIyh6oy5pxQ7aJTXMI11D7qoMQ6eQ+wIOWTK5TDjEr22E6t2YWtgHpSS58wptv K+Fh5SUK4YAyNTNwAYrSaTeBB8DLtFOK5QfJSt38Q3KqwMLRS0eBxjZ6euheAeMlAuRX qZvgj5yHmkuD7vu9No8oOJ1qUMveyvSJAKF9XT+kWpsdn21od/bX7GxvJfqnzJG3ALaF q+TQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=7hF3rMk8apVOOgsCxEsnYa+onZZgR9Ac90HGkbjlBXY=; b=paSQ+EwxBSlI+BNOILXnQpz19oqdQ25fbGDyThfhbs5ED3/s8bQfYng9FEHxBSn7B2 qagGM3s/HleYL6SoPcjXDrCxc2fH10WU5GmcgZeiaHDoTAhRznNSU7L3Qg6hsduHTI7g Nb/d5ymxTXxUwaPeRoKjNhqkzi1oEmATGPMcHQQUmQe1z6AmqRxjhq0plb0KGyCFnzYO BDmuZ79Ip0ty8pv/2SuLkekrY4K7I+u1R35uEdYga3oC0B/vp762k3rtnQCpeFwvs+U2 p3yEgtcXskRcYtlc6UXNhh/lbrX8wvh2tAHE5a6qSYdBEn6epPuijvZypjKrhd1QVlgB B33w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linbit-com.20210112.gappssmtp.com header.s=20210112 header.b=WYj28GSb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y19-20020a50e613000000b004612505ece4si15532921edm.483.2022.11.09.05.42.33; Wed, 09 Nov 2022 05:42:58 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linbit-com.20210112.gappssmtp.com header.s=20210112 header.b=WYj28GSb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230041AbiKINfT (ORCPT + 99 others); Wed, 9 Nov 2022 08:35:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57924 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230000AbiKINfP (ORCPT ); Wed, 9 Nov 2022 08:35:15 -0500 Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D01C92E9E2 for ; Wed, 9 Nov 2022 05:35:12 -0800 (PST) Received: by mail-ej1-x630.google.com with SMTP id kt23so46631766ejc.7 for ; Wed, 09 Nov 2022 05:35:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linbit-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7hF3rMk8apVOOgsCxEsnYa+onZZgR9Ac90HGkbjlBXY=; b=WYj28GSbjnk/0i6VzKp+MEOsvjkvjUeibMvRcUfW8smL/+shwJz5l4p4w/58lBmgHG wAF7yA/kX6SFBy/U6gtEdb2TLYEoDejIGD9TO7iJI1aWWq8v8x/2NQCeQYOYCSOlVih+ jm9H+EOdFouIINDlH9a730ZzHbq3ZLklBfiD4xL4AkgO9e6NYGl216ncs2uqfDGc2sG/ jNtDphsOaZbNoguEdcNTKrSxD8fofe7WvgLClYsd+29wF/MgfETVfX2rzwwzOvbffDkC yZT1XLShDODK9R/wOvvSJbtwJdYfW2ofQea00HyXshfebDO+YAgfBWaQF2qqyeIuWtUT RO1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7hF3rMk8apVOOgsCxEsnYa+onZZgR9Ac90HGkbjlBXY=; b=3/AS565IRK2yfy3eMy4og32f5YV2qZVyNA/6Y0K0vgUCCFrl/rC74ke2RwyTiq4brk 2R0efUyl+YN+V5Pkk6peka3GCSOMOPPM1qwbIGwXPrUflEXVzLHto452d8SPrVufFVRV zqW9pAI1qiB8xbnqRwo1r6I8uykAJaQKmQGTkPQsen6HEVGbyocBm5ixgFkznfY3t6lk 2Bu/To+8TtR0z6ZLTopBF9ihel5bo14BBO8YbuG6mqHVizsyO3S8YrmyNugOWRk/XanT 9gr1shLM2AmxFOmWIHBTaDTxzQSybeIJnBnE7zVfDDU6dIhW7G1ZlAdK4CRrhZ5NlDUy NAhQ== X-Gm-Message-State: ACrzQf3OZpmzZp5jPVL9ufO/fjrW623WeDjHufSijmlnVB/N4FOFDBEp yRRGp3f1yYPmO7SgNMZgr0wq3w== X-Received: by 2002:a17:906:3b48:b0:7ad:fa67:1daa with SMTP id h8-20020a1709063b4800b007adfa671daamr38243729ejf.653.1668000911374; Wed, 09 Nov 2022 05:35:11 -0800 (PST) Received: from localhost.localdomain (h082218028181.host.wavenet.at. [82.218.28.181]) by smtp.gmail.com with ESMTPSA id lx8-20020a170906af0800b00782fbb7f5f7sm5867463ejb.113.2022.11.09.05.35.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Nov 2022 05:35:11 -0800 (PST) From: =?utf-8?q?Christoph_B=C3=B6hmwalder?= To: Jens Axboe Cc: drbd-dev@lists.linbit.com, linux-kernel@vger.kernel.org, Lars Ellenberg , Philipp Reisner , linux-block@vger.kernel.org, Joel Colledge , =?utf-8?q?Christoph_B=C3=B6hmwald?= =?utf-8?q?er?= Subject: [PATCH 2/3] drbd: disable discard support if granularity > max Date: Wed, 9 Nov 2022 14:34:52 +0100 Message-Id: <20221109133453.51652-3-christoph.boehmwalder@linbit.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221109133453.51652-1-christoph.boehmwalder@linbit.com> References: <20221109133453.51652-1-christoph.boehmwalder@linbit.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749026213242075281?= X-GMAIL-MSGID: =?utf-8?q?1749026213242075281?= From: Philipp Reisner The discard_granularity describes the minimum unit of a discard. If that is larger than the maximal discard size, we need to disable discards completely. Reviewed-by: Joel Colledge Signed-off-by: Philipp Reisner Signed-off-by: Christoph Böhmwalder --- drivers/block/drbd/drbd_nl.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c index 249eba7d21c2..63f589926d85 100644 --- a/drivers/block/drbd/drbd_nl.c +++ b/drivers/block/drbd/drbd_nl.c @@ -1256,6 +1256,18 @@ static void fixup_write_zeroes(struct drbd_device *device, struct request_queue q->limits.max_write_zeroes_sectors = 0; } +static void fixup_discard_support(struct drbd_device *device, struct request_queue *q) +{ + unsigned int max_discard = device->rq_queue->limits.max_discard_sectors; + unsigned int discard_granularity = + device->rq_queue->limits.discard_granularity >> SECTOR_SHIFT; + + if (discard_granularity > max_discard) { + blk_queue_discard_granularity(q, 0); + blk_queue_max_discard_sectors(q, 0); + } +} + static void drbd_setup_queue_param(struct drbd_device *device, struct drbd_backing_dev *bdev, unsigned int max_bio_size, struct o_qlim *o) { @@ -1288,6 +1300,7 @@ static void drbd_setup_queue_param(struct drbd_device *device, struct drbd_backi disk_update_readahead(device->vdisk); } fixup_write_zeroes(device, q); + fixup_discard_support(device, q); } void drbd_reconsider_queue_parameters(struct drbd_device *device, struct drbd_backing_dev *bdev, struct o_qlim *o) From patchwork Wed Nov 9 13:34:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christoph_B=C3=B6hmwalder?= X-Patchwork-Id: 17548 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp341878wru; Wed, 9 Nov 2022 05:47:51 -0800 (PST) X-Google-Smtp-Source: AMsMyM4JP7F8idsV9XXX3JIUzA/6EHVXGih9F86J/PRLm1K96kwoG6p6l2En+s+QPXA5e5anEJwg X-Received: by 2002:a05:6a00:8c5:b0:56d:67d8:10a with SMTP id s5-20020a056a0008c500b0056d67d8010amr21846616pfu.28.1668001671529; Wed, 09 Nov 2022 05:47:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668001671; cv=none; d=google.com; s=arc-20160816; b=yZ6qe0Lxxuz9ZhgUVOh5EXLoZNe2LKAZeCAeE1qUAuzyyVMm0ucnzKgwe48/q/jItG 4HVTnvnDE8/RG4AjyLAw8QjhBvwgeNBl7ZsQCXZYRDtLFu5pSra7epNH1mA23DTT+2st /z0qZ9cws46p1j1U07YZR/aOPAC3mHYx+Ay8aV1v0xsqzqrstPEREHWwXKw7B6bnVFrB hlW+vv4B2iDfe7iJ9VEkp+dg9IYdZjWezF+Qku1ucfaFFLdl5USVEYyPghhWNU584Lf+ nqRD/P8mX1C12Cbcwyu2pD8qe9RpjtRdx+yViJB1BwOW0ejAECjNqB4cTtSx/brtUbor DFKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=iWe+5u2rhx99ZPm+uxu8vcq2qF8FxcNBmLJ9xXsjgYQ=; b=X9mGeH8MJnoFX3qrADg9VU72M787C0CS0rfqH7RPjX8qbVpq/Uh7hNsdbGGmxbAHEa COuR3WCAsiiYRTT/4eKTePhDIdVguhqv7JrYJrXwl2/KjZjtVqvIgmh6YUlu+UG2n0pf j2dvOlA7tdPXcYv9SIBr3iI0Ir1znXw9MqdVv1iOKD1Sc65AGPyzmuJOEKm6H8pBH8Ug iRxvWfXnazfiemRHGSeIm23R67tYUN+mAIDNHwyNWl2FqnnhaRsyMYMAHECxh6Uqauk6 KifhODtpQRBYRl6wCe+NH0mPdKktbUc4GrKU0bgwxS9ZtHKPXDbmlELss9B9kPCoJzL9 8e+Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linbit-com.20210112.gappssmtp.com header.s=20210112 header.b=AP13A9Wk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id y9-20020a17090322c900b0017f641cac01si20590233plg.27.2022.11.09.05.47.36; Wed, 09 Nov 2022 05:47:51 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linbit-com.20210112.gappssmtp.com header.s=20210112 header.b=AP13A9Wk; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230216AbiKINfW (ORCPT + 99 others); Wed, 9 Nov 2022 08:35:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57924 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230155AbiKINfR (ORCPT ); Wed, 9 Nov 2022 08:35:17 -0500 Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 066762E9D0 for ; Wed, 9 Nov 2022 05:35:15 -0800 (PST) Received: by mail-ej1-x636.google.com with SMTP id f27so46697458eje.1 for ; Wed, 09 Nov 2022 05:35:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linbit-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iWe+5u2rhx99ZPm+uxu8vcq2qF8FxcNBmLJ9xXsjgYQ=; b=AP13A9WkHMJ3O7/MGQkFOlZ4M+10TdavqFR7l74hQmBAVZOOW8N6Mqnj/k7kFxQKO1 rGT4cTRGDd8SMjF2r88N5vuz2URbYnEWqdtHWR82wv4pVyFDaCpZrY2HtjeS3HY07vux SkJaQEB2UA282vynElxl49j/7VVm4kNRQGkXry1INd9vjamQf75RCf636JW+dO5GkRS9 /GetEte/TyLenuoRg8zOXImlFlF0VjNbrS0Qt2fWDEgu6YSBIxOI7m4fHheNI98x1YH0 8eAoXuII2ysyIjh9S11RHvJoiLPipAsg4qCpN55wB2lWPwtpfidWkIOoRfXv/syXONBX PqmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iWe+5u2rhx99ZPm+uxu8vcq2qF8FxcNBmLJ9xXsjgYQ=; b=fC0qg3qZm2k4y2n6wU6dMezxQ9ROyZLWUDRfzlmIGSB9xlks9JfnD4IOjzeQ3GaALX O9B8DkENWf4ifK2TgNc/sZ7FCGmu4W/1qz8mFH5O9YHyQEqc2BfXtYEadOQDaoIUYdi3 nxeVX3BVa889ia7Meypm492WrliDxMwhEicgJSFRDRgv05ruXPC0su0EEE0RXujiyNFf sClrhmGhTJfp/peo9jBABbQtScNKGyzJrz/K06n/wwYG8W5fNfWZ24Bl9GH2mvigx3+G RnFPpetBt4oZuIVLLWODZsda6ygoFMg8hTELf23cDumP+XhC8yFXak9j9ek/x+Yngznq 50EA== X-Gm-Message-State: ACrzQf3dNVQ0BEHkTJXaTMEh/sODdc9OJbdc/yIUhkQz221uUgrQsX3l m9snc45twjiAGc1K5QLjeUyfxz4ez2doIZOf X-Received: by 2002:a17:907:60c8:b0:78d:bc8d:8457 with SMTP id hv8-20020a17090760c800b0078dbc8d8457mr55342620ejc.418.1668000913542; Wed, 09 Nov 2022 05:35:13 -0800 (PST) Received: from localhost.localdomain (h082218028181.host.wavenet.at. [82.218.28.181]) by smtp.gmail.com with ESMTPSA id lx8-20020a170906af0800b00782fbb7f5f7sm5867463ejb.113.2022.11.09.05.35.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Nov 2022 05:35:13 -0800 (PST) From: =?utf-8?q?Christoph_B=C3=B6hmwalder?= To: Jens Axboe Cc: drbd-dev@lists.linbit.com, linux-kernel@vger.kernel.org, Lars Ellenberg , Philipp Reisner , linux-block@vger.kernel.org, =?utf-8?q?Christoph_B=C3=B6hmwalder?= , Joel Colledge Subject: [PATCH RESEND 3/3] drbd: Store op in drbd_peer_request Date: Wed, 9 Nov 2022 14:34:53 +0100 Message-Id: <20221109133453.51652-4-christoph.boehmwalder@linbit.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221109133453.51652-1-christoph.boehmwalder@linbit.com> References: <20221109133453.51652-1-christoph.boehmwalder@linbit.com> MIME-Version: 1.0 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1749026520468008902?= X-GMAIL-MSGID: =?utf-8?q?1749026520468008902?= (Sort of) cherry-picked from the out-of-tree drbd9 branch. Original commit message by Joel Colledge: This simplifies drbd_submit_peer_request by removing most of the arguments. It also makes the treatment of the op better aligned with that in struct bio. Determine fault_type dynamically using information which is already available instead of passing it in as a parameter. Note: The opf in receive_rs_deallocated was changed from REQ_OP_WRITE_ZEROES to REQ_OP_DISCARD. This was required in the out-of-tree module, and does not matter in-tree. The opf is ignored anyway in drbd_submit_peer_request, since the discard/zero-out is decided by the EE_TRIM flag. Signed-off-by: Joel Colledge Signed-off-by: Christoph Böhmwalder --- drivers/block/drbd/drbd_int.h | 8 ++- drivers/block/drbd/drbd_receiver.c | 81 +++++++++++++++++------------- drivers/block/drbd/drbd_worker.c | 4 +- 3 files changed, 53 insertions(+), 40 deletions(-) diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h index 4d661282ff41..913cf4c55cba 100644 --- a/drivers/block/drbd/drbd_int.h +++ b/drivers/block/drbd/drbd_int.h @@ -395,6 +395,7 @@ struct drbd_peer_request { struct drbd_peer_device *peer_device; struct drbd_epoch *epoch; /* for writes */ struct page *pages; + blk_opf_t opf; atomic_t pending_bios; struct drbd_interval i; /* see comments on ee flag bits below */ @@ -406,6 +407,10 @@ struct drbd_peer_request { }; }; +/* Equivalent to bio_op and req_op. */ +#define peer_req_op(peer_req) \ + ((peer_req)->opf & REQ_OP_MASK) + /* ee flag bits. * While corresponding bios are in flight, the only modification will be * set_bit WAS_ERROR, which has to be atomic. @@ -1545,8 +1550,7 @@ extern void drbd_send_acks_wf(struct work_struct *ws); extern bool drbd_rs_c_min_rate_throttle(struct drbd_device *device); extern bool drbd_rs_should_slow_down(struct drbd_device *device, sector_t sector, bool throttle_if_app_is_waiting); -extern int drbd_submit_peer_request(struct drbd_device *, - struct drbd_peer_request *, blk_opf_t, int); +extern int drbd_submit_peer_request(struct drbd_peer_request *peer_req); extern int drbd_free_peer_reqs(struct drbd_device *, struct list_head *); extern struct drbd_peer_request *drbd_alloc_peer_req(struct drbd_peer_device *, u64, sector_t, unsigned int, diff --git a/drivers/block/drbd/drbd_receiver.c b/drivers/block/drbd/drbd_receiver.c index ee69d50ba4fd..93d6df4dc5a4 100644 --- a/drivers/block/drbd/drbd_receiver.c +++ b/drivers/block/drbd/drbd_receiver.c @@ -1603,9 +1603,19 @@ static void drbd_issue_peer_discard_or_zero_out(struct drbd_device *device, stru drbd_endio_write_sec_final(peer_req); } +static int peer_request_fault_type(struct drbd_peer_request *peer_req) +{ + if (peer_req_op(peer_req) == REQ_OP_READ) { + return peer_req->flags & EE_APPLICATION ? + DRBD_FAULT_DT_RD : DRBD_FAULT_RS_RD; + } else { + return peer_req->flags & EE_APPLICATION ? + DRBD_FAULT_DT_WR : DRBD_FAULT_RS_WR; + } +} + /** * drbd_submit_peer_request() - * @device: DRBD device. * @peer_req: peer request * * May spread the pages to multiple bios, @@ -1619,10 +1629,9 @@ static void drbd_issue_peer_discard_or_zero_out(struct drbd_device *device, stru * on certain Xen deployments. */ /* TODO allocate from our own bio_set. */ -int drbd_submit_peer_request(struct drbd_device *device, - struct drbd_peer_request *peer_req, - const blk_opf_t opf, const int fault_type) +int drbd_submit_peer_request(struct drbd_peer_request *peer_req) { + struct drbd_device *device = peer_req->peer_device->device; struct bio *bios = NULL; struct bio *bio; struct page *page = peer_req->pages; @@ -1667,7 +1676,18 @@ int drbd_submit_peer_request(struct drbd_device *device, * generated bio, but a bio allocated on behalf of the peer. */ next_bio: - bio = bio_alloc(device->ldev->backing_bdev, nr_pages, opf, GFP_NOIO); + /* _DISCARD, _WRITE_ZEROES handled above. + * REQ_OP_FLUSH (empty flush) not expected, + * should have been mapped to a "drbd protocol barrier". + * REQ_OP_SECURE_ERASE: I don't see how we could ever support that. + */ + if (!(peer_req_op(peer_req) == REQ_OP_WRITE || + peer_req_op(peer_req) == REQ_OP_READ)) { + drbd_err(device, "Invalid bio op received: 0x%x\n", peer_req->opf); + return -EINVAL; + } + + bio = bio_alloc(device->ldev->backing_bdev, nr_pages, peer_req->opf, GFP_NOIO); /* > peer_req->i.sector, unless this is the first bio */ bio->bi_iter.bi_sector = sector; bio->bi_private = peer_req; @@ -1697,7 +1717,7 @@ int drbd_submit_peer_request(struct drbd_device *device, bios = bios->bi_next; bio->bi_next = NULL; - drbd_submit_bio_noacct(device, fault_type, bio); + drbd_submit_bio_noacct(device, peer_request_fault_type(peer_req), bio); } while (bios); return 0; } @@ -2051,6 +2071,7 @@ static int recv_resync_read(struct drbd_peer_device *peer_device, sector_t secto * respective _drbd_clear_done_ee */ peer_req->w.cb = e_end_resync_block; + peer_req->opf = REQ_OP_WRITE; peer_req->submit_jif = jiffies; spin_lock_irq(&device->resource->req_lock); @@ -2058,8 +2079,7 @@ static int recv_resync_read(struct drbd_peer_device *peer_device, sector_t secto spin_unlock_irq(&device->resource->req_lock); atomic_add(pi->size >> 9, &device->rs_sect_ev); - if (drbd_submit_peer_request(device, peer_req, REQ_OP_WRITE, - DRBD_FAULT_RS_WR) == 0) + if (drbd_submit_peer_request(peer_req) == 0) return 0; /* don't care for the reason here */ @@ -2375,16 +2395,6 @@ static int wait_for_and_update_peer_seq(struct drbd_peer_device *peer_device, co return ret; } -/* see also bio_flags_to_wire() - * DRBD_REQ_*, because we need to semantically map the flags to data packet - * flags and back. We may replicate to other kernel versions. */ -static blk_opf_t wire_flags_to_bio_flags(u32 dpf) -{ - return (dpf & DP_RW_SYNC ? REQ_SYNC : 0) | - (dpf & DP_FUA ? REQ_FUA : 0) | - (dpf & DP_FLUSH ? REQ_PREFLUSH : 0); -} - static enum req_op wire_flags_to_bio_op(u32 dpf) { if (dpf & DP_ZEROES) @@ -2395,6 +2405,15 @@ static enum req_op wire_flags_to_bio_op(u32 dpf) return REQ_OP_WRITE; } +/* see also bio_flags_to_wire() */ +static blk_opf_t wire_flags_to_bio(struct drbd_connection *connection, u32 dpf) +{ + return wire_flags_to_bio_op(dpf) | + (dpf & DP_RW_SYNC ? REQ_SYNC : 0) | + (dpf & DP_FUA ? REQ_FUA : 0) | + (dpf & DP_FLUSH ? REQ_PREFLUSH : 0); +} + static void fail_postponed_requests(struct drbd_device *device, sector_t sector, unsigned int size) { @@ -2538,8 +2557,6 @@ static int receive_Data(struct drbd_connection *connection, struct packet_info * struct drbd_peer_request *peer_req; struct p_data *p = pi->data; u32 peer_seq = be32_to_cpu(p->seq_num); - enum req_op op; - blk_opf_t op_flags; u32 dp_flags; int err, tp; @@ -2578,11 +2595,10 @@ static int receive_Data(struct drbd_connection *connection, struct packet_info * peer_req->flags |= EE_APPLICATION; dp_flags = be32_to_cpu(p->dp_flags); - op = wire_flags_to_bio_op(dp_flags); - op_flags = wire_flags_to_bio_flags(dp_flags); + peer_req->opf = wire_flags_to_bio(connection, dp_flags); if (pi->cmd == P_TRIM) { D_ASSERT(peer_device, peer_req->i.size > 0); - D_ASSERT(peer_device, op == REQ_OP_DISCARD); + D_ASSERT(peer_device, peer_req_op(peer_req) == REQ_OP_DISCARD); D_ASSERT(peer_device, peer_req->pages == NULL); /* need to play safe: an older DRBD sender * may mean zero-out while sending P_TRIM. */ @@ -2590,7 +2606,7 @@ static int receive_Data(struct drbd_connection *connection, struct packet_info * peer_req->flags |= EE_ZEROOUT; } else if (pi->cmd == P_ZEROES) { D_ASSERT(peer_device, peer_req->i.size > 0); - D_ASSERT(peer_device, op == REQ_OP_WRITE_ZEROES); + D_ASSERT(peer_device, peer_req_op(peer_req) == REQ_OP_WRITE_ZEROES); D_ASSERT(peer_device, peer_req->pages == NULL); /* Do (not) pass down BLKDEV_ZERO_NOUNMAP? */ if (dp_flags & DP_DISCARD) @@ -2677,8 +2693,7 @@ static int receive_Data(struct drbd_connection *connection, struct packet_info * peer_req->flags |= EE_CALL_AL_COMPLETE_IO; } - err = drbd_submit_peer_request(device, peer_req, op | op_flags, - DRBD_FAULT_DT_WR); + err = drbd_submit_peer_request(peer_req); if (!err) return 0; @@ -2789,7 +2804,6 @@ static int receive_DataRequest(struct drbd_connection *connection, struct packet struct drbd_peer_request *peer_req; struct digest_info *di = NULL; int size, verb; - unsigned int fault_type; struct p_block_req *p = pi->data; peer_device = conn_peer_device(connection, pi->vnr); @@ -2849,11 +2863,11 @@ static int receive_DataRequest(struct drbd_connection *connection, struct packet put_ldev(device); return -ENOMEM; } + peer_req->opf = REQ_OP_READ; switch (pi->cmd) { case P_DATA_REQUEST: peer_req->w.cb = w_e_end_data_req; - fault_type = DRBD_FAULT_DT_RD; /* application IO, don't drbd_rs_begin_io */ peer_req->flags |= EE_APPLICATION; goto submit; @@ -2867,14 +2881,12 @@ static int receive_DataRequest(struct drbd_connection *connection, struct packet fallthrough; case P_RS_DATA_REQUEST: peer_req->w.cb = w_e_end_rsdata_req; - fault_type = DRBD_FAULT_RS_RD; /* used in the sector offset progress display */ device->bm_resync_fo = BM_SECT_TO_BIT(sector); break; case P_OV_REPLY: case P_CSUM_RS_REQUEST: - fault_type = DRBD_FAULT_RS_RD; di = kmalloc(sizeof(*di) + pi->size, GFP_NOIO); if (!di) goto out_free_e; @@ -2923,7 +2935,6 @@ static int receive_DataRequest(struct drbd_connection *connection, struct packet (unsigned long long)sector); } peer_req->w.cb = w_e_end_ov_req; - fault_type = DRBD_FAULT_RS_RD; break; default: @@ -2975,8 +2986,7 @@ static int receive_DataRequest(struct drbd_connection *connection, struct packet submit: update_receiver_timing_details(connection, drbd_submit_peer_request); inc_unacked(device); - if (drbd_submit_peer_request(device, peer_req, REQ_OP_READ, - fault_type) == 0) + if (drbd_submit_peer_request(peer_req) == 0) return 0; /* don't care for the reason here */ @@ -4947,7 +4957,6 @@ static int receive_rs_deallocated(struct drbd_connection *connection, struct pac if (get_ldev(device)) { struct drbd_peer_request *peer_req; - const enum req_op op = REQ_OP_WRITE_ZEROES; peer_req = drbd_alloc_peer_req(peer_device, ID_SYNCER, sector, size, 0, GFP_NOIO); @@ -4957,6 +4966,7 @@ static int receive_rs_deallocated(struct drbd_connection *connection, struct pac } peer_req->w.cb = e_end_resync_block; + peer_req->opf = REQ_OP_DISCARD; peer_req->submit_jif = jiffies; peer_req->flags |= EE_TRIM; @@ -4965,8 +4975,7 @@ static int receive_rs_deallocated(struct drbd_connection *connection, struct pac spin_unlock_irq(&device->resource->req_lock); atomic_add(pi->size >> 9, &device->rs_sect_ev); - err = drbd_submit_peer_request(device, peer_req, op, - DRBD_FAULT_RS_WR); + err = drbd_submit_peer_request(peer_req); if (err) { spin_lock_irq(&device->resource->req_lock); diff --git a/drivers/block/drbd/drbd_worker.c b/drivers/block/drbd/drbd_worker.c index 0bb1a900c2d5..c69beefc9d5c 100644 --- a/drivers/block/drbd/drbd_worker.c +++ b/drivers/block/drbd/drbd_worker.c @@ -400,13 +400,13 @@ static int read_for_csum(struct drbd_peer_device *peer_device, sector_t sector, goto defer; peer_req->w.cb = w_e_send_csum; + peer_req->opf = REQ_OP_READ; spin_lock_irq(&device->resource->req_lock); list_add_tail(&peer_req->w.list, &device->read_ee); spin_unlock_irq(&device->resource->req_lock); atomic_add(size >> 9, &device->rs_sect_ev); - if (drbd_submit_peer_request(device, peer_req, REQ_OP_READ, - DRBD_FAULT_RS_RD) == 0) + if (drbd_submit_peer_request(peer_req) == 0) return 0; /* If it failed because of ENOMEM, retry should help. If it failed