Message ID | 20231007012817.3052558-1-sarthakkukreti@chromium.org |
---|---|
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:a888:0:b0:403:3b70:6f57 with SMTP id x8csp692246vqo; Fri, 6 Oct 2023 18:28:53 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE9K/+ySCPqV8b56AlverW71rlQup/mbcye4IHCnvzYi/HdHkJkpLoArLAtkz0qTZDO+rr2 X-Received: by 2002:aca:1b15:0:b0:3ad:fe8d:dfae with SMTP id b21-20020aca1b15000000b003adfe8ddfaemr9116573oib.57.1696642133328; Fri, 06 Oct 2023 18:28:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696642133; cv=none; d=google.com; s=arc-20160816; b=p5q/iZgCz0TMPowRNxqAo21+fmlt+FfyHE4KjU752TuegFlKhRsSDdflDQuybvwups uTSoz2n9qfVgf0swvwEjoZWcsc9eJf1SdL6TZqyDQZBcw2g8dwrMDQi6f5bZT71M/p2D UXI/99mN5QqbyvS8W//QUmTAFCvga5WdmyqYFt805SZiCF2LfzOgvgiOFf9MOn9ViFyh hpTf1zVtylK9uoKbLxP8ex5/Hz7217QdWny2r7IOCs+3k7GwVOTJRwsnNX8napjfcM/0 PqVpIstDFskQb8spTgxenChpRQKLa91tsDVfMaL6XxG3tCEYZtFdNT8Xg7aym02qcQZQ +Ogg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=sPAOchZ3lviZQWz7F/yVzJ/5u6K9fS7pELXL8VtCc9c=; fh=LP/ElqP5atwyI2q2hCf+Eqqm/1Qm/oXEwaFb5LVgt4U=; b=el9sPcISnp+LqfpYPnB9tkosBTyQu0Ca2EoBFyJSV0Z2yYjOKdvSLut1OhjJrgmseC rQbWvTSQFqOHB9GzcSf7rEAyToQq1VA5lACjQYym4IM9F5ip+j+a5B/1Gk1Db0XnQjq+ Wdla/1lDV09zCctSiQKLYeFm1Awivz0L1JfWMmegHeQvu/3T6f5oo4W9H9SVLcBlejym nRlX3EMkTbJUVeYZIXPD/JtQkeSn4k3eZhhz3kuHdeMyqxyWF+lzsSVb9EoNepS5BImg VMxdeggHVapoVgCWSrTPqn3KwTpPGCatxiAbr//UIRBkh7Y2TGJjAsJD9BJTQcQYjcWv yjeg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=BB+jwWHP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from morse.vger.email (morse.vger.email. [2620:137:e000::3:1]) by mx.google.com with ESMTPS id s35-20020a634523000000b00573f9d84ecbsi4758714pga.387.2023.10.06.18.28.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Oct 2023 18:28:53 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) client-ip=2620:137:e000::3:1; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=BB+jwWHP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id D553C8042C33; Fri, 6 Oct 2023 18:28:48 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234039AbjJGB21 (ORCPT <rfc822;ezelljr.billy@gmail.com> + 18 others); Fri, 6 Oct 2023 21:28:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53372 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233556AbjJGB2Z (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Fri, 6 Oct 2023 21:28:25 -0400 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 98F3DBD for <linux-kernel@vger.kernel.org>; Fri, 6 Oct 2023 18:28:24 -0700 (PDT) Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-1c874b43123so23046925ad.2 for <linux-kernel@vger.kernel.org>; Fri, 06 Oct 2023 18:28:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1696642104; x=1697246904; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=sPAOchZ3lviZQWz7F/yVzJ/5u6K9fS7pELXL8VtCc9c=; b=BB+jwWHPZKoC+SFdcT4TIJ0MVh4GMwMAeh9vXdKr0exaDqj/uLEU5WfLtt/OytF56U cmIKL4nzncZ4Mlz/UWnu+96fR2gV33xPR1GhH+sCjipxEcrhIB2Qc1rkM505dMLdOif3 90XFavyzJ35MiqABFI9ylzog9/FqQvDspR9GI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696642104; x=1697246904; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=sPAOchZ3lviZQWz7F/yVzJ/5u6K9fS7pELXL8VtCc9c=; b=boEs5SqPNiUV2KGjuBB8JpJ5N+hDs3/FhsYIhLMRAkwx4eV3va0iGDaKni1APnkXJt AH1a5kPdiXybpFme1KG1jFYci2KAPjbxfK8cXaC+VukYqr7fasjXi3INoEVICv3JF1oX 0g1lqqFmFq8yIcns4iw2JJqtiW7DecttecY/FQz/W6OeHgmYA+XBSeQUtSn75G8fPfw9 TrBMahkZmHudj8Ib3XxoXVEixwdT7p1FXzwDCNlveJy0aDK9tZrk8X665RjHxqSkyZzz sSboLgbU1O26xYLo20Fl527+shz6vBlks+hGYxuAGVnkHtJra1pUBfbNYTcfcfeIc8+6 y7wA== X-Gm-Message-State: AOJu0YzkiD4hn2lqmbJaJEmlOm2E4IWGYu+AaOXl5Qic5o4lD2WfG0tx BK8/xkLwbzyK2INRRqMlFTCAgQ== X-Received: by 2002:a17:903:228f:b0:1c7:66a4:27ba with SMTP id b15-20020a170903228f00b001c766a427bamr11470478plh.48.1696642104031; Fri, 06 Oct 2023 18:28:24 -0700 (PDT) Received: from localhost ([2620:15c:9d:2:138c:8976:eb4a:a91c]) by smtp.gmail.com with UTF8SMTPSA id q13-20020a170902dacd00b001b8b2a6c4a4sm4575373plx.172.2023.10.06.18.28.22 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 06 Oct 2023 18:28:23 -0700 (PDT) From: Sarthak Kukreti <sarthakkukreti@chromium.org> To: dm-devel@redhat.com, linux-block@vger.kernel.org, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org Cc: Jens Axboe <axboe@kernel.dk>, Alasdair Kergon <agk@redhat.com>, Mike Snitzer <snitzer@kernel.org>, Christoph Hellwig <hch@infradead.org>, Brian Foster <bfoster@redhat.com>, Theodore Ts'o <tytso@mit.edu>, Andreas Dilger <adilger.kernel@dilger.ca>, Bart Van Assche <bvanassche@google.com>, "Darrick J. Wong" <djwong@kernel.org>, Dave Chinner <david@fromorbit.com>, Sarthak Kukreti <sarthakkukreti@chromium.org> Subject: [PATCH v8 0/5] Introduce provisioning primitives Date: Fri, 6 Oct 2023 18:28:12 -0700 Message-ID: <20231007012817.3052558-1-sarthakkukreti@chromium.org> X-Mailer: git-send-email 2.42.0.609.gbb76f46606-goog MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=2.7 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Fri, 06 Oct 2023 18:28:48 -0700 (PDT) X-Spam-Level: ** X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1779058221718472524 X-GMAIL-MSGID: 1779058221718472524 |
Series |
Introduce provisioning primitives
|
|
Message
Sarthak Kukreti
Oct. 7, 2023, 1:28 a.m. UTC
Hi, This patch series is version 8 of the patch series to introduce block-level provisioning mechanism (original [1]), which is useful for provisioning space across thinly provisioned storage architectures (loop devices backed by sparse files, dm-thin devices, virtio-blk). This series has minimal changes over v7[2]. This patch series is rebased from the linux-dm/dm-6.5-provision-support [1] on to (cac405a3bfa2 Merge tag 'for-6.6-rc3-tag'). In addition, there's an additional patch to allow passing through an unshare intent via REQ_OP_PROVISION (suggested by Darrick in [4]). [1] Original: https://lore.kernel.org/lkml/20220915164826.1396245-1-sarthakkukreti@google.com/ [2] v7 (last series): https://lore.kernel.org/linux-fsdevel/20230518223326.18744-1-sarthakkukreti@chromium.org/ [3] linux-dm/dm-6.5-provision-suppport tree: https://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git/log/?h=dm-6.5-provision-support (with the last two WIP patches for dm-thinpool dropped as per discussion with maintainers). [4] https://lore.kernel.org/linux-fsdevel/20230522163710.GA11607@frogsfrogsfrogs/ Changes from v7: - Drop dm-thinpool (will be independently developed with snapshot support) and dm-snapshot (will not be supported) from the series. - (By snitzer@kernel.org) Fixes for block device provision limits. - (Suggested by djwong@kernel.org) Add mechanism to pass unshare intent via REQ_OP_PROVISION Sarthak Kukreti (5): block: Don't invalidate pagecache for invalid falloc modes block: Introduce provisioning primitives loop: Add support for provision requests dm: Add block provisioning support block: Pass unshare intent via REQ_OP_PROVISION block/blk-core.c | 5 +++ block/blk-lib.c | 55 ++++++++++++++++++++++++++++++++ block/blk-merge.c | 18 +++++++++++ block/blk-settings.c | 19 +++++++++++ block/blk-sysfs.c | 9 ++++++ block/bounce.c | 1 + block/fops.c | 33 ++++++++++++++++---- drivers/block/loop.c | 59 ++++++++++++++++++++++++++++++++--- drivers/md/dm-crypt.c | 4 ++- drivers/md/dm-linear.c | 1 + drivers/md/dm-table.c | 23 ++++++++++++++ drivers/md/dm.c | 7 +++++ include/linux/bio.h | 6 ++-- include/linux/blk_types.h | 8 ++++- include/linux/blkdev.h | 17 ++++++++++ include/linux/device-mapper.h | 17 ++++++++++ 16 files changed, 268 insertions(+), 14 deletions(-)
Comments
On Fri, Oct 06, 2023 at 06:28:12PM -0700, Sarthak Kukreti wrote: > Hi, > > This patch series is version 8 of the patch series to introduce > block-level provisioning mechanism (original [1]), which is useful for provisioning > space across thinly provisioned storage architectures (loop devices > backed by sparse files, dm-thin devices, virtio-blk). This series has > minimal changes over v7[2]. > > This patch series is rebased from the linux-dm/dm-6.5-provision-support [1] on to > (cac405a3bfa2 Merge tag 'for-6.6-rc3-tag'). In addition, there's an > additional patch to allow passing through an unshare intent via REQ_OP_PROVISION > (suggested by Darrick in [4]). The XFS patches I just posted were smoke tested a while back against loop devices and then forward ported to this patchset. Good for testing that userspace driven file preallocation gets propagated by the filesystem down to the backing device correctly and that subsequent IO to the file then does the right thing (e.g. fio testing using fallocate() to set up the files being written to).... -Dave.
On Sun, Oct 8, 2023 at 4:50 PM Dave Chinner <david@fromorbit.com> wrote: > > On Fri, Oct 06, 2023 at 06:28:12PM -0700, Sarthak Kukreti wrote: > > Hi, > > > > This patch series is version 8 of the patch series to introduce > > block-level provisioning mechanism (original [1]), which is useful for provisioning > > space across thinly provisioned storage architectures (loop devices > > backed by sparse files, dm-thin devices, virtio-blk). This series has > > minimal changes over v7[2]. > > > > This patch series is rebased from the linux-dm/dm-6.5-provision-support [1] on to > > (cac405a3bfa2 Merge tag 'for-6.6-rc3-tag'). In addition, there's an > > additional patch to allow passing through an unshare intent via REQ_OP_PROVISION > > (suggested by Darrick in [4]). > > The XFS patches I just posted were smoke tested a while back against > loop devices and then forward ported to this patchset. Good for > testing that userspace driven file preallocation gets propagated by > the filesystem down to the backing device correctly and that > subsequent IO to the file then does the right thing (e.g. fio > testing using fallocate() to set up the files being written to).... > Thanks! I've been testing with a WIP patch for ext4, I'll give your patches a try. Once we are closer to submitting the filesystem support, we can formalize the test into an xfstest (sparse file + loop + filesystem, fallocate() file, check the size of the underlying sparse file). Best Sarthak - Sarthak > -Dave. > -- > Dave Chinner > david@fromorbit.com
On Tue, Oct 10, 2023 at 03:42:53PM -0700, Sarthak Kukreti wrote: > On Sun, Oct 8, 2023 at 4:50 PM Dave Chinner <david@fromorbit.com> wrote: > > > > On Fri, Oct 06, 2023 at 06:28:12PM -0700, Sarthak Kukreti wrote: > > > Hi, > > > > > > This patch series is version 8 of the patch series to introduce > > > block-level provisioning mechanism (original [1]), which is useful for provisioning > > > space across thinly provisioned storage architectures (loop devices > > > backed by sparse files, dm-thin devices, virtio-blk). This series has > > > minimal changes over v7[2]. > > > > > > This patch series is rebased from the linux-dm/dm-6.5-provision-support [1] on to > > > (cac405a3bfa2 Merge tag 'for-6.6-rc3-tag'). In addition, there's an > > > additional patch to allow passing through an unshare intent via REQ_OP_PROVISION > > > (suggested by Darrick in [4]). > > > > The XFS patches I just posted were smoke tested a while back against > > loop devices and then forward ported to this patchset. Good for > > testing that userspace driven file preallocation gets propagated by > > the filesystem down to the backing device correctly and that > > subsequent IO to the file then does the right thing (e.g. fio > > testing using fallocate() to set up the files being written to).... > > > > Thanks! I've been testing with a WIP patch for ext4, I'll give your > patches a try. Once we are closer to submitting the filesystem > support, we can formalize the test into an xfstest (sparse file + loop > + filesystem, fallocate() file, check the size of the underlying > sparse file). That's not really a valid test - there are so many optional filesystem behaviours that can change the layout of the backing file for the same upper filesystem operations. What we actually need to test is the ENOSPC guarantees, not that fallocate has been called by the loop device. i.e. that ENOSPC is propagated from the underlying filesystem though the loop device to the application running on the upper filesystem appropriately. e.g. when the lower filesystem is at ENOSPC, the writes into provisioned space in the loop device backing file continue to succeed without ENOSPC being reported to the upper filesystem. i.e. this needs to be tested from the perspective of the API presented to the upper filesystem, not by running an upper fs operation and then trying to infer correct behaviour by peering at the state of the lower filesystem... Cheers, Dave.