[v9,0/3,v9,0/3] Introduce provisioning primitives

  Hi,

This patch series is version 9 of the patch series to introduce
block-level provisioning mechanism (original [1]), which is useful for
provisioning space across thinly provisioned storage architectures (loop
devices backed by sparse files, dm-thin devices, virtio-blk). This
series has minimal changes over v8[2], with a couple of patches dropped
(suggested by Dave).

This patch series is rebased from the linux-dm/dm-6.5-provision-support
[3] on to (a12deb44f973 Merge tag 'input-for-v6.7-rc0' ...). The final 
patch in the series is a blktest (suggested by Dave in 4) which was used
to test out the provisioning flow for loop devices on sparse files on an
ext4 filesystem.

[1] Original: https://lore.kernel.org/lkml/20220915164826.1396245-1-sarthakkukreti@google.com/
[2] v8 (last series): https://lore.kernel.org/lkml/20231007012817.3052558-1-sarthakkukreti@chromium.org/
[3] linux-dm/dm-6.5-provision-suppport tree:
https://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git/log/?h=dm-6.5-provision-support
(with the last two WIP patches for dm-thinpool dropped as per discussion with
maintainers).
[4] https://lore.kernel.org/lkml/ZSXono3GkXhgrZ1T@dread.disaster.area/

Changes from v8:
- Dropped fallocate() bugfix: merged separately.
- Dropped patch for unshare intent (suggested by Dave).
- More comments in loop device patch to point out the implementation
  constraint and how it affects filesystems with copy-on-write
  semantics.
- Added blktest "loop/010" (rebased on top of ) for testing.

Sarthak Kukreti (4):
  block: Introduce provisioning primitives
  dm: Add block provisioning support
  loop: Add support for provision requests
  loop/010: Add test for mode 0 fallocate on loop devices

 block/blk-core.c              |  5 ++++
 block/blk-lib.c               | 51 +++++++++++++++++++++++++++++++++++
 block/blk-merge.c             | 18 +++++++++++++
 block/blk-settings.c          | 19 +++++++++++++
 block/blk-sysfs.c             |  9 +++++++
 block/bounce.c                |  1 +
 block/fops.c                  |  5 ++++
 drivers/block/loop.c          | 39 ++++++++++++++++++++++++---
 drivers/md/dm-crypt.c         |  4 ++-
 drivers/md/dm-linear.c        |  1 +
 drivers/md/dm-table.c         | 23 ++++++++++++++++
 drivers/md/dm.c               |  7 +++++
 include/linux/bio.h           |  6 +++--
 include/linux/blk_types.h     |  5 +++-
 include/linux/blkdev.h        | 16 +++++++++++
 include/linux/device-mapper.h | 17 ++++++++++++
 16 files changed, 219 insertions(+), 7 deletions(-)
  

On Thu, Nov 09, 2023 at 05:01:35PM -0800, Sarthak Kukreti wrote:
> Hi,
> 
> This patch series is version 9 of the patch series to introduce
> block-level provisioning mechanism (original [1]), which is useful for
> provisioning space across thinly provisioned storage architectures (loop
> devices backed by sparse files, dm-thin devices, virtio-blk). This
> series has minimal changes over v8[2], with a couple of patches dropped
> (suggested by Dave).
> 
> This patch series is rebased from the linux-dm/dm-6.5-provision-support
> [3] on to (a12deb44f973 Merge tag 'input-for-v6.7-rc0' ...). The final 
> patch in the series is a blktest (suggested by Dave in 4) which was used
> to test out the provisioning flow for loop devices on sparse files on an
> ext4 filesystem.

What happened to the XFS patch I sent to support provisioning for
fallocate() operations through XFS?

Cheers,

Dave.
  

On Fri, Nov 10, 2023 at 4:56 PM Dave Chinner <david@fromorbit.com> wrote:
>
> On Thu, Nov 09, 2023 at 05:01:35PM -0800, Sarthak Kukreti wrote:
> > Hi,
> >
> > This patch series is version 9 of the patch series to introduce
> > block-level provisioning mechanism (original [1]), which is useful for
> > provisioning space across thinly provisioned storage architectures (loop
> > devices backed by sparse files, dm-thin devices, virtio-blk). This
> > series has minimal changes over v8[2], with a couple of patches dropped
> > (suggested by Dave).
> >
> > This patch series is rebased from the linux-dm/dm-6.5-provision-support
> > [3] on to (a12deb44f973 Merge tag 'input-for-v6.7-rc0' ...). The final
> > patch in the series is a blktest (suggested by Dave in 4) which was used
> > to test out the provisioning flow for loop devices on sparse files on an
> > ext4 filesystem.
>
> What happened to the XFS patch I sent to support provisioning for
> fallocate() operations through XFS?
>
Apologies, I missed out on mentioning that the XFS patches work well
with loop devices.

I might have misunderstood: were those patches only for sanity testing
or would you prefer that I send those out as a part of this series? I
can whip up a quick v10 if so!

Cheers

Sarthak


> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
  

On Mon, Nov 13, 2023 at 01:26:51PM -0800, Sarthak Kukreti wrote:
> On Fri, Nov 10, 2023 at 4:56 PM Dave Chinner <david@fromorbit.com> wrote:
> >
> > On Thu, Nov 09, 2023 at 05:01:35PM -0800, Sarthak Kukreti wrote:
> > > Hi,
> > >
> > > This patch series is version 9 of the patch series to introduce
> > > block-level provisioning mechanism (original [1]), which is useful for
> > > provisioning space across thinly provisioned storage architectures (loop
> > > devices backed by sparse files, dm-thin devices, virtio-blk). This
> > > series has minimal changes over v8[2], with a couple of patches dropped
> > > (suggested by Dave).
> > >
> > > This patch series is rebased from the linux-dm/dm-6.5-provision-support
> > > [3] on to (a12deb44f973 Merge tag 'input-for-v6.7-rc0' ...). The final
> > > patch in the series is a blktest (suggested by Dave in 4) which was used
> > > to test out the provisioning flow for loop devices on sparse files on an
> > > ext4 filesystem.
> >
> > What happened to the XFS patch I sent to support provisioning for
> > fallocate() operations through XFS?
> >
> Apologies, I missed out on mentioning that the XFS patches work well
> with loop devices.
> 
> I might have misunderstood: were those patches only for sanity testing
> or would you prefer that I send those out as a part of this series? I
> can whip up a quick v10 if so!

I was implying that if you are going to be adding support to random
block devices for people to actually test out, then you should be
adding support to filesystems and writing new -fstests- to ensure
that loop devices are actually provisioning blocks at exactly the
locations that correspond to the physical file extents the
filesystem provisioned, too.

-Dave.
  

On Mon, Nov 20, 2023 at 12:33 PM Dave Chinner <david@fromorbit.com> wrote:
>
> On Mon, Nov 13, 2023 at 01:26:51PM -0800, Sarthak Kukreti wrote:
> > On Fri, Nov 10, 2023 at 4:56 PM Dave Chinner <david@fromorbit.com> wrote:
> > >
> > > On Thu, Nov 09, 2023 at 05:01:35PM -0800, Sarthak Kukreti wrote:
> > > > Hi,
> > > >
> > > > This patch series is version 9 of the patch series to introduce
> > > > block-level provisioning mechanism (original [1]), which is useful for
> > > > provisioning space across thinly provisioned storage architectures (loop
> > > > devices backed by sparse files, dm-thin devices, virtio-blk). This
> > > > series has minimal changes over v8[2], with a couple of patches dropped
> > > > (suggested by Dave).
> > > >
> > > > This patch series is rebased from the linux-dm/dm-6.5-provision-support
> > > > [3] on to (a12deb44f973 Merge tag 'input-for-v6.7-rc0' ...). The final
> > > > patch in the series is a blktest (suggested by Dave in 4) which was used
> > > > to test out the provisioning flow for loop devices on sparse files on an
> > > > ext4 filesystem.
> > >
> > > What happened to the XFS patch I sent to support provisioning for
> > > fallocate() operations through XFS?
> > >
> > Apologies, I missed out on mentioning that the XFS patches work well
> > with loop devices.
> >
> > I might have misunderstood: were those patches only for sanity testing
> > or would you prefer that I send those out as a part of this series? I
> > can whip up a quick v10 if so!
>
> I was implying that if you are going to be adding support to random
> block devices for people to actually test out, then you should be
> adding support to filesystems and writing new -fstests- to ensure
> that loop devices are actually provisioning blocks at exactly the
> locations that correspond to the physical file extents the
> filesystem provisioned, too.
>

Fair enough, let me work on an additional fstests patch to validate that.

Best
Sarthak

> -Dave.
> --
> Dave Chinner
> david@fromorbit.com