mbox series

[0/5] erofs: basic sub-page compressed data support

Message ID 20231206091057.87027-1-hsiangkao@linux.alibaba.com
Headers
Series erofs: basic sub-page compressed data support |

Message

Gao Xiang Dec. 6, 2023, 9:10 a.m. UTC 
  Hi folks,

Recently, there are two new cases so that we need to add a preliminary
sub-page block support for compressed files;

 - As Android folks requested, Android ecosystem itself is now switching
   to 16k page size for their arm64 devices.  They needs an option of
   4k-block image compatibility on their new 16k devices;

 - Some arm64 cloud servers use 64k page size for their optimized
   workloads, but 4k-block EROFS container images need to be parsed too.

So this patchset mainly addresses the requirements above with a very
very simple approach as a start: just allocate short-lived temporary
buffers all the time to keep compressed data if sub-page blocks are
identified.  In other words, no inplace/cache decompression for
the preliminary support.

This patchset survives EROFS stress test on my own testfarms.

Thanks,
Gao Xiang

Gao Xiang (5):
  erofs: support I/O submission for sub-page compressed blocks
  erofs: record `pclustersize` in bytes instead of pages
  erofs: fix up compacted indexes for block size < 4096
  erofs: refine z_erofs_transform_plain() for sub-page block support
  erofs: enable sub-page compressed block support

 fs/erofs/decompressor.c |  81 +++++++++------
 fs/erofs/inode.c        |   6 +-
 fs/erofs/zdata.c        | 224 ++++++++++++++++++----------------------
 fs/erofs/zmap.c         |  32 +++---
 4 files changed, 169 insertions(+), 174 deletions(-)

Comments

Yue Hu Dec. 11, 2023, 6:50 a.m. UTC | #1 
On Wed,  6 Dec 2023 17:10:52 +0800
Gao Xiang <hsiangkao@linux.alibaba.com> wrote:

> Hi folks,
> 
> Recently, there are two new cases so that we need to add a preliminary
> sub-page block support for compressed files;
> 
>  - As Android folks requested, Android ecosystem itself is now switching
>    to 16k page size for their arm64 devices.  They needs an option of
>    4k-block image compatibility on their new 16k devices;
> 
>  - Some arm64 cloud servers use 64k page size for their optimized
>    workloads, but 4k-block EROFS container images need to be parsed too.
> 
> So this patchset mainly addresses the requirements above with a very
> very simple approach as a start: just allocate short-lived temporary
> buffers all the time to keep compressed data if sub-page blocks are
> identified.  In other words, no inplace/cache decompression for
> the preliminary support.
> 
> This patchset survives EROFS stress test on my own testfarms.
> 
> Thanks,
> Gao Xiang
> 
> Gao Xiang (5):
>   erofs: support I/O submission for sub-page compressed blocks
>   erofs: record `pclustersize` in bytes instead of pages
>   erofs: fix up compacted indexes for block size < 4096
>   erofs: refine z_erofs_transform_plain() for sub-page block support
>   erofs: enable sub-page compressed block support
> 
>  fs/erofs/decompressor.c |  81 +++++++++------
>  fs/erofs/inode.c        |   6 +-
>  fs/erofs/zdata.c        | 224 ++++++++++++++++++----------------------
>  fs/erofs/zmap.c         |  32 +++---
>  4 files changed, 169 insertions(+), 174 deletions(-)
> 

Reviewed-by: Yue Hu <huyue2@coolpad.com>
Chao Yu Dec. 14, 2023, 2:58 p.m. UTC | #2 
On 2023/12/6 17:10, Gao Xiang wrote:
> Hi folks,
> 
> Recently, there are two new cases so that we need to add a preliminary
> sub-page block support for compressed files;
> 
>   - As Android folks requested, Android ecosystem itself is now switching
>     to 16k page size for their arm64 devices.  They needs an option of
>     4k-block image compatibility on their new 16k devices;
> 
>   - Some arm64 cloud servers use 64k page size for their optimized
>     workloads, but 4k-block EROFS container images need to be parsed too.
> 
> So this patchset mainly addresses the requirements above with a very
> very simple approach as a start: just allocate short-lived temporary
> buffers all the time to keep compressed data if sub-page blocks are
> identified.  In other words, no inplace/cache decompression for
> the preliminary support.
> 
> This patchset survives EROFS stress test on my own testfarms.

Looks good to me, feel free to add:

Reviewed-by: Chao Yu <chao@kernel.org>

Thanks,

> 
> Thanks,
> Gao Xiang
> 
> Gao Xiang (5):
>    erofs: support I/O submission for sub-page compressed blocks
>    erofs: record `pclustersize` in bytes instead of pages
>    erofs: fix up compacted indexes for block size < 4096
>    erofs: refine z_erofs_transform_plain() for sub-page block support
>    erofs: enable sub-page compressed block support
> 
>   fs/erofs/decompressor.c |  81 +++++++++------
>   fs/erofs/inode.c        |   6 +-
>   fs/erofs/zdata.c        | 224 ++++++++++++++++++----------------------
>   fs/erofs/zmap.c         |  32 +++---
>   4 files changed, 169 insertions(+), 174 deletions(-)
>