erofs: update documentation
Commit Message
- update new features like bloom filter and DEFLATE.
- add documentation for the long xattr name prefixes, which was
landed upstream since v6.4.
Signed-off-by: Jingbo Xu <jefflexu@linux.alibaba.com>
---
Documentation/filesystems/erofs.rst | 40 ++++++++++++++++++++++++++---
1 file changed, 37 insertions(+), 3 deletions(-)
Comments
On Thu, Sep 28, 2023 at 09:16:00PM +0800, Jingbo Xu wrote:
> - update new features like bloom filter and DEFLATE.
>
> - add documentation for the long xattr name prefixes, which was
> landed upstream since v6.4.
>
> Signed-off-by: Jingbo Xu <jefflexu@linux.alibaba.com>
> ---
> Documentation/filesystems/erofs.rst | 40 ++++++++++++++++++++++++++---
> 1 file changed, 37 insertions(+), 3 deletions(-)
>
> diff --git a/Documentation/filesystems/erofs.rst b/Documentation/filesystems/erofs.rst
> index 4654ee57c1d5..522183737be6 100644
> --- a/Documentation/filesystems/erofs.rst
> +++ b/Documentation/filesystems/erofs.rst
> @@ -58,12 +58,14 @@ Here are the main features of EROFS:
>
> - Support extended attributes as an option;
>
> + - Support a bloom filter that speeds up negative extended attribute lookups;
> +
> - Support POSIX.1e ACLs by using extended attributes;
>
> - Support transparent data compression as an option:
> - LZ4 and MicroLZMA algorithms can be used on a per-file basis; In addition,
> - inplace decompression is also supported to avoid bounce compressed buffers
> - and page cache thrashing.
> + LZ4, MicroLZMA and DEFLATE algorithms can be used on a per-file basis; In
> + addition, inplace decompression is also supported to avoid bounce compressed
> + buffers and unnecessary page cache thrashing.
>
> - Support chunk-based data deduplication and rolling-hash compressed data
> deduplication;
> @@ -268,6 +270,38 @@ details.)
>
> By the way, chunk-based files are all uncompressed for now.
>
> +Long extended attribute name prefixes
> +-------------------------------------
> +There are use cases that extended attributes with different values can have
^ where
> +only a few common prefixes (such as overlayfs xattrs). The predefined prefixes
> +works inefficiently in both image size and runtime performance in such cases.
^ work
> +
> +The long xattr name prefixes feature is introduced to address this issue. The
> +overall idea is that, apart from the existing predefined prefixes, the xattr
> +entry could also refer to user specified long xattr name prefixes, e.g.
^ user-specified
> +"trusted.overlay.".
> +
> +When referring to a long xattr name prefix, the highest bit (bit 7) of
> +erofs_xattr_entry.e_name_index is set, while the lower bits (bit 0-6) as a whole
> +represents the index of the referred long name prefix among all long name
> +prefixes. Therefore, only the trailing part of the name apart from the long
> +xattr name prefix is stored in erofs_xattr_entry.e_name, which could be empty if
> +the full xattr name matches exactly as its long xattr name prefix.
> +
> +All long xattr prefixes are stored one by one in the packed inode as long as
> +the packed inode is valid, or meta inode otherwise. The xattr_prefix_count (of
^ the meta inode
> +on-disk superblock) indicates the total number of the long xattr name prefixes,
^ the on-disk superblock
> +while (xattr_prefix_start * 4) indicates the start offset of long name prefixes
> +in the packed/meta inode. Note that, long extended attribute name prefixes is
^ are
> +disabled if xattr_prefix_count is 0.
> +
> +Each long name prefix is stored in the format: ALIGN({__le16 len, data}, 4),
> +where len represents the total size of the data part. The data part is actually
> +represented by 'struct erofs_xattr_long_prefix', where base_index represents the
> +index of the predefined xattr name prefix, e.g. EROFS_XATTR_INDEX_TRUSTED for
> +"trusted.overlay." long name prefix, while the infix string kepts the string
^ keeps
Thanks,
Gao Xiang
@@ -58,12 +58,14 @@ Here are the main features of EROFS:
- Support extended attributes as an option;
+ - Support a bloom filter that speeds up negative extended attribute lookups;
+
- Support POSIX.1e ACLs by using extended attributes;
- Support transparent data compression as an option:
- LZ4 and MicroLZMA algorithms can be used on a per-file basis; In addition,
- inplace decompression is also supported to avoid bounce compressed buffers
- and page cache thrashing.
+ LZ4, MicroLZMA and DEFLATE algorithms can be used on a per-file basis; In
+ addition, inplace decompression is also supported to avoid bounce compressed
+ buffers and unnecessary page cache thrashing.
- Support chunk-based data deduplication and rolling-hash compressed data
deduplication;
@@ -268,6 +270,38 @@ details.)
By the way, chunk-based files are all uncompressed for now.
+Long extended attribute name prefixes
+-------------------------------------
+There are use cases that extended attributes with different values can have
+only a few common prefixes (such as overlayfs xattrs). The predefined prefixes
+works inefficiently in both image size and runtime performance in such cases.
+
+The long xattr name prefixes feature is introduced to address this issue. The
+overall idea is that, apart from the existing predefined prefixes, the xattr
+entry could also refer to user specified long xattr name prefixes, e.g.
+"trusted.overlay.".
+
+When referring to a long xattr name prefix, the highest bit (bit 7) of
+erofs_xattr_entry.e_name_index is set, while the lower bits (bit 0-6) as a whole
+represents the index of the referred long name prefix among all long name
+prefixes. Therefore, only the trailing part of the name apart from the long
+xattr name prefix is stored in erofs_xattr_entry.e_name, which could be empty if
+the full xattr name matches exactly as its long xattr name prefix.
+
+All long xattr prefixes are stored one by one in the packed inode as long as
+the packed inode is valid, or meta inode otherwise. The xattr_prefix_count (of
+on-disk superblock) indicates the total number of the long xattr name prefixes,
+while (xattr_prefix_start * 4) indicates the start offset of long name prefixes
+in the packed/meta inode. Note that, long extended attribute name prefixes is
+disabled if xattr_prefix_count is 0.
+
+Each long name prefix is stored in the format: ALIGN({__le16 len, data}, 4),
+where len represents the total size of the data part. The data part is actually
+represented by 'struct erofs_xattr_long_prefix', where base_index represents the
+index of the predefined xattr name prefix, e.g. EROFS_XATTR_INDEX_TRUSTED for
+"trusted.overlay." long name prefix, while the infix string kepts the string
+after stripping the short prefix, e.g. "overlay." for the example above.
+
Data compression
----------------
EROFS implements fixed-sized output compression which generates fixed-sized