From patchwork Wed Nov 30 09:56:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gao Xiang X-Patchwork-Id: 27710 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp838694wrr; Wed, 30 Nov 2022 02:19:32 -0800 (PST) X-Google-Smtp-Source: AA0mqf5VJ3hlvjM7RuxIUTqUJbWc6ocgh0Xs6cEmJM8Upw21+Ts2Piszs2ObKV+uNO2/1DCbEgQR X-Received: by 2002:a17:906:3952:b0:7b9:2a28:f6ff with SMTP id g18-20020a170906395200b007b92a28f6ffmr32821269eje.61.1669803572376; Wed, 30 Nov 2022 02:19:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669803572; cv=none; d=google.com; s=arc-20160816; b=InFy+U5is7waadA7zpPwpBiOqSjO3E/lPdPl1cxPm159bJsCsL3tNhrNgPPTUXzEGF lz/6t+nuj8w3v/VWiUy1S+WtNxgdL5Gg77mNRNH1VtRwaZmig9JPkT9nI0VUr3ocZlXO 64i8+eqLcgmplLf00DuUqqNzfNW35TSik53Vy8I9f0i3CKIrgWwrwTwiuE4ZRFbCTokA mKxN8C8gmNyl9Ypv8RsbQt5d4tPJFVFEvpU7bKTh5OgqmCEpDLosiZb8lia7Q03C5IQ6 bMwLC7sQzi6dru5wCJyQQNgroFSR4Nq7YM+0pKvZ63bndRDA7LvYcbi+5f+vPumPn5tO ay7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from; bh=zg7cZb5nT7GSKDl8+pqxmWT78tqpAHGaKOGrEy9tk+M=; b=Lap4xo76THshi4dTJIEZmlicr9xS46LdLt0q6ksXmcCC7LbymvTzP8vaf9zRNiJuRj xbDzj4ip5ZycxudH2QyNmlJzpmBPICgYi0T8b+JRhMjVkM5m8ofJdD7o504XSQW7YuK1 Y+cslhNVzGkkDvduHlS0v/6/YQXQZRJlfhmJ/WaneAWLw+rRigNwf6IPwGhA7ijuljLu GG79Ft7kQGtrKtvB9NwnIQrciNLxKNVf+z//C8r5BEY6R7YjcBuAynZqDhuX++BCarka afkbtWiQQJK4XPi0iP5ylKFL9stNgvVh5g1ygR7HKuTF/cCdrEbHflnMAHYnY4GOHb+R Uufg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i6-20020a1709064fc600b0078d93325645si1083125ejw.405.2022.11.30.02.19.09; Wed, 30 Nov 2022 02:19:32 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235185AbiK3J4Y (ORCPT + 99 others); Wed, 30 Nov 2022 04:56:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56660 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235099AbiK3J4U (ORCPT ); Wed, 30 Nov 2022 04:56:20 -0500 Received: from out30-42.freemail.mail.aliyun.com (out30-42.freemail.mail.aliyun.com [115.124.30.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9C4442E684 for ; Wed, 30 Nov 2022 01:56:13 -0800 (PST) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R161e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046050;MF=hsiangkao@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0VW2qBWd_1669802166; Received: from e18g06460.et15sqa.tbsite.net(mailfrom:hsiangkao@linux.alibaba.com fp:SMTPD_---0VW2qBWd_1669802166) by smtp.aliyun-inc.com; Wed, 30 Nov 2022 17:56:11 +0800 From: Gao Xiang To: linux-erofs@lists.ozlabs.org Cc: LKML , Chao Yu , Gao Xiang Subject: [PATCH] erofs: update documentation Date: Wed, 30 Nov 2022 17:56:05 +0800 Message-Id: <20221130095605.4656-1-hsiangkao@linux.alibaba.com> X-Mailer: git-send-email 2.24.4 MIME-Version: 1.0 X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2, SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1750915950797112722?= X-GMAIL-MSGID: =?utf-8?q?1750915950797112722?= - Refine highlights for main features; - Add multi-reference pclusters and fragment description. Signed-off-by: Gao Xiang Reviewed-by: Yue Hu Reviewed-by: Chao Yu --- Documentation/filesystems/erofs.rst | 35 ++++++++++++++++++----------- 1 file changed, 22 insertions(+), 13 deletions(-) diff --git a/Documentation/filesystems/erofs.rst b/Documentation/filesystems/erofs.rst index 05e03d54af1a..82af67fdaf99 100644 --- a/Documentation/filesystems/erofs.rst +++ b/Documentation/filesystems/erofs.rst @@ -30,12 +30,17 @@ It is implemented to be a better choice for the following scenarios: especially for those embedded devices with limited memory and high-density hosts with numerous containers. -Here is the main features of EROFS: +Here are the main features of EROFS: - Little endian on-disk design; - - 4KiB block size and 32-bit block addresses, therefore 16TiB address space - at most for now; + - Block-based and file-based distribution over fscache are supported; + + - Support multiple devices to refer to external blobs, which can be used + for container images; + + - 4KiB block size and 32-bit block addresses for each device, therefore + 16TiB address space at most for now; - Two inode layouts for different requirements: @@ -50,28 +55,29 @@ Here is the main features of EROFS: Metadata reserved 8 bytes 18 bytes ===================== ============ ====================================== - - Metadata and data could be mixed as an option; - - - Support extended attributes (xattrs) as an option; + - Support extended attributes as an option; - - Support tailpacking data and xattr inline compared to byte-addressed - unaligned metadata or smaller block size alternatives; - - - Support POSIX.1e ACLs by using xattrs; + - Support POSIX.1e ACLs by using extended attributes; - Support transparent data compression as an option: LZ4 and MicroLZMA algorithms can be used on a per-file basis; In addition, inplace decompression is also supported to avoid bounce compressed buffers and page cache thrashing. + - Support chunk-based data deduplication and rolling-hash compressed data + deduplication; + + - Support tailpacking inline compared to byte-addressed unaligned metadata + or smaller block size alternatives; + + - Support merging tail-end data into a special inode as fragments. + - Support direct I/O on uncompressed files to avoid double caching for loop devices; - Support FSDAX on uncompressed images for secure containers and ramdisks in order to get rid of unnecessary page cache. - - Support multiple devices for multi blob container images; - - Support file-based on-demand loading with the Fscache infrastructure. The following git tree provides the file system user-space tools under @@ -259,7 +265,7 @@ By the way, chunk-based files are all uncompressed for now. Data compression ---------------- -EROFS implements LZ4 fixed-sized output compression which generates fixed-sized +EROFS implements fixed-sized output compression which generates fixed-sized compressed data blocks from variable-sized input in contrast to other existing fixed-sized input solutions. Relatively higher compression ratios can be gotten by using fixed-sized output compression since nowadays popular data compression @@ -314,3 +320,6 @@ to understand its delta0 is constantly 1, as illustrated below:: If another HEAD follows a HEAD lcluster, there is no room to record CBLKCNT, but it's easy to know the size of such pcluster is 1 lcluster as well. + +Since Linux v6.1, each pcluster can be used for multiple variable-sized extents, +therefore it can be used for compressed data deduplication.