From patchwork Fri Nov 4 07:26:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jingbo Xu X-Patchwork-Id: 1153 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp228187wru; Fri, 4 Nov 2022 00:30:38 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5JdaYbgcEsbTbreEy/drT+tuCr9YU74R9bctB9MjtUz/6NYHoJ3p50zo8Fo6fICcp77biV X-Received: by 2002:a17:902:c94b:b0:187:3da6:37ab with SMTP id i11-20020a170902c94b00b001873da637abmr15880255pla.50.1667547037902; Fri, 04 Nov 2022 00:30:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667547037; cv=none; d=google.com; s=arc-20160816; b=i0MG8mw7XcaatFmVioIvuyMgqYjjrXXDCotdBPoKkigAWe9kHx9IpZ+HNn2LLxDKOW VcnG9dqt4zFUSotESyUa4XdL/svtxu5AQKr2U+Yic7fnWF9tHdbGG0o6KNvJoachDKTU ur6GQfhAWKrqvFI0kNKlCfB2z2GxjRY+THjNBv/pMmnPXgliDAsj/Mhqt7vy9w+0c1gt O86mhZ4Vgqbiep/Mw7KDjgpAf8yOrvgPADmP1+SEIBJDf9qxnqxHPBfBDESYGK/ZprSX I4txuEIseDT5HwqHca7Ho+7tUT6k1tZQ24q1SneKsF1yQ06KXcollqCf2eQ/WHbvp85q 3uLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from; bh=8NKvZKy6wQ74zo3zDbaRR8/FR7zM1f7nDKYCLgNLsqU=; b=jvtNMbmWg7uamDUBGxn/3lqMCag+pT8lD2LUlCfyUZidSy6PWaFmafVt0S4jwAT58T DoxISLQUruGOBL95pw20A114HiJBkg+vJuHYyLWNj4eYgK4CGGkGV5iJGESpPEW98Iy7 JgMLg/d7PcNkQ8naYl7HWjnPRvCxW+2LtLnIbzCft+cgKci7d/xriHTfyy968w378HAX T1OpQ2b9quGOl+YXbKxVby4JiNBme4bfZ1z7RH3JLF8qZ76UERp9yy5mOxJeCoS41U9d TBCrYNMxepaUd6DKsKAIksGX7c4CbG+uXSnNB4SpqQ2LIxkJ+aNy4D0rHoJsOAywA1ne xMQA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p2-20020a170902e74200b00170f6f728d4si4020606plf.543.2022.11.04.00.30.24; Fri, 04 Nov 2022 00:30:37 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230389AbiKDH0o (ORCPT + 99 others); Fri, 4 Nov 2022 03:26:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53370 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229974AbiKDH0m (ORCPT ); Fri, 4 Nov 2022 03:26:42 -0400 Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com [115.124.30.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 358896364; Fri, 4 Nov 2022 00:26:41 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R181e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=jefflexu@linux.alibaba.com;NM=1;PH=DS;RN=8;SR=0;TI=SMTPD_---0VTwBRBs_1667546797; Received: from localhost(mailfrom:jefflexu@linux.alibaba.com fp:SMTPD_---0VTwBRBs_1667546797) by smtp.aliyun-inc.com; Fri, 04 Nov 2022 15:26:38 +0800 From: Jingbo Xu To: dhowells@redhat.com, jlayton@kernel.org, xiang@kernel.org, chao@kernel.org, linux-cachefs@redhat.com, linux-erofs@lists.ozlabs.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: [PATCH 0/2] fscache,cachefiles: add prepare_ondemand_read() interface Date: Fri, 4 Nov 2022 15:26:35 +0800 Message-Id: <20221104072637.72375-1-jefflexu@linux.alibaba.com> X-Mailer: git-send-email 2.19.1.6.gb485710b MIME-Version: 1.0 X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2, SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748549803062996630?= X-GMAIL-MSGID: =?utf-8?q?1748549803062996630?= [Rationale] =========== Fscache has been landed as a generic caching management framework in the Linux kernel for decades. It aims to manage cache data availability or fetch data if needed. Currently it's mainly used for network fses, but in principle the main caching subsystem can be used more widely. We do really like fscache framework and we believe it'd be better to reuse such framework if possible instead of duplicating other alternatives for better maintenance and testing. Therefore for our container image use cases, we applied the existing fscache to implement on-demand read for erofs in the past months. For more details, also see [1]. In short, here each erofs filesystem is composed of multiple blobs (or devices). Each blob corresponds to one fscache cookie to strictly follow on-disk format and implement the image downloading in a deterministic manner, which means it has a unique checksum and is signed by vendors. Data of each erofs inode can be scattered among multiple blobs (cookie) since erofs supports chunk-level deduplication. In this case, each erofs inode can correspond to multiple cookies, and there's a logical to physical offset mapping between the logical offset in erofs inode and the physical offset in the backing file. As described above, per-cookie netfs model can not be used here directly. Instead, we'd like to propose/decouple a simple set of raw fscache APIs, to access cache for all fses to use. We believe it's useful since it's like the relationship between raw bio and iomap, both of which are useful for local fses. fscache_read() seems a reasonable candidate and is enough for such use case. In addition, the on-demand read feature relies on .prepare_read() to reuse the hole detecting logic as much as possible. However, after fscache/netfs rework, libnetfs is preferred to access fscache, making .prepare_read() closely coupled with libnetfs, or more precisely, netfs_io_subrequest. [What We Do] ============ As we discussed previously, we propose a new interface, i,e, .prepare_ondemand_read() dedicated for the on-demand read scenarios, which is independent on netfs_io_subrequest. The netfs will still use the original .prepare_read() as usual. And as we discussed, in the near future, prepare_read() will get enhanced and more information will be needed and then returned to callers. Thus netfs_io_subrequest is retained as the aggregation for all parameters needed as the internal implementation inside Cachefiles. Jingbo Xu (2): fscache,cachefiles: add prepare_ondemand_read() callback erofs: switch to prepare_ondemand_read() in fscache mode fs/cachefiles/io.c | 42 ++++- fs/erofs/fscache.c | 257 +++++++++++------------------- include/linux/netfs.h | 7 + include/trace/events/cachefiles.h | 4 +- 4 files changed, 135 insertions(+), 175 deletions(-)