From patchwork Thu Oct 27 08:35:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jingbo Xu X-Patchwork-Id: 767 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp105801wru; Thu, 27 Oct 2022 01:36:21 -0700 (PDT) X-Google-Smtp-Source: AMsMyM502S+zjINwyQQ26JWbtdcT3VPCC/NVtlbCXt+ttjMlahIhsrVjolTRwpSHexdFAjGYVOv0 X-Received: by 2002:a17:902:e952:b0:17c:7aaa:c679 with SMTP id b18-20020a170902e95200b0017c7aaac679mr48379795pll.74.1666859781353; Thu, 27 Oct 2022 01:36:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666859781; cv=none; d=google.com; s=arc-20160816; b=ru9XLgaWpOiGjyUi47NcBJVXL2daOxlQrYBLc8sgTzQpQsC3OeXT7k+1zlzs/a0apD /PToa/hC1wzDCkJ8+P8Tb7ioBRLSu6I9wbtt9r/lHAiuhdgbPg7ZqY1i0KaG0gs843Rz XAXyd5W+UQ//COzNATTWsQ5pKTGug0RT/0NL5lcm4faZy8mf7HWgF7yGu0PrJVM+t1Ow mqq1KRtJgE/0yJfVYNiGSeHMAP8pYcOO5moBfnn8qi0iep+8Mlvo23hGC2Zft4XZhzus d9BH54jZUYJstvDNjW4jYIY3Rcw6tTLc7GcScKEyU7845Bjf0W/DZ18aqEioT88oGwIp HXmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from; bh=fHmAUyqvyUbbn1GD5U216fkN4k2nNYzOuMI0JrnhnfM=; b=Jcl8fGMhvgmmDJUC+wUEeqDcU2J+piZcA001hhlYwMvlBpc44URIAW5gsDnr4oSzTO RSfVYRS3YpYBwVw+fYMuqkLtAK1QCCv7T8T4nogUyGo2G/5eUZ5JcqZM0BbQPyIiQowK YUWtBcPulswMasDIfhBgRS7Y18/XZN4cfp5tuf18f/TuBhBsihrOeIo2IUG9uud6LQfV N+6UrLwFlTLBBIyT2cWW1dRIi8KKzqDTdUqW3wvygDfrEOMPZ9EHI6YPnPaqdscbdkHf wJVtXAQFIAh+I+bR2SZlqOdzBcKZd9VZimth0p4atWiagzp2pDkzROeKPLiJyqKfnZ6y a+4w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 13-20020a170902c24d00b0017514e57d7esi750184plg.576.2022.10.27.01.36.08; Thu, 27 Oct 2022 01:36:21 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234886AbiJ0If7 (ORCPT + 99 others); Thu, 27 Oct 2022 04:35:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51156 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234486AbiJ0Ifz (ORCPT ); Thu, 27 Oct 2022 04:35:55 -0400 Received: from out30-57.freemail.mail.aliyun.com (out30-57.freemail.mail.aliyun.com [115.124.30.57]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DDA7689AEB; Thu, 27 Oct 2022 01:35:51 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R121e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=jefflexu@linux.alibaba.com;NM=1;PH=DS;RN=9;SR=0;TI=SMTPD_---0VTAl7YM_1666859747; Received: from localhost(mailfrom:jefflexu@linux.alibaba.com fp:SMTPD_---0VTAl7YM_1666859747) by smtp.aliyun-inc.com; Thu, 27 Oct 2022 16:35:48 +0800 From: Jingbo Xu To: dhowells@redhat.com, jlayton@kernel.org, linux-cachefs@redhat.com, linux-erofs@lists.ozlabs.org Cc: linux-cifs@vger.kernel.org, linux-nfs@vger.kernel.org, linux-afs@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 0/9] fscache,netfs: decouple raw fscache APIs from libnetfs Date: Thu, 27 Oct 2022 16:35:38 +0800 Message-Id: <20221027083547.46933-1-jefflexu@linux.alibaba.com> X-Mailer: git-send-email 2.19.1.6.gb485710b MIME-Version: 1.0 X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1747829161918475525?= X-GMAIL-MSGID: =?utf-8?q?1747829161918475525?= Git tree: https://github.com/lostjeffle/linux.git jingbo/clean-fscache-v1-clean-netfs Gitweb: https://github.com/lostjeffle/linux/commits/jingbo/clean-fscache-v1-clean-netfs [Rationale] =========== Fscache has been landed as a generic caching management framework in the Linux kernel for decades. It aims to manage cache data availability or fetch data if needed. Currently it's mainly used for network fses, but in principle the main caching subsystem can be used more widely. We do really like fscache framework and we believe it'd be better to reuse such framework if possible instead of duplicating other alternatives for better maintenance and testing. Therefore for our container image use cases, we applied the existing fscache to implement on-demand read for erofs in the past months. For more details, also see [1]. In short, here each erofs filesystem is composed of multiple blobs (or devices). Each blob corresponds to one fscache cookie to strictly follow on-disk format and implement the image downloading in a deterministic manner, which means it has a unique checksum and is signed by vendors. Data of each erofs inode can be scattered among multiple blobs (cookie) since erofs supports chunk-level deduplication. In this case, each erofs inode can correspond to multiple cookies, and there's a logical to physical offset mapping between the logical offset in erofs inode and the physical offset in the backing file. As described above, per-cookie netfs model can not be used here directly. Instead, we'd like to propose/decouple a simple set of raw fscache APIs, e.g. fscache_read(), to access cache for all fses to use. We believe it's useful since it's like the relationship between raw bio and iomap, both of which are useful for local fses. However, after fscache/netfs rework, libnetfs is preferred to access fscache, making fscache closely coupled with libnetfs. Therefore, a more simple neutral raw fscache APIs is shown here which is independent to libnetfs for those who are not using libnetfs. [Patchset Organization] ======================= patch 1: decouple prepare_read() from netfs_io_subrequest patch 2-9: All structures related to cache accessing will be transformed with "fscache_" prefix, and defined in fscache namespace. The whole transition is divided into separate patches to facilitate code review, with each patch transforming one structure. It is worth noting that the structure declaration is temporarily placed in netfs.h, and will be moved to fscache.h when all related structures are transformed to "fscache_" prefixed finally. The reason is that, in the intermediate state during the transition, the declarations of related structures are scattered among fscache.h and netfs.h. This will cause a bidirectional reference of these two headers, and compilation error then. As a work around, keep the declaration in netfs.h temporarily, until all structures are transformed and then moved to fscache.h (in patch 9). [Following cleanup for erofs] ============================= We will also cleanup erofs based on fscache_read() so that it won't rely on netfs internals anymore. For example, erofs can implement its own request completion routine, so that it can get rid of reliance on netfs_io_request and netfs_io_subrequest. [1] https://lore.kernel.org/all/Yoj1AcHoBPqir++H@debian/ Jingbo Xu (9): fscache: decouple prepare_read() from netfs_io_subrequest fscache,netfs: rename netfs_io_source as fscache_io_source fscache,netfs: rename netfs_io_terminated_t as fscache_io_terminated_t fscache,netfs: rename netfs_read_from_hole as fscache_read_from_hole fscache,netfs: rename netfs_cache_ops as fscache_ops fscache,netfs: rename netfs_cache_resources as fscache_resources fscache,netfs: define flags for prepare_read() fscache,netfs: move PageFsCache() family helpers to fscache.h fscache,netfs: move "fscache_" prefixed structures to fscache.h fs/afs/internal.h | 2 +- fs/cachefiles/interface.c | 2 +- fs/cachefiles/internal.h | 8 +- fs/cachefiles/io.c | 84 ++++++------ fs/cifs/fscache.c | 8 +- fs/erofs/fscache.c | 17 ++- fs/fscache/io.c | 18 +-- fs/netfs/buffered_read.c | 2 +- fs/netfs/io.c | 67 +++++----- fs/nfs/fscache.c | 6 +- fs/nfs/fscache.h | 2 +- include/linux/fscache-cache.h | 8 +- include/linux/fscache.h | 211 +++++++++++++++++++++++++++--- include/linux/netfs.h | 173 +----------------------- include/trace/events/cachefiles.h | 27 ++-- include/trace/events/netfs.h | 14 +- 16 files changed, 330 insertions(+), 319 deletions(-)