From patchwork Mon Feb 13 15:32:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 56337 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2411172wrn; Mon, 13 Feb 2023 07:35:51 -0800 (PST) X-Google-Smtp-Source: AK7set+17/omeun8hTtvD+yYUg5TDlaRU7C8xSFIjWtThhGEcMqcWBEvel/dY7w0FPR43AvEYSsl X-Received: by 2002:a17:90b:380f:b0:233:c758:9a1c with SMTP id mq15-20020a17090b380f00b00233c7589a1cmr8666777pjb.12.1676302550872; Mon, 13 Feb 2023 07:35:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676302550; cv=none; d=google.com; s=arc-20160816; b=pszCfW6Ay2FtYz3MCWx23Su+VpMcKm0Th7QijGaFuWj5TsCbvy7zge18zOWpAPml8L PhfrZf1lYBIRd3qD/kg4U2P7ZIZ0fmlEEIlpWJQg2N9VUjs6g5CZMTsS6+sXZK/s9kmi 9+0dPKB2gHdD5rprTg/mNs3/Qw9L2xLF6FdmGI70f+C7PyWN37D24gQgV+vbLdw2hxhc E6VNvipieVGpFsccZAg3NBk1tSEOS0Sbz90Hi5OYWm44FZq3ZA5xSZNtvW4aNYFOpZRi CXtT6BJpsYWnswDZGy/yjlamaLb+gi0o5FtSITYPQFlSCk1/Gvpurc96TnRbPCUr+lIQ Rzgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=jiirWQEzjOJm38ClQocxHDg9OYUUCbqX/zh69yaIxOk=; b=OdAXEMyTEQ2iLcNCE94WHOpymkpZhJHoV/rmGzaQvaWwK4SsJ7bRHd2zY0bObrjMuh Owq4sKKxoBsiZTxvNjEEKNooHrIKmglnOVINIbVnQ1yxb+/PYUKAen2OIULtVA6LykwP EZafJxj0Xi6mFGNDEY6UHtP56gZjEjCbEnq4Y8C6a7M77a8QMg/rVi7/E26IOmgL3BRp kWEKJFWDmUYddqk8B76cluWidARmtVDjoj7sOE0SW5idZVBgA9MfejKUYlyRdd8bMsYM 1lXgJQgX1d1nDQpro8qTAfn7TYhVuyhradtbZdd7SQWsdEYj5IJTVEoR+2ygYx6uV4kC Votw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=N13fLyZo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w11-20020a17090a4f4b00b00233cb4c4d23si6687239pjl.119.2023.02.13.07.35.37; Mon, 13 Feb 2023 07:35:50 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=N13fLyZo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230074AbjBMPeD (ORCPT + 99 others); Mon, 13 Feb 2023 10:34:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47312 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229597AbjBMPd7 (ORCPT ); Mon, 13 Feb 2023 10:33:59 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 00AD29765 for ; Mon, 13 Feb 2023 07:33:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676302392; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jiirWQEzjOJm38ClQocxHDg9OYUUCbqX/zh69yaIxOk=; b=N13fLyZoBTqSFYf3Aky6nf7/WXr80LdKuyJnUJxgnda2OB9W5nIoVfvMjzpI9IQruJSx5j yDAWi5kZhrsavt1c8JnR8WW/BHCzk7XL8fH++4HUxDdd0ZpbQtbBtcdu8wFWdMdJTl7zwM C18V+qpn6Nmm1NE4mmf7egHioNVbG6I= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-312-m02BLPiNNVOcc7akYvkMdA-1; Mon, 13 Feb 2023 10:33:09 -0500 X-MC-Unique: m02BLPiNNVOcc7akYvkMdA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 104AE1C04339; Mon, 13 Feb 2023 15:33:09 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.24]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1907918EC7; Mon, 13 Feb 2023 15:33:07 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v2 1/4] splice: Rename new splice functions Date: Mon, 13 Feb 2023 15:32:58 +0000 Message-Id: <20230213153301.2338806-2-dhowells@redhat.com> In-Reply-To: <20230213153301.2338806-1-dhowells@redhat.com> References: <20230213153301.2338806-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757730623472967652?= X-GMAIL-MSGID: =?utf-8?q?1757730623472967652?= Rename generic_file_buffered_splice_read() to filemap_splice_read(). Rename generic_file_direct_splice_read() to direct_splice_read(). Requested-by: Christoph Hellwig Signed-off-by: David Howells Reviewed-by: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Reviewed-by: David Hildenbrand --- fs/splice.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index 2717078949a2..91b9e2cb9e03 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -287,9 +287,9 @@ void splice_shrink_spd(struct splice_pipe_desc *spd) * Splice data from an O_DIRECT file into pages and then add them to the output * pipe. */ -static ssize_t generic_file_direct_splice_read(struct file *in, loff_t *ppos, - struct pipe_inode_info *pipe, - size_t len, unsigned int flags) +static ssize_t direct_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) { struct iov_iter to; struct bio_vec *bv; @@ -417,10 +417,9 @@ static size_t splice_folio_into_pipe(struct pipe_inode_info *pipe, * Splice folios from the pagecache of a buffered (ie. non-O_DIRECT) file into * a pipe. */ -static ssize_t generic_file_buffered_splice_read(struct file *in, loff_t *ppos, - struct pipe_inode_info *pipe, - size_t len, - unsigned int flags) +static ssize_t filemap_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) { struct folio_batch fbatch; size_t total_spliced = 0, used, npages; @@ -529,8 +528,8 @@ ssize_t generic_file_splice_read(struct file *in, loff_t *ppos, if (unlikely(!len)) return 0; if (in->f_flags & O_DIRECT) - return generic_file_direct_splice_read(in, ppos, pipe, len, flags); - return generic_file_buffered_splice_read(in, ppos, pipe, len, flags); + return direct_splice_read(in, ppos, pipe, len, flags); + return filemap_splice_read(in, ppos, pipe, len, flags); } EXPORT_SYMBOL(generic_file_splice_read); From patchwork Mon Feb 13 15:32:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 56339 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2412957wrn; Mon, 13 Feb 2023 07:39:29 -0800 (PST) X-Google-Smtp-Source: AK7set/BBHYjBYfYXnUEH5JSAmKH0hbfrr8vT39LXOTP8dUfsmJGv4RGke1PabZO3yrQ6A/wnl6v X-Received: by 2002:a17:90a:1a19:b0:230:dcdb:c779 with SMTP id 25-20020a17090a1a1900b00230dcdbc779mr27635337pjk.24.1676302769057; Mon, 13 Feb 2023 07:39:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676302769; cv=none; d=google.com; s=arc-20160816; b=REnN8PpH8AqN3fVkWnscokq+BImdFuhlxOjPhh6RJxfw1D5u150gw7xnvT+a+tjDOm 8sfyCsy6eV2NQSjIh2tnixokvAAXrYxYDcyPNOoSDL+4lX65AG3AE7Q42rtpVsdLhAL3 rMLncm7JviiRxSqCDE8Us6mt7ygilSyr8bRpKSc3vYLUnH7WzNHrXZIsAIiF7KxmdA2Q AK9vr6KVeHW0dnPad2DMafE3g0lf9ibHN+GA8PSeX9cKH85xFs5NRLUBzuc9KoFZPnQf u+xHTLUwIjWFehSa3tiVSN0fkEgnVIJVhZa0LXiCzcfKt0bGtgvJqdOkttcQiGxkroMC UGCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=+b2VMeGpZTyrheqYPKTaHyxDIQ3yZeSE4YqjD4ZyLaI=; b=RLXYY7cWart+DDK+pCl81P2M5A2zmmnZ6WO3TZTKrEv+1ESrJeNu/HYvY7T40Nz/1D PuKD4z31MzC6A+bL12ae7+J2wUtWbNWNwYWJIlc9KVasL6LxNbPCqqU3Gd3Gwar5+1G1 rQLSJ/DZDxQ4rARe51UlLZtgqeEEfwqGfREOAkJHT+lgHP4kbjWy58TA6TyKTIXRTo+P jbWl2spkA6UfdhGSlEPgeMgvX2NGHoO797QS6uubzfnRbBCxfK5BRbKF0P3qV04wrlnt COzCj8etigjFQ9RmaJonl9lDZwb6r0AlD2MWjblR/4wc5kGfLbqorCSx0S2XNvKaoqWb iIkA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=TYUYc5r8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id pi17-20020a17090b1e5100b002341fa8d93asi435973pjb.92.2023.02.13.07.39.16; Mon, 13 Feb 2023 07:39:29 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=TYUYc5r8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231258AbjBMPeY (ORCPT + 99 others); Mon, 13 Feb 2023 10:34:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231142AbjBMPeL (ORCPT ); Mon, 13 Feb 2023 10:34:11 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BB93B13D57 for ; Mon, 13 Feb 2023 07:33:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676302400; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+b2VMeGpZTyrheqYPKTaHyxDIQ3yZeSE4YqjD4ZyLaI=; b=TYUYc5r8NERANBbWIYT9I0TWQxPO7U385qverDzvK+CuAFfqrAi0MUY9y79QUyYn/rOXxr mryLukZBU7xAKlmEd5X77Fy+9nw6qBznsWBI6X0D3An+WfI3Qn+JsMTOT/iHJBvwmwhth5 txoMVBAMtUQWb8P+EnZEZ5yVOKWH52E= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-256-1ODbYshqMwmxxNEz7syx6g-1; Mon, 13 Feb 2023 10:33:12 -0500 X-MC-Unique: 1ODbYshqMwmxxNEz7syx6g-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id ACFD985A588; Mon, 13 Feb 2023 15:33:11 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.24]) by smtp.corp.redhat.com (Postfix) with ESMTP id BCD6240B40C9; Mon, 13 Feb 2023 15:33:09 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v2 2/4] splice: Provide pipe_head_buf() helper Date: Mon, 13 Feb 2023 15:32:59 +0000 Message-Id: <20230213153301.2338806-3-dhowells@redhat.com> In-Reply-To: <20230213153301.2338806-1-dhowells@redhat.com> References: <20230213153301.2338806-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757730851938113934?= X-GMAIL-MSGID: =?utf-8?q?1757730851938113934?= Provide a helper, pipe_head_buf(), to get the current head buffer from a pipe. Implement this as a wrapper around a more general function, pipe_buf(), that gets a specified buffer. Requested-by: Christoph Hellwig Signed-off-by: David Howells Reviewed-by: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Reviewed-by: David Hildenbrand --- fs/splice.c | 9 +++------ include/linux/pipe_fs_i.h | 20 ++++++++++++++++++++ 2 files changed, 23 insertions(+), 6 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index 91b9e2cb9e03..7c0ff187f87a 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -295,7 +295,6 @@ static ssize_t direct_splice_read(struct file *in, loff_t *ppos, struct bio_vec *bv; struct kiocb kiocb; struct page **pages; - unsigned int head; ssize_t ret; size_t used, npages, chunk, remain, reclaim; int i; @@ -358,9 +357,8 @@ static ssize_t direct_splice_read(struct file *in, loff_t *ppos, } /* Push the remaining pages into the pipe. */ - head = pipe->head; for (i = 0; i < npages; i++) { - struct pipe_buffer *buf = &pipe->bufs[head & (pipe->ring_size - 1)]; + struct pipe_buffer *buf = pipe_head_buf(pipe); chunk = min_t(size_t, remain, PAGE_SIZE); *buf = (struct pipe_buffer) { @@ -369,10 +367,9 @@ static ssize_t direct_splice_read(struct file *in, loff_t *ppos, .offset = 0, .len = chunk, }; - head++; + pipe->head++; remain -= chunk; } - pipe->head = head; kfree(bv); return ret; @@ -394,7 +391,7 @@ static size_t splice_folio_into_pipe(struct pipe_inode_info *pipe, while (spliced < size && !pipe_full(pipe->head, pipe->tail, pipe->max_usage)) { - struct pipe_buffer *buf = &pipe->bufs[pipe->head & (pipe->ring_size - 1)]; + struct pipe_buffer *buf = pipe_head_buf(pipe); size_t part = min_t(size_t, PAGE_SIZE - offset, size - spliced); *buf = (struct pipe_buffer) { diff --git a/include/linux/pipe_fs_i.h b/include/linux/pipe_fs_i.h index 6cb65df3e3ba..d2c3f16cf6b1 100644 --- a/include/linux/pipe_fs_i.h +++ b/include/linux/pipe_fs_i.h @@ -156,6 +156,26 @@ static inline bool pipe_full(unsigned int head, unsigned int tail, return pipe_occupancy(head, tail) >= limit; } +/** + * pipe_buf - Return the pipe buffer for the specified slot in the pipe ring + * @pipe: The pipe to access + * @slot: The slot of interest + */ +static inline struct pipe_buffer *pipe_buf(const struct pipe_inode_info *pipe, + unsigned int slot) +{ + return &pipe->bufs[slot & (pipe->ring_size - 1)]; +} + +/** + * pipe_head_buf - Return the pipe buffer at the head of the pipe ring + * @pipe: The pipe to access + */ +static inline struct pipe_buffer *pipe_head_buf(const struct pipe_inode_info *pipe) +{ + return pipe_buf(pipe, pipe->head); +} + /** * pipe_buf_get - get a reference to a pipe_buffer * @pipe: the pipe that the buffer belongs to From patchwork Mon Feb 13 15:33:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 56338 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2412774wrn; Mon, 13 Feb 2023 07:39:03 -0800 (PST) X-Google-Smtp-Source: AK7set9i9KGHKO/6Fs1DZkVRw9QMdsgicN6F8llwV+D0m0qJMmzVfBlypSAjIp11yFp4zxSf46Mh X-Received: by 2002:a05:6a21:3605:b0:c0:df5b:e9e7 with SMTP id yg5-20020a056a21360500b000c0df5be9e7mr18912021pzb.60.1676302742772; Mon, 13 Feb 2023 07:39:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676302742; cv=none; d=google.com; s=arc-20160816; b=H6RErsp0UM/raNKHJt+NF+7aU9z1iQ1lGUuL2WTRPFIHlIi1Axtx9Os6NiHm433oRS q8hUAADfO+eInQ6fGVAtN4qQUJWhCBP8ukOW/ChdIyOXf4zNyhuWzUNuuSTmNtTR4M8b L0laOANNpkmzOV/FTdCot/ZAm1dltYNPUPR+CptMxzgaXf8oP7MaTYFstvD0QITHCElR /VEofBcuYdhapnQFrmw4bkWEZv0gShmKPrcMv4/0L3Yjm7/++v9wMksMLE9WJDmcM9NJ TyaX7y95p6f6XDmq1gnbyLeZuitzlwHpZ6xYqZrN/g2W1hpEa4iuozKL6QrC96cBJL4Y A0MA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=xJXt4KNNFIdMvSSPmBpCRCAi8g0JFMLHorMyvAgfics=; b=urJfn/3XYh78bvdE3eQL/IpL0a+d985eiUHfUj0oNju/nXFT/Ug3vVZRJ5A/XW4c6A 8OLZPAyvObqaUgZs787tc1CObt7wIxV2tE7zN0z5tl74WgtrRUSOyNLc7hPMvvhyvLuy YSM7joaLVleLSKvCDH9AdcZOitJ/UAAnWE8VEmKzlYBP3V3CvCaFwyCTMkUZapE6mXxt kiDUJ2sxiSzM0+xLCeePb/dIQqlk1Q4rIfLYwm8tag1xabf15E3jPdg+FnutXO04cVgu ZT0rZYg53ILULzERrernG3oAuam/txH+sYLqYYnNeqc0GP+epwgah8DhFVoRpOm4+Jvd Hz1g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=EoZ4+zo9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j72-20020a636e4b000000b004df4af6841asi13765742pgc.84.2023.02.13.07.38.48; Mon, 13 Feb 2023 07:39:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=EoZ4+zo9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231273AbjBMPe1 (ORCPT + 99 others); Mon, 13 Feb 2023 10:34:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47402 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231187AbjBMPeN (ORCPT ); Mon, 13 Feb 2023 10:34:13 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4CD0216339 for ; Mon, 13 Feb 2023 07:33:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676302403; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xJXt4KNNFIdMvSSPmBpCRCAi8g0JFMLHorMyvAgfics=; b=EoZ4+zo9eJbbHbmh7cOYqBY15uCH//mg80tA1UsDa2VJo76jZvkIjSW/BwT+ZRDgFGK9J+ I3XQoSf0k/NdNaWDKdEMeVz0UEj0sRhji2G+ePR2ksVD5mvntEY8WeLURGMhomuSTtSLpn WUNLalUoiMgJJLreQnvcBZbDJit/yKg= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-325-ffjTGi4LPyGsidvz5tnMXw-1; Mon, 13 Feb 2023 10:33:15 -0500 X-MC-Unique: ffjTGi4LPyGsidvz5tnMXw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 590D43C14851; Mon, 13 Feb 2023 15:33:14 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.24]) by smtp.corp.redhat.com (Postfix) with ESMTP id 53FA22026D4B; Mon, 13 Feb 2023 15:33:12 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v2 3/4] splice: Use init_sync_kiocb() in filemap_splice_read() Date: Mon, 13 Feb 2023 15:33:00 +0000 Message-Id: <20230213153301.2338806-4-dhowells@redhat.com> In-Reply-To: <20230213153301.2338806-1-dhowells@redhat.com> References: <20230213153301.2338806-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757730824923020146?= X-GMAIL-MSGID: =?utf-8?q?1757730824923020146?= Use init_sync_kiocb() in filemap_splice_read() rather than open coding it. Requested-by: Christoph Hellwig Signed-off-by: David Howells cc: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Reviewed-by: David Hildenbrand --- Notes: ver #2) - Don't attempt to filter IOCB_* flags. fs/splice.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index 7c0ff187f87a..4ea63d6a9040 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -419,15 +419,14 @@ static ssize_t filemap_splice_read(struct file *in, loff_t *ppos, size_t len, unsigned int flags) { struct folio_batch fbatch; + struct kiocb iocb; size_t total_spliced = 0, used, npages; loff_t isize, end_offset; bool writably_mapped; int i, error = 0; - struct kiocb iocb = { - .ki_filp = in, - .ki_pos = *ppos, - }; + init_sync_kiocb(&iocb, in); + iocb.ki_pos = *ppos; /* Work out how much data we can actually add into the pipe */ used = pipe_occupancy(pipe->head, pipe->tail); From patchwork Mon Feb 13 15:33:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 56340 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2417731wrn; Mon, 13 Feb 2023 07:49:16 -0800 (PST) X-Google-Smtp-Source: AK7set+WRDaWSx0CcEBHCe48uFd47GlQdZsokueFqnRwX0LiSKPnXFWycbpqw88HjAcJEap+6lNq X-Received: by 2002:a17:90a:4a8d:b0:22b:b78f:ba04 with SMTP id f13-20020a17090a4a8d00b0022bb78fba04mr28656951pjh.41.1676303356081; Mon, 13 Feb 2023 07:49:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676303356; cv=none; d=google.com; s=arc-20160816; b=yr5vSY7K000Zp24tMehsqUoMiQ84+0sxDD1k86dBvUMDnfjSI49U8/Kfyw2NVc2AOs urBpVUPlHKyXSvPi9VEMttSXzulumEkja4nGbHB9bCS6ReHuuWcN0/JkcPIQjASrHalm TREXN3NAEFIlrmmtxSXZS1OckHKatcC0wTLYJOpqXEJgeKe24FC+UThYsFm2WaFyW8oi 2sKr0BEVGL7QYWmSOpDG1XEqE1rxqGMrg5rVvLqzVyUot0jTvi4suQX8YjSiXN4DcaV/ tvtOAIR8JZL6u5wc977zHtUG48/EFv8ncRuZwVkbc1EWQvxZmTgAMOqIR6WkHtRj7sCZ LZpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=49IJ7O8KaG9bUjSCVtwVGeadJYJCODKyAnH6L7XF9aY=; b=VyjH7PXtUV0UtHIO7fQk7ayWm6g+oR/v9hIvpKdFVxHdaJ+7gVU2SoYJGLNfppaNVK 32uLzuBgwhcRHgEX9Z2RhWWpj4fwCqJX2hCeoInaTjtpnTbfJU4Qgx/K2/xbjUx1L2re QOhbWbHtqdanbVxAZqmmwqq6q+CJsm6iD0oQK32UJ8pwZ8hpoq/p08m5qME/9lN2iTNb pr0NWBcQxyJHR+ZbbMOgZZmrsTTFFPGHqa/9f4kcdfYSXLCWiKaxuK1JIybNy+2Mi9DQ kfe4PKx+61Q4f+l8H0gNKYvuSygRyQPrIM3TuW1Q0QkdbSR3EY4I8LtI9epbm5Oc8Ebx qi7A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=HkyClUdn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h11-20020a17090acf0b00b00233c7c617e7si7022994pju.101.2023.02.13.07.49.03; Mon, 13 Feb 2023 07:49:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=HkyClUdn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231281AbjBMPe3 (ORCPT + 99 others); Mon, 13 Feb 2023 10:34:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47390 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231190AbjBMPeN (ORCPT ); Mon, 13 Feb 2023 10:34:13 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DED8215CB6 for ; Mon, 13 Feb 2023 07:33:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676302402; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=49IJ7O8KaG9bUjSCVtwVGeadJYJCODKyAnH6L7XF9aY=; b=HkyClUdnj/TQCXccDXpy9R0b6hVJt/PqCfubjNk+VqNYXDT2ebgGFSYW4A+MxU4qx6RDty Fr6NJYF092OUk+lkmFvSgbgpZEYy+osFfYDkHQVbc/k3KEWeJytW8O04wrc+Ha1zsRpDs5 8AnQ/eQ3Mb9or/cVmYbJHESUO/j1nns= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-443-1yMYszxON0mvnfJn-xDpZA-1; Mon, 13 Feb 2023 10:33:17 -0500 X-MC-Unique: 1yMYszxON0mvnfJn-xDpZA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0864D18A6464; Mon, 13 Feb 2023 15:33:17 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.24]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1696CC16022; Mon, 13 Feb 2023 15:33:14 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v2 4/4] splice: Move filemap_read_splice() to mm/filemap.c Date: Mon, 13 Feb 2023 15:33:01 +0000 Message-Id: <20230213153301.2338806-5-dhowells@redhat.com> In-Reply-To: <20230213153301.2338806-1-dhowells@redhat.com> References: <20230213153301.2338806-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757731467980189639?= X-GMAIL-MSGID: =?utf-8?q?1757731467980189639?= Move filemap_read_splice() to mm/filemap.c and make filemap_get_pages() static again. Requested-by: Christoph Hellwig Signed-off-by: David Howells Reviewed-by: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Reviewed-by: David Hildenbrand --- fs/splice.c | 127 ------------------------------------- include/linux/pagemap.h | 2 - include/linux/splice.h | 4 ++ mm/filemap.c | 137 ++++++++++++++++++++++++++++++++++++++-- 4 files changed, 135 insertions(+), 135 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index 4ea63d6a9040..341cd8fb47a8 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -375,133 +375,6 @@ static ssize_t direct_splice_read(struct file *in, loff_t *ppos, return ret; } -/* - * Splice subpages from a folio into a pipe. - */ -static size_t splice_folio_into_pipe(struct pipe_inode_info *pipe, - struct folio *folio, - loff_t fpos, size_t size) -{ - struct page *page; - size_t spliced = 0, offset = offset_in_folio(folio, fpos); - - page = folio_page(folio, offset / PAGE_SIZE); - size = min(size, folio_size(folio) - offset); - offset %= PAGE_SIZE; - - while (spliced < size && - !pipe_full(pipe->head, pipe->tail, pipe->max_usage)) { - struct pipe_buffer *buf = pipe_head_buf(pipe); - size_t part = min_t(size_t, PAGE_SIZE - offset, size - spliced); - - *buf = (struct pipe_buffer) { - .ops = &page_cache_pipe_buf_ops, - .page = page, - .offset = offset, - .len = part, - }; - folio_get(folio); - pipe->head++; - page++; - spliced += part; - offset = 0; - } - - return spliced; -} - -/* - * Splice folios from the pagecache of a buffered (ie. non-O_DIRECT) file into - * a pipe. - */ -static ssize_t filemap_splice_read(struct file *in, loff_t *ppos, - struct pipe_inode_info *pipe, - size_t len, unsigned int flags) -{ - struct folio_batch fbatch; - struct kiocb iocb; - size_t total_spliced = 0, used, npages; - loff_t isize, end_offset; - bool writably_mapped; - int i, error = 0; - - init_sync_kiocb(&iocb, in); - iocb.ki_pos = *ppos; - - /* Work out how much data we can actually add into the pipe */ - used = pipe_occupancy(pipe->head, pipe->tail); - npages = max_t(ssize_t, pipe->max_usage - used, 0); - len = min_t(size_t, len, npages * PAGE_SIZE); - - folio_batch_init(&fbatch); - - do { - cond_resched(); - - if (*ppos >= i_size_read(file_inode(in))) - break; - - iocb.ki_pos = *ppos; - error = filemap_get_pages(&iocb, len, &fbatch, true); - if (error < 0) - break; - - /* - * i_size must be checked after we know the pages are Uptodate. - * - * Checking i_size after the check allows us to calculate - * the correct value for "nr", which means the zero-filled - * part of the page is not copied back to userspace (unless - * another truncate extends the file - this is desired though). - */ - isize = i_size_read(file_inode(in)); - if (unlikely(*ppos >= isize)) - break; - end_offset = min_t(loff_t, isize, *ppos + len); - - /* - * Once we start copying data, we don't want to be touching any - * cachelines that might be contended: - */ - writably_mapped = mapping_writably_mapped(in->f_mapping); - - for (i = 0; i < folio_batch_count(&fbatch); i++) { - struct folio *folio = fbatch.folios[i]; - size_t n; - - if (folio_pos(folio) >= end_offset) - goto out; - folio_mark_accessed(folio); - - /* - * If users can be writing to this folio using arbitrary - * virtual addresses, take care of potential aliasing - * before reading the folio on the kernel side. - */ - if (writably_mapped) - flush_dcache_folio(folio); - - n = splice_folio_into_pipe(pipe, folio, *ppos, len); - if (!n) - goto out; - len -= n; - total_spliced += n; - *ppos += n; - in->f_ra.prev_pos = *ppos; - if (pipe_full(pipe->head, pipe->tail, pipe->max_usage)) - goto out; - } - - folio_batch_release(&fbatch); - } while (len); - -out: - folio_batch_release(&fbatch); - file_accessed(in); - - return total_spliced ? total_spliced : error; -} - /** * generic_file_splice_read - splice data from file to a pipe * @in: file to splice from diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 3a7bdb35acff..29e1f9e76eb6 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -748,8 +748,6 @@ struct page *read_cache_page(struct address_space *, pgoff_t index, filler_t *filler, struct file *file); extern struct page * read_cache_page_gfp(struct address_space *mapping, pgoff_t index, gfp_t gfp_mask); -int filemap_get_pages(struct kiocb *iocb, size_t count, - struct folio_batch *fbatch, bool need_uptodate); static inline struct page *read_mapping_page(struct address_space *mapping, pgoff_t index, struct file *file) diff --git a/include/linux/splice.h b/include/linux/splice.h index a55179fd60fc..691c44ef5c0b 100644 --- a/include/linux/splice.h +++ b/include/linux/splice.h @@ -67,6 +67,10 @@ typedef int (splice_actor)(struct pipe_inode_info *, struct pipe_buffer *, typedef int (splice_direct_actor)(struct pipe_inode_info *, struct splice_desc *); +ssize_t filemap_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags); + extern ssize_t splice_from_pipe(struct pipe_inode_info *, struct file *, loff_t *, size_t, unsigned int, splice_actor *); diff --git a/mm/filemap.c b/mm/filemap.c index 6970be64a3e0..e1ee267675d2 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -42,6 +42,8 @@ #include #include #include +#include +#include #include #include #include "internal.h" @@ -2576,12 +2578,8 @@ static int filemap_readahead(struct kiocb *iocb, struct file *file, return 0; } -/* - * Extract some folios from the pagecache of a file, reading those pages from - * the backing store if necessary and waiting for them. - */ -int filemap_get_pages(struct kiocb *iocb, size_t count, - struct folio_batch *fbatch, bool need_uptodate) +static int filemap_get_pages(struct kiocb *iocb, size_t count, + struct folio_batch *fbatch, bool need_uptodate) { struct file *filp = iocb->ki_filp; struct address_space *mapping = filp->f_mapping; @@ -2845,6 +2843,133 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) } EXPORT_SYMBOL(generic_file_read_iter); +/* + * Splice subpages from a folio into a pipe. + */ +static size_t splice_folio_into_pipe(struct pipe_inode_info *pipe, + struct folio *folio, + loff_t fpos, size_t size) +{ + struct page *page; + size_t spliced = 0, offset = offset_in_folio(folio, fpos); + + page = folio_page(folio, offset / PAGE_SIZE); + size = min(size, folio_size(folio) - offset); + offset %= PAGE_SIZE; + + while (spliced < size && + !pipe_full(pipe->head, pipe->tail, pipe->max_usage)) { + struct pipe_buffer *buf = pipe_head_buf(pipe); + size_t part = min_t(size_t, PAGE_SIZE - offset, size - spliced); + + *buf = (struct pipe_buffer) { + .ops = &page_cache_pipe_buf_ops, + .page = page, + .offset = offset, + .len = part, + }; + folio_get(folio); + pipe->head++; + page++; + spliced += part; + offset = 0; + } + + return spliced; +} + +/* + * Splice folios from the pagecache of a buffered (ie. non-O_DIRECT) file into + * a pipe. + */ +ssize_t filemap_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + struct folio_batch fbatch; + struct kiocb iocb; + size_t total_spliced = 0, used, npages; + loff_t isize, end_offset; + bool writably_mapped; + int i, error = 0; + + init_sync_kiocb(&iocb, in); + iocb.ki_pos = *ppos; + + /* Work out how much data we can actually add into the pipe */ + used = pipe_occupancy(pipe->head, pipe->tail); + npages = max_t(ssize_t, pipe->max_usage - used, 0); + len = min_t(size_t, len, npages * PAGE_SIZE); + + folio_batch_init(&fbatch); + + do { + cond_resched(); + + if (*ppos >= i_size_read(file_inode(in))) + break; + + iocb.ki_pos = *ppos; + error = filemap_get_pages(&iocb, len, &fbatch, true); + if (error < 0) + break; + + /* + * i_size must be checked after we know the pages are Uptodate. + * + * Checking i_size after the check allows us to calculate + * the correct value for "nr", which means the zero-filled + * part of the page is not copied back to userspace (unless + * another truncate extends the file - this is desired though). + */ + isize = i_size_read(file_inode(in)); + if (unlikely(*ppos >= isize)) + break; + end_offset = min_t(loff_t, isize, *ppos + len); + + /* + * Once we start copying data, we don't want to be touching any + * cachelines that might be contended: + */ + writably_mapped = mapping_writably_mapped(in->f_mapping); + + for (i = 0; i < folio_batch_count(&fbatch); i++) { + struct folio *folio = fbatch.folios[i]; + size_t n; + + if (folio_pos(folio) >= end_offset) + goto out; + folio_mark_accessed(folio); + + /* + * If users can be writing to this folio using arbitrary + * virtual addresses, take care of potential aliasing + * before reading the folio on the kernel side. + */ + if (writably_mapped) + flush_dcache_folio(folio); + + n = splice_folio_into_pipe(pipe, folio, *ppos, len); + if (!n) + goto out; + len -= n; + total_spliced += n; + *ppos += n; + in->f_ra.prev_pos = *ppos; + if (pipe_full(pipe->head, pipe->tail, pipe->max_usage)) + goto out; + } + + folio_batch_release(&fbatch); + } while (len); + +out: + folio_batch_release(&fbatch); + file_accessed(in); + + return total_spliced ? total_spliced : error; +} + static inline loff_t folio_seek_hole_data(struct xa_state *xas, struct address_space *mapping, struct folio *folio, loff_t start, loff_t end, bool seek_data)