Message ID | 20230214083710.2547248-1-dhowells@redhat.com |
---|---|
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:eb09:0:0:0:0:0 with SMTP id s9csp2845700wrn; Tue, 14 Feb 2023 00:44:03 -0800 (PST) X-Google-Smtp-Source: AK7set9mWmwwpYOlyIxY80G+7xqNezkKbSc/XPXjU/xoXWktok4Jyzxm52c+rO7CqBbbivmnF4YY X-Received: by 2002:a17:902:f549:b0:196:8292:e879 with SMTP id h9-20020a170902f54900b001968292e879mr2266926plf.1.1676364243014; Tue, 14 Feb 2023 00:44:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1676364242; cv=none; d=google.com; s=arc-20160816; b=jEIi2uA2bJ9N/zKIRaZYZSnGZYRMv/fCGzWsI58Y6+HDEejk/4VC9WqcCfrNXkEws2 pEwxX6QLFw4XU3OfACxk9wBIc/GQY25D2PckydnFovAUkpbPB7Tp2hcO+IWE679G/Gnt 7cuN9oWwi13pnPQnyHGZpb+7ntE+UH5A5eY3zExPYQfKmLGMho8os5350YkB2wzo9J/f aWCk18TCHEN67IwYSn+ejWkSnmHy93OOcFENLLccIC/K/ESSJDP5y7TzDa/9/h3glL3K m6XQdaMNeMJ6F5OkIC+QMTs93yPmqnSJ5cBgZ6G6yOUWj/jcQ7EOG0V9A1h+U6nJiuuX bnmg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=Vq3Sj7rlsNdLjcqHDgs0XkW5jVa94T8TrtK6zXmHKG4=; b=Vh+lDulucH0V0ABSJyQXJ9eQZqAI3dYstRHhQ8el6H6FcnSSMwLanpO4Ljqp99f7AK Aci2UOSiFEf+PPF00UlMRLqLlDWnh9dXpNXQI1zgJpKfJlfc0bZW/HRK2E+pfdA/U+AC ms1UqWyq9w8tkC4blGMBZSQvqFCGblQxmy4i4jZVptc+MHy7WxaYeLVrYdsAtAKXrLKh z/gt7pTuiguftM38WyMC7PmbLOKV/Yh74Cm8TBZsMDLUcKSIMcUCjryOXT30fH6UTJ/a KipnWf9ao6KyO79ZX2SR1SI4NZAYuAvhC2A7Xg/fjW1QcR1OxKdb/dOk3VU4Gny1I2+a bEvg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=iH+nUlBC; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z24-20020a1709028f9800b00182c510591asi13315704plo.246.2023.02.14.00.43.50; Tue, 14 Feb 2023 00:44:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=iH+nUlBC; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232039AbjBNIiG (ORCPT <rfc822;henrikjoshmiranda3@gmail.com> + 99 others); Tue, 14 Feb 2023 03:38:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40744 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231779AbjBNIiE (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Tue, 14 Feb 2023 03:38:04 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5E32FC178 for <linux-kernel@vger.kernel.org>; Tue, 14 Feb 2023 00:37:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676363838; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=Vq3Sj7rlsNdLjcqHDgs0XkW5jVa94T8TrtK6zXmHKG4=; b=iH+nUlBCZ76lKU4+M3HxJ8XomuUkNywbP86qwv/9jfjS+iqk77wQi924pmsWNX9Smyoeq2 vZhRHdMbPlxHm9SooBdbJOaKq9nw5wLyxx77o2eNTq2IQkub7eZVJ2XIs+PHcMaQhmAmhI oWPvoUHx+URqL4olDosTgOlK8wFEVDQ= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-132-bCtj-sb0O7-xSZu8Wwp9jg-1; Tue, 14 Feb 2023 03:37:15 -0500 X-MC-Unique: bCtj-sb0O7-xSZu8Wwp9jg-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 52454857A93; Tue, 14 Feb 2023 08:37:14 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.24]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4236A492B03; Tue, 14 Feb 2023 08:37:12 +0000 (UTC) From: David Howells <dhowells@redhat.com> To: Jens Axboe <axboe@kernel.dk>, Al Viro <viro@zeniv.linux.org.uk>, Christoph Hellwig <hch@infradead.org> Cc: David Howells <dhowells@redhat.com>, Matthew Wilcox <willy@infradead.org>, Jan Kara <jack@suse.cz>, Jeff Layton <jlayton@kernel.org>, David Hildenbrand <david@redhat.com>, Jason Gunthorpe <jgg@nvidia.com>, Logan Gunthorpe <logang@deltatee.com>, Hillf Danton <hdanton@sina.com>, linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v3 0/5] iov_iter: Adjust styling/location of new splice functions Date: Tue, 14 Feb 2023 08:37:05 +0000 Message-Id: <20230214083710.2547248-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1757795312753917690?= X-GMAIL-MSGID: =?utf-8?q?1757795312753917690?= |
Series |
iov_iter: Adjust styling/location of new splice functions
|
|
Message
David Howells
Feb. 14, 2023, 8:37 a.m. UTC
Hi Jens, Al, Christoph, Here are patches to make some changes that Christoph requested[1] to the new generic file splice functions that I implemented[2]. Apart from one functional change, they just altering the styling and move one of the functions to a different file: (1) Rename the main functions: generic_file_buffered_splice_read() -> filemap_splice_read() generic_file_direct_splice_read() -> direct_splice_read() (2) Abstract out the calculation of the location of the head pipe buffer into a helper function in linux/pipe_fs_i.h. (3) Use init_sync_kiocb() in filemap_splice_read(). This is where the functional change is. Some kiocb fields are then filled in where they were set to 0 before, including setting ki_flags from f_iocb_flags. (4) Move filemap_splice_read() to mm/filemap.c. filemap_get_pages() can then be made static again. (5) Fix splice-read for a number of filesystems that don't provide a ->read_folio() op and for which filemap_get_pages() cannot be used. I've pushed the patches here also: https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=iov-extract-3 I've also updated worked the changes into the commits on my iov-extract branch if that would be preferable, though that means Jens would need to update his for-6.3/iov-extract again. David Link: https://lore.kernel.org/r/Y+n0n2UE8BQa/OwW@infradead.org/ [1] Link: https://lore.kernel.org/r/20230207171305.3716974-1-dhowells@redhat.com/ [2] Changes ======= ver #3) - Fix filesystems/drivers that don't have ->read_folio(). ver #2) - Don't attempt to filter IOCB_* flags in filemap_splice_read(). Link: https://lore.kernel.org/r/20230213134619.2198965-1-dhowells@redhat.com/ # v1 David Howells (5): splice: Rename new splice functions splice: Provide pipe_head_buf() helper splice: Use init_sync_kiocb() in filemap_splice_read() splice: Move filemap_read_splice() to mm/filemap.c shmem, overlayfs, coda, tty, proc, kernfs, random: Fix splice-read drivers/char/random.c | 4 +- drivers/tty/tty_io.c | 4 +- fs/coda/file.c | 36 +++++++++- fs/kernfs/file.c | 2 +- fs/overlayfs/file.c | 36 +++++++++- fs/proc/inode.c | 4 +- fs/proc/proc_sysctl.c | 2 +- fs/proc_namespace.c | 6 +- fs/splice.c | 146 ++------------------------------------ include/linux/fs.h | 6 ++ include/linux/pagemap.h | 2 - include/linux/pipe_fs_i.h | 20 ++++++ mm/filemap.c | 136 +++++++++++++++++++++++++++++++++-- mm/internal.h | 6 ++ mm/shmem.c | 124 +++++++++++++++++++++++++++++++- 15 files changed, 373 insertions(+), 161 deletions(-)
Comments
On 14.02.23 09:37, David Howells wrote: > Hi Jens, Al, Christoph, > > Here are patches to make some changes that Christoph requested[1] to the > new generic file splice functions that I implemented[2]. Apart from one > functional change, they just altering the styling and move one of the > functions to a different file: > > (1) Rename the main functions: > > generic_file_buffered_splice_read() -> filemap_splice_read() > generic_file_direct_splice_read() -> direct_splice_read() > > (2) Abstract out the calculation of the location of the head pipe buffer > into a helper function in linux/pipe_fs_i.h. > > (3) Use init_sync_kiocb() in filemap_splice_read(). > > This is where the functional change is. Some kiocb fields are then > filled in where they were set to 0 before, including setting ki_flags > from f_iocb_flags. > > (4) Move filemap_splice_read() to mm/filemap.c. filemap_get_pages() can > then be made static again. > > (5) Fix splice-read for a number of filesystems that don't provide a > ->read_folio() op and for which filemap_get_pages() cannot be used. > > I've pushed the patches here also: > > https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=iov-extract-3 > > I've also updated worked the changes into the commits on my iov-extract > branch if that would be preferable, though that means Jens would need to > update his for-6.3/iov-extract again. > > David > > Link: https://lore.kernel.org/r/Y+n0n2UE8BQa/OwW@infradead.org/ [1] > Link: https://lore.kernel.org/r/20230207171305.3716974-1-dhowells@redhat.com/ [2] > > Changes > ======= > ver #3) > - Fix filesystems/drivers that don't have ->read_folio(). > > ver #2) > - Don't attempt to filter IOCB_* flags in filemap_splice_read(). > > Link: https://lore.kernel.org/r/20230213134619.2198965-1-dhowells@redhat.com/ # v1 > You ignored my RB's :( .. but unrelated, what's the plan with this now? As Jens mentioned, it might be better to wait for 6.4 for the full series, in which case folding this series into the other series would be better.
On 2/14/23 2:07?AM, David Hildenbrand wrote: > On 14.02.23 09:37, David Howells wrote: >> Hi Jens, Al, Christoph, >> >> Here are patches to make some changes that Christoph requested[1] to the >> new generic file splice functions that I implemented[2]. Apart from one >> functional change, they just altering the styling and move one of the >> functions to a different file: >> >> (1) Rename the main functions: >> >> generic_file_buffered_splice_read() -> filemap_splice_read() >> generic_file_direct_splice_read() -> direct_splice_read() >> >> (2) Abstract out the calculation of the location of the head pipe buffer >> into a helper function in linux/pipe_fs_i.h. >> >> (3) Use init_sync_kiocb() in filemap_splice_read(). >> >> This is where the functional change is. Some kiocb fields are then >> filled in where they were set to 0 before, including setting ki_flags >> from f_iocb_flags. >> >> (4) Move filemap_splice_read() to mm/filemap.c. filemap_get_pages() can >> then be made static again. >> >> (5) Fix splice-read for a number of filesystems that don't provide a >> ->read_folio() op and for which filemap_get_pages() cannot be used. >> >> I've pushed the patches here also: >> >> https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=iov-extract-3 >> >> I've also updated worked the changes into the commits on my iov-extract >> branch if that would be preferable, though that means Jens would need to >> update his for-6.3/iov-extract again. >> >> David >> >> Link: https://lore.kernel.org/r/Y+n0n2UE8BQa/OwW@infradead.org/ [1] >> Link: https://lore.kernel.org/r/20230207171305.3716974-1-dhowells@redhat.com/ [2] >> >> Changes >> ======= >> ver #3) >> - Fix filesystems/drivers that don't have ->read_folio(). >> >> ver #2) >> - Don't attempt to filter IOCB_* flags in filemap_splice_read(). >> >> Link: https://lore.kernel.org/r/20230213134619.2198965-1-dhowells@redhat.com/ # v1 >> > > You ignored my RB's :( > > .. but unrelated, what's the plan with this now? As Jens mentioned, it > might be better to wait for 6.4 for the full series, in which case > folding this series into the other series would be better. That is indeed the question, and unanswered so far... Let's turn it into one clean series, and get it stuffed into for-next and most likely target 6.4 for inclusion at this point.
Jens Axboe <axboe@kernel.dk> wrote: > That is indeed the question, and unanswered so far... Let's turn it into > one clean series, and get it stuffed into for-next and most likely > target 6.4 for inclusion at this point. I was waiting to see if the patch worked for Daniel (which it does) and Guenter (no answer yet) before answering. It appears to fix shmem - I've tested it with: dd if=/dev/zero of=/tmp/sparse count=1 seek=401 bs=4096 just-splice /tmp/sparse 11234000 | sha1sum where just-splice.c is attached (note that piping the output into another program is important to make the splice work). Meanwhile, I'm working on working the changes into my patchset at appropriate points. David --- #define _GNU_SOURCE #include <stdio.h> #include <stdlib.h> #include <string.h> #include <unistd.h> #include <fcntl.h> #include <sys/sendfile.h> #include <sys/wait.h> static char *prog; int main(int argc, char *argv[]) { unsigned int iflags = 0; ssize_t spliced, remain; int in, out; prog = argv[0]; if (argc > 1 && strcmp(argv[1], "-d") == 0) { iflags |= O_DIRECT; argv++; argc--; } if (argc != 3 || !argv[1][0] || !argv[2][0]) { fprintf(stderr, "Usage: %s <file> <amount>\n", prog); exit(2); } in = open(argv[1], O_RDONLY | O_NOFOLLOW | iflags); if (in < 0) { perror("open"); exit(1); } remain = strtoul(argv[2], NULL, 0); while (remain > 0) { spliced = splice(in, NULL, 1, NULL, remain, 0); if (spliced < 0) { perror("splice"); exit(1); } if (spliced == 0) break; remain -= spliced; } exit(0); }