Message ID | 20230331160914.1608208-34-dhowells@redhat.com |
---|---|
State | New |
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp692268vqo; Fri, 31 Mar 2023 09:36:16 -0700 (PDT) X-Google-Smtp-Source: AKy350Z/K8ZHifBPk+Rm5+NCXIj2v227h1MltoMudKvsmsNaveQMydghLa9kHj15qvuGyfc357A5 X-Received: by 2002:a17:906:f193:b0:92b:eca6:43fc with SMTP id gs19-20020a170906f19300b0092beca643fcmr28250364ejb.64.1680280576246; Fri, 31 Mar 2023 09:36:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680280576; cv=none; d=google.com; s=arc-20160816; b=VcPBTmyn9V+VeoY8wgdj5+i9BM7cLMEXNN3zLI8d3xxzeNC8qFc6+K3PkOqIE37Gi7 oo/FxuXH0WFoLgL3SkKWznuhCaCvz1NThYkz359xW2ujvpHIrdqyeI8pBwerUJIAX0T+ vvrk+RYDyhP9PRVmQDDwD9UidWL4mnyKrSHTBz5QJk3HmMpVHe0yYH+ThrKoxi3CKJmr inpw+e9R6xGe82nedSpZmguHZUGMZE3QU1ugcAPdLyuuG10R5Gvpr25Xu8vRt0vXockz 4PP4kWjK553ViFBBnoLZmy6Y49WX6c8vkFEa1K/dMN2GKQatd3lqBTyEzNOa1V8bxCQQ mUeg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=KbTdePCQuY00Eh43/mGhCoyrcL0EDX5k9L0XaPYIsA0=; b=qkUTmDgy5QGB5DiUIq6p/gRu1y1UR7c0va2OcMqOye/LR1ZZxHQifknR2hXeU0QBIz //xF3eAp4ZqbzS2JR8cQR44oEGVz7eZE89QIc82jN3+Ecf9zKB3+GEg6cb0BvEGfdKUX AgoAHQnyCWW9AVhwRk6Nu1AvcRyuxCZUHDQ+YxSGrSef3pI5Jt4l4vWjXVbNe9kbfPJX 1BomFYKPFy0/pqurtdhJQ8/EhqTDTL01mCEyTFZWhF99GjJs1O0frSY0KQ0gjPIH4/AY 1KLddYEQgJfh2QL1CW7O/IVIoyo/qx5GD6Msg8Yp5CeJ/C5rnYnuSZb99FOjmIh478+f lW9A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=DNrYTrhc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id lv21-20020a170906bc9500b00947797f2840si1078731ejb.320.2023.03.31.09.35.52; Fri, 31 Mar 2023 09:36:16 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=DNrYTrhc; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233153AbjCaQQs (ORCPT <rfc822;dexuan.linux@gmail.com> + 99 others); Fri, 31 Mar 2023 12:16:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42198 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233391AbjCaQQa (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Fri, 31 Mar 2023 12:16:30 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA8C42658C for <linux-kernel@vger.kernel.org>; Fri, 31 Mar 2023 09:11:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1680279057; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KbTdePCQuY00Eh43/mGhCoyrcL0EDX5k9L0XaPYIsA0=; b=DNrYTrhcDpY+5WJqIl+waxiTz4PVv9bTa4MGMutlH6CHO6bxVH8cJuEPnDRMaLUmseHKM7 A3vwdzrc145tImcq6UJP4biY8cP6JfS+2GVhEMLjwt+SeT1I+o3johwwKvolHrlVagztYY XUl2hzO4a4SBgvO0cXmLGcu3oPCmHmk= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-615--qgn__EoP-eE5uusdvkQ2A-1; Fri, 31 Mar 2023 12:10:54 -0400 X-MC-Unique: -qgn__EoP-eE5uusdvkQ2A-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7FD543802AC1; Fri, 31 Mar 2023 16:10:52 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 63D4851FF; Fri, 31 Mar 2023 16:10:50 +0000 (UTC) From: David Howells <dhowells@redhat.com> To: Matthew Wilcox <willy@infradead.org>, "David S. Miller" <davem@davemloft.net>, Eric Dumazet <edumazet@google.com>, Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com> Cc: David Howells <dhowells@redhat.com>, Al Viro <viro@zeniv.linux.org.uk>, Christoph Hellwig <hch@infradead.org>, Jens Axboe <axboe@kernel.dk>, Jeff Layton <jlayton@kernel.org>, Christian Brauner <brauner@kernel.org>, Chuck Lever III <chuck.lever@oracle.com>, Linus Torvalds <torvalds@linux-foundation.org>, netdev@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Tom Herbert <tom@herbertland.com>, Tom Herbert <tom@quantonium.net> Subject: [PATCH v3 33/55] kcm: Support MSG_SPLICE_PAGES Date: Fri, 31 Mar 2023 17:08:52 +0100 Message-Id: <20230331160914.1608208-34-dhowells@redhat.com> In-Reply-To: <20230331160914.1608208-1-dhowells@redhat.com> References: <20230331160914.1608208-1-dhowells@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Spam-Status: No, score=-0.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1761901885533544298?= X-GMAIL-MSGID: =?utf-8?q?1761901885533544298?= |
Series |
splice, net: Replace sendpage with sendmsg(MSG_SPLICE_PAGES)
|
|
Commit Message
David Howells
March 31, 2023, 4:08 p.m. UTC
Make AF_KCM sendmsg() support MSG_SPLICE_PAGES. This causes pages to be
spliced from the source iterator if possible and copied otherwise.
This allows ->sendpage() to be replaced by something that can handle
multiple multipage folios in a single transaction.
Signed-off-by: David Howells <dhowells@redhat.com>
cc: Tom Herbert <tom@herbertland.com>
cc: Tom Herbert <tom@quantonium.net>
cc: Jakub Kicinski <kuba@kernel.org>
cc: Eric Dumazet <edumazet@google.com>
cc: "David S. Miller" <davem@davemloft.net>
cc: Paolo Abeni <pabeni@redhat.com>
cc: Jens Axboe <axboe@kernel.dk>
cc: Matthew Wilcox <willy@infradead.org>
cc: netdev@vger.kernel.org
---
net/kcm/kcmsock.c | 89 ++++++++++++++++++++++++++++++++++++++---------
1 file changed, 72 insertions(+), 17 deletions(-)
Comments
Hi Tom, I found a test program for AF_KCM: https://gist.githubusercontent.com/peo3/fd0e266a3852d3422c08854aba96bff5/raw/98e02e120bd4b4bc5d499c4510e5879bb3a023d7/kcm-sample.c I don't suppose you have a version that compiles? It seems that the userland BPF API has changed. Thanks, David
Okay, I have a test program for AF_KCM that builds and works up to a point. However, it doesn't seem to work for two reasons: (1) When it clones a socket with SIOCKCMCLONE, it doesn't set the LSM context on the new socket. This results in EACCES if, say, SELinux is enforcing. ioctl(8, SIOCPROTOPRIVATE, 0x7ffe17cc3b24) = 0 ioctl(9, SIOCPROTOPRIVATE, 0x7ffe17cc3b24) = -1 EACCES (Permission denied) from the SIOCKCMATTACH ioctl, so this won't work on a number of Linux distributions, such as Fedora and RHEL. (2) Assuming SELinux is set to non-enforcing mode, it then fails when trying to attach the cloned KCM socket to the TCP socket: ioctl(8, SIOCPROTOPRIVATE, 0x7ffddfb64f84) = 0 ioctl(9, SIOCPROTOPRIVATE, 0x7ffddfb64f84) = -1 EALREADY (Operation already in progress) again from the SIOCKCMATTACH ioctl. This seems to be because the TCP socket (csock in kcm_attach() in the kernel) has already got sk_user_data set from the first ioctl on fd 8: if (csk->sk_user_data) { write_unlock_bh(&csk->sk_callback_lock); kmem_cache_free(kcm_psockp, psock); err = -EALREADY; goto out; } Now, this could be a bug in the test program. Since both fds 8 and 9 should correspond to the same multiplexor, presumably the TCP socket only needs attaching once (note that the TCP socket is obtained from accept() in this case). David --- /* * A sample program of KCM. * * $ gcc -lbcc kcm-sample.c * $ ./a.out 10000 * * https://gist.github.com/dhowells/24b87fdf731884ed9ca19e9840c0c894 */ #include <err.h> #include <errno.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <unistd.h> #include <netdb.h> #include <poll.h> #include <sys/ioctl.h> #include <sys/socket.h> #include <sys/types.h> #include <sys/wait.h> #include <netinet/in.h> /* libbcc */ #include <bcc/bcc_common.h> #include <bcc/libbpf.h> #include <bpf/bpf.h> #include <linux/bpf.h> /* From linux/kcm.h */ struct kcm_clone { int fd; }; struct kcm_attach { int fd; int bpf_fd; }; #ifndef AF_KCM /* From linux/socket.h */ #define AF_KCM 41 /* Kernel Connection Multiplexor*/ #endif #ifndef KCMPROTO_CONNECTED /* From linux/kcm.h */ #define KCMPROTO_CONNECTED 0 #endif #ifndef SIOCKCMCLONE /* From linux/sockios.h */ #define SIOCPROTOPRIVATE 0x89E0 /* to 89EF */ /* From linux/kcm.h */ #define SIOCKCMATTACH (SIOCPROTOPRIVATE + 0) #define SIOCKCMCLONE (SIOCPROTOPRIVATE + 2) #endif struct my_proto { struct _hdr { uint32_t len; } hdr; char data[32]; }; const char *bpf_prog_string = " \ ssize_t bpf_prog1(struct __sk_buff *skb) \ { \ return load_half(skb, 0) + 4; \ } \ "; int servsock_init(int port) { int s, error; struct sockaddr_in addr; s = socket(AF_INET, SOCK_STREAM, 0); addr.sin_family = AF_INET; addr.sin_port = htons(port); addr.sin_addr.s_addr = INADDR_ANY; error = bind(s, (struct sockaddr *)&addr, sizeof(addr)); if (error == -1) err(EXIT_FAILURE, "bind"); error = listen(s, 10); if (error == -1) err(EXIT_FAILURE, "listen"); return s; } int bpf_init(void) { int fd, map_fd; void *mod; int key; long long value = 0; mod = bpf_module_create_c_from_string(bpf_prog_string, 0, NULL, 0, 0, NULL); fd = bcc_prog_load( BPF_PROG_TYPE_SOCKET_FILTER, "bpf_prog1", bpf_function_start(mod, "bpf_prog1"), bpf_function_size(mod, "bpf_prog1"), bpf_module_license(mod), bpf_module_kern_version(mod), 0, NULL, 0); if (fd == -1) exit(1); return fd; } void client(int port) { int s, error; struct sockaddr_in addr; struct hostent *host; struct my_proto my_msg; int len; printf("client is starting\n"); s = socket(AF_INET, SOCK_STREAM, 0); if (s == -1) err(EXIT_FAILURE, "socket"); memset(&addr, 0, sizeof(addr)); addr.sin_family = AF_INET; addr.sin_port = htons(port); host = gethostbyname("localhost"); if (host == NULL) err(EXIT_FAILURE, "gethostbyname"); memcpy(&addr.sin_addr, host->h_addr, host->h_length); error = connect(s, (struct sockaddr *)&addr, sizeof(addr)); if (error == -1) err(EXIT_FAILURE, "connect"); len = sprintf(my_msg.data, "hello"); my_msg.data[len] = '\0'; my_msg.hdr.len = htons(len + 1); len = write(s, &my_msg, sizeof(my_msg.hdr) + len + 1); if (error == -1) err(EXIT_FAILURE, "write"); printf("client sent data\n"); printf("client is waiting a reply\n"); len = read(s, &my_msg, sizeof(my_msg)); if (error == -1) err(EXIT_FAILURE, "read"); printf("\"%s\" from server\n", my_msg.data); printf("client received data\n"); close(s); } int kcm_init(void) { int kcmfd; kcmfd = socket(AF_KCM, SOCK_DGRAM, KCMPROTO_CONNECTED); if (kcmfd == -1) err(EXIT_FAILURE, "socket(AF_KCM)"); return kcmfd; } int kcm_clone(int kcmfd) { int error; struct kcm_clone clone_info; memset(&clone_info, 0, sizeof(clone_info)); error = ioctl(kcmfd, SIOCKCMCLONE, &clone_info); if (error == -1) err(EXIT_FAILURE, "ioctl(SIOCKCMCLONE)"); return clone_info.fd; } int kcm_attach(int kcmfd, int csock, int bpf_prog_fd) { int error; struct kcm_attach attach_info; memset(&attach_info, 0, sizeof(attach_info)); attach_info.fd = csock; attach_info.bpf_fd = bpf_prog_fd; error = ioctl(kcmfd, SIOCKCMATTACH, &attach_info); if (error == -1) err(EXIT_FAILURE, "ioctl(SIOCKCMATTACH)"); } void process(int kcmfd0, int kcmfd1) { struct my_proto my_msg; int error, len; struct pollfd fds[2]; struct msghdr msg; struct iovec iov; int fd; fds[0].fd = kcmfd0; fds[0].events = POLLIN; fds[0].revents = 0; fds[1].fd = kcmfd1; fds[1].events = POLLIN; fds[1].revents = 0; printf("server is waiting data\n"); error = poll(fds, 1, -1); if (error == -1) err(EXIT_FAILURE, "poll"); if (fds[0].revents & POLLIN) fd = fds[0].fd; else if (fds[1].revents & POLLIN) fd = fds[1].fd; iov.iov_base = &my_msg; iov.iov_len = sizeof(my_msg); memset(&msg, 0, sizeof(msg)); msg.msg_iov = &iov; msg.msg_iovlen = 1; printf("server is receiving data\n"); len = recvmsg(fd, &msg, 0); if (len == -1) err(EXIT_FAILURE, "recvmsg"); printf("\"%s\" from client\n", my_msg.data); printf("server received data\n"); len = sprintf(my_msg.data, "goodbye"); my_msg.data[len] = '\0'; my_msg.hdr.len = htons(len + 1); len = sendmsg(fd, &msg, 0); if (len == -1) err(EXIT_FAILURE, "sendmsg"); } void server(int tcpfd, int bpf_prog_fd) { int kcmfd0, error, kcmfd1; struct sockaddr_in client; int len, csock; printf("server is starting\n"); kcmfd0 = kcm_init(); kcmfd1 = kcm_clone(kcmfd0); len = sizeof(client); csock = accept(tcpfd, (struct sockaddr *)&client, &len); if (csock == -1) err(EXIT_FAILURE, "accept"); kcm_attach(kcmfd0, csock, bpf_prog_fd); kcm_attach(kcmfd1, csock, bpf_prog_fd); process(kcmfd0, kcmfd1); close(kcmfd0); close(kcmfd1); } int main(int argc, char **argv) { int error, tcpfd, bpf_prog_fd; pid_t pid; int pipefd[2]; int dummy; if (argc != 2) { fprintf(stderr, "Format %s <port>\n", argv[0]); exit(2); } error = pipe(pipefd); if (error == -1) err(EXIT_FAILURE, "pipe"); pid = fork(); if (pid == -1) err(EXIT_FAILURE, "fork"); if (pid == 0) { /* wait for server's ready */ read(pipefd[0], &dummy, sizeof(dummy)); client(atoi(argv[1])); exit(0); } tcpfd = servsock_init(atoi(argv[1])); bpf_prog_fd = bpf_init(); /* tell ready */ write(pipefd[1], &dummy, sizeof(dummy)); server(tcpfd, bpf_prog_fd); waitpid(pid, NULL, 0); close(bpf_prog_fd); close(tcpfd); return 0; }
diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c index cfe828bd7fc6..0a3f79d81595 100644 --- a/net/kcm/kcmsock.c +++ b/net/kcm/kcmsock.c @@ -989,29 +989,84 @@ static int kcm_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) merge = false; } - copy = min_t(int, msg_data_left(msg), - pfrag->size - pfrag->offset); + if (msg->msg_flags & MSG_SPLICE_PAGES) { + struct page *page = NULL, **pages = &page; + size_t off; + bool put = false; + + err = iov_iter_extract_pages(&msg->msg_iter, &pages, + INT_MAX, 1, 0, &off); + if (err <= 0) { + err = err ?: -EIO; + goto out_error; + } + copy = err; - if (!sk_wmem_schedule(sk, copy)) - goto wait_for_memory; + if (skb_can_coalesce(skb, i, page, off)) { + skb_frag_size_add(&skb_shinfo(skb)->frags[i - 1], copy); + goto coalesced; + } - err = skb_copy_to_page_nocache(sk, &msg->msg_iter, skb, - pfrag->page, - pfrag->offset, - copy); - if (err) - goto out_error; + if (!sk_wmem_schedule(sk, copy)) { + iov_iter_revert(&msg->msg_iter, copy); + goto wait_for_memory; + } + + if (!sendpage_ok(page)) { + const void *p = kmap_local_page(page); + void *q; - /* Update the skb. */ - if (merge) { - skb_frag_size_add(&skb_shinfo(skb)->frags[i - 1], copy); + q = page_frag_memdup(NULL, p + off, copy, + sk->sk_allocation, ULONG_MAX); + kunmap_local(p); + if (!q) { + iov_iter_revert(&msg->msg_iter, copy); + err = -ENOMEM; + goto out_error; + } + page = virt_to_page(q); + off = offset_in_page(q); + put = true; + } + + skb_fill_page_desc_noacc(skb, i, page, off, copy); + if (put) + put_page(page); +coalesced: + skb_shinfo(skb)->flags |= SKBFL_SHARED_FRAG; + skb->len += copy; + skb->data_len += copy; + skb->truesize += copy; + sk->sk_wmem_queued += copy; + sk_mem_charge(sk, copy); + + if (head != skb) + head->truesize += copy; } else { - skb_fill_page_desc(skb, i, pfrag->page, - pfrag->offset, copy); - get_page(pfrag->page); + copy = min_t(int, msg_data_left(msg), + pfrag->size - pfrag->offset); + if (!sk_wmem_schedule(sk, copy)) + goto wait_for_memory; + + err = skb_copy_to_page_nocache(sk, &msg->msg_iter, skb, + pfrag->page, + pfrag->offset, + copy); + if (err) + goto out_error; + + /* Update the skb. */ + if (merge) { + skb_frag_size_add(&skb_shinfo(skb)->frags[i - 1], copy); + } else { + skb_fill_page_desc(skb, i, pfrag->page, + pfrag->offset, copy); + get_page(pfrag->page); + } + + pfrag->offset += copy; } - pfrag->offset += copy; copied += copy; if (head != skb) { head->len += copy;