From patchwork Mon Jul 17 21:00:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 121614 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c923:0:b0:3e4:2afc:c1 with SMTP id j3csp1359238vqt; Mon, 17 Jul 2023 14:42:51 -0700 (PDT) X-Google-Smtp-Source: APBJJlHCOYGGl6PPEDEfXZN5Bva0phP9VPvlXslrzzlJ1gvyWqRxEj2I4TPXFXJOV8Bo420oDwsn X-Received: by 2002:a05:6402:744:b0:50b:c693:70af with SMTP id p4-20020a056402074400b0050bc69370afmr11606623edy.2.1689630170866; Mon, 17 Jul 2023 14:42:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689630170; cv=none; d=google.com; s=arc-20160816; b=UKSHujRVdFrABE0Dc1Qx+agwkL8VZbXwabZ5+Vp+3Ph9EnU9YvxV0WhYRTHq8vUKAj 1VN/ObqGTessjjH8SonWFy3j1J3nLVUIA4Z8gd8MYJjgiHHTEJZ1Zx07zicdMYAlfhCm 2SfkXE+dkOPW6XzjNHlg6aFaAhkGzR8lL2g39fMJzCcJoH5VVVrDKpTiauHf+pSekSKD GGf9vo5L0zd3PM5tYMb7K5H2lwHnsP9gNMeJE0n+iChqPT8pEgyMb0eGy7kmWLXicqcF OAb5CnSk1aDxN8jij/VY+8q1SoEYarTLJdnLsbQX8lyaiGAAW+Uv7vHkrYqtYQkD77Z2 0IjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=cd1IOAcuYzKWX8jCc2ym7vhBB+ONo5JKwu2/WLdhWSw=; fh=IiLlUAYZQ4hWuQbi3V9KE00VdjFsw4tg4Y6JCUFRLn0=; b=BNKYKoZiPHqojA3EWF80cbGsVfpj9RnhjD0fXxoo89mKBzVpddCJv9kXNlU6EZEdyJ nYqynqlarzE8mGBqS11l1tO+IHPHNq0fJO3j2sn821XClYRDlIu/TknNZ2pnNdGCONj6 l396bBESThzC9arRVLeN4Ac3Zczf1idyik81Kd0Z/P7dPwJaJhcJD8U3DCaR0iGSZ5wK NWpA8uJazIjWZOD5i8zaRzWJKPkl1YyZlICL+Wv9aoJWYhBsEN289KST9mVRhe66nFaO i1xwsxwPQ4bC7jbrF5MgCGpoNYvDmVmETdxshDCiXaIeq9gmxgj8+UI6b4SsxZ37BPmA 1UUg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=Mrcy55bf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id t18-20020a05640203d200b005219817ce6bsi185621edw.88.2023.07.17.14.42.26; Mon, 17 Jul 2023 14:42:50 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=Mrcy55bf; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230297AbjGQVGl (ORCPT + 99 others); Mon, 17 Jul 2023 17:06:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49146 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231197AbjGQVGa (ORCPT ); Mon, 17 Jul 2023 17:06:30 -0400 Received: from mx1.sberdevices.ru (mx1.sberdevices.ru [37.18.73.165]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 537E2118; Mon, 17 Jul 2023 14:06:28 -0700 (PDT) Received: from p-infra-ksmg-sc-msk01 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id 4B2F2100011; Tue, 18 Jul 2023 00:06:26 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru 4B2F2100011 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1689627986; bh=cd1IOAcuYzKWX8jCc2ym7vhBB+ONo5JKwu2/WLdhWSw=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=Mrcy55bfLFbLLvpRvwd83v57Tp8ZGr2+g3dd+dQ+3yVaJCK3lPbJf2sPoRoJtTx1J 1kWLkqEjrhBrpzhOIubxhippkRtpbGRXi+S1WQMRnRZqREwwEB/7NxcDLTLADruAUL bhkE7qBujk3RWmmfYItQ3H51SOSi8B1/z8crN+p410Tz0rOz0Pkf7CEiWoiMUYHdfg 3bKzcVwFRNM5qp9H7kU9qwoplZ7b9CHGm5mWTDLGI9FueYUT3e/OLtxirjY02lkr3B N81lLdk7xP3paNQYZHHU2WrZpY0aHEiBkcbobBGHlFe2UBmWicrF9BijDNasZjsWHQ 1Uo/TPhbOftSw== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Tue, 18 Jul 2023 00:06:26 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Tue, 18 Jul 2023 00:06:25 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [PATCH net-next v1 1/4] vsock/virtio/vhost: read data from non-linear skb Date: Tue, 18 Jul 2023 00:00:48 +0300 Message-ID: <20230717210051.856388-2-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230717210051.856388-1-AVKrasnov@sberdevices.ru> References: <20230717210051.856388-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178666 [Jul 17 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 523 523 523027ce26ed1d9067f7a52a4756a876e54db27c, {Tracking_from_domain_doesnt_match_to}, 100.64.160.123:7.1.2;sberdevices.ru:5.0.1,7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;127.0.0.199:7.1.2;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/17 13:15:00 #21627216 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1771705646261444817 X-GMAIL-MSGID: 1771705646261444817 This is preparation patch for MSG_ZEROCOPY support. It adds handling of non-linear skbs by replacing direct calls of 'memcpy_to_msg()' with 'skb_copy_datagram_iter()'. Main advantage of the second one is that it can handle paged part of the skb by using 'kmap()' on each page, but if there are no pages in the skb, it behaves like simple copying to iov iterator. This patch also adds new field to the control block of skb - this value shows current offset in the skb to read next portion of data (it doesn't matter linear it or not). Idea behind this field is that 'skb_copy_datagram_iter()' handles both types of skb internally - it just needs an offset from which to copy data from the given skb. This offset is incremented on each read from skb. This approach allows to avoid special handling of non-linear skbs: 1) We can't call 'skb_pull()' on it, because it updates 'data' pointer. 2) We need to update 'data_len' also on each read from this skb. Signed-off-by: Arseniy Krasnov Reviewed-by: Stefano Garzarella --- Changelog: v5(big patchset) -> v1: * Merge 'virtio_transport_common.c' and 'vhost/vsock.c' patches into this single patch. * Commit message update: grammar fix and remark that this patch is MSG_ZEROCOPY preparation. * Use 'min_t()' instead of comparison using '<>' operators. drivers/vhost/vsock.c | 14 ++++++++----- include/linux/virtio_vsock.h | 1 + net/vmw_vsock/virtio_transport_common.c | 27 ++++++++++++++++--------- 3 files changed, 28 insertions(+), 14 deletions(-) diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c index 817d377a3f36..8c917be32b5d 100644 --- a/drivers/vhost/vsock.c +++ b/drivers/vhost/vsock.c @@ -114,6 +114,7 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock, struct sk_buff *skb; unsigned out, in; size_t nbytes; + u32 frag_off; int head; skb = virtio_vsock_skb_dequeue(&vsock->send_pkt_queue); @@ -156,7 +157,8 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock, } iov_iter_init(&iov_iter, ITER_DEST, &vq->iov[out], in, iov_len); - payload_len = skb->len; + frag_off = VIRTIO_VSOCK_SKB_CB(skb)->frag_off; + payload_len = skb->len - frag_off; hdr = virtio_vsock_hdr(skb); /* If the packet is greater than the space available in the @@ -197,8 +199,10 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock, break; } - nbytes = copy_to_iter(skb->data, payload_len, &iov_iter); - if (nbytes != payload_len) { + if (skb_copy_datagram_iter(skb, + frag_off, + &iov_iter, + payload_len)) { kfree_skb(skb); vq_err(vq, "Faulted on copying pkt buf\n"); break; @@ -212,13 +216,13 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock, vhost_add_used(vq, head, sizeof(*hdr) + payload_len); added = true; - skb_pull(skb, payload_len); + VIRTIO_VSOCK_SKB_CB(skb)->frag_off += payload_len; total_len += payload_len; /* If we didn't send all the payload we can requeue the packet * to send it with the next available buffer. */ - if (skb->len > 0) { + if (VIRTIO_VSOCK_SKB_CB(skb)->frag_off < skb->len) { hdr->flags |= cpu_to_le32(flags_to_restore); /* We are queueing the same skb to handle diff --git a/include/linux/virtio_vsock.h b/include/linux/virtio_vsock.h index c58453699ee9..17dbb7176e37 100644 --- a/include/linux/virtio_vsock.h +++ b/include/linux/virtio_vsock.h @@ -12,6 +12,7 @@ struct virtio_vsock_skb_cb { bool reply; bool tap_delivered; + u32 frag_off; }; #define VIRTIO_VSOCK_SKB_CB(skb) ((struct virtio_vsock_skb_cb *)((skb)->cb)) diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c index b769fc258931..1a376f808ae6 100644 --- a/net/vmw_vsock/virtio_transport_common.c +++ b/net/vmw_vsock/virtio_transport_common.c @@ -355,7 +355,7 @@ virtio_transport_stream_do_peek(struct vsock_sock *vsk, spin_lock_bh(&vvs->rx_lock); skb_queue_walk_safe(&vvs->rx_queue, skb, tmp) { - off = 0; + off = VIRTIO_VSOCK_SKB_CB(skb)->frag_off; if (total == len) break; @@ -370,7 +370,10 @@ virtio_transport_stream_do_peek(struct vsock_sock *vsk, */ spin_unlock_bh(&vvs->rx_lock); - err = memcpy_to_msg(msg, skb->data + off, bytes); + err = skb_copy_datagram_iter(skb, off, + &msg->msg_iter, + bytes); + if (err) goto out; @@ -413,25 +416,28 @@ virtio_transport_stream_do_dequeue(struct vsock_sock *vsk, while (total < len && !skb_queue_empty(&vvs->rx_queue)) { skb = skb_peek(&vvs->rx_queue); - bytes = len - total; - if (bytes > skb->len) - bytes = skb->len; + bytes = min_t(size_t, len - total, + skb->len - VIRTIO_VSOCK_SKB_CB(skb)->frag_off); /* sk_lock is held by caller so no one else can dequeue. * Unlock rx_lock since memcpy_to_msg() may sleep. */ spin_unlock_bh(&vvs->rx_lock); - err = memcpy_to_msg(msg, skb->data, bytes); + err = skb_copy_datagram_iter(skb, + VIRTIO_VSOCK_SKB_CB(skb)->frag_off, + &msg->msg_iter, bytes); + if (err) goto out; spin_lock_bh(&vvs->rx_lock); total += bytes; - skb_pull(skb, bytes); - if (skb->len == 0) { + VIRTIO_VSOCK_SKB_CB(skb)->frag_off += bytes; + + if (skb->len == VIRTIO_VSOCK_SKB_CB(skb)->frag_off) { u32 pkt_len = le32_to_cpu(virtio_vsock_hdr(skb)->len); virtio_transport_dec_rx_pkt(vvs, pkt_len); @@ -503,7 +509,10 @@ static int virtio_transport_seqpacket_do_dequeue(struct vsock_sock *vsk, */ spin_unlock_bh(&vvs->rx_lock); - err = memcpy_to_msg(msg, skb->data, bytes_to_copy); + err = skb_copy_datagram_iter(skb, 0, + &msg->msg_iter, + bytes_to_copy); + if (err) { /* Copy of message failed. Rest of * fragments will be freed without copy. From patchwork Mon Jul 17 21:00:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 121607 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c923:0:b0:3e4:2afc:c1 with SMTP id j3csp1350150vqt; Mon, 17 Jul 2023 14:19:21 -0700 (PDT) X-Google-Smtp-Source: APBJJlFFnlIKA65JAC4T5Wa11toQAIuTzWdkmDrjBVS/SLUSiUxKAUSmv5NWn/heSnAxMHRSWfQa X-Received: by 2002:a17:903:22d0:b0:1b3:ec08:f8e2 with SMTP id y16-20020a17090322d000b001b3ec08f8e2mr14625952plg.49.1689628761015; Mon, 17 Jul 2023 14:19:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689628761; cv=none; d=google.com; s=arc-20160816; b=QCcB+6Z5YG9jr/TwhTqQKSAW4rcPWJK59vACeMXxvTraL7b046aRHdp4KeSrxZ1foO 04kAeYsTwlIp0Bt8JgQrg9yGwSnClEOIjeIf6kxdnaBq2qUxDgHTxIvgfsrxJBv9Uowi uQfEyf06d3E72Qt1U3a0ichsP2knN+wmKcQuDspTq3DNkgKML1CqawcBtbg/rTG0aNTM AVVMG9ZZukN/x1nAfPFqjB+QgLpCciNEUhhziYGNKY/6hu50gcTOTJibkVpU+x59305h mC2f/LtJ4FqRYStrfmNH7SHasxOe8YvYEKgT/HiLlrJQs+qQ8q5K1Doadbz5TS5rnyBu f9Dw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=tkRDIjgwoi7c8HuFjXoUTj2zb+RizwwV2GwfLVc5LoM=; fh=IiLlUAYZQ4hWuQbi3V9KE00VdjFsw4tg4Y6JCUFRLn0=; b=TAB2SXA3MX5xer4jYYVA4EG+ESYpRIctY8eR2NSzAaV9nG0XlvhQ6jPCqiUAaz7EI4 +2Fsh5wEwhd8u6iQ1Ao9TFByfkdghKlkaU1kwbCcOyxbXGyecT62xKqvrEqZlkD4OxuD Zvq4RGNv3Qy3nwdoXIonQUE9Z9YrqBtZl3+16MKWwaHvRXfCQPVQxEdsf3T7P8QRKsQE ioKP8E5lX4G4EGJUIFMbZlBAXNb35g6v4f8vV53wvGoJLtARzdVhiReAA++OX2MrFOjg 4AnETGY9ujql9OqQf6oaBfL7pgBj6gaTjJcbBqzDoN8ubVjJ0PmZ/h63A7HqaMJu/tSD 0ZiA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=DIQc9ZlI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l1-20020a170902d04100b001b06e17610fsi406056pll.321.2023.07.17.14.19.07; Mon, 17 Jul 2023 14:19:20 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=DIQc9ZlI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231342AbjGQVGd (ORCPT + 99 others); Mon, 17 Jul 2023 17:06:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49148 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229935AbjGQVGa (ORCPT ); Mon, 17 Jul 2023 17:06:30 -0400 Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7505810C4; Mon, 17 Jul 2023 14:06:28 -0700 (PDT) Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id 0B05512001B; Tue, 18 Jul 2023 00:06:27 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru 0B05512001B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1689627987; bh=tkRDIjgwoi7c8HuFjXoUTj2zb+RizwwV2GwfLVc5LoM=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=DIQc9ZlIEx9naGM3kgorMd06oFu39dW10QPSa1CUYFX8wrRg5XVCS9J3fT2mLMOpH Duh0/it+8cVjNXqoKq1KcyPLkQ49zkapiZYJLyBmYeOZAzQgiR0iO4i536qGemscOf fB/X7DCWF6Zi3m3AbNqARb16wULoloKFRrldNZsV8FtnGNU99H70SJsfFEPPZZjkpN e0Zd58tYP8tswwN3ZZj/BKhn3lkel7WQcB4ZPmRjzhK93Yo9HoYVgCOpZPcxH9y5GG 3znWRCb5E1C11c6c9Xjg1oTAvvF4XBvjZxwIZB3jvKzMSA23O5PihJX199ElcSBz1u 0yYTKZXxOK23A== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Tue, 18 Jul 2023 00:06:26 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Tue, 18 Jul 2023 00:06:25 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [PATCH net-next v1 2/4] vsock/virtio: support to send non-linear skb Date: Tue, 18 Jul 2023 00:00:49 +0300 Message-ID: <20230717210051.856388-3-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230717210051.856388-1-AVKrasnov@sberdevices.ru> References: <20230717210051.856388-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178666 [Jul 17 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 523 523 523027ce26ed1d9067f7a52a4756a876e54db27c, {Tracking_from_domain_doesnt_match_to}, 127.0.0.199:7.1.2;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;sberdevices.ru:5.0.1,7.1.1;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/17 13:15:00 #21627216 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1771704167685498292 X-GMAIL-MSGID: 1771704167685498292 For non-linear skb use its pages from fragment array as buffers in virtio tx queue. These pages are already pinned by 'get_user_pages()' during such skb creation. Signed-off-by: Arseniy Krasnov Reviewed-by: Stefano Garzarella --- net/vmw_vsock/virtio_transport.c | 40 +++++++++++++++++++++++++++----- 1 file changed, 34 insertions(+), 6 deletions(-) diff --git a/net/vmw_vsock/virtio_transport.c b/net/vmw_vsock/virtio_transport.c index e95df847176b..6cbb45bb12d2 100644 --- a/net/vmw_vsock/virtio_transport.c +++ b/net/vmw_vsock/virtio_transport.c @@ -100,7 +100,9 @@ virtio_transport_send_pkt_work(struct work_struct *work) vq = vsock->vqs[VSOCK_VQ_TX]; for (;;) { - struct scatterlist hdr, buf, *sgs[2]; + /* +1 is for packet header. */ + struct scatterlist *sgs[MAX_SKB_FRAGS + 1]; + struct scatterlist bufs[MAX_SKB_FRAGS + 1]; int ret, in_sg = 0, out_sg = 0; struct sk_buff *skb; bool reply; @@ -111,12 +113,38 @@ virtio_transport_send_pkt_work(struct work_struct *work) virtio_transport_deliver_tap_pkt(skb); reply = virtio_vsock_skb_reply(skb); + sg_init_one(&bufs[out_sg], virtio_vsock_hdr(skb), + sizeof(*virtio_vsock_hdr(skb))); + sgs[out_sg] = &bufs[out_sg]; + out_sg++; + + if (!skb_is_nonlinear(skb)) { + if (skb->len > 0) { + sg_init_one(&bufs[out_sg], skb->data, skb->len); + sgs[out_sg] = &bufs[out_sg]; + out_sg++; + } + } else { + struct skb_shared_info *si; + int i; + + si = skb_shinfo(skb); + + for (i = 0; i < si->nr_frags; i++) { + skb_frag_t *skb_frag = &si->frags[i]; + void *va = page_to_virt(skb_frag->bv_page); - sg_init_one(&hdr, virtio_vsock_hdr(skb), sizeof(*virtio_vsock_hdr(skb))); - sgs[out_sg++] = &hdr; - if (skb->len > 0) { - sg_init_one(&buf, skb->data, skb->len); - sgs[out_sg++] = &buf; + /* We will use 'page_to_virt()' for userspace page here, + * because virtio layer will call 'virt_to_phys()' later + * to fill buffer descriptor. We don't touch memory at + * "virtual" address of this page. + */ + sg_init_one(&bufs[out_sg], + va + skb_frag->bv_offset, + skb_frag->bv_len); + sgs[out_sg] = &bufs[out_sg]; + out_sg++; + } } ret = virtqueue_add_sgs(vq, sgs, out_sg, in_sg, skb, GFP_KERNEL); From patchwork Mon Jul 17 21:00:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 121606 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c923:0:b0:3e4:2afc:c1 with SMTP id j3csp1350012vqt; Mon, 17 Jul 2023 14:18:59 -0700 (PDT) X-Google-Smtp-Source: APBJJlFg0oRiksIztfcP7VBAQVuAP84tS7icx9mOXQ+GpXyLYeCE3bZFVSzCBTkaOm9iuywzHbxw X-Received: by 2002:a17:902:9a4b:b0:1b8:64a9:ea60 with SMTP id x11-20020a1709029a4b00b001b864a9ea60mr11364676plv.53.1689628739350; Mon, 17 Jul 2023 14:18:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689628739; cv=none; d=google.com; s=arc-20160816; b=wEn8AQ8bVNGJggTCtobLE+Gt8Ng5M6ppjKHRxaoQZ69BRRShxP4fJSXfATXq/gzAs2 FLgqn6a6O0qqCsvuQHObw9TjQV1oFhdni+RH5iTWKCnJsIYSoNT20/eM65Nq0bclidVE PcM5m5ABkg8f0ZBsDF9GhoqNICJ3I4YNXw5tPNqPxXkgEpsWsG8ACATIZj1KiI7fAdtD YM1aBJl0aZ8oh+DaJ7WOgOCiyqBmx2fwIts6GfilNUWEIUoY6iPSZCUZM2VtpTNCk0ZF DVM38zK2TaP4ZFQxTPsh34VK8zFal4JIvb/FduYp+fiFg7O1cmeLhW4aHY1acF6aAHPk jMlQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=ydsf/FQJHsRd7VM9sT5BCFNwDtuvxxMDFyVsLpjPMoM=; fh=IiLlUAYZQ4hWuQbi3V9KE00VdjFsw4tg4Y6JCUFRLn0=; b=U6NQRYsDeJu2SYX5dWoKAR+RNHLYjrNLc2b46nGZsD0/3NUurGT9a2hwJgxQbxwMHc 8Lw4be+wXpds+L24o1X61s9kAeFcUNgU1+H+2VO2nxc1DLoOl7LJjU+E7EVWbOXeya58 EHvmh12wMu4n1QGSARvL4oFAGDwXqItZFXla1oa+V4El/fMi4fsGibJamOZl/cObhh8y H3zheK8W5nHkthy7SFjU1VcLclt67XH9okLeNW4Ag9fGoLPeV2DUUESgBUy8xXJ9faKU 5NwtYPW4+hZLYeERc/mgVlPfmwQiTaBlhzSeMKdFI7YKXcgUuEF3rkcJoDbgMrA0NDYj 5lPA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=D7k6OrpH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q17-20020a170902dad100b001b85a47d2b7si468114plx.422.2023.07.17.14.18.45; Mon, 17 Jul 2023 14:18:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=D7k6OrpH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231361AbjGQVGg (ORCPT + 99 others); Mon, 17 Jul 2023 17:06:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49154 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230463AbjGQVGa (ORCPT ); Mon, 17 Jul 2023 17:06:30 -0400 Received: from mx1.sberdevices.ru (mx1.sberdevices.ru [37.18.73.165]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 16B6A10C7; Mon, 17 Jul 2023 14:06:29 -0700 (PDT) Received: from p-infra-ksmg-sc-msk01 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id AF783100014; Tue, 18 Jul 2023 00:06:27 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru AF783100014 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1689627987; bh=ydsf/FQJHsRd7VM9sT5BCFNwDtuvxxMDFyVsLpjPMoM=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=D7k6OrpHzQF+wJs6pKpyovgILTHKOLATgpbvzNYaf49uj6rEgCjI1U4oOxCA9Aglp PZE+2b8Ho/JO2ipGpYAHwZfJckcCq/XWdbt/G19o4VFcMuH22m/owszREODoiG5ieA Dwm+CXcswySVWCMp410awLpkuftHmBFRv+X1GNhvLUq2UowT45OO940MAGLw6vPIxe 0t0+e7iK+SFJZC1YdiEJkY37F6yAznUBMbcmLTSPikHp/qdeyEWvFayi8l6dcEfMpA 8yS8aqFLl8+RMBbZKbwpFakHa6WC0LRs+o0p1w7jROMir3s54AmP6MCiNNkOwick+q jaBLTVvXOd/5w== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Tue, 18 Jul 2023 00:06:27 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Tue, 18 Jul 2023 00:06:26 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [PATCH net-next v1 3/4] vsock/virtio: non-linear skb handling for tap Date: Tue, 18 Jul 2023 00:00:50 +0300 Message-ID: <20230717210051.856388-4-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230717210051.856388-1-AVKrasnov@sberdevices.ru> References: <20230717210051.856388-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178666 [Jul 17 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 523 523 523027ce26ed1d9067f7a52a4756a876e54db27c, {Tracking_from_domain_doesnt_match_to}, 100.64.160.123:7.1.2;sberdevices.ru:5.0.1,7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;127.0.0.199:7.1.2;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/17 13:15:00 #21627216 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1771704144855498742 X-GMAIL-MSGID: 1771704144855498742 For tap device new skb is created and data from the current skb is copied to it. This adds copying data from non-linear skb to new the skb. Signed-off-by: Arseniy Krasnov Reviewed-by: Stefano Garzarella --- net/vmw_vsock/virtio_transport_common.c | 31 ++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-) diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c index 1a376f808ae6..26a4d10da205 100644 --- a/net/vmw_vsock/virtio_transport_common.c +++ b/net/vmw_vsock/virtio_transport_common.c @@ -106,6 +106,27 @@ virtio_transport_alloc_skb(struct virtio_vsock_pkt_info *info, return NULL; } +static void virtio_transport_copy_nonlinear_skb(const struct sk_buff *skb, + void *dst, + size_t len) +{ + struct iov_iter iov_iter = { 0 }; + struct kvec kvec; + size_t to_copy; + + kvec.iov_base = dst; + kvec.iov_len = len; + + iov_iter.iter_type = ITER_KVEC; + iov_iter.kvec = &kvec; + iov_iter.nr_segs = 1; + + to_copy = min_t(size_t, len, skb->len); + + skb_copy_datagram_iter(skb, VIRTIO_VSOCK_SKB_CB(skb)->frag_off, + &iov_iter, to_copy); +} + /* Packet capture */ static struct sk_buff *virtio_transport_build_skb(void *opaque) { @@ -114,7 +135,6 @@ static struct sk_buff *virtio_transport_build_skb(void *opaque) struct af_vsockmon_hdr *hdr; struct sk_buff *skb; size_t payload_len; - void *payload_buf; /* A packet could be split to fit the RX buffer, so we can retrieve * the payload length from the header and the buffer pointer taking @@ -122,7 +142,6 @@ static struct sk_buff *virtio_transport_build_skb(void *opaque) */ pkt_hdr = virtio_vsock_hdr(pkt); payload_len = pkt->len; - payload_buf = pkt->data; skb = alloc_skb(sizeof(*hdr) + sizeof(*pkt_hdr) + payload_len, GFP_ATOMIC); @@ -165,7 +184,13 @@ static struct sk_buff *virtio_transport_build_skb(void *opaque) skb_put_data(skb, pkt_hdr, sizeof(*pkt_hdr)); if (payload_len) { - skb_put_data(skb, payload_buf, payload_len); + if (skb_is_nonlinear(pkt)) { + void *data = skb_put(skb, payload_len); + + virtio_transport_copy_nonlinear_skb(pkt, data, payload_len); + } else { + skb_put_data(skb, pkt->data, payload_len); + } } return skb; From patchwork Mon Jul 17 21:00:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 121615 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c923:0:b0:3e4:2afc:c1 with SMTP id j3csp1364015vqt; Mon, 17 Jul 2023 14:57:59 -0700 (PDT) X-Google-Smtp-Source: APBJJlFFVeMciLkRe/vWWh9SSvkPAj875mC2Akr5X4rpiscRZPeYIwVCenD4/qGLDP4WJetMhjag X-Received: by 2002:a05:6a00:cc4:b0:676:20f8:be41 with SMTP id b4-20020a056a000cc400b0067620f8be41mr14948019pfv.16.1689631079201; Mon, 17 Jul 2023 14:57:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689631079; cv=none; d=google.com; s=arc-20160816; b=d0inB7GjqYZUCHIjgKjCPS5Tn16kDQ2mGY/FJDI5WUYrsagC8tLFyBAkyNiigffM+h animDe1Spviiktbt3BAq0aCGOqoxT9kG1Hl7lLS7SperUMQlSXNYad4oH6/7B+CfQx46 +dVMrtqaQVzAmhT6A4UcX0F6rkTJLXqyjX+mEzvYcX9huI6qKGHLD9FHwrR6cmJHEwtS 8bJRawJKHFbveIJf80Gt5whIzaS+4IWx10I7aVqUxTRxbwPc1LNAWcE4jYg/nvgeUUlJ cYyN09CnJN7rGkreQoXfs/GFJL2EBNp4Z4IdF8RjOC9R77sALp2WFiKEqJSoVkf/GlW+ ICAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=+WCTkftOvzPrKMBqiHt1VoahHKl1KwYo/xOW7LbIL4g=; fh=IiLlUAYZQ4hWuQbi3V9KE00VdjFsw4tg4Y6JCUFRLn0=; b=n36ZUN9DQk5umxKtKRfX2oViGdcD1bR4CL4fZXTs764IgKEwHGi1ju/2g4cfIW+eUr U2mDTMWgOymG6KWCT4EvBhE6FGp4qzETXK8lcusdFkL9N5/fsNnddEDNaBRoKNDIRIhP 5zTmmhcm5RCVZUnAsOxy9cgEL751Fxt19y2i5MCLZWNdb0CLK/B2mvFInaAqzF4AjzT/ XHnIlsshfJid6PCscxbKIK6ljjHiXSoJ+3u5b8q3veFQ+MsgfQdlClKGOCP7R+pAfAl3 QcqMyulgBli9wmKB8KvX2p2cECSDCqYv/C2UJuU8f4nC3+BdM/JtuzkM0pCdnHoP/VRc HeEQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=l7AAznvb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s63-20020a637742000000b0054405637a7esi400735pgc.801.2023.07.17.14.57.45; Mon, 17 Jul 2023 14:57:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=l7AAznvb; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231363AbjGQVGo (ORCPT + 99 others); Mon, 17 Jul 2023 17:06:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49162 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229541AbjGQVGb (ORCPT ); Mon, 17 Jul 2023 17:06:31 -0400 Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CED25121; Mon, 17 Jul 2023 14:06:29 -0700 (PDT) Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id 7193512001C; Tue, 18 Jul 2023 00:06:28 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru 7193512001C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1689627988; bh=+WCTkftOvzPrKMBqiHt1VoahHKl1KwYo/xOW7LbIL4g=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=l7AAznvb2+3tfH7jhXx717h00sLfFJKSuEyAl6vFWLCFLUeet0GIIOTIQFANgmSS8 VU3NRBZWe+tHy5sRzVStMGiTMJAS3Jl1/OjuhcVmZKXfJDnaZEFfS/NJb+hf8LVYkM Jyt+3tOIOJP1N8nB3CqZUjBosuzBc2d17DDKtHyYFMklDqoy69UjxMsnmpMXWI/BMA QiwbhylAvtJfAjr9lvhTD/LqU/pmiSQDSDAt9i7v7gaZDDf1ifVZorGc1/XeulFTpf BQnFePHdd6mKqrXJ7UXZZR7YMXESpDwQjZKgGPIlXXZyY7v0lTRZA5b7JVGJcGV/nG XhvjFCLC/Reyw== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Tue, 18 Jul 2023 00:06:28 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Tue, 18 Jul 2023 00:06:27 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [PATCH net-next v1 4/4] vsock/virtio: MSG_ZEROCOPY flag support Date: Tue, 18 Jul 2023 00:00:51 +0300 Message-ID: <20230717210051.856388-5-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230717210051.856388-1-AVKrasnov@sberdevices.ru> References: <20230717210051.856388-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178666 [Jul 17 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 523 523 523027ce26ed1d9067f7a52a4756a876e54db27c, {Tracking_from_domain_doesnt_match_to}, 127.0.0.199:7.1.2;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;sberdevices.ru:5.0.1,7.1.1;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/17 13:15:00 #21627216 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1771706598492001631 X-GMAIL-MSGID: 1771706598492001631 This adds handling of MSG_ZEROCOPY flag on transmission path: if this flag is set and zerocopy transmission is possible, then non-linear skb will be created and filled with the pages of user's buffer. Pages of user's buffer are locked in memory by 'get_user_pages()'. Second thing that this patch does is replace type of skb owning: instead of calling 'skb_set_owner_sk_safe()' it calls 'skb_set_owner_w()'. Reason of this change is that '__zerocopy_sg_from_iter()' increments 'sk_wmem_alloc' of socket, so to decrease this field correctly proper skb destructor is needed: 'sock_wfree()'. This destructor is set by 'skb_set_owner_w()'. Signed-off-by: Arseniy Krasnov --- Changelog: v5(big patchset) -> v1: * Refactorings of 'if' conditions. * Remove extra blank line. * Remove 'frag_off' field unneeded init. * Add function 'virtio_transport_fill_skb()' which fills both linear and non-linear skb with provided data. net/vmw_vsock/virtio_transport_common.c | 260 ++++++++++++++++++------ 1 file changed, 197 insertions(+), 63 deletions(-) diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c index 26a4d10da205..1fb0a0f694c6 100644 --- a/net/vmw_vsock/virtio_transport_common.c +++ b/net/vmw_vsock/virtio_transport_common.c @@ -37,73 +37,115 @@ virtio_transport_get_ops(struct vsock_sock *vsk) return container_of(t, struct virtio_transport, transport); } -/* Returns a new packet on success, otherwise returns NULL. - * - * If NULL is returned, errp is set to a negative errno. - */ -static struct sk_buff * -virtio_transport_alloc_skb(struct virtio_vsock_pkt_info *info, - size_t len, - u32 src_cid, - u32 src_port, - u32 dst_cid, - u32 dst_port) -{ - const size_t skb_len = VIRTIO_VSOCK_SKB_HEADROOM + len; - struct virtio_vsock_hdr *hdr; - struct sk_buff *skb; - void *payload; - int err; +static bool virtio_transport_can_zcopy(struct virtio_vsock_pkt_info *info, + size_t max_to_send) +{ + struct iov_iter *iov_iter; - skb = virtio_vsock_alloc_skb(skb_len, GFP_KERNEL); - if (!skb) - return NULL; + if (!info->msg) + return false; - hdr = virtio_vsock_hdr(skb); - hdr->type = cpu_to_le16(info->type); - hdr->op = cpu_to_le16(info->op); - hdr->src_cid = cpu_to_le64(src_cid); - hdr->dst_cid = cpu_to_le64(dst_cid); - hdr->src_port = cpu_to_le32(src_port); - hdr->dst_port = cpu_to_le32(dst_port); - hdr->flags = cpu_to_le32(info->flags); - hdr->len = cpu_to_le32(len); + iov_iter = &info->msg->msg_iter; - if (info->msg && len > 0) { - payload = skb_put(skb, len); - err = memcpy_from_msg(payload, info->msg, len); - if (err) - goto out; + /* Data is simple buffer. */ + if (iter_is_ubuf(iov_iter)) + return true; - if (msg_data_left(info->msg) == 0 && - info->type == VIRTIO_VSOCK_TYPE_SEQPACKET) { - hdr->flags |= cpu_to_le32(VIRTIO_VSOCK_SEQ_EOM); + if (!iter_is_iovec(iov_iter)) + return false; - if (info->msg->msg_flags & MSG_EOR) - hdr->flags |= cpu_to_le32(VIRTIO_VSOCK_SEQ_EOR); - } + if (iov_iter->iov_offset) + return false; + + /* We can't send whole iov. */ + if (iov_iter->count > max_to_send) + return false; + + return true; +} + +static int virtio_transport_init_zcopy_skb(struct vsock_sock *vsk, + struct sk_buff *skb, + struct msghdr *msg, + bool zerocopy) +{ + struct ubuf_info *uarg; + + if (msg->msg_ubuf) { + uarg = msg->msg_ubuf; + net_zcopy_get(uarg); + } else { + struct iov_iter *iter = &msg->msg_iter; + struct ubuf_info_msgzc *uarg_zc; + int len; + + /* Only ITER_IOVEC or ITER_UBUF are allowed and + * checked before. + */ + if (iter_is_iovec(iter)) + len = iov_length(iter->__iov, iter->nr_segs); + else + len = iter->count; + + uarg = msg_zerocopy_realloc(sk_vsock(vsk), + len, + NULL); + if (!uarg) + return -1; + + uarg_zc = uarg_to_msgzc(uarg); + uarg_zc->zerocopy = zerocopy ? 1 : 0; } - if (info->reply) - virtio_vsock_skb_set_reply(skb); + skb_zcopy_init(skb, uarg); - trace_virtio_transport_alloc_pkt(src_cid, src_port, - dst_cid, dst_port, - len, - info->type, - info->op, - info->flags); + return 0; +} - if (info->vsk && !skb_set_owner_sk_safe(skb, sk_vsock(info->vsk))) { - WARN_ONCE(1, "failed to allocate skb on vsock socket with sk_refcnt == 0\n"); - goto out; +static int virtio_transport_fill_skb(struct sk_buff *skb, + struct virtio_vsock_pkt_info *info, + size_t len, + bool zcopy) +{ + if (zcopy) { + return __zerocopy_sg_from_iter(info->msg, NULL, skb, + &info->msg->msg_iter, + len); + } else { + void *payload; + int err; + + payload = skb_put(skb, len); + err = memcpy_from_msg(payload, info->msg, len); + if (err) + return -1; + + if (msg_data_left(info->msg)) + return 0; + + return 0; } +} - return skb; +static void virtio_transport_init_hdr(struct sk_buff *skb, + struct virtio_vsock_pkt_info *info, + u32 src_cid, + u32 src_port, + u32 dst_cid, + u32 dst_port, + size_t len) +{ + struct virtio_vsock_hdr *hdr; -out: - kfree_skb(skb); - return NULL; + hdr = virtio_vsock_hdr(skb); + hdr->type = cpu_to_le16(info->type); + hdr->op = cpu_to_le16(info->op); + hdr->src_cid = cpu_to_le64(src_cid); + hdr->dst_cid = cpu_to_le64(dst_cid); + hdr->src_port = cpu_to_le32(src_port); + hdr->dst_port = cpu_to_le32(dst_port); + hdr->flags = cpu_to_le32(info->flags); + hdr->len = cpu_to_le32(len); } static void virtio_transport_copy_nonlinear_skb(const struct sk_buff *skb, @@ -214,6 +256,70 @@ static u16 virtio_transport_get_type(struct sock *sk) return VIRTIO_VSOCK_TYPE_SEQPACKET; } +static struct sk_buff *virtio_transport_alloc_skb(struct vsock_sock *vsk, + struct virtio_vsock_pkt_info *info, + size_t payload_len, + bool zcopy, + u32 dst_cid, + u32 dst_port, + u32 src_cid, + u32 src_port) +{ + struct sk_buff *skb; + size_t skb_len; + + skb_len = VIRTIO_VSOCK_SKB_HEADROOM; + + if (!zcopy) + skb_len += payload_len; + + skb = virtio_vsock_alloc_skb(skb_len, GFP_KERNEL); + if (!skb) + return NULL; + + virtio_transport_init_hdr(skb, info, src_cid, src_port, + dst_cid, dst_port, + payload_len); + + /* Set owner here, because '__zerocopy_sg_from_iter()' uses + * owner of skb without check to update 'sk_wmem_alloc'. + */ + if (vsk) + skb_set_owner_w(skb, sk_vsock(vsk)); + + if (info->msg && payload_len > 0) { + int err; + + err = virtio_transport_fill_skb(skb, info, payload_len, zcopy); + if (err) + goto out; + + if (info->type == VIRTIO_VSOCK_TYPE_SEQPACKET) { + struct virtio_vsock_hdr *hdr = virtio_vsock_hdr(skb); + + hdr->flags |= cpu_to_le32(VIRTIO_VSOCK_SEQ_EOM); + + if (info->msg->msg_flags & MSG_EOR) + hdr->flags |= cpu_to_le32(VIRTIO_VSOCK_SEQ_EOR); + } + } + + if (info->reply) + virtio_vsock_skb_set_reply(skb); + + trace_virtio_transport_alloc_pkt(src_cid, src_port, + dst_cid, dst_port, + payload_len, + info->type, + info->op, + info->flags); + + return skb; +out: + kfree_skb(skb); + return NULL; +} + /* This function can only be used on connecting/connected sockets, * since a socket assigned to a transport is required. * @@ -222,10 +328,12 @@ static u16 virtio_transport_get_type(struct sock *sk) static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, struct virtio_vsock_pkt_info *info) { + u32 max_skb_len = VIRTIO_VSOCK_MAX_PKT_BUF_SIZE; u32 src_cid, src_port, dst_cid, dst_port; const struct virtio_transport *t_ops; struct virtio_vsock_sock *vvs; u32 pkt_len = info->pkt_len; + bool can_zcopy = false; u32 rest_len; int ret; @@ -254,22 +362,48 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, if (pkt_len == 0 && info->op == VIRTIO_VSOCK_OP_RW) return pkt_len; + if (info->msg) { + /* If zerocopy is not enabled by 'setsockopt()', we behave as + * there is no MSG_ZEROCOPY flag set. + */ + if (!sock_flag(sk_vsock(vsk), SOCK_ZEROCOPY)) + info->msg->msg_flags &= ~MSG_ZEROCOPY; + + if (info->msg->msg_flags & MSG_ZEROCOPY) + can_zcopy = virtio_transport_can_zcopy(info, pkt_len); + + if (can_zcopy) + max_skb_len = min_t(u32, VIRTIO_VSOCK_MAX_PKT_BUF_SIZE, + (MAX_SKB_FRAGS * PAGE_SIZE)); + } + rest_len = pkt_len; do { struct sk_buff *skb; size_t skb_len; - skb_len = min_t(u32, VIRTIO_VSOCK_MAX_PKT_BUF_SIZE, rest_len); + skb_len = min(max_skb_len, rest_len); - skb = virtio_transport_alloc_skb(info, skb_len, - src_cid, src_port, - dst_cid, dst_port); + skb = virtio_transport_alloc_skb(vsk, info, skb_len, can_zcopy, + dst_cid, dst_port, + src_cid, src_port); if (!skb) { ret = -ENOMEM; break; } + /* This is last skb to send this portion of data. */ + if (info->msg && info->msg->msg_flags & MSG_ZEROCOPY && + skb_len == rest_len && info->op == VIRTIO_VSOCK_OP_RW) { + if (virtio_transport_init_zcopy_skb(vsk, skb, + info->msg, + can_zcopy)) { + ret = -ENOMEM; + break; + } + } + virtio_transport_inc_tx_pkt(vvs, skb); ret = t_ops->send_pkt(skb); @@ -934,11 +1068,11 @@ static int virtio_transport_reset_no_sock(const struct virtio_transport *t, if (!t) return -ENOTCONN; - reply = virtio_transport_alloc_skb(&info, 0, - le64_to_cpu(hdr->dst_cid), - le32_to_cpu(hdr->dst_port), + reply = virtio_transport_alloc_skb(NULL, &info, 0, false, le64_to_cpu(hdr->src_cid), - le32_to_cpu(hdr->src_port)); + le32_to_cpu(hdr->src_port), + le64_to_cpu(hdr->dst_cid), + le32_to_cpu(hdr->dst_port)); if (!reply) return -ENOMEM;