From patchwork Tue Jul 18 18:02:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 122189 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c923:0:b0:3e4:2afc:c1 with SMTP id j3csp1926695vqt; Tue, 18 Jul 2023 11:17:29 -0700 (PDT) X-Google-Smtp-Source: APBJJlFaImyKYZdFvh+sIxQRD02VVz5u8F+eor4rAHuNlmQDi46hXpfhZST/ZSQMPlrcKq6Z5h+k X-Received: by 2002:a05:6512:1150:b0:4f8:49a8:a0e2 with SMTP id m16-20020a056512115000b004f849a8a0e2mr12616766lfg.16.1689704249597; Tue, 18 Jul 2023 11:17:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689704249; cv=none; d=google.com; s=arc-20160816; b=tqVt2zO237MdPSoeUg1IKYBnYmnYXZPzXn8+5cJF8JqmkFGUUIhVB0UmFey6aYaKTN 9o83CV0ujSBh1FVDJki/oFbtN6MC9ERh0uVYf6RpPRaZT/EFaT+et5D7CZ9zJFL6XRmS /Q2N2CWl+PKl7OOw840VhA8c9Nzlgs+kxh2sec2wW+tzLYQARRxjAXAILrKqbI3TVE2Q f6btanWjWlagkCaioxYnscjbEi8lNZR3Kesq9vtzvLQYJETH6rVgK/CUjtUmXFaHUd8K MjIxJF9Il+JmEk/LwaoING74W1HCcRBmfe/cK5C7Vwb+v0KiP/Xkj13ckMqDjgFYWLcL GoQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=orXX14OPfIpbdl+OCofg0u3TAslJ4Vpwl3Ko0PpRkoo=; fh=IiLlUAYZQ4hWuQbi3V9KE00VdjFsw4tg4Y6JCUFRLn0=; b=mJo3SxCdCikXq8JJKvpb6tjnXx6tiitybk6ILKvVAn6xRS1HjtosecEcq+GK5q8ZIh 9ov2kYS23/6AlZm7iCxjEXoUe4KwcRRLDCJmUcp1r8p4wSzemkUeqbPwv7FxmMAG5+Rx 0kak9ifkrYoZgowO8jCg0MbFtMcMaHqQDxvQ35Z1HV9Sl2YMvoCqbYamxJLI70KOx25f W3dFHCzyjWp1F/7ZFQQrmv07dYuzif18SGPgQF3WCq4KcJpCryHicpsm9goPmdc9P36W 4dnTiEyJNEo8IFU7QHBB1XJH1wImOkvfCjia9jWzf6vIe84VNKM6IbYq5SqBd1/Tt/jK W+WQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=DWsNDh8E; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b13-20020aa7d48d000000b0051de5968bd8si1641926edr.443.2023.07.18.11.17.05; Tue, 18 Jul 2023 11:17:29 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=DWsNDh8E; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231352AbjGRSI0 (ORCPT + 99 others); Tue, 18 Jul 2023 14:08:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232261AbjGRSIW (ORCPT ); Tue, 18 Jul 2023 14:08:22 -0400 Received: from mx1.sberdevices.ru (mx1.sberdevices.ru [37.18.73.165]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B5CECBD; Tue, 18 Jul 2023 11:08:19 -0700 (PDT) Received: from p-infra-ksmg-sc-msk01 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id B18D9100015; Tue, 18 Jul 2023 21:08:17 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru B18D9100015 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1689703697; bh=orXX14OPfIpbdl+OCofg0u3TAslJ4Vpwl3Ko0PpRkoo=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=DWsNDh8EyKKDlXduPHGWbR3nW6Ul9ozHPbriaTRdeI4Hfm8W9PpTqjKRsyDwjVoet Xw0H7RiSnKBU9frnb6+WrB9UFfop5Q7PRJQzIz1fyybojQpUESxRn0/eZGOi4E3Xz8 eiyMeveXWBQT2gsg6SYZb797NzQg91/F6K9dp512HMJfWsKE9PbNM2WFsEiN4EbJH4 r4xAvXrBlvUq1R5oiWv5paOPtm/wOBZzdyp3NqB8TBeFPe0/oXB6sed/8ws8ZSDsYV uEvruieSfl0UOXzuA7k63K9tqKrnW1qu0Q83dDcqVehf8KX7acWZ7hT0v66dpWTCJv tU+ZFvlmTih7g== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Tue, 18 Jul 2023 21:08:17 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Tue, 18 Jul 2023 21:08:16 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [PATCH net-next v2 1/4] vsock/virtio/vhost: read data from non-linear skb Date: Tue, 18 Jul 2023 21:02:34 +0300 Message-ID: <20230718180237.3248179-2-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230718180237.3248179-1-AVKrasnov@sberdevices.ru> References: <20230718180237.3248179-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m02.sberdevices.ru (172.16.192.103) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178700 [Jul 18 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 524 524 9753033d6953787301affc41bead8ed49c47b39d, {Tracking_from_domain_doesnt_match_to}, p-i-exch-sc-m01.sberdevices.ru:7.1.1,5.0.1;127.0.0.199:7.1.2;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;sberdevices.ru:7.1.1,5.0.1;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/18 13:44:00 #21634370 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1771783323145394465 X-GMAIL-MSGID: 1771783323145394465 This is preparation patch for MSG_ZEROCOPY support. It adds handling of non-linear skbs by replacing direct calls of 'memcpy_to_msg()' with 'skb_copy_datagram_iter()'. Main advantage of the second one is that it can handle paged part of the skb by using 'kmap()' on each page, but if there are no pages in the skb, it behaves like simple copying to iov iterator. This patch also adds new field to the control block of skb - this value shows current offset in the skb to read next portion of data (it doesn't matter linear it or not). Idea behind this field is that 'skb_copy_datagram_iter()' handles both types of skb internally - it just needs an offset from which to copy data from the given skb. This offset is incremented on each read from skb. This approach allows to avoid special handling of non-linear skbs: 1) We can't call 'skb_pull()' on it, because it updates 'data' pointer. 2) We need to update 'data_len' also on each read from this skb. Signed-off-by: Arseniy Krasnov Reviewed-by: Stefano Garzarella --- Changelog: v5(big patchset) -> v1: * Merge 'virtio_transport_common.c' and 'vhost/vsock.c' patches into this single patch. * Commit message update: grammar fix and remark that this patch is MSG_ZEROCOPY preparation. * Use 'min_t()' instead of comparison using '<>' operators. v1 -> v2: * R-b tag added. drivers/vhost/vsock.c | 14 ++++++++----- include/linux/virtio_vsock.h | 1 + net/vmw_vsock/virtio_transport_common.c | 27 ++++++++++++++++--------- 3 files changed, 28 insertions(+), 14 deletions(-) diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c index 817d377a3f36..8c917be32b5d 100644 --- a/drivers/vhost/vsock.c +++ b/drivers/vhost/vsock.c @@ -114,6 +114,7 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock, struct sk_buff *skb; unsigned out, in; size_t nbytes; + u32 frag_off; int head; skb = virtio_vsock_skb_dequeue(&vsock->send_pkt_queue); @@ -156,7 +157,8 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock, } iov_iter_init(&iov_iter, ITER_DEST, &vq->iov[out], in, iov_len); - payload_len = skb->len; + frag_off = VIRTIO_VSOCK_SKB_CB(skb)->frag_off; + payload_len = skb->len - frag_off; hdr = virtio_vsock_hdr(skb); /* If the packet is greater than the space available in the @@ -197,8 +199,10 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock, break; } - nbytes = copy_to_iter(skb->data, payload_len, &iov_iter); - if (nbytes != payload_len) { + if (skb_copy_datagram_iter(skb, + frag_off, + &iov_iter, + payload_len)) { kfree_skb(skb); vq_err(vq, "Faulted on copying pkt buf\n"); break; @@ -212,13 +216,13 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock, vhost_add_used(vq, head, sizeof(*hdr) + payload_len); added = true; - skb_pull(skb, payload_len); + VIRTIO_VSOCK_SKB_CB(skb)->frag_off += payload_len; total_len += payload_len; /* If we didn't send all the payload we can requeue the packet * to send it with the next available buffer. */ - if (skb->len > 0) { + if (VIRTIO_VSOCK_SKB_CB(skb)->frag_off < skb->len) { hdr->flags |= cpu_to_le32(flags_to_restore); /* We are queueing the same skb to handle diff --git a/include/linux/virtio_vsock.h b/include/linux/virtio_vsock.h index c58453699ee9..17dbb7176e37 100644 --- a/include/linux/virtio_vsock.h +++ b/include/linux/virtio_vsock.h @@ -12,6 +12,7 @@ struct virtio_vsock_skb_cb { bool reply; bool tap_delivered; + u32 frag_off; }; #define VIRTIO_VSOCK_SKB_CB(skb) ((struct virtio_vsock_skb_cb *)((skb)->cb)) diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c index b769fc258931..1a376f808ae6 100644 --- a/net/vmw_vsock/virtio_transport_common.c +++ b/net/vmw_vsock/virtio_transport_common.c @@ -355,7 +355,7 @@ virtio_transport_stream_do_peek(struct vsock_sock *vsk, spin_lock_bh(&vvs->rx_lock); skb_queue_walk_safe(&vvs->rx_queue, skb, tmp) { - off = 0; + off = VIRTIO_VSOCK_SKB_CB(skb)->frag_off; if (total == len) break; @@ -370,7 +370,10 @@ virtio_transport_stream_do_peek(struct vsock_sock *vsk, */ spin_unlock_bh(&vvs->rx_lock); - err = memcpy_to_msg(msg, skb->data + off, bytes); + err = skb_copy_datagram_iter(skb, off, + &msg->msg_iter, + bytes); + if (err) goto out; @@ -413,25 +416,28 @@ virtio_transport_stream_do_dequeue(struct vsock_sock *vsk, while (total < len && !skb_queue_empty(&vvs->rx_queue)) { skb = skb_peek(&vvs->rx_queue); - bytes = len - total; - if (bytes > skb->len) - bytes = skb->len; + bytes = min_t(size_t, len - total, + skb->len - VIRTIO_VSOCK_SKB_CB(skb)->frag_off); /* sk_lock is held by caller so no one else can dequeue. * Unlock rx_lock since memcpy_to_msg() may sleep. */ spin_unlock_bh(&vvs->rx_lock); - err = memcpy_to_msg(msg, skb->data, bytes); + err = skb_copy_datagram_iter(skb, + VIRTIO_VSOCK_SKB_CB(skb)->frag_off, + &msg->msg_iter, bytes); + if (err) goto out; spin_lock_bh(&vvs->rx_lock); total += bytes; - skb_pull(skb, bytes); - if (skb->len == 0) { + VIRTIO_VSOCK_SKB_CB(skb)->frag_off += bytes; + + if (skb->len == VIRTIO_VSOCK_SKB_CB(skb)->frag_off) { u32 pkt_len = le32_to_cpu(virtio_vsock_hdr(skb)->len); virtio_transport_dec_rx_pkt(vvs, pkt_len); @@ -503,7 +509,10 @@ static int virtio_transport_seqpacket_do_dequeue(struct vsock_sock *vsk, */ spin_unlock_bh(&vvs->rx_lock); - err = memcpy_to_msg(msg, skb->data, bytes_to_copy); + err = skb_copy_datagram_iter(skb, 0, + &msg->msg_iter, + bytes_to_copy); + if (err) { /* Copy of message failed. Rest of * fragments will be freed without copy. From patchwork Tue Jul 18 18:02:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 122190 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c923:0:b0:3e4:2afc:c1 with SMTP id j3csp1927485vqt; Tue, 18 Jul 2023 11:19:08 -0700 (PDT) X-Google-Smtp-Source: APBJJlHK3QsLvq6PqKCB7SFIOf3gaTdJbFirQ+eud2UhaFrwxssVQplV+FAHVy76RGZIrkxfYq0U X-Received: by 2002:a17:90a:46c7:b0:263:3ccc:dfe4 with SMTP id x7-20020a17090a46c700b002633cccdfe4mr41481pjg.1.1689704348093; Tue, 18 Jul 2023 11:19:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689704348; cv=none; d=google.com; s=arc-20160816; b=ihb8oidlhwcnUKVWbk1RX1sKz9ULT6weODA/6CDyS9PCv+/blmfm1cLuOqB0tBFbs0 TqYiqpC2RdXKpzmrZOBSjCpPUt+dplyq8Fj39l2cxuFSytWh94OHQZaLCssnD42wxdyi qgUr/BOwwFyZlpUHnj8p7CZYQXQhHoNslhaCHUEVPYYs4if3dSRqVoebQ1KxCvi+qKh+ pEL12n34mbtkjznhA655wW6bTYo7vTeUbf56msT5/PJzc8WGSQecu/u96UsYgGnynwTz Nb92WxfU0ltTflZHFFPnqF7byoxx0DTrNZ7bvMUhoDE5Xdiis0wNeeXFjbReXyzrxZVW roLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=tkRDIjgwoi7c8HuFjXoUTj2zb+RizwwV2GwfLVc5LoM=; fh=IiLlUAYZQ4hWuQbi3V9KE00VdjFsw4tg4Y6JCUFRLn0=; b=K2I5rTELhmIB/eqkPeepAFmLEe2k8kaMk8rQDRG2HljNraagunlE82o96qU2PNxYvK SZW8EUhY+5QcfqT0OqClDGWCsAQAk6lx2lW+jRQEYVq1a1goGUUhOpRLnSOx6iJ05JUP g3hE+Kwc3+4sCSKOhJMUPmP1tuRpsNCvzNxE4QTMJ++myzws5e7oaPBwAR/vszPiEqwi wy+kE+aXly3fmLpSbxCWctvuhq4wHqXTWyWYdkzEu9BYJ5d47Of/2gHvX8sKVtP+PWrN 013gPRVZ+DPRBGbkNGRmyRLBEhmmhbQ582q20vEH/1f8u4igdiGud/xb+lsvdhVTY68r FHjg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=Mmo1l+fK; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id in7-20020a17090b438700b002636ac73ffdsi6202369pjb.63.2023.07.18.11.18.24; Tue, 18 Jul 2023 11:19:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=Mmo1l+fK; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232950AbjGRSIf (ORCPT + 99 others); Tue, 18 Jul 2023 14:08:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46684 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232692AbjGRSI0 (ORCPT ); Tue, 18 Jul 2023 14:08:26 -0400 Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DB7991733; Tue, 18 Jul 2023 11:08:22 -0700 (PDT) Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id 6845612005D; Tue, 18 Jul 2023 21:08:18 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru 6845612005D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1689703698; bh=tkRDIjgwoi7c8HuFjXoUTj2zb+RizwwV2GwfLVc5LoM=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=Mmo1l+fK/0woxIqGezL0OflmOtkdmUafpFBEg3/9bvYgCR7Fp4ZhhkFtIUyv+/pRe mvuvuj26b29QPvvpXlg7xa27Yw6OdcS1L8MzFCLqNWKBCWuGSTdnviaWl7XSSoi/0g 0bT1hAUvJ7F6hWyPb80Cj4g608o+tmuIBrUeSe3kRHulnQIj6tkehQKtiBenhQFVmh qQHhC3Tf/C/rRfDWhLh77fruHotLzGYCMAFIkoTPZz+lVn1f4hMav0Rer+a3emR4Of Q5uY2BqL76CPyb9Ro3zL2wBFlR6xUQmdnnAT2mCk+nUMuP32qNhx6nS4iYYVzefZvg AIgucssW0R1Yg== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Tue, 18 Jul 2023 21:08:18 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Tue, 18 Jul 2023 21:08:17 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [PATCH net-next v2 2/4] vsock/virtio: support to send non-linear skb Date: Tue, 18 Jul 2023 21:02:35 +0300 Message-ID: <20230718180237.3248179-3-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230718180237.3248179-1-AVKrasnov@sberdevices.ru> References: <20230718180237.3248179-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m02.sberdevices.ru (172.16.192.103) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178700 [Jul 18 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 524 524 9753033d6953787301affc41bead8ed49c47b39d, {Tracking_from_domain_doesnt_match_to}, 100.64.160.123:7.1.2;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:7.1.1,5.0.1;127.0.0.199:7.1.2;sberdevices.ru:7.1.1,5.0.1, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/18 13:44:00 #21634370 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1771783426073482041 X-GMAIL-MSGID: 1771783426073482041 For non-linear skb use its pages from fragment array as buffers in virtio tx queue. These pages are already pinned by 'get_user_pages()' during such skb creation. Signed-off-by: Arseniy Krasnov Reviewed-by: Stefano Garzarella --- net/vmw_vsock/virtio_transport.c | 40 +++++++++++++++++++++++++++----- 1 file changed, 34 insertions(+), 6 deletions(-) diff --git a/net/vmw_vsock/virtio_transport.c b/net/vmw_vsock/virtio_transport.c index e95df847176b..6cbb45bb12d2 100644 --- a/net/vmw_vsock/virtio_transport.c +++ b/net/vmw_vsock/virtio_transport.c @@ -100,7 +100,9 @@ virtio_transport_send_pkt_work(struct work_struct *work) vq = vsock->vqs[VSOCK_VQ_TX]; for (;;) { - struct scatterlist hdr, buf, *sgs[2]; + /* +1 is for packet header. */ + struct scatterlist *sgs[MAX_SKB_FRAGS + 1]; + struct scatterlist bufs[MAX_SKB_FRAGS + 1]; int ret, in_sg = 0, out_sg = 0; struct sk_buff *skb; bool reply; @@ -111,12 +113,38 @@ virtio_transport_send_pkt_work(struct work_struct *work) virtio_transport_deliver_tap_pkt(skb); reply = virtio_vsock_skb_reply(skb); + sg_init_one(&bufs[out_sg], virtio_vsock_hdr(skb), + sizeof(*virtio_vsock_hdr(skb))); + sgs[out_sg] = &bufs[out_sg]; + out_sg++; + + if (!skb_is_nonlinear(skb)) { + if (skb->len > 0) { + sg_init_one(&bufs[out_sg], skb->data, skb->len); + sgs[out_sg] = &bufs[out_sg]; + out_sg++; + } + } else { + struct skb_shared_info *si; + int i; + + si = skb_shinfo(skb); + + for (i = 0; i < si->nr_frags; i++) { + skb_frag_t *skb_frag = &si->frags[i]; + void *va = page_to_virt(skb_frag->bv_page); - sg_init_one(&hdr, virtio_vsock_hdr(skb), sizeof(*virtio_vsock_hdr(skb))); - sgs[out_sg++] = &hdr; - if (skb->len > 0) { - sg_init_one(&buf, skb->data, skb->len); - sgs[out_sg++] = &buf; + /* We will use 'page_to_virt()' for userspace page here, + * because virtio layer will call 'virt_to_phys()' later + * to fill buffer descriptor. We don't touch memory at + * "virtual" address of this page. + */ + sg_init_one(&bufs[out_sg], + va + skb_frag->bv_offset, + skb_frag->bv_len); + sgs[out_sg] = &bufs[out_sg]; + out_sg++; + } } ret = virtqueue_add_sgs(vq, sgs, out_sg, in_sg, skb, GFP_KERNEL); From patchwork Tue Jul 18 18:02:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 122191 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c923:0:b0:3e4:2afc:c1 with SMTP id j3csp1929918vqt; Tue, 18 Jul 2023 11:23:52 -0700 (PDT) X-Google-Smtp-Source: APBJJlGdRhPcejnpji84VZPUX0kyPgNMQbp4vIOXSHE1vIG8PX2JXlFnJUNMkDIt0H/GVQL9smCg X-Received: by 2002:a05:6a00:1142:b0:682:f529:6d69 with SMTP id b2-20020a056a00114200b00682f5296d69mr17956123pfm.7.1689704631914; Tue, 18 Jul 2023 11:23:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689704631; cv=none; d=google.com; s=arc-20160816; b=ZDb3LVoKCkg4Qq6/TzScpjluhqmFNCYo67ougc7U0HhUed4CNaZKuzzVaV4cX9LlyS gsq6eeG4reLrHaQL3avhZaeicZThcnHwnblfGnS+r5vUTM9OL5sJt9L7l9bOy95KliSs fA3/inmClRmBEfLtYDBoHDpXgZpFWUOhTcp6AHGPqT5L8Div92Rm+E143D2Mf1CwlTY9 MD3iBakPHUGxg7NZPZVBC0rVptr8F5pu9ICVFkJMCh3w3OPclc2H7nGfpADxZ70xIRoh Qv08Zp42B7j9UaM7hOz/C5GdXmPMOsXC74Seo/FTsTdaYGpSk/FaT3hlD34oazNHNHSm Meyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=ydsf/FQJHsRd7VM9sT5BCFNwDtuvxxMDFyVsLpjPMoM=; fh=IiLlUAYZQ4hWuQbi3V9KE00VdjFsw4tg4Y6JCUFRLn0=; b=Duq5zMVTO+tY1CbkebZzo4zql6jCIpglldHPRuG7J++5OLPBUBfPAn2q3eo5N5TPuv QCecq44Bpqobgm2YJJemE0gC19kvyIJK4JPHpGkGxHAFi8UOp8+ChJ6N1qWHBdN1E+QA zFqLlpPwr8ZXmPhHCUTJ2HItIYh2MNgUmkBFSOp7I8mPOOygR/kN3TIDhgyIbIAPPBA3 UVYxO+1UVuf/YvVs5tErbG6QWsO5l1VtXPrMTRRGN9b3f9jU8UBk9jqfBVWhcg3c749e 2WWsol79iHbRd3WCj22zrzE6Ua22hf2W8PimtQsw26SoZliVUfvYYEUQIidJmLqehLGJ EiFw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=QdmAcxZ4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z13-20020aa7888d000000b006863085981esi150965pfe.68.2023.07.18.11.23.37; Tue, 18 Jul 2023 11:23:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=QdmAcxZ4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232864AbjGRSIa (ORCPT + 99 others); Tue, 18 Jul 2023 14:08:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46566 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232255AbjGRSIX (ORCPT ); Tue, 18 Jul 2023 14:08:23 -0400 Received: from mx1.sberdevices.ru (mx1.sberdevices.ru [37.18.73.165]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A4A89173A; Tue, 18 Jul 2023 11:08:20 -0700 (PDT) Received: from p-infra-ksmg-sc-msk01 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id 2530210002B; Tue, 18 Jul 2023 21:08:19 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru 2530210002B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1689703699; bh=ydsf/FQJHsRd7VM9sT5BCFNwDtuvxxMDFyVsLpjPMoM=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=QdmAcxZ4jjyGocIhAeJ3L3OxrEu9iuioIW9m8bO1PTfi6tnDUp7aIbepyVDs4KZ4n Zs2Hv9GeQaY25iXuMSg1VIlZu3urtvCHn8O8bkgnfb4Zwf7evhr3314L73ZfDRAPhg gmcVt+OhGWa6uy8QxbL5hq8QI0Sy5EWyHR3JUUih7ykL+YBvRZuU4yUzz7mjOZ7ZfR eV79LUdEIKClT+nQZpwIoV9bKnzefN7dmOKDfgNE09l/tYcYBxpkMUGrinrXKuu5WN eX2CSX26BMR/0GckJex4Ft0ieG0qVMut9h4JX3YULQwswApFa8cefPNr8ocK0q2anz EcgODJNYY/4vg== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Tue, 18 Jul 2023 21:08:19 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Tue, 18 Jul 2023 21:08:18 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [PATCH net-next v2 3/4] vsock/virtio: non-linear skb handling for tap Date: Tue, 18 Jul 2023 21:02:36 +0300 Message-ID: <20230718180237.3248179-4-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230718180237.3248179-1-AVKrasnov@sberdevices.ru> References: <20230718180237.3248179-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m02.sberdevices.ru (172.16.192.103) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178700 [Jul 18 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 524 524 9753033d6953787301affc41bead8ed49c47b39d, {Tracking_from_domain_doesnt_match_to}, p-i-exch-sc-m01.sberdevices.ru:7.1.1,5.0.1;127.0.0.199:7.1.2;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;sberdevices.ru:7.1.1,5.0.1;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/18 13:44:00 #21634370 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1771783724363630194 X-GMAIL-MSGID: 1771783724363630194 For tap device new skb is created and data from the current skb is copied to it. This adds copying data from non-linear skb to new the skb. Signed-off-by: Arseniy Krasnov Reviewed-by: Stefano Garzarella --- net/vmw_vsock/virtio_transport_common.c | 31 ++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-) diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c index 1a376f808ae6..26a4d10da205 100644 --- a/net/vmw_vsock/virtio_transport_common.c +++ b/net/vmw_vsock/virtio_transport_common.c @@ -106,6 +106,27 @@ virtio_transport_alloc_skb(struct virtio_vsock_pkt_info *info, return NULL; } +static void virtio_transport_copy_nonlinear_skb(const struct sk_buff *skb, + void *dst, + size_t len) +{ + struct iov_iter iov_iter = { 0 }; + struct kvec kvec; + size_t to_copy; + + kvec.iov_base = dst; + kvec.iov_len = len; + + iov_iter.iter_type = ITER_KVEC; + iov_iter.kvec = &kvec; + iov_iter.nr_segs = 1; + + to_copy = min_t(size_t, len, skb->len); + + skb_copy_datagram_iter(skb, VIRTIO_VSOCK_SKB_CB(skb)->frag_off, + &iov_iter, to_copy); +} + /* Packet capture */ static struct sk_buff *virtio_transport_build_skb(void *opaque) { @@ -114,7 +135,6 @@ static struct sk_buff *virtio_transport_build_skb(void *opaque) struct af_vsockmon_hdr *hdr; struct sk_buff *skb; size_t payload_len; - void *payload_buf; /* A packet could be split to fit the RX buffer, so we can retrieve * the payload length from the header and the buffer pointer taking @@ -122,7 +142,6 @@ static struct sk_buff *virtio_transport_build_skb(void *opaque) */ pkt_hdr = virtio_vsock_hdr(pkt); payload_len = pkt->len; - payload_buf = pkt->data; skb = alloc_skb(sizeof(*hdr) + sizeof(*pkt_hdr) + payload_len, GFP_ATOMIC); @@ -165,7 +184,13 @@ static struct sk_buff *virtio_transport_build_skb(void *opaque) skb_put_data(skb, pkt_hdr, sizeof(*pkt_hdr)); if (payload_len) { - skb_put_data(skb, payload_buf, payload_len); + if (skb_is_nonlinear(pkt)) { + void *data = skb_put(skb, payload_len); + + virtio_transport_copy_nonlinear_skb(pkt, data, payload_len); + } else { + skb_put_data(skb, pkt->data, payload_len); + } } return skb; From patchwork Tue Jul 18 18:02:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 122192 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c923:0:b0:3e4:2afc:c1 with SMTP id j3csp1931474vqt; Tue, 18 Jul 2023 11:26:57 -0700 (PDT) X-Google-Smtp-Source: APBJJlFccFNFDNtSK46daaqpfn8ESOOBvrUgZWzFpsgpOb7pyRbuKsBbY630hRcM2c3jA0MsSlcX X-Received: by 2002:a17:90a:6f65:b0:263:5d42:79f7 with SMTP id d92-20020a17090a6f6500b002635d4279f7mr15538532pjk.23.1689704817165; Tue, 18 Jul 2023 11:26:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689704817; cv=none; d=google.com; s=arc-20160816; b=e1D5twb8c+drE+70zhWHcx06sMB98wFcUjTn46poxx2xGpNjYJquPb925VXItMqRlL 0YdEMPACWVt8vWnUTsW+WtIYHjOsI527Wxtm9psDV7iYsq08AF6vNA7XiP6BHrCBqoV/ dwmC9C9uG6BGKzpujME9tyAUYVhDoh6t8jGXicgwjFn4CJr+1feAgHTmz0hbPYKhfUl/ 6HGxPctSLPEvMKl/avR5CwIWiHlZzeSpyo0DFcgpSB6HSa644/KrDR2ZPiWbrpHEMRKh JwbIakESaY4+Byl8vPqjIyInFje/WIUtmjRTXezws9nBhm0uuSoD0L1gn19hZj4v9PbR eUZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=CazwbxwNAW72PaLQtK+kzSkBtO6i4jTyUwPfJe4YbKo=; fh=IiLlUAYZQ4hWuQbi3V9KE00VdjFsw4tg4Y6JCUFRLn0=; b=O1oAWX9r1qrgY0qj333U2flWmHhJPD/bsgK044XuFnNzRtFIBsuG0Tw4/VwSi5hvrI cfgP6mF3m7RePu5oyUPtJZRPiF9IfnCC/1+NbTOHj2Yy9TQCr3O0SO69SAzN6S5yJnht 9orIZd12LJ5szH9vKQFqRTpDUpQUBIE+6R8gFtdG9mIVJ/iWiKYGq0dWzpj+kAqwLhld 5eM0bMNlhIvLgrQFnkkJtd+eogKVHJPJTHTLbzLD+V9hIEQhdV+VaqG+OyjCGLuT0RET qDlLFIZ07BHXTzzk7RHnAD1G9j0y/wNtBbFHVCx7cKEX7x6uRQ7TLjgvNs760iZoQs+7 6jDA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=m5kaId+a; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q3-20020a17090a68c300b002632c1fc68dsi1535695pjj.72.2023.07.18.11.26.43; Tue, 18 Jul 2023 11:26:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=m5kaId+a; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232896AbjGRSIc (ORCPT + 99 others); Tue, 18 Jul 2023 14:08:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46638 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232202AbjGRSIY (ORCPT ); Tue, 18 Jul 2023 14:08:24 -0400 Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8AFD21984; Tue, 18 Jul 2023 11:08:21 -0700 (PDT) Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id 0256D120060; Tue, 18 Jul 2023 21:08:20 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru 0256D120060 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1689703700; bh=CazwbxwNAW72PaLQtK+kzSkBtO6i4jTyUwPfJe4YbKo=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=m5kaId+avQ4+V+mhT79zMmjFBHqhkVIZ0MsbuNjL0d2w2nhaJD2Vpe3gEOBDnPazF JJGcNyA3cR9lyhTbSnriK6vvJctAgg1pSPcKCSawNKQ3cvGkjhSdyMcA7EBHnbyJtH WMxDD3QehczPiRhrKTvpf6xktfGt8iRbQVpSqo0JOcQ9T9xJ+Pvlj2Kv2l35XfbZXm HNlrwUbWBCKKqEcS7ozArWTk181sLJAoT5wDGFqSyd2UVa1cTzRG3W5gjBXLmetMEW UovgGp1DsGOZZQm4L+hfI3iC8FVX/aPUdXJER9Qc0FAypv+vBx9yNwKJLaPmiZZrdY e8Bhb8nMNaGYA== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Tue, 18 Jul 2023 21:08:19 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Tue, 18 Jul 2023 21:08:19 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [PATCH net-next v2 4/4] vsock/virtio: MSG_ZEROCOPY flag support Date: Tue, 18 Jul 2023 21:02:37 +0300 Message-ID: <20230718180237.3248179-5-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230718180237.3248179-1-AVKrasnov@sberdevices.ru> References: <20230718180237.3248179-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m02.sberdevices.ru (172.16.192.103) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178700 [Jul 18 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 524 524 9753033d6953787301affc41bead8ed49c47b39d, {Tracking_from_domain_doesnt_match_to}, 100.64.160.123:7.1.2;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:7.1.1,5.0.1;127.0.0.199:7.1.2;sberdevices.ru:7.1.1,5.0.1, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/18 13:44:00 #21634370 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1771783917947529583 X-GMAIL-MSGID: 1771783917947529583 This adds handling of MSG_ZEROCOPY flag on transmission path: if this flag is set and zerocopy transmission is possible, then non-linear skb will be created and filled with the pages of user's buffer. Pages of user's buffer are locked in memory by 'get_user_pages()'. Second thing that this patch does is replace type of skb owning: instead of calling 'skb_set_owner_sk_safe()' it calls 'skb_set_owner_w()'. Reason of this change is that '__zerocopy_sg_from_iter()' increments 'sk_wmem_alloc' of socket, so to decrease this field correctly proper skb destructor is needed: 'sock_wfree()'. This destructor is set by 'skb_set_owner_w()'. Signed-off-by: Arseniy Krasnov --- Changelog: v5(big patchset) -> v1: * Refactorings of 'if' conditions. * Remove extra blank line. * Remove 'frag_off' field unneeded init. * Add function 'virtio_transport_fill_skb()' which fills both linear and non-linear skb with provided data. v1 -> v2: * Use original order of last four arguments in 'virtio_transport_alloc_skb()'. net/vmw_vsock/virtio_transport_common.c | 250 ++++++++++++++++++------ 1 file changed, 192 insertions(+), 58 deletions(-) diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c index 26a4d10da205..dbe503205d53 100644 --- a/net/vmw_vsock/virtio_transport_common.c +++ b/net/vmw_vsock/virtio_transport_common.c @@ -37,73 +37,115 @@ virtio_transport_get_ops(struct vsock_sock *vsk) return container_of(t, struct virtio_transport, transport); } -/* Returns a new packet on success, otherwise returns NULL. - * - * If NULL is returned, errp is set to a negative errno. - */ -static struct sk_buff * -virtio_transport_alloc_skb(struct virtio_vsock_pkt_info *info, - size_t len, - u32 src_cid, - u32 src_port, - u32 dst_cid, - u32 dst_port) -{ - const size_t skb_len = VIRTIO_VSOCK_SKB_HEADROOM + len; - struct virtio_vsock_hdr *hdr; - struct sk_buff *skb; - void *payload; - int err; +static bool virtio_transport_can_zcopy(struct virtio_vsock_pkt_info *info, + size_t max_to_send) +{ + struct iov_iter *iov_iter; - skb = virtio_vsock_alloc_skb(skb_len, GFP_KERNEL); - if (!skb) - return NULL; + if (!info->msg) + return false; - hdr = virtio_vsock_hdr(skb); - hdr->type = cpu_to_le16(info->type); - hdr->op = cpu_to_le16(info->op); - hdr->src_cid = cpu_to_le64(src_cid); - hdr->dst_cid = cpu_to_le64(dst_cid); - hdr->src_port = cpu_to_le32(src_port); - hdr->dst_port = cpu_to_le32(dst_port); - hdr->flags = cpu_to_le32(info->flags); - hdr->len = cpu_to_le32(len); + iov_iter = &info->msg->msg_iter; - if (info->msg && len > 0) { - payload = skb_put(skb, len); - err = memcpy_from_msg(payload, info->msg, len); - if (err) - goto out; + /* Data is simple buffer. */ + if (iter_is_ubuf(iov_iter)) + return true; - if (msg_data_left(info->msg) == 0 && - info->type == VIRTIO_VSOCK_TYPE_SEQPACKET) { - hdr->flags |= cpu_to_le32(VIRTIO_VSOCK_SEQ_EOM); + if (!iter_is_iovec(iov_iter)) + return false; - if (info->msg->msg_flags & MSG_EOR) - hdr->flags |= cpu_to_le32(VIRTIO_VSOCK_SEQ_EOR); - } + if (iov_iter->iov_offset) + return false; + + /* We can't send whole iov. */ + if (iov_iter->count > max_to_send) + return false; + + return true; +} + +static int virtio_transport_init_zcopy_skb(struct vsock_sock *vsk, + struct sk_buff *skb, + struct msghdr *msg, + bool zerocopy) +{ + struct ubuf_info *uarg; + + if (msg->msg_ubuf) { + uarg = msg->msg_ubuf; + net_zcopy_get(uarg); + } else { + struct iov_iter *iter = &msg->msg_iter; + struct ubuf_info_msgzc *uarg_zc; + int len; + + /* Only ITER_IOVEC or ITER_UBUF are allowed and + * checked before. + */ + if (iter_is_iovec(iter)) + len = iov_length(iter->__iov, iter->nr_segs); + else + len = iter->count; + + uarg = msg_zerocopy_realloc(sk_vsock(vsk), + len, + NULL); + if (!uarg) + return -1; + + uarg_zc = uarg_to_msgzc(uarg); + uarg_zc->zerocopy = zerocopy ? 1 : 0; } - if (info->reply) - virtio_vsock_skb_set_reply(skb); + skb_zcopy_init(skb, uarg); - trace_virtio_transport_alloc_pkt(src_cid, src_port, - dst_cid, dst_port, - len, - info->type, - info->op, - info->flags); + return 0; +} - if (info->vsk && !skb_set_owner_sk_safe(skb, sk_vsock(info->vsk))) { - WARN_ONCE(1, "failed to allocate skb on vsock socket with sk_refcnt == 0\n"); - goto out; +static int virtio_transport_fill_skb(struct sk_buff *skb, + struct virtio_vsock_pkt_info *info, + size_t len, + bool zcopy) +{ + if (zcopy) { + return __zerocopy_sg_from_iter(info->msg, NULL, skb, + &info->msg->msg_iter, + len); + } else { + void *payload; + int err; + + payload = skb_put(skb, len); + err = memcpy_from_msg(payload, info->msg, len); + if (err) + return -1; + + if (msg_data_left(info->msg)) + return 0; + + return 0; } +} - return skb; +static void virtio_transport_init_hdr(struct sk_buff *skb, + struct virtio_vsock_pkt_info *info, + u32 src_cid, + u32 src_port, + u32 dst_cid, + u32 dst_port, + size_t len) +{ + struct virtio_vsock_hdr *hdr; -out: - kfree_skb(skb); - return NULL; + hdr = virtio_vsock_hdr(skb); + hdr->type = cpu_to_le16(info->type); + hdr->op = cpu_to_le16(info->op); + hdr->src_cid = cpu_to_le64(src_cid); + hdr->dst_cid = cpu_to_le64(dst_cid); + hdr->src_port = cpu_to_le32(src_port); + hdr->dst_port = cpu_to_le32(dst_port); + hdr->flags = cpu_to_le32(info->flags); + hdr->len = cpu_to_le32(len); } static void virtio_transport_copy_nonlinear_skb(const struct sk_buff *skb, @@ -214,6 +256,70 @@ static u16 virtio_transport_get_type(struct sock *sk) return VIRTIO_VSOCK_TYPE_SEQPACKET; } +static struct sk_buff *virtio_transport_alloc_skb(struct vsock_sock *vsk, + struct virtio_vsock_pkt_info *info, + size_t payload_len, + bool zcopy, + u32 src_cid, + u32 src_port, + u32 dst_cid, + u32 dst_port) +{ + struct sk_buff *skb; + size_t skb_len; + + skb_len = VIRTIO_VSOCK_SKB_HEADROOM; + + if (!zcopy) + skb_len += payload_len; + + skb = virtio_vsock_alloc_skb(skb_len, GFP_KERNEL); + if (!skb) + return NULL; + + virtio_transport_init_hdr(skb, info, src_cid, src_port, + dst_cid, dst_port, + payload_len); + + /* Set owner here, because '__zerocopy_sg_from_iter()' uses + * owner of skb without check to update 'sk_wmem_alloc'. + */ + if (vsk) + skb_set_owner_w(skb, sk_vsock(vsk)); + + if (info->msg && payload_len > 0) { + int err; + + err = virtio_transport_fill_skb(skb, info, payload_len, zcopy); + if (err) + goto out; + + if (info->type == VIRTIO_VSOCK_TYPE_SEQPACKET) { + struct virtio_vsock_hdr *hdr = virtio_vsock_hdr(skb); + + hdr->flags |= cpu_to_le32(VIRTIO_VSOCK_SEQ_EOM); + + if (info->msg->msg_flags & MSG_EOR) + hdr->flags |= cpu_to_le32(VIRTIO_VSOCK_SEQ_EOR); + } + } + + if (info->reply) + virtio_vsock_skb_set_reply(skb); + + trace_virtio_transport_alloc_pkt(src_cid, src_port, + dst_cid, dst_port, + payload_len, + info->type, + info->op, + info->flags); + + return skb; +out: + kfree_skb(skb); + return NULL; +} + /* This function can only be used on connecting/connected sockets, * since a socket assigned to a transport is required. * @@ -222,10 +328,12 @@ static u16 virtio_transport_get_type(struct sock *sk) static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, struct virtio_vsock_pkt_info *info) { + u32 max_skb_len = VIRTIO_VSOCK_MAX_PKT_BUF_SIZE; u32 src_cid, src_port, dst_cid, dst_port; const struct virtio_transport *t_ops; struct virtio_vsock_sock *vvs; u32 pkt_len = info->pkt_len; + bool can_zcopy = false; u32 rest_len; int ret; @@ -254,15 +362,30 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, if (pkt_len == 0 && info->op == VIRTIO_VSOCK_OP_RW) return pkt_len; + if (info->msg) { + /* If zerocopy is not enabled by 'setsockopt()', we behave as + * there is no MSG_ZEROCOPY flag set. + */ + if (!sock_flag(sk_vsock(vsk), SOCK_ZEROCOPY)) + info->msg->msg_flags &= ~MSG_ZEROCOPY; + + if (info->msg->msg_flags & MSG_ZEROCOPY) + can_zcopy = virtio_transport_can_zcopy(info, pkt_len); + + if (can_zcopy) + max_skb_len = min_t(u32, VIRTIO_VSOCK_MAX_PKT_BUF_SIZE, + (MAX_SKB_FRAGS * PAGE_SIZE)); + } + rest_len = pkt_len; do { struct sk_buff *skb; size_t skb_len; - skb_len = min_t(u32, VIRTIO_VSOCK_MAX_PKT_BUF_SIZE, rest_len); + skb_len = min(max_skb_len, rest_len); - skb = virtio_transport_alloc_skb(info, skb_len, + skb = virtio_transport_alloc_skb(vsk, info, skb_len, can_zcopy, src_cid, src_port, dst_cid, dst_port); if (!skb) { @@ -270,6 +393,17 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, break; } + /* This is last skb to send this portion of data. */ + if (info->msg && info->msg->msg_flags & MSG_ZEROCOPY && + skb_len == rest_len && info->op == VIRTIO_VSOCK_OP_RW) { + if (virtio_transport_init_zcopy_skb(vsk, skb, + info->msg, + can_zcopy)) { + ret = -ENOMEM; + break; + } + } + virtio_transport_inc_tx_pkt(vvs, skb); ret = t_ops->send_pkt(skb); @@ -934,7 +1068,7 @@ static int virtio_transport_reset_no_sock(const struct virtio_transport *t, if (!t) return -ENOTCONN; - reply = virtio_transport_alloc_skb(&info, 0, + reply = virtio_transport_alloc_skb(NULL, &info, 0, false, le64_to_cpu(hdr->dst_cid), le32_to_cpu(hdr->dst_port), le64_to_cpu(hdr->src_cid),