From patchwork Sat Jul 1 06:22:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 114970 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp10866841vqr; Fri, 30 Jun 2023 23:40:44 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6NzfKrZAFuBEKanibvfNbpa3Z8ykbx7ad1H3I5U3E4PfJokZZ76EciSaocSPbdfnHCVXTS X-Received: by 2002:a05:6870:d1c6:b0:1b0:84df:50ec with SMTP id b6-20020a056870d1c600b001b084df50ecmr6732173oac.50.1688193644506; Fri, 30 Jun 2023 23:40:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688193644; cv=none; d=google.com; s=arc-20160816; b=cOR58hmF6FjxvatgjFZpmwOS/g+GVfS1bpSmeUcXUUft70JM41UrN4vdjiboaEkthv Jln0fBiD6AnhpLfjxTttjGVpVlww6pnsX5xorjC/rlu1pdjL83m+oGWHO2gYWWConUU7 gTu3seqkzAs0wcJL35nj+7RYAIQM/+LPEK7l+I4MYnJWGHFQKS88f3jlDnik5Otwuo2M rq6MYKKDQ+tP2FMw/dcGgXOTy4IyvwPFy74s+tNe1Vo6Gnh43rH/Q0o6gS+ld8IVoJOi NnWyUPdP33G34tEgxW/ElTC7yhEUjpLQkZPVUq5nokAzhdzv24xfiy/wKRfQ09L0TATU oj2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=KtR+snAmemDx/QdrmLGfvVQlJ5kcP+i64AE/km7ksjQ=; fh=PzWIi6JmlJNekYqVy23433fKO6OE4+V62ucRx731HUI=; b=tegIvRXoGHq6KFPaaMGbcFIVMPqoFtQiT8a00i7qd3dJ675MkYtzyND6hwv6hdFzqZ lyRHQIndqebCeTtOs0sRjspGveZx8jqUKNRM8bdM9YlhRJG/dc8nigc8+KGmEdUo4zRc eD4GPebtDATYVzhwST8OyJ+B2R1MQmUu7fP8KkeIjthF7GiF5tf1Oh5EeVsmzpiYFP7r Fs9JNFdwoFPg9EBz2E83Hq4b1ahHt6fP81AdATFn7HMXq0AIaRP2m6d3FgaqcJnaHRDz hGnrHZFiVEcOlxTeTD8Qc5SbLRCpjqsvvJcqKsKhF6idRPIVu/jLCV/NQb3p2VpCTofh VS8g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=de9t4ZvQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m16-20020a17090a159000b00260d0644358si15639174pja.187.2023.06.30.23.40.28; Fri, 30 Jun 2023 23:40:44 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=de9t4ZvQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229611AbjGAGjE (ORCPT + 99 others); Sat, 1 Jul 2023 02:39:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39652 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229480AbjGAGjC (ORCPT ); Sat, 1 Jul 2023 02:39:02 -0400 X-Greylist: delayed 619 seconds by postgrey-1.37 at lindbergh.monkeyblade.net; Fri, 30 Jun 2023 23:38:59 PDT Received: from mx1.sberdevices.ru (mx1.sberdevices.ru [37.18.73.165]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8C7C2118; Fri, 30 Jun 2023 23:38:59 -0700 (PDT) Received: from p-infra-ksmg-sc-msk01 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id E3CAB100003; Sat, 1 Jul 2023 09:28:35 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru E3CAB100003 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1688192915; bh=KtR+snAmemDx/QdrmLGfvVQlJ5kcP+i64AE/km7ksjQ=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=de9t4ZvQkesOquo/75JI+uekiz4u7gN/D7MHlurmTNJLny5T5WHUjW+iBWXXYprfF x322yXOf1z2B6rRE89rfoG6GAcrk5eQnzGbJeuOzWX5BrES/3na8lEyVYKSXqm4OVp /wBagkhN1Kfrh0xHN5f0p2S3VrV3gj/rlB8GRtiF2sj35HtIS72MoUcajdoC+Zhpxm bYWWAgAGlfUYZgh/9j/dxRuFoqsmYasc1a6joEI2a6OcznCHabcFrKVoOCcH4eCsIy HgfwUExrOu4uv2WoIUZej05eDizgZ9E7/d8wP/7gmpQLJLQOUQqLUjvkCTqsI/ZL8l wkmu8V93wix4g== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Sat, 1 Jul 2023 09:28:35 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Sat, 1 Jul 2023 09:28:16 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [RFC PATCH v5 01/17] vsock/virtio: read data from non-linear skb Date: Sat, 1 Jul 2023 09:22:54 +0300 Message-ID: <20230701062310.3397129-2-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> References: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178380 [Jun 30 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 517 517 b0056c19d8e10afbb16cb7aad7258dedb0179a79, {Tracking_from_domain_doesnt_match_to}, p-i-exch-sc-m01.sberdevices.ru:7.1.1,5.0.1;sberdevices.ru:7.1.1,5.0.1;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;127.0.0.199:7.1.2;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/01 04:02:00 #21597763 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770199338854873664?= X-GMAIL-MSGID: =?utf-8?q?1770199338854873664?= This is preparation patch for non-linear skbuff handling. It replaces direct calls of 'memcpy_to_msg()' with 'skb_copy_datagram_iter()'. Main advantage of the second one is that is can handle paged part of the skb by using 'kmap()' on each page, but if there are no pages in the skb, it behaves like simple copying to iov iterator. This patch also adds new field to the control block of skb - this value shows current offset in the skb to read next portion of data (it doesn't matter linear it or not). Idea is that 'skb_copy_datagram_iter()' handles both types of skb internally - it just needs an offset from which to copy data from the given skb. This offset is incremented on each read from skb. This approach allows to avoid special handling of non-linear skbs: 1) We can't call 'skb_pull()' on it, because it updates 'data' pointer. 2) We need to update 'data_len' also on each read from this skb. Signed-off-by: Arseniy Krasnov --- Changelog: v4 -> v5: * Use local variable for 'frag_off' in stream dequeue calback. * R-b from Bobby Eshleman removed due to patch update. include/linux/virtio_vsock.h | 1 + net/vmw_vsock/virtio_transport_common.c | 30 ++++++++++++++++++------- 2 files changed, 23 insertions(+), 8 deletions(-) diff --git a/include/linux/virtio_vsock.h b/include/linux/virtio_vsock.h index c58453699ee9..17dbb7176e37 100644 --- a/include/linux/virtio_vsock.h +++ b/include/linux/virtio_vsock.h @@ -12,6 +12,7 @@ struct virtio_vsock_skb_cb { bool reply; bool tap_delivered; + u32 frag_off; }; #define VIRTIO_VSOCK_SKB_CB(skb) ((struct virtio_vsock_skb_cb *)((skb)->cb)) diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c index b769fc258931..e5683af23e60 100644 --- a/net/vmw_vsock/virtio_transport_common.c +++ b/net/vmw_vsock/virtio_transport_common.c @@ -355,7 +355,7 @@ virtio_transport_stream_do_peek(struct vsock_sock *vsk, spin_lock_bh(&vvs->rx_lock); skb_queue_walk_safe(&vvs->rx_queue, skb, tmp) { - off = 0; + off = VIRTIO_VSOCK_SKB_CB(skb)->frag_off; if (total == len) break; @@ -370,7 +370,10 @@ virtio_transport_stream_do_peek(struct vsock_sock *vsk, */ spin_unlock_bh(&vvs->rx_lock); - err = memcpy_to_msg(msg, skb->data + off, bytes); + err = skb_copy_datagram_iter(skb, off, + &msg->msg_iter, + bytes); + if (err) goto out; @@ -411,27 +414,35 @@ virtio_transport_stream_do_dequeue(struct vsock_sock *vsk, } while (total < len && !skb_queue_empty(&vvs->rx_queue)) { + u32 skb_rest_len; + skb = skb_peek(&vvs->rx_queue); bytes = len - total; - if (bytes > skb->len) - bytes = skb->len; + skb_rest_len = skb->len - VIRTIO_VSOCK_SKB_CB(skb)->frag_off; + + if (bytes > skb_rest_len) + bytes = skb_rest_len; /* sk_lock is held by caller so no one else can dequeue. * Unlock rx_lock since memcpy_to_msg() may sleep. */ spin_unlock_bh(&vvs->rx_lock); - err = memcpy_to_msg(msg, skb->data, bytes); + err = skb_copy_datagram_iter(skb, + VIRTIO_VSOCK_SKB_CB(skb)->frag_off, + &msg->msg_iter, bytes); + if (err) goto out; spin_lock_bh(&vvs->rx_lock); total += bytes; - skb_pull(skb, bytes); - if (skb->len == 0) { + VIRTIO_VSOCK_SKB_CB(skb)->frag_off += bytes; + + if (skb->len == VIRTIO_VSOCK_SKB_CB(skb)->frag_off) { u32 pkt_len = le32_to_cpu(virtio_vsock_hdr(skb)->len); virtio_transport_dec_rx_pkt(vvs, pkt_len); @@ -503,7 +514,10 @@ static int virtio_transport_seqpacket_do_dequeue(struct vsock_sock *vsk, */ spin_unlock_bh(&vvs->rx_lock); - err = memcpy_to_msg(msg, skb->data, bytes_to_copy); + err = skb_copy_datagram_iter(skb, 0, + &msg->msg_iter, + bytes_to_copy); + if (err) { /* Copy of message failed. Rest of * fragments will be freed without copy. From patchwork Sat Jul 1 06:22:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 114988 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp10878304vqr; Sat, 1 Jul 2023 00:11:59 -0700 (PDT) X-Google-Smtp-Source: APBJJlHTEa5ahsBYALpUoAJXfRRxPZcB7gXw8WoB2VO7kuwycNvcNHvQDCEnYkk65lUAm7W258vO X-Received: by 2002:a05:6359:a23:b0:134:c1eb:8744 with SMTP id el35-20020a0563590a2300b00134c1eb8744mr4863030rwb.9.1688195518932; Sat, 01 Jul 2023 00:11:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688195518; cv=none; d=google.com; s=arc-20160816; b=JtTWp3mN/x2oOufBJO+Nk6yo5HKkj0mXaRhg+r1Me/t/TDXih7cYMqyr3/yRFg8vdk O0Ii636+4Q2b6b9AqjSsmLNEbby1+8rcVGimYgXsjPijKI+agvKGUsNalNPM2xswGxzp c7ld4d3V1sDn5DPalbz9j0tmgPkvkev7kdTVGiyvxxyEerAef2nVf2TF4cTSrquwOa6L wGTh8UNGxHlwrcf9oFxv/jR+TKc3i0indLrrrkCmXTpZkoSRkIPW8x63/EFetS8mBn4a qWQEsOAHw2F/62Q+gs+oF0IxPNwdt0gjisOj51souYHgrdU6GVoX7XnPCxA9sKaXpLcy x/RQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=C4H+wApJQrD5oBvAIFQn1rVYvXO7cAQ8ZFLo0nPthkM=; fh=PzWIi6JmlJNekYqVy23433fKO6OE4+V62ucRx731HUI=; b=VVZ8tKJdNYaUKitOSRpf6Xs+63GPakjjN+XFhm0/c0/BzSNSnUTDApF47Poji572OX F6tUwOXyBlEVP/Wn+q2xYmxoDjOlkx5P9Xml6dN0p+ArcOyORdI95E9m0A6hpQALNcGK JYJOfq/dLXHffg0D8MX0xe8v1rli43PpnPsS8Bhkz6Xw88+SxdWPLukufiX1Pw4kikDn OJdWaJDX5IaXq4HSAyoUnBhkgpBkbnI5K3Qs3pg8NgOOX9r/p+zM3oexd6ILB6m+qqEq yskaqPGOSsh9OiZA9B7m87zBQQ6Gu57RgMuuRZlswkxTkzhJMdhspcFTT5R4nXenpRb5 2z0w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=fwkxqMMY; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p4-20020a056a000a0400b0067537349056si12341756pfh.342.2023.07.01.00.11.44; Sat, 01 Jul 2023 00:11:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=fwkxqMMY; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230213AbjGAGj5 (ORCPT + 99 others); Sat, 1 Jul 2023 02:39:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39974 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229760AbjGAGjY (ORCPT ); Sat, 1 Jul 2023 02:39:24 -0400 Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EC2BF4239; Fri, 30 Jun 2023 23:39:13 -0700 (PDT) Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id AAF74120005; Sat, 1 Jul 2023 09:28:37 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru AAF74120005 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1688192917; bh=C4H+wApJQrD5oBvAIFQn1rVYvXO7cAQ8ZFLo0nPthkM=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=fwkxqMMYnMcZ5yna0XDVnWh2qjWaBZQ8XYiVxyQQJazczF8l2tcmpCnsiVBkOGdIu YRznkoboE5UTywwbkP+v1I5FRRLnfr04np/1WAPWvuB0qTz/YK/WI7Ndb4UlKMXjAK YA3T+j8tP+wJ7IiOgf43u2pBEaT2uPM4O3ASmn4DyCV5L5aux9Y6fIULInUWtMFU15 3pNWgruA5tSEyWisq+L6UZB3tB76dSzRgB2gj0FX5Y1raoAvCNy6vR6xcW5NEufhbN DjlrkGyWEsB8TOkOJvKpbn4tjDNUGa7fd8PiTjhUoiyBHqt6EL2Xlse4zN8sHTlXas CSlMqos6CjWzg== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Sat, 1 Jul 2023 09:28:36 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Sat, 1 Jul 2023 09:28:17 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [RFC PATCH v5 02/17] vhost/vsock: read data from non-linear skb Date: Sat, 1 Jul 2023 09:22:55 +0300 Message-ID: <20230701062310.3397129-3-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> References: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178380 [Jun 30 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 517 517 b0056c19d8e10afbb16cb7aad7258dedb0179a79, {Tracking_from_domain_doesnt_match_to}, sberdevices.ru:5.0.1,7.1.1;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;127.0.0.199:7.1.2;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/01 00:58:00 #21597365 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770201304377985465?= X-GMAIL-MSGID: =?utf-8?q?1770201304377985465?= This adds copying to guest's virtio buffers from non-linear skbs. Such skbs are created by protocol layer when MSG_ZEROCOPY flags is used. It replaces call of 'copy_to_iter()' to 'skb_copy_datagram_iter()'- second function can read data from non-linear skb. Also this patch uses field 'frag_off' from skb control block. This field shows current offset to read data from skb which could be both linear or not. Signed-off-by: Arseniy Krasnov --- Changelog: v4 -> v5: * Use local variable for 'frag_off'. * Update commit message by adding some details about 'frag_off' field. * R-b from Bobby Eshleman removed due to patch update. drivers/vhost/vsock.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c index 6578db78f0ae..cb00e0e059e4 100644 --- a/drivers/vhost/vsock.c +++ b/drivers/vhost/vsock.c @@ -114,6 +114,7 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock, struct sk_buff *skb; unsigned out, in; size_t nbytes; + u32 frag_off; int head; skb = virtio_vsock_skb_dequeue(&vsock->send_pkt_queue); @@ -156,7 +157,8 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock, } iov_iter_init(&iov_iter, ITER_DEST, &vq->iov[out], in, iov_len); - payload_len = skb->len; + frag_off = VIRTIO_VSOCK_SKB_CB(skb)->frag_off; + payload_len = skb->len - frag_off; hdr = virtio_vsock_hdr(skb); /* If the packet is greater than the space available in the @@ -197,8 +199,10 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock, break; } - nbytes = copy_to_iter(skb->data, payload_len, &iov_iter); - if (nbytes != payload_len) { + if (skb_copy_datagram_iter(skb, + frag_off, + &iov_iter, + payload_len)) { kfree_skb(skb); vq_err(vq, "Faulted on copying pkt buf\n"); break; @@ -212,13 +216,13 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock, vhost_add_used(vq, head, sizeof(*hdr) + payload_len); added = true; - skb_pull(skb, payload_len); + VIRTIO_VSOCK_SKB_CB(skb)->frag_off += payload_len; total_len += payload_len; /* If we didn't send all the payload we can requeue the packet * to send it with the next available buffer. */ - if (skb->len > 0) { + if (VIRTIO_VSOCK_SKB_CB(skb)->frag_off < skb->len) { hdr->flags |= cpu_to_le32(flags_to_restore); /* We are queueing the same skb to handle From patchwork Sat Jul 1 06:22:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 114977 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp10870840vqr; Fri, 30 Jun 2023 23:53:14 -0700 (PDT) X-Google-Smtp-Source: APBJJlHlKMIQdD6xDzeUwxyRRoBVQP0sM57CDCDZdC5eh+IuIeqZMKrRxHQx5x7zrLqMampz/yfm X-Received: by 2002:a17:902:7608:b0:1b8:2c6f:3248 with SMTP id k8-20020a170902760800b001b82c6f3248mr3948059pll.39.1688194393881; Fri, 30 Jun 2023 23:53:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688194393; cv=none; d=google.com; s=arc-20160816; b=vMf1KD9GMpENVovAF5QVlVTNog4AwlZU65c3mMfxCIXd22u75123jGhfPW+kfHYiSz mxtWH8IL6ZEYA5JjLMsnPoeqHkHf7gQROxJ/fOQsgvUUkUa6UoxCJQ4rsJ8aQMBZHedC UdKwvymEUGKYRN+7vzlEZRtPuI06ldor6XHKjZ22jz2Euihq4RuUIJDMPJFTsDLKz9jZ nsCk5EIDQAL3FM6q+afebAZJN7ElkS3x3qE1UIGuaHdrjFH/6F4pG4yriCIkA2HuwMK2 W21DGlUOEBeG5AuEmZKMXG94qWnZYhtE86KoCzUCqkxKKW73y7rQUa/+1W5ZfD2R8dGz C8Xg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=HVWROl+BBXUa2BfH+uJoxZPqiGPfyFPGa1/4nNCI6t8=; fh=PzWIi6JmlJNekYqVy23433fKO6OE4+V62ucRx731HUI=; b=mttvT1dUiz2cHVNg7olqZXAutvrAy4mffCXY7MBdo7RXhnXr8I7nWG4b+JrWdoeatq ZDklhc8YMLBesWScKPzGnZOkWwN4230B2aX+tXoiUnAfHWmdyhciX48kaFcFwXjnopRq ACb5/ki3bQHp2P1VEc+hsJ0N8lKwukCcdUIarvdooi8XOGEjwTvM+Q3k0yaGHQwRPgMp ZNe2NzQo9NH0r6Qg1qOlmSIREh+kQZxuZUa3ulC5JhNDSMowMwwIufw0nojbKUu4HgwB I3AluEr+Dz7+znbWvKyT+us4riNf06Z3usjq5RYPYJ0BEMHreDlGajissDHTIF04L1O/ 1GBg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=jOiCiWey; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q16-20020a170902bd9000b001b674055d72si13341541pls.621.2023.06.30.23.52.59; Fri, 30 Jun 2023 23:53:13 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=jOiCiWey; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231360AbjGAGlO (ORCPT + 99 others); Sat, 1 Jul 2023 02:41:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40290 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230168AbjGAGjr (ORCPT ); Sat, 1 Jul 2023 02:39:47 -0400 Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C5C27423F; Fri, 30 Jun 2023 23:39:37 -0700 (PDT) Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id 175F0120007; Sat, 1 Jul 2023 09:28:40 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru 175F0120007 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1688192920; bh=HVWROl+BBXUa2BfH+uJoxZPqiGPfyFPGa1/4nNCI6t8=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=jOiCiWeyGK3J/4fy6lO/51CaJ3v5r9LtDE7nmfVnVbYbARCjGWiAPKGUbLYkmfjhy 9GPOcI4Wfa0DDY3bbiQr6y46gPYXD+zYKfjP70bnGoOBXkNpIE29SFsyUlEGLrEUIY CQqHk8HcsVRYIOpZvd3PIEy+mL2ZX2tSaG5xucfmZzlS8o3mMFOIBzhkrrfmpg+gm4 k7mj2ciRXmsJUjPg0UE/B/+RdkEUuDXIq6pxbkKB39n2WU9JG1CGfhXBzGIRZbKFQH uT4eVftLLVrS9+7PH+FXKVY60jKeRoKHH5m8/2sbQgTn+FihIqQ18qFGeNfCpnOyrX eYWRMgNjcS5rA== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Sat, 1 Jul 2023 09:28:38 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Sat, 1 Jul 2023 09:28:18 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [RFC PATCH v5 03/17] vsock/virtio: support to send non-linear skb Date: Sat, 1 Jul 2023 09:22:56 +0300 Message-ID: <20230701062310.3397129-4-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> References: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178380 [Jun 30 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 517 517 b0056c19d8e10afbb16cb7aad7258dedb0179a79, {Tracking_from_domain_doesnt_match_to}, sberdevices.ru:5.0.1,7.1.1;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;127.0.0.199:7.1.2;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/01 00:58:00 #21597365 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770200124695553375?= X-GMAIL-MSGID: =?utf-8?q?1770200124695553375?= For non-linear skb use its pages from fragment array as buffers in virtio tx queue. These pages are already pinned by 'get_user_pages()' during such skb creation. Signed-off-by: Arseniy Krasnov --- Changelog: v4 -> v5: * Use 'out_sgs' variable to index 'bufs', not only 'sgs'. * Move smaller branch above, see 'if (!skb_is_nonlinear(skb)'). * Remove blank line. * R-b from Bobby Eshleman removed due to patch update. net/vmw_vsock/virtio_transport.c | 40 +++++++++++++++++++++++++++----- 1 file changed, 34 insertions(+), 6 deletions(-) diff --git a/net/vmw_vsock/virtio_transport.c b/net/vmw_vsock/virtio_transport.c index e95df847176b..6cbb45bb12d2 100644 --- a/net/vmw_vsock/virtio_transport.c +++ b/net/vmw_vsock/virtio_transport.c @@ -100,7 +100,9 @@ virtio_transport_send_pkt_work(struct work_struct *work) vq = vsock->vqs[VSOCK_VQ_TX]; for (;;) { - struct scatterlist hdr, buf, *sgs[2]; + /* +1 is for packet header. */ + struct scatterlist *sgs[MAX_SKB_FRAGS + 1]; + struct scatterlist bufs[MAX_SKB_FRAGS + 1]; int ret, in_sg = 0, out_sg = 0; struct sk_buff *skb; bool reply; @@ -111,12 +113,38 @@ virtio_transport_send_pkt_work(struct work_struct *work) virtio_transport_deliver_tap_pkt(skb); reply = virtio_vsock_skb_reply(skb); + sg_init_one(&bufs[out_sg], virtio_vsock_hdr(skb), + sizeof(*virtio_vsock_hdr(skb))); + sgs[out_sg] = &bufs[out_sg]; + out_sg++; + + if (!skb_is_nonlinear(skb)) { + if (skb->len > 0) { + sg_init_one(&bufs[out_sg], skb->data, skb->len); + sgs[out_sg] = &bufs[out_sg]; + out_sg++; + } + } else { + struct skb_shared_info *si; + int i; + + si = skb_shinfo(skb); + + for (i = 0; i < si->nr_frags; i++) { + skb_frag_t *skb_frag = &si->frags[i]; + void *va = page_to_virt(skb_frag->bv_page); - sg_init_one(&hdr, virtio_vsock_hdr(skb), sizeof(*virtio_vsock_hdr(skb))); - sgs[out_sg++] = &hdr; - if (skb->len > 0) { - sg_init_one(&buf, skb->data, skb->len); - sgs[out_sg++] = &buf; + /* We will use 'page_to_virt()' for userspace page here, + * because virtio layer will call 'virt_to_phys()' later + * to fill buffer descriptor. We don't touch memory at + * "virtual" address of this page. + */ + sg_init_one(&bufs[out_sg], + va + skb_frag->bv_offset, + skb_frag->bv_len); + sgs[out_sg] = &bufs[out_sg]; + out_sg++; + } } ret = virtqueue_add_sgs(vq, sgs, out_sg, in_sg, skb, GFP_KERNEL); From patchwork Sat Jul 1 06:22:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 114984 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp10871149vqr; Fri, 30 Jun 2023 23:53:58 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5frQqS1Qy+6OY3ohBtAKz1g1MvmO1MIWV6yXNNM9Cf25hk+5iw5wbnL5SQUWDVcF9r1MFM X-Received: by 2002:a17:90a:c78a:b0:262:ecdd:2799 with SMTP id gn10-20020a17090ac78a00b00262ecdd2799mr12252409pjb.8.1688194438345; Fri, 30 Jun 2023 23:53:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688194438; cv=none; d=google.com; s=arc-20160816; b=VVmGVcH+lfyUag59iPKDJYj+UBO/adxAFmw6nkoz8KTvbdhHK6VgXqr7qoBDzb3DpZ dL2QzmWqbtTLBGd1BFHLDxckV0UGz+NI2c01RcercoDa7tMNMu4yuiJFiVmra1Brtz// OX2dCZM2goRaTH6Ufmff4+m3rCv6WZibdvasqkJMv4t3n93Z+XoxH7yGzz7StYE04K/O 1B1MbyMdn1tv+2jGaB/6Ds0S6MHgEylqBo3pAeNTljhzZj+l8QgE9Pv3mQ29w3krs/dR XCYrWWBdG3X4fcSNwx7UN3vOAUP96XAq7eGYYqQ2FaAa9QmLW7SH6k9YjwAuVFrZcmd7 WQrg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=DaiKPxzYsqdAcaKBay/Cxep1Ax7Uwh6y3azpG4f7wPI=; fh=PzWIi6JmlJNekYqVy23433fKO6OE4+V62ucRx731HUI=; b=HN2uDM4a84/r7qzbGCDhPEDCM2c+kLt9Z0xGm8OJHuvEDdZ7FOkrgoDH49Y2SrvbT2 saxZ1Kcpa7waxSRaKfVSyW7Pa9dUK67H97jTouCyInyT3J7DuC5QplN73LmpAmSAy4yU iozanD4JK6TiO64emtyq78xr3GMRWy7dUsnOpBQ8eVjdGYvKLsLy/szGWMpY1yRJ+j0B 9b6j5foNM9Ze08AtzFfo1l3MOLHknuCY8VMb0Ls8Ti4pYbmfbLJVu+im2Tr/XXMEsYri 1MK9LU3X07+8cgrWLlUD6l0Irgpz7VSo4LdsmGvjcSmYyibxaqVQT/+cSthZH2sXvMho 30AA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=K8gArG2v; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k60-20020a17090a3ec200b002563251c8e7si2170053pjc.128.2023.06.30.23.53.42; Fri, 30 Jun 2023 23:53:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=K8gArG2v; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231342AbjGAGlG (ORCPT + 99 others); Sat, 1 Jul 2023 02:41:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39724 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230166AbjGAGjr (ORCPT ); Sat, 1 Jul 2023 02:39:47 -0400 Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 91EEE4239; Fri, 30 Jun 2023 23:39:34 -0700 (PDT) Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id 4E81B120008; Sat, 1 Jul 2023 09:28:41 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru 4E81B120008 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1688192921; bh=DaiKPxzYsqdAcaKBay/Cxep1Ax7Uwh6y3azpG4f7wPI=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=K8gArG2va7fDnIbHexfVRCq04H6pTCpLyy+yMGtzg6MwzUUBIn7ehWBNKp17FtA2w MpU2dGFmAhXqkWOt9ScsBdis1GoYH83ND/pqLQBiUDEMEf+qGSXdxPbrgYdSO5Xbc3 Y17nRV5cy4gRELPFYLwPHxQ+8D5VqPRfiSDilpWxAUV21FbPzP1GFzUN+CgEKvpJi/ b69wXFcqpn0X9NFGc4ERrb2LfxGpCSm9GLN24zL3ZRSjpC4NEcxP1bayWAO2A3vt4m cJBn3/0LCZz+rkxDSOXQjOcelXOz/IGUTJeu7O4SG5cydESrpeviLdzynzgP/EyScG QHHLcmJ5ybjhg== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Sat, 1 Jul 2023 09:28:40 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Sat, 1 Jul 2023 09:28:19 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [RFC PATCH v5 04/17] vsock/virtio: non-linear skb handling for tap Date: Sat, 1 Jul 2023 09:22:57 +0300 Message-ID: <20230701062310.3397129-5-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> References: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178380 [Jun 30 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 517 517 b0056c19d8e10afbb16cb7aad7258dedb0179a79, {Tracking_from_domain_doesnt_match_to}, sberdevices.ru:5.0.1,7.1.1;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;127.0.0.199:7.1.2;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/01 00:58:00 #21597365 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770200171581404852?= X-GMAIL-MSGID: =?utf-8?q?1770200171581404852?= For tap device new skb is created and data from the current skb is copied to it. This adds copying data from non-linear skb to new the skb. Signed-off-by: Arseniy Krasnov --- Changelog: v4 -> v5: * Make 'skb' pointer constant because it is source. net/vmw_vsock/virtio_transport_common.c | 31 ++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-) diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c index e5683af23e60..dfc48b56d0a2 100644 --- a/net/vmw_vsock/virtio_transport_common.c +++ b/net/vmw_vsock/virtio_transport_common.c @@ -106,6 +106,27 @@ virtio_transport_alloc_skb(struct virtio_vsock_pkt_info *info, return NULL; } +static void virtio_transport_copy_nonlinear_skb(const struct sk_buff *skb, + void *dst, + size_t len) +{ + struct iov_iter iov_iter = { 0 }; + struct kvec kvec; + size_t to_copy; + + kvec.iov_base = dst; + kvec.iov_len = len; + + iov_iter.iter_type = ITER_KVEC; + iov_iter.kvec = &kvec; + iov_iter.nr_segs = 1; + + to_copy = min_t(size_t, len, skb->len); + + skb_copy_datagram_iter(skb, VIRTIO_VSOCK_SKB_CB(skb)->frag_off, + &iov_iter, to_copy); +} + /* Packet capture */ static struct sk_buff *virtio_transport_build_skb(void *opaque) { @@ -114,7 +135,6 @@ static struct sk_buff *virtio_transport_build_skb(void *opaque) struct af_vsockmon_hdr *hdr; struct sk_buff *skb; size_t payload_len; - void *payload_buf; /* A packet could be split to fit the RX buffer, so we can retrieve * the payload length from the header and the buffer pointer taking @@ -122,7 +142,6 @@ static struct sk_buff *virtio_transport_build_skb(void *opaque) */ pkt_hdr = virtio_vsock_hdr(pkt); payload_len = pkt->len; - payload_buf = pkt->data; skb = alloc_skb(sizeof(*hdr) + sizeof(*pkt_hdr) + payload_len, GFP_ATOMIC); @@ -165,7 +184,13 @@ static struct sk_buff *virtio_transport_build_skb(void *opaque) skb_put_data(skb, pkt_hdr, sizeof(*pkt_hdr)); if (payload_len) { - skb_put_data(skb, payload_buf, payload_len); + if (skb_is_nonlinear(pkt)) { + void *data = skb_put(skb, payload_len); + + virtio_transport_copy_nonlinear_skb(pkt, data, payload_len); + } else { + skb_put_data(skb, pkt->data, payload_len); + } } return skb; From patchwork Sat Jul 1 06:22:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 114972 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp10870258vqr; Fri, 30 Jun 2023 23:51:31 -0700 (PDT) X-Google-Smtp-Source: APBJJlGNXDWb9iubrn+2D91p4/wL6J5j0DEy6TtKXyfZc3l0JTzzxf/uEweZTDrLdRvKLEvbV9AJ X-Received: by 2002:a92:d94d:0:b0:345:d3eb:470b with SMTP id l13-20020a92d94d000000b00345d3eb470bmr4511562ilq.17.1688194291595; Fri, 30 Jun 2023 23:51:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688194291; cv=none; d=google.com; s=arc-20160816; b=tTLsbATYJdffyoMjrpzrtv+STKI/a5ugj4BomNeYZlfwkRMJ0bd5gowFlyCYFHlV5l 07aerApPNUylvTYsa5+ge+ZVb+XH7p3kJR0CYTuro9iQyzIJXffYHyGnG71aC5ztj5LU MK8BWgt4WGHA0qe2FzEdgMAHVjvwlyiFD/c20Mu2Zl8knRs9b7LlueNnNI50TuLHpa3v hCjDvsFdQZ32O9XQFOcRLIqMf8Ra+nd2txvaB5/bG53da1kRmr/YyQor3SI/9Iz2HLwC ZJSpmpJD1x4VGn74ObVebDJ5FJxfnv1Y0KJHuk45fcGk0LLYSdrji23bAJS6BhWsNPHv ATjg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=m9zois+QCrFB15Af+wiZukpNNaD+yGxiy2VOPPrcIAU=; fh=PzWIi6JmlJNekYqVy23433fKO6OE4+V62ucRx731HUI=; b=BnIUw/Imq84tBfgqHVAPzI1r2sZ5OJkltvW+I+DAqvHkIr2z/kgDObMpbg4fKO059W LvHMVKpIt63z2JWPsHsz5r8hmfgN1LRBUb391CpIeI7p+DxWNyYf3K6Mgb/kEny9BfND 9Q6j7FScVX2RR3qKcVjtZE3/MmOH0PXvQ6EVWkx1NQ2pUkPqGnyfotPfvTnsuGNnh6yz Z10+avuibMuT61OhOPyDUy6nIdfvoHzNalEqH1n/qYqO2Aw/vqUq3Apf6Kf7oTXx9fRK G6I93QhSTeYJBzQsFAlLXr5F2ShCXLXiBUIuw+F7unsPKRkk2ZyYU34HqkK5hP8F/baM BaOA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b="qIJ/oIgj"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m1-20020a170902bb8100b001ae0691dfdbsi12878993pls.158.2023.06.30.23.51.14; Fri, 30 Jun 2023 23:51:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b="qIJ/oIgj"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231395AbjGAGlV (ORCPT + 99 others); Sat, 1 Jul 2023 02:41:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40294 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230174AbjGAGjr (ORCPT ); Sat, 1 Jul 2023 02:39:47 -0400 Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C594A423E; Fri, 30 Jun 2023 23:39:37 -0700 (PDT) Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id 7B077120009; Sat, 1 Jul 2023 09:28:42 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru 7B077120009 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1688192922; bh=m9zois+QCrFB15Af+wiZukpNNaD+yGxiy2VOPPrcIAU=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=qIJ/oIgjgGzGAhCK4Y+6JfmvwJN8ZZ+c1tti3+FFyEa/pJcJWYNUDr8+zFd0gyTrL nXt1suiH31AwYeD657OyY537AaMdDe+TNebr0ZZVprazRlZrAFgqXbBA4WVi5jIyXf 2yAD/VDAyq1lCpPceP+pOyfx3tumQUi88xiAY+mWTkB2qPnjEIp+VfYeB/muyHXwp8 9/LpsOC35wYKuIlfDQ/Hfqvf0owqlgIVQ+L+Q5q0/QhX9Lvq3Uc2w8Re0U2eNVvG6c WteAJbkaMB7/wEnOo2u8GCDvU8wySMjtc76B0vZGv1ZJI3Epm6iYeGq8XCe9zhw2Lu NP3LmolXeNG5g== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Sat, 1 Jul 2023 09:28:41 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Sat, 1 Jul 2023 09:28:19 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [RFC PATCH v5 05/17] vsock/virtio: MSG_ZEROCOPY flag support Date: Sat, 1 Jul 2023 09:22:58 +0300 Message-ID: <20230701062310.3397129-6-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> References: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178380 [Jun 30 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 517 517 b0056c19d8e10afbb16cb7aad7258dedb0179a79, {Tracking_from_domain_doesnt_match_to}, sberdevices.ru:5.0.1,7.1.1;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;127.0.0.199:7.1.2;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/01 00:58:00 #21597365 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770200017091368962?= X-GMAIL-MSGID: =?utf-8?q?1770200017091368962?= This adds handling of MSG_ZEROCOPY flag on transmission path: if this flag is set and zerocopy transmission is possible, then non-linear skb will be created and filled with the pages of user's buffer. Pages of user's buffer are locked in memory by 'get_user_pages()'. Second thing that this patch does is replace type of skb owning: instead of calling 'skb_set_owner_sk_safe()' it calls 'skb_set_owner_w()'. Reason of this change is that '__zerocopy_sg_from_iter()' increments 'sk_wmem_alloc' of socket, so to decrease this field correctly proper skb destructor is needed: 'sock_wfree()'. This destructor is set by 'skb_set_owner_w()'. Signed-off-by: Arseniy Krasnov --- Changelog: v4 -> v5: * Remove unused 'vsk' arg from 'virtio_transport_fill_linear_skb()'. * Remove old comment 'Returns a new packet on success...'. * Commit message update by adding details why this patch uses 'skb_set_owner_w()' instead of previous 'skb_set_owner_sk_safe()'. * 'hdr' variable declaration and assignment in a single line. * Rename 'max_skb_cap' to 'max_skb_len'. * Use 'info->msg->msg_flags' to check MSG_ZEROCOPY flag instead of field 'flags' of struct 'virtio_vsock_pkt_info'. This was a bug. net/vmw_vsock/virtio_transport_common.c | 262 ++++++++++++++++++------ 1 file changed, 200 insertions(+), 62 deletions(-) diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c index dfc48b56d0a2..2269530ea737 100644 --- a/net/vmw_vsock/virtio_transport_common.c +++ b/net/vmw_vsock/virtio_transport_common.c @@ -37,27 +37,99 @@ virtio_transport_get_ops(struct vsock_sock *vsk) return container_of(t, struct virtio_transport, transport); } -/* Returns a new packet on success, otherwise returns NULL. - * - * If NULL is returned, errp is set to a negative errno. - */ -static struct sk_buff * -virtio_transport_alloc_skb(struct virtio_vsock_pkt_info *info, - size_t len, - u32 src_cid, - u32 src_port, - u32 dst_cid, - u32 dst_port) -{ - const size_t skb_len = VIRTIO_VSOCK_SKB_HEADROOM + len; - struct virtio_vsock_hdr *hdr; - struct sk_buff *skb; +static bool virtio_transport_can_zcopy(struct virtio_vsock_pkt_info *info, + size_t max_to_send) +{ + struct iov_iter *iov_iter; + + if (!info->msg) + return false; + + iov_iter = &info->msg->msg_iter; + + /* Data is simple buffer. */ + if (iter_is_ubuf(iov_iter)) + return true; + + if (!iter_is_iovec(iov_iter)) + return false; + + if (iov_iter->iov_offset) + return false; + + /* We can't send whole iov. */ + if (iov_iter->count > max_to_send) + return false; + + return true; +} + +static int virtio_transport_init_zcopy_skb(struct vsock_sock *vsk, + struct sk_buff *skb, + struct msghdr *msg, + bool zerocopy) +{ + struct ubuf_info *uarg; + + if (msg->msg_ubuf) { + uarg = msg->msg_ubuf; + net_zcopy_get(uarg); + } else { + struct iov_iter *iter = &msg->msg_iter; + struct ubuf_info_msgzc *uarg_zc; + int len; + + /* Only ITER_IOVEC or ITER_UBUF are allowed and + * checked before. + */ + if (iter_is_iovec(iter)) + len = iov_length(iter->__iov, iter->nr_segs); + else + len = iter->count; + + uarg = msg_zerocopy_realloc(sk_vsock(vsk), + len, + NULL); + + if (!uarg) + return -1; + + uarg_zc = uarg_to_msgzc(uarg); + uarg_zc->zerocopy = zerocopy ? 1 : 0; + } + + skb_zcopy_init(skb, uarg); + + return 0; +} + +static int virtio_transport_fill_linear_skb(struct sk_buff *skb, + struct virtio_vsock_pkt_info *info, + size_t len) +{ void *payload; int err; - skb = virtio_vsock_alloc_skb(skb_len, GFP_KERNEL); - if (!skb) - return NULL; + payload = skb_put(skb, len); + err = memcpy_from_msg(payload, info->msg, len); + if (err) + return -1; + + if (msg_data_left(info->msg)) + return 0; + + return 0; +} + +static void virtio_transport_init_hdr(struct sk_buff *skb, + struct virtio_vsock_pkt_info *info, + u32 src_cid, + u32 src_port, + u32 dst_cid, + u32 dst_port, + size_t len) +{ + struct virtio_vsock_hdr *hdr; hdr = virtio_vsock_hdr(skb); hdr->type = cpu_to_le16(info->type); @@ -68,42 +140,6 @@ virtio_transport_alloc_skb(struct virtio_vsock_pkt_info *info, hdr->dst_port = cpu_to_le32(dst_port); hdr->flags = cpu_to_le32(info->flags); hdr->len = cpu_to_le32(len); - - if (info->msg && len > 0) { - payload = skb_put(skb, len); - err = memcpy_from_msg(payload, info->msg, len); - if (err) - goto out; - - if (msg_data_left(info->msg) == 0 && - info->type == VIRTIO_VSOCK_TYPE_SEQPACKET) { - hdr->flags |= cpu_to_le32(VIRTIO_VSOCK_SEQ_EOM); - - if (info->msg->msg_flags & MSG_EOR) - hdr->flags |= cpu_to_le32(VIRTIO_VSOCK_SEQ_EOR); - } - } - - if (info->reply) - virtio_vsock_skb_set_reply(skb); - - trace_virtio_transport_alloc_pkt(src_cid, src_port, - dst_cid, dst_port, - len, - info->type, - info->op, - info->flags); - - if (info->vsk && !skb_set_owner_sk_safe(skb, sk_vsock(info->vsk))) { - WARN_ONCE(1, "failed to allocate skb on vsock socket with sk_refcnt == 0\n"); - goto out; - } - - return skb; - -out: - kfree_skb(skb); - return NULL; } static void virtio_transport_copy_nonlinear_skb(const struct sk_buff *skb, @@ -214,6 +250,79 @@ static u16 virtio_transport_get_type(struct sock *sk) return VIRTIO_VSOCK_TYPE_SEQPACKET; } +static struct sk_buff *virtio_transport_alloc_skb(struct vsock_sock *vsk, + struct virtio_vsock_pkt_info *info, + size_t payload_len, + bool zcopy, + u32 dst_cid, + u32 dst_port, + u32 src_cid, + u32 src_port) +{ + struct sk_buff *skb; + size_t skb_len; + + skb_len = VIRTIO_VSOCK_SKB_HEADROOM; + + if (!zcopy) + skb_len += payload_len; + + skb = virtio_vsock_alloc_skb(skb_len, GFP_KERNEL); + if (!skb) + return NULL; + + virtio_transport_init_hdr(skb, info, src_cid, src_port, + dst_cid, dst_port, + payload_len); + + /* Set owner here, because '__zerocopy_sg_from_iter()' uses + * owner of skb without check to update 'sk_wmem_alloc'. + */ + if (vsk) + skb_set_owner_w(skb, sk_vsock(vsk)); + + if (info->msg && payload_len > 0) { + int err; + + if (zcopy) { + err = __zerocopy_sg_from_iter(info->msg, NULL, skb, + &info->msg->msg_iter, + payload_len); + } else { + err = virtio_transport_fill_linear_skb(skb, info, payload_len); + } + + if (err) + goto out; + + VIRTIO_VSOCK_SKB_CB(skb)->frag_off = 0; + + if (info->type == VIRTIO_VSOCK_TYPE_SEQPACKET) { + struct virtio_vsock_hdr *hdr = virtio_vsock_hdr(skb); + + hdr->flags |= cpu_to_le32(VIRTIO_VSOCK_SEQ_EOM); + + if (info->msg->msg_flags & MSG_EOR) + hdr->flags |= cpu_to_le32(VIRTIO_VSOCK_SEQ_EOR); + } + } + + if (info->reply) + virtio_vsock_skb_set_reply(skb); + + trace_virtio_transport_alloc_pkt(src_cid, src_port, + dst_cid, dst_port, + payload_len, + info->type, + info->op, + info->flags); + + return skb; +out: + kfree_skb(skb); + return NULL; +} + /* This function can only be used on connecting/connected sockets, * since a socket assigned to a transport is required. * @@ -226,6 +335,8 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, const struct virtio_transport *t_ops; struct virtio_vsock_sock *vvs; u32 pkt_len = info->pkt_len; + bool can_zcopy = false; + u32 max_skb_len; u32 rest_len; int ret; @@ -254,22 +365,49 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk, if (pkt_len == 0 && info->op == VIRTIO_VSOCK_OP_RW) return pkt_len; + /* If zerocopy is not enabled by 'setsockopt()', we behave as + * there is no MSG_ZEROCOPY flag set. + */ + if (info->msg && !sock_flag(sk_vsock(vsk), SOCK_ZEROCOPY)) + info->msg->msg_flags &= ~MSG_ZEROCOPY; + + if (info->msg && info->msg->msg_flags & MSG_ZEROCOPY) + can_zcopy = virtio_transport_can_zcopy(info, pkt_len); + + if (can_zcopy) + max_skb_len = min_t(u32, VIRTIO_VSOCK_MAX_PKT_BUF_SIZE, + (MAX_SKB_FRAGS * PAGE_SIZE)); + else + max_skb_len = VIRTIO_VSOCK_MAX_PKT_BUF_SIZE; + rest_len = pkt_len; do { struct sk_buff *skb; size_t skb_len; - skb_len = min_t(u32, VIRTIO_VSOCK_MAX_PKT_BUF_SIZE, rest_len); + skb_len = min(max_skb_len, rest_len); - skb = virtio_transport_alloc_skb(info, skb_len, - src_cid, src_port, - dst_cid, dst_port); + skb = virtio_transport_alloc_skb(vsk, info, skb_len, can_zcopy, + dst_cid, dst_port, + src_cid, src_port); if (!skb) { ret = -ENOMEM; break; } + /* This is last skb to send this portion of data. */ + if (skb_len == rest_len && info->msg && + info->msg->msg_flags & MSG_ZEROCOPY && + info->op == VIRTIO_VSOCK_OP_RW) { + if (virtio_transport_init_zcopy_skb(vsk, skb, + info->msg, + can_zcopy)) { + ret = -ENOMEM; + break; + } + } + virtio_transport_inc_tx_pkt(vvs, skb); ret = t_ops->send_pkt(skb); @@ -939,11 +1077,11 @@ static int virtio_transport_reset_no_sock(const struct virtio_transport *t, if (!t) return -ENOTCONN; - reply = virtio_transport_alloc_skb(&info, 0, - le64_to_cpu(hdr->dst_cid), - le32_to_cpu(hdr->dst_port), + reply = virtio_transport_alloc_skb(NULL, &info, 0, false, le64_to_cpu(hdr->src_cid), - le32_to_cpu(hdr->src_port)); + le32_to_cpu(hdr->src_port), + le64_to_cpu(hdr->dst_cid), + le32_to_cpu(hdr->dst_port)); if (!reply) return -ENOMEM; From patchwork Sat Jul 1 06:22:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 114975 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp10870600vqr; Fri, 30 Jun 2023 23:52:28 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6RWrnKAdS2QzloqIA9fQouSlP8B5t76A639PNvOp9b72R8fQ+/uzPQqJdTIzcg9CZ1GBXv X-Received: by 2002:a05:6808:1cb:b0:39e:a606:544e with SMTP id x11-20020a05680801cb00b0039ea606544emr4684429oic.32.1688194348459; Fri, 30 Jun 2023 23:52:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688194348; cv=none; d=google.com; s=arc-20160816; b=MO3fskBqkYBp6SV1UWWyvzTohpoI8PkQMomBYkax2juLs2citHlTy4zEMnTM6yNgq5 QaH2DV9pRbQkSH/T5LgfQSkGlEMnXg4yv0SnCGrGof9j4F694cp4ctOlBnJMBmijDPPA +RvkkvbvTj39u1gvkJeEURvQNQerH20mNi6kBXIrj8F47t9mFCnvCjhpxEihKXOnzB+6 J1RIdr9qb7+Dk+dtwZ2Ni9B/ZG2pkf0NYAgUiA9AN4Q8ZCh0JW1R2ShvS2XUA0iwDE0E dfo3sTYG4HDN9z2oFwRqazDIC5ga/OJkYKNiNraEZZoyAR0sr+EA9VGlkHGbf0ElZZRa o2LQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=2BWXHhctpAkbTJhGKfSJxyOEOoSV2p85OEMm5gTeQ6k=; fh=PzWIi6JmlJNekYqVy23433fKO6OE4+V62ucRx731HUI=; b=1BFy5j1BJ/rNK4ZiTBqcLaZ1fKNO1bolO9WdIByOIUgx/0EecMANMJ936dcQk/hhYT alrxCcPX/5J9IgM2hBSMDVP3JvIdC60SyxaOW7ysN4Wwq13iemP0StCvfw1sAx60lx7r l1Yl6M+iHZ1wcu43W8hydlIhDWcUcVzvmy/HKeJIRppcbfVV+s30NXG9HPnPjJTtDJis 0vxX/B1TJzVf2K2axHPZw2Uijam35HQOCfLj6mJcV4PpczOfgbQ/dgJ/DImNarGr1yo0 6wHbHhw8+/HHm+Wb4Xkox/AqhT9kDBUugsN/kAjxvc8vDv0mXZAwsUEKnNVQ0TCdTfcV eX2Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=Td0B7dNn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id ay24-20020a056a00301800b0068219eb27a9si4904990pfb.233.2023.06.30.23.52.14; Fri, 30 Jun 2023 23:52:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=Td0B7dNn; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229942AbjGAGkc (ORCPT + 99 others); Sat, 1 Jul 2023 02:40:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40134 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230059AbjGAGjj (ORCPT ); Sat, 1 Jul 2023 02:39:39 -0400 Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8190D4232; Fri, 30 Jun 2023 23:39:33 -0700 (PDT) Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id A0E3612000C; Sat, 1 Jul 2023 09:28:43 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru A0E3612000C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1688192923; bh=2BWXHhctpAkbTJhGKfSJxyOEOoSV2p85OEMm5gTeQ6k=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=Td0B7dNnXU2D9+IJBugEC1PQ5fUymlSr2OUd8FYFXT2E92rwaCQWnBqjPRN//cmKh TX78gLil0vrt2GyeuEKBMbF0YGLfXQxdJlPFDqWF0B0HqEqlaw0JrUIuHh2cCzzrZz LnG+al4rJIaBPhCNtAmvbOKXw76L+truHziLvxVz9T/KRxNm9+U62Csk/yKLuwPRQ0 WSUcuYUfqN3eQBTDS2HOfM/4UeLmey7Br867OgrEiZd1NCHD/VRp0E10w+l5JSk13t vRvKvBNtV5aONJdpFqI0rZGW56ZPVSdLuGPmN+2ZGA0D67W7Ux8UEw7C6WvYFAKx6x j3gl7GaGBw1sA== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Sat, 1 Jul 2023 09:28:42 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Sat, 1 Jul 2023 09:28:20 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [RFC PATCH v5 06/17] vsock: fix EPOLLERR set on non-empty error queue Date: Sat, 1 Jul 2023 09:22:59 +0300 Message-ID: <20230701062310.3397129-7-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> References: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178380 [Jun 30 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 517 517 b0056c19d8e10afbb16cb7aad7258dedb0179a79, {Tracking_from_domain_doesnt_match_to}, sberdevices.ru:5.0.1,7.1.1;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;127.0.0.199:7.1.2;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/01 00:58:00 #21597365 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770200076795233572?= X-GMAIL-MSGID: =?utf-8?q?1770200076795233572?= If socket's error queue is not empty, EPOLLERR must be set. Otherwise, reader of error queue won't detect data in it using EPOLLERR bit. Currently for AF_VSOCK this is reproducible only with MSG_ZEROCOPY, as this feature is the only user of an error queue of the socket. Fixes: d021c344051a ("VSOCK: Introduce VM Sockets") Signed-off-by: Arseniy Krasnov --- Changelog: v4 -> v5: * Change commit message as Fix patch. Also add details that this problem could be reproduced only with MSG_ZEROCOPY transmission mode. net/vmw_vsock/af_vsock.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/vmw_vsock/af_vsock.c b/net/vmw_vsock/af_vsock.c index efb8a0937a13..45fd20c4ed50 100644 --- a/net/vmw_vsock/af_vsock.c +++ b/net/vmw_vsock/af_vsock.c @@ -1030,7 +1030,7 @@ static __poll_t vsock_poll(struct file *file, struct socket *sock, poll_wait(file, sk_sleep(sk), wait); mask = 0; - if (sk->sk_err) + if (sk->sk_err || !skb_queue_empty_lockless(&sk->sk_error_queue)) /* Signify that there has been an error on this socket. */ mask |= EPOLLERR; From patchwork Sat Jul 1 06:23:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 114983 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp10871142vqr; Fri, 30 Jun 2023 23:53:57 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7hP6oiiPUhXLXxn6mOkJYao+5v7JzmOAb6atJl+c73CPInG5k3a+o4+WRg+4Y1nVI7d/Bu X-Received: by 2002:a05:6808:f8f:b0:3a3:6f89:5de1 with SMTP id o15-20020a0568080f8f00b003a36f895de1mr6323889oiw.49.1688194437368; Fri, 30 Jun 2023 23:53:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688194437; cv=none; d=google.com; s=arc-20160816; b=nc95jJe2eGtXOjypp8y5ZtNBKT9pq9+EEm2wlMjNeX3wRhC6q/ngcKip/j2e8dye9j 3VQ1Xrjx3rQ22YNyob3kYIltdlWtilx7/t8GVGYyPxo1/h9mlwRMUigjOPOZ6dUrV/oT IOtSb7JMnnsonmmR8nYVDTG7DfUAXlwAGCnpOyqGSln6MzW4Tv5daQ9iY6klIH0YrLmJ Mfeuro8gdvpbsl60gsS81OzAwWIhYyIIPMeyKze0xLZDoZk3dFOEXHcReBjMmGK1tIJ6 BNp/p4ZV6owxVt25aJqlvnlgxV9E5XXay9Qp4uu/dwP2JXLZSVgF8m0pcGpFvtcPRTmF WClA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=UprXoHq7Bqi5L60D4m4n3ECZ9Zuve/BBPpn+ZPhJids=; fh=PzWIi6JmlJNekYqVy23433fKO6OE4+V62ucRx731HUI=; b=kU742Y6W3PFAoMmFi+ZWyJwgPqqNRqmiE6Q2XHsFwnBW/8+zbEU9UWLqX+Jsp4eYo1 Tn9Tvog6jBJQ7FM7wzt1+JZg0DRGObvSSM9soPzLMzv5Re4QDAABg9ufCvPo8S425VVI Xj7f9QHTMC578ajMJ1d8dRwbEd3iC863WgUgdkrPfDyhdOaivGAwf0fujWP1LRTWa2z4 TKkqJeTU67jJTA+b8v7ehnxg/C6vKFwItaYrGt+68o968Lc05m1c4mn/3cvs6wtVe5TH QMxBxUl4TMlk51V2W1eV5/jqs+FJYEbeAWPGFAxPUDODJ3ADOMKXJrHs6PuP2BjIeGTa 2Kww== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=l1fIFjg0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id br12-20020a056a00440c00b0063b8a054e06si12701458pfb.259.2023.06.30.23.53.42; Fri, 30 Jun 2023 23:53:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=l1fIFjg0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230086AbjGAGjl (ORCPT + 99 others); Sat, 1 Jul 2023 02:39:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39942 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229890AbjGAGjW (ORCPT ); Sat, 1 Jul 2023 02:39:22 -0400 Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E03F44237; Fri, 30 Jun 2023 23:39:13 -0700 (PDT) Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id C676812000E; Sat, 1 Jul 2023 09:28:44 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru C676812000E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1688192924; bh=UprXoHq7Bqi5L60D4m4n3ECZ9Zuve/BBPpn+ZPhJids=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=l1fIFjg0bsJNU8nXttK7I4WaCQhvF7u05F5ojLkYPS9hlepzoGV+7p9G4WEr2Gd1f UWoGSnj3vJs85PBvdYEJ3POiPFc71m+MxKRMWLhk1ESiOJTGmLDs/uTxJKvyP0WJb1 iodDilyoFW8/gnEGRFy6ssfduFkiTan+wWA5gQnIjJIYdewHeCvQovw8q6L+unj/B0 Iev6nWrOOK4GbVY8zCt4LQTplv8PBdvnOtl6j5MLR40/qgjnD6yZmUPqZbyvsDnZ+v n/rW/YZnQ+DdGhqhKPCzvHgaUoeFT1b0tOrUWcrVhDRYlJmHYGfP50Yf7dl6NdrzD1 a1zdi0o4E4P8w== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Sat, 1 Jul 2023 09:28:43 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Sat, 1 Jul 2023 09:28:21 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [RFC PATCH v5 07/17] vsock: read from socket's error queue Date: Sat, 1 Jul 2023 09:23:00 +0300 Message-ID: <20230701062310.3397129-8-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> References: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178380 [Jun 30 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 517 517 b0056c19d8e10afbb16cb7aad7258dedb0179a79, {Tracking_from_domain_doesnt_match_to}, sberdevices.ru:5.0.1,7.1.1;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;127.0.0.199:7.1.2;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/01 00:58:00 #21597365 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770200170554369013?= X-GMAIL-MSGID: =?utf-8?q?1770200170554369013?= This adds handling of MSG_ERRQUEUE input flag in receive call. This flag is used to read socket's error queue instead of data queue. Possible scenario of error queue usage is receiving completions for transmission with MSG_ZEROCOPY flag. This patch also adds 'SOL_VSOCK' define. Signed-off-by: Arseniy Krasnov --- Changelog: v4 -> v5: * Update commit message by adding sentence that 'SOL_VSOCK' is also added. include/linux/socket.h | 1 + net/vmw_vsock/af_vsock.c | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/include/linux/socket.h b/include/linux/socket.h index bd1cc3238851..d79efd026880 100644 --- a/include/linux/socket.h +++ b/include/linux/socket.h @@ -382,6 +382,7 @@ struct ucred { #define SOL_MPTCP 284 #define SOL_MCTP 285 #define SOL_SMC 286 +#define SOL_VSOCK 287 /* IPX options */ #define IPX_TYPE 1 diff --git a/net/vmw_vsock/af_vsock.c b/net/vmw_vsock/af_vsock.c index 45fd20c4ed50..07803d9fbf6d 100644 --- a/net/vmw_vsock/af_vsock.c +++ b/net/vmw_vsock/af_vsock.c @@ -110,6 +110,7 @@ #include #include #include +#include static int __vsock_bind(struct sock *sk, struct sockaddr_vm *addr); static void vsock_sk_destruct(struct sock *sk); @@ -2135,6 +2136,10 @@ vsock_connectible_recvmsg(struct socket *sock, struct msghdr *msg, size_t len, int err; sk = sock->sk; + + if (unlikely(flags & MSG_ERRQUEUE)) + return sock_recv_errqueue(sk, msg, len, SOL_VSOCK, 0); + vsk = vsock_sk(sk); err = 0; From patchwork Sat Jul 1 06:23:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 114980 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp10871089vqr; Fri, 30 Jun 2023 23:53:53 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7x+UCNbI6T1hxnNiKgggfw2rca6EKfucDkSpjpr59KQqL1M9IaFn4GXdEDISr9MCm4Ga40 X-Received: by 2002:a9d:4e98:0:b0:6af:a471:1ab3 with SMTP id v24-20020a9d4e98000000b006afa4711ab3mr5443896otk.4.1688194433364; Fri, 30 Jun 2023 23:53:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688194433; cv=none; d=google.com; s=arc-20160816; b=YClRwMGHkHfG1tPEPIMYL9sEfBguM5ITvk/5wxdFwUCEUhPhjrnSsUjlHlIjZxEjuo uKHDBICiFvAlJop6UYV54eT7DzTNnQPQq3SrF1g+CrM/pBTItfXYvwtGTclYVJijMnHl 1SPUupt3UOqIL1mzg02koEx+GA8R26U0JRA0OWMrHVKGd5rAD0LbWkBq+HEeWFNrkNjq 8TiS3ZVtDZo9k54wH2TPG3oogj7MxzU2hH3W/hSaKdtNGMp6hnwHnN++/LJ1aPbyef5w DHRaZoFX9qVoHPcAr6QtU1f2tmGndu3yjyqoSqMjCpFBW9D6FDhQtQ8bTnuwI9B/KQJA cPLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=dXr+GIu8Zm+IAJzZ6mwmY2cDmvd5mo+b/fOvW2U8kdk=; fh=PzWIi6JmlJNekYqVy23433fKO6OE4+V62ucRx731HUI=; b=xJPden+R+hSU2ZOqO+DK/1i8rjSvZTtYgJT8ZIguqKag2guYqyvYyBjchWOnSqONoO x/Ptyi2Oh5vyj8laKz8cAzAGMzSWDAPFb82XDj/W1dA4rxj6iCzGcN9b9YF2MEo9c1xO f2qpEngW9I/tg9BKxyeLVMWJDVlBLOFBMuavEhgBA7W/tAWsELLx3KUjPkj8Jbt1ZB32 cYZRc8GqtgFiQtolyjsW8Z950AO/JIUOeMDEJKknW0nU9axhg1eK3TWcgxZ9JetWgSSH iv69kJz30focTFcBQ0LuOwiDTPiMs3Dinpjq6kXtUhecsyWRke6qyDBRB+eQLQneuoOX 9smA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=QZrmFC4P; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j16-20020a63e750000000b0053eeed3d26asi13699620pgk.688.2023.06.30.23.53.38; Fri, 30 Jun 2023 23:53:53 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=QZrmFC4P; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230154AbjGAGjq (ORCPT + 99 others); Sat, 1 Jul 2023 02:39:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39936 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229891AbjGAGjW (ORCPT ); Sat, 1 Jul 2023 02:39:22 -0400 Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BBFBA4236; Fri, 30 Jun 2023 23:39:13 -0700 (PDT) Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id 0471F12000F; Sat, 1 Jul 2023 09:28:46 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru 0471F12000F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1688192926; bh=dXr+GIu8Zm+IAJzZ6mwmY2cDmvd5mo+b/fOvW2U8kdk=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=QZrmFC4PlO3klITr+HkONqkpFls4BdprnTeeTQTqOtzY1jdkG1gkcCealFq6Or26O cq5Sqv/oe02NjDDJvpERpE8gXUWUlHSy5CkgLkeEhbqsCPGF5Hdn5vB9IXmHWcSJiq fYEHxrDxcaNlslan/4xH3DsCwOQtYsLLLqpE7Q1tymrdtuOY5mJbppN0AF3wZI5uda SUYo6QVINahM8F8t+O7TpP4rl0hpYxKBbzUxALHBOQnnDXllhCqqMZInb0TDCCKkpP 9KxE73F/TFZkgm7Et72x0Np2D8ADKlNW4sp1kt4tAXh5sW3n4f6MgUaByYoEhAIIK3 8jZo4ZUH2Iveg== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Sat, 1 Jul 2023 09:28:44 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Sat, 1 Jul 2023 09:28:21 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [RFC PATCH v5 08/17] vsock: check for MSG_ZEROCOPY support on send Date: Sat, 1 Jul 2023 09:23:01 +0300 Message-ID: <20230701062310.3397129-9-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> References: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178380 [Jun 30 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 517 517 b0056c19d8e10afbb16cb7aad7258dedb0179a79, {Tracking_from_domain_doesnt_match_to}, sberdevices.ru:5.0.1,7.1.1;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;127.0.0.199:7.1.2;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/01 00:58:00 #21597365 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770200165975426742?= X-GMAIL-MSGID: =?utf-8?q?1770200165975426742?= This feature totally depends on transport, so if transport doesn't support it, return error. Signed-off-by: Arseniy Krasnov --- include/net/af_vsock.h | 7 +++++++ net/vmw_vsock/af_vsock.c | 6 ++++++ 2 files changed, 13 insertions(+) diff --git a/include/net/af_vsock.h b/include/net/af_vsock.h index 0e7504a42925..ec09edc5f3a0 100644 --- a/include/net/af_vsock.h +++ b/include/net/af_vsock.h @@ -177,6 +177,9 @@ struct vsock_transport { /* Read a single skb */ int (*read_skb)(struct vsock_sock *, skb_read_actor_t); + + /* Zero-copy. */ + bool (*msgzerocopy_allow)(void); }; /**** CORE ****/ @@ -243,4 +246,8 @@ static inline void __init vsock_bpf_build_proto(void) {} #endif +static inline bool vsock_msgzerocopy_allow(const struct vsock_transport *t) +{ + return t->msgzerocopy_allow && t->msgzerocopy_allow(); +} #endif /* __AF_VSOCK_H__ */ diff --git a/net/vmw_vsock/af_vsock.c b/net/vmw_vsock/af_vsock.c index 07803d9fbf6d..033006e1b5ad 100644 --- a/net/vmw_vsock/af_vsock.c +++ b/net/vmw_vsock/af_vsock.c @@ -1824,6 +1824,12 @@ static int vsock_connectible_sendmsg(struct socket *sock, struct msghdr *msg, goto out; } + if (msg->msg_flags & MSG_ZEROCOPY && + !vsock_msgzerocopy_allow(transport)) { + err = -EOPNOTSUPP; + goto out; + } + /* Wait for room in the produce queue to enqueue our user's data. */ timeout = sock_sndtimeo(sk, msg->msg_flags & MSG_DONTWAIT); From patchwork Sat Jul 1 06:23:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 114973 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp10870515vqr; Fri, 30 Jun 2023 23:52:14 -0700 (PDT) X-Google-Smtp-Source: APBJJlF51Nbj4gMk8gKTkAf90hJHPyZAXUkZlIeTsPTqGoakKLpxABhx+yQVAK9JZCcEwe8dNY0O X-Received: by 2002:a05:6a00:1a8e:b0:682:54b9:1093 with SMTP id e14-20020a056a001a8e00b0068254b91093mr6185779pfv.15.1688194334227; Fri, 30 Jun 2023 23:52:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688194334; cv=none; d=google.com; s=arc-20160816; b=xCk8qt04y8RgTkVEGFBV7efM0OxeOzQnNLjj5l+7SbDa5N8yvVcff4Wnd+65WRZc6S 1D1MjK7b73doKHK3Gp1wPDTUxGamujI/Ha1rZpo12TpHdPGhvhaFu6kpPYlT3f/T/hdc 0wHIquGjwC31jilQT1pzAjpB0/ILWuQsmXekcZzRPQHlrxLWRwGWS+Dpy//6WcMrjNQR cLp56oUdR4y5ltksiW9VHOS1ew/DA2wYuwr3EPy8ZpT8Rx+ItVEWyRssjpnFZYhvD4jE QwOsua7cj9U6KVgShQJ3q31JVcp1MCe636bybJT6knmoBGwZb+8TKVa5jL80IohX93bO 5SKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=HekjpGnl2zxNT3T4PtMzqPreuhYkvcvhGbre4mbN788=; fh=PzWIi6JmlJNekYqVy23433fKO6OE4+V62ucRx731HUI=; b=fdtIEjIF/J2QMuXBG7lJdmlZ1xDqGPk3vblblrmRfd/Mpm3few7DVJGOJ9pxqI8Ncp fKQ0yU2D8rgnq2L2Y1c9tMQDaMe2qqZpIYK49DgMEMZVLSVwZzZ+Hh34Nye7dx1mgLZm VIUpfN21CXwSeAPAPSCNWAS1pUIfQR4ImpSPcS9FBJJMz9UJ/gnLVnoeNd+g/rHNidqA G5ScJ0tNf9u2bXt8mh4gIbj+h5EuiSQv9bn4iB0AyZmr5UFfsfOUXImqnAa4fcuHvdFM f0cK6jasR3f+PUCti7ISR/2WCW8bdVOQLdi0kV7pbPBmQR4Rk/HkxlVON6hKCfrympuO r6jg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b="YS9Y/4Nq"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id cp24-20020a056a00349800b006728f1913cfsi12276110pfb.336.2023.06.30.23.51.59; Fri, 30 Jun 2023 23:52:14 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b="YS9Y/4Nq"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230182AbjGAGjv (ORCPT + 99 others); Sat, 1 Jul 2023 02:39:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229847AbjGAGjW (ORCPT ); Sat, 1 Jul 2023 02:39:22 -0400 Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E9E4423A; Fri, 30 Jun 2023 23:39:13 -0700 (PDT) Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id 5B4E0120013; Sat, 1 Jul 2023 09:28:48 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru 5B4E0120013 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1688192928; bh=HekjpGnl2zxNT3T4PtMzqPreuhYkvcvhGbre4mbN788=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=YS9Y/4Nq7QyLj3q0n+STThG3zG7NIO38sMUdlqQsBxBY+Cs1l5DBwRZsSAsRN4qRD shFpx71KLAqGJBbdewqAEsO6Va/sEsGXmIuRwGzoRuR01heD8DhBDgiJj0mchqHtz4 TBOtk/TfvNg1yBztQsG4uG86DGGCjLD4/YResK+CY5bXUNKtyO2xJkn/LZk+C9Ywjn ixaYrmI/Mh/Sj4akm2f2g+fFG4qF+8Ec5fZ+VMTZ9Sg+hqFeh1XTYoTORvnX3XtqdN 0Ftws5OlfFrST+0RHu1x0uO7wyt/sNSAUL1BoWeztc0/gjgDYdAGx9H7aYiY9HlBVG QeRWR9VkFy61w== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Sat, 1 Jul 2023 09:28:47 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Sat, 1 Jul 2023 09:28:22 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [RFC PATCH v5 09/17] vsock: enable SOCK_SUPPORT_ZC bit Date: Sat, 1 Jul 2023 09:23:02 +0300 Message-ID: <20230701062310.3397129-10-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> References: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178380 [Jun 30 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 517 517 b0056c19d8e10afbb16cb7aad7258dedb0179a79, {Tracking_from_domain_doesnt_match_to}, sberdevices.ru:5.0.1,7.1.1;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;127.0.0.199:7.1.2;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/01 00:58:00 #21597365 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770200062536889381?= X-GMAIL-MSGID: =?utf-8?q?1770200062536889381?= This bit is used by io_uring in case of zerocopy tx mode. io_uring code checks, that socket has this feature. This patch sets it in two places: 1) For socket in 'connect()' call. 2) For new socket which is returned by 'accept()' call. Signed-off-by: Arseniy Krasnov --- net/vmw_vsock/af_vsock.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/net/vmw_vsock/af_vsock.c b/net/vmw_vsock/af_vsock.c index 033006e1b5ad..da22ae0ef477 100644 --- a/net/vmw_vsock/af_vsock.c +++ b/net/vmw_vsock/af_vsock.c @@ -1406,6 +1406,9 @@ static int vsock_connect(struct socket *sock, struct sockaddr *addr, goto out; } + if (vsock_msgzerocopy_allow(transport)) + set_bit(SOCK_SUPPORT_ZC, &sk->sk_socket->flags); + err = vsock_auto_bind(vsk); if (err) goto out; @@ -1560,6 +1563,9 @@ static int vsock_accept(struct socket *sock, struct socket *newsock, int flags, } else { newsock->state = SS_CONNECTED; sock_graft(connected, newsock); + if (vsock_msgzerocopy_allow(vconnected->transport)) + set_bit(SOCK_SUPPORT_ZC, + &connected->sk_socket->flags); } release_sock(connected); From patchwork Sat Jul 1 06:23:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 114971 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp10867271vqr; Fri, 30 Jun 2023 23:42:15 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6DUaeAO2AGth+RmhVc7fYyzqFlYU8xg7J5wgBAay6Po/iZIaO9Y1J6JSlFHx37oFr0V1K7 X-Received: by 2002:a05:6808:d4e:b0:3a3:7e8d:b0b with SMTP id w14-20020a0568080d4e00b003a37e8d0b0bmr6138054oik.22.1688193734855; Fri, 30 Jun 2023 23:42:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688193734; cv=none; d=google.com; s=arc-20160816; b=yaCQu1BP+9Bqs8O5YeWoe8kJa5RTHYhzVza2cTBMj+Vz0ZSHc31xbljNYZQLNUm5zQ EG28ZBfsLdbQxnHUXrJ/2DP0tUJzvnZEmfZ5aGzOYjzIx8qv0PmHEX0L06QXzUD5wddY joteazuM1ZB4pZBhNb+jYrNIDlPSClROC3ojiFeHvdHwd10mRDxh0YCs6si4N3PxmjKQ kME8Fq+NWBZLeOqEym5bf9uZif1CGg+yb3DYbfo4mwA1LPSvKfl8hf+x8TsjhGCVL8tr NbltUJhuIOCvITyRO1WqXnWqH90AjROVhHvtTvnGzTKwLMTPdizmhzRjMMH7YsLBqsqN xfyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=tAX9BS9Qmyzlmpr1k17sJ0Kzs+6bGB2QVrF/rbwEyDA=; fh=PzWIi6JmlJNekYqVy23433fKO6OE4+V62ucRx731HUI=; b=jzIlr6B1GjjMxcwg82h22WuIfQrqpthgV+z4BXeHZ+f9wTcmQ2+5p6JMZ36V+s95Ht Hi4muJtNz/Lz5LENVmzg/NvWczxM0yhv+HgkPE6jbM5wc+D7PdLy6uN9p4DzZ2B+joVe xdr1A3Gdk9rWi8elt7BGfIV0k5CyHTSZLSBc9rflZ52Pwk2ZIc1m6qJGQrbO0+AvMrEE xVm4cR0tgSdoRcJ45RsaKhmnmclqQcJOrSBjRmWQQ0ycICAk/h6KD/xcc2B4PUbAxTUw y2CdZfxUNOyfQqzs4lwRixgbjXozsUD0lI0kaOjatZ15SqvGVIj4t0Y3KWVSS/Fku3cd tAKA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=DeNDBFfp; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g18-20020aa796b2000000b00682689dbdd1si1880858pfk.278.2023.06.30.23.42.00; Fri, 30 Jun 2023 23:42:14 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=DeNDBFfp; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231254AbjGAGkw (ORCPT + 99 others); Sat, 1 Jul 2023 02:40:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40274 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230141AbjGAGjq (ORCPT ); Sat, 1 Jul 2023 02:39:46 -0400 Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 76D66423A; Fri, 30 Jun 2023 23:39:34 -0700 (PDT) Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id 88B55120014; Sat, 1 Jul 2023 09:28:49 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru 88B55120014 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1688192929; bh=tAX9BS9Qmyzlmpr1k17sJ0Kzs+6bGB2QVrF/rbwEyDA=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=DeNDBFfpH9IbbeqVzIvq/we2164xAwwjrKwlFH+Dmnq2ALgwrGUuw3tYz/xsDshbM f5lfBjuZk+iHsxiKiMxySwecX6WFoc93NmNLuPghjYUFBWBXDsBrGG1Oaj0BPjHE5b epwl2FwoIzFf4iS0NIMlK/6tsS/3CE5RbHTOiqwJUnDHIZZsdtVg4s/AwZ9/7atxed EcHa8n625gGiTEzV+Dva/q5pWjtANlW/vGqh2IqLEBRCpl6y/TDHtwl/g/QmppQkYp tc0kbqoy1taoo6I7RpiJQBt10bcjY54G31G009/VvCyeIsy9Znu4MwsCfbUFvlYfpa k+xm0Rt16aaYg== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Sat, 1 Jul 2023 09:28:48 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Sat, 1 Jul 2023 09:28:23 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [RFC PATCH v5 10/17] vhost/vsock: support MSG_ZEROCOPY for transport Date: Sat, 1 Jul 2023 09:23:03 +0300 Message-ID: <20230701062310.3397129-11-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> References: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178380 [Jun 30 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 517 517 b0056c19d8e10afbb16cb7aad7258dedb0179a79, {Tracking_from_domain_doesnt_match_to}, sberdevices.ru:5.0.1,7.1.1;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;127.0.0.199:7.1.2;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/01 00:58:00 #21597365 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770199433759217029?= X-GMAIL-MSGID: =?utf-8?q?1770199433759217029?= Add 'msgzerocopy_allow()' callback for vhost transport. Signed-off-by: Arseniy Krasnov --- Changelog: v4 -> v5: * Move 'msgzerocopy_allow' right after seqpacket callbacks. drivers/vhost/vsock.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c index cb00e0e059e4..3fd0ab0c0edc 100644 --- a/drivers/vhost/vsock.c +++ b/drivers/vhost/vsock.c @@ -398,6 +398,11 @@ static bool vhost_vsock_more_replies(struct vhost_vsock *vsock) return val < vq->num; } +static bool vhost_transport_msgzerocopy_allow(void) +{ + return true; +} + static bool vhost_transport_seqpacket_allow(u32 remote_cid); static struct virtio_transport vhost_transport = { @@ -431,6 +436,8 @@ static struct virtio_transport vhost_transport = { .seqpacket_allow = vhost_transport_seqpacket_allow, .seqpacket_has_data = virtio_transport_seqpacket_has_data, + .msgzerocopy_allow = vhost_transport_msgzerocopy_allow, + .notify_poll_in = virtio_transport_notify_poll_in, .notify_poll_out = virtio_transport_notify_poll_out, .notify_recv_init = virtio_transport_notify_recv_init, From patchwork Sat Jul 1 06:23:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 114981 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp10871115vqr; Fri, 30 Jun 2023 23:53:55 -0700 (PDT) X-Google-Smtp-Source: APBJJlEtf218p3DckTopnL9eNRnhTKA1r+vAsZwct11nMVVL9lrhbXZR/Uqft/A/RVLCnOvaBwkd X-Received: by 2002:a05:6214:226b:b0:62f:f2f0:2af3 with SMTP id gs11-20020a056214226b00b0062ff2f02af3mr7231810qvb.41.1688194435249; Fri, 30 Jun 2023 23:53:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688194435; cv=none; d=google.com; s=arc-20160816; b=O6t1Flv/TXJpAUbgmxqd4s+ICEla0pMhxIYCndLpxtdFw2TNQmdWHv5BdlG+9Tw3Vo aIl34HSphBxgSi+nFzzQbuf5hitr3Il2OZFam1kFv0CsXBJpTysqq4K6ocCFR86AbYIP 3uF6jF7Wo7gMvpbui6N61I2zv7vP+NkAmX0ZR9pxkNzpoaWzyxLAc/ptY7RVfnkZm60d U/oaoNh/95JNAoxJXjuewrWcoZEJ2YNsj09fUAiGQnSZkudBy6HEARkD7LITrTylv0C9 HzzPIioY7P5fx0cyJVFBMV6lksyV+xoaXWIC05awA+MMJSyfxn4m/FK6kPCqVif9u4za ExpA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=/zPSdfUGbdjD3WsNJ+2VRQman2eX/Wi/L4491xiMsn8=; fh=PzWIi6JmlJNekYqVy23433fKO6OE4+V62ucRx731HUI=; b=g3ncrUFwIel75zGbK5Fs2l3Z4C7P1JFIDEiBSbQ8KVooTyTagHrhpKKDoJDBh/R1Al 59O+fkLBW4F2FWkslWH7U0RppFuJWBOGcztUp0wCbJtWinGIpcW2dKVOKoxq+QYr6Riq lGAldf9cNjR+JgUskV0D7CWl274AXD21dqC8F9n+ajuwjeWZU40f7Zkrk8BCpw88T+uy ETEQY4Myu/WiM2M5YjIvCLOd+gUUDmzN+PpsNay+Gg01jGhgGlPNHF9H4i+JWEJwxdYd 9Ojpcid0N8dbbYU4tlAaCPi7QE6fYjsG24Gtw/rnql6uWc0J7Lni0RCkBJ1T4aysm12P 0TSg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=XeiUz6z8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id r20-20020a62e414000000b006688c47e5b4si965627pfh.399.2023.06.30.23.53.40; Fri, 30 Jun 2023 23:53:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=XeiUz6z8; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230083AbjGAGk1 (ORCPT + 99 others); Sat, 1 Jul 2023 02:40:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40002 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229919AbjGAGjj (ORCPT ); Sat, 1 Jul 2023 02:39:39 -0400 Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 707504230; Fri, 30 Jun 2023 23:39:33 -0700 (PDT) Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id B6EB8120015; Sat, 1 Jul 2023 09:28:50 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru B6EB8120015 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1688192930; bh=/zPSdfUGbdjD3WsNJ+2VRQman2eX/Wi/L4491xiMsn8=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=XeiUz6z8OFifNh3zimAno63/Bw/k8RpV0A9NPKs+Sytm+fqRWusR47ky9AdkOaMUL k/ujvy7fAYTN1CdbZ/o0fETmkCXszW2sJdhDZS515C6Wjz9IEAHGwCA05mItWtjubZ RSZ09D3j6KZaOlrhxygDQgQBFdux+Fz4pGHDrHRFCAiZRIpQBBS/3VEStHba5qT8/1 ewC3dai3Vk3NdAaIFZGRLPqH4H781OID3Dm9cyC+/T/KdM43TCsbglOKxcdeqimsNO ay7xhHNdydb0Oo7Xn6U2H5x/7+n8kJsq0HPb8Bml3J+jluK8OcpObdJnRfesRu/WoQ Sg+iDRiqSGDAA== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Sat, 1 Jul 2023 09:28:49 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Sat, 1 Jul 2023 09:28:24 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [RFC PATCH v5 11/17] vsock/virtio: support MSG_ZEROCOPY for transport Date: Sat, 1 Jul 2023 09:23:04 +0300 Message-ID: <20230701062310.3397129-12-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> References: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178380 [Jun 30 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 517 517 b0056c19d8e10afbb16cb7aad7258dedb0179a79, {Tracking_from_domain_doesnt_match_to}, sberdevices.ru:5.0.1,7.1.1;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;127.0.0.199:7.1.2;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/01 00:58:00 #21597365 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770200168450022810?= X-GMAIL-MSGID: =?utf-8?q?1770200168450022810?= Add 'msgzerocopy_allow()' callback for virtio transport. Signed-off-by: Arseniy Krasnov --- Changelog: v4 -> v5: * Move 'msgzerocopy_allow' right after seqpacket callbacks. net/vmw_vsock/virtio_transport.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/net/vmw_vsock/virtio_transport.c b/net/vmw_vsock/virtio_transport.c index 6cbb45bb12d2..8d3e9f441fa1 100644 --- a/net/vmw_vsock/virtio_transport.c +++ b/net/vmw_vsock/virtio_transport.c @@ -441,6 +441,11 @@ static void virtio_vsock_rx_done(struct virtqueue *vq) queue_work(virtio_vsock_workqueue, &vsock->rx_work); } +static bool virtio_transport_msgzerocopy_allow(void) +{ + return true; +} + static bool virtio_transport_seqpacket_allow(u32 remote_cid); static struct virtio_transport virtio_transport = { @@ -474,6 +479,8 @@ static struct virtio_transport virtio_transport = { .seqpacket_allow = virtio_transport_seqpacket_allow, .seqpacket_has_data = virtio_transport_seqpacket_has_data, + .msgzerocopy_allow = virtio_transport_msgzerocopy_allow, + .notify_poll_in = virtio_transport_notify_poll_in, .notify_poll_out = virtio_transport_notify_poll_out, .notify_recv_init = virtio_transport_notify_recv_init, From patchwork Sat Jul 1 06:23:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 114978 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp10870866vqr; Fri, 30 Jun 2023 23:53:18 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5mNsNeWef15H1YWTQ8KWoKl4CMHTreFrKv6CNFRgUKQ2rTfyoeV/dkdClTDa04CDTnmR/y X-Received: by 2002:a05:6808:2a68:b0:3a1:e132:dd8b with SMTP id fu8-20020a0568082a6800b003a1e132dd8bmr4090728oib.41.1688194398198; Fri, 30 Jun 2023 23:53:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688194398; cv=none; d=google.com; s=arc-20160816; b=J97VoQcO5YumbCDheoOiTDa4XnA1kq/Df01EfmCYoKjjEaTj9DZPA07gtrQ3Aca5Vp nQqXbtfazWfMgb5J4Avfl3cTic/f1F1268Zpe+RKi6ClX5ATBQfpK48kxX9qK1yi4fby HAOTB+0tWmrrT3jx8cY16bAOsjyOvWr2kkncUADt7r6ojDcX+HyCu432HOMnijI67k6T 7DxBfqNwuX4uG+8cqetB6XeT+t2LeYhGLPxQwnhBfBGYxDygIEBLRgf8/XJAa5vtKIUj Itb5wJmZOTbzQDhLZ/9LH4pZH/a8/6su3TTyKYC53lQxqHGbX4qGorfb4ihss3ILxO/M N1LA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=/JpJG9jMN3yK/D5/IqeMHJNNEU4HVxLFDCFOJJsYMMQ=; fh=PzWIi6JmlJNekYqVy23433fKO6OE4+V62ucRx731HUI=; b=UL8WExVKMbPVk3CpjRGrrnK13eTLtwr5YbvZ2VzdgCdvgAfVbAGp37p+NlgjcOUIYG o0onIgcxSLj9rM/vHeUQEktUimeGQ84iUP3/JDSxuP/fySdbQHu3978UrJT4QPWxEegS CPhnJ2RS6B6FWhNuVCO1kLoYvXoctMfXxj11FfknzORro5S2xWFm76jwNHU75gbXiI/6 QFLOHl5UEpdqubA8rEAIYsSkA3+prvIcvG3p0XOYZh3imT4c5nNZxBHNHTSI2LQTaeqV YuBEo7rWbN46tYJohcFaU89VmqEH8yOtWlSu+Ilfvgzo/wtzidqRhxVAC3w3hhymF7zv ytUg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=pBVbca5i; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 30-20020a17090a001e00b002613720fcd5si16325178pja.35.2023.06.30.23.53.04; Fri, 30 Jun 2023 23:53:18 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=pBVbca5i; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230008AbjGAGmg (ORCPT + 99 others); Sat, 1 Jul 2023 02:42:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40822 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230282AbjGAGli (ORCPT ); Sat, 1 Jul 2023 02:41:38 -0400 Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 45D1A49D3; Fri, 30 Jun 2023 23:39:55 -0700 (PDT) Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id E5C77120016; Sat, 1 Jul 2023 09:28:51 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru E5C77120016 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1688192931; bh=/JpJG9jMN3yK/D5/IqeMHJNNEU4HVxLFDCFOJJsYMMQ=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=pBVbca5isHLQIE/+QNwxi7WmfWNANfDLUUZMcl8M7DM10EWZSlT28kTUp99xtsIA+ aFZgB2FKCZ0UweRLRfjijAz62syihhdOt1vvc91iMRkqY33+Q2FhVOxFpYt/wpHdCm /tmZzIJfet6kEFMj00Gwv+OoElj1HWZ4/T1YqM+TQ7Goc60R6O2p/XnxjdF9jblYpp pJx1AoniReeDq7y9lnxIf5QnEVfZK/F9sEBf6BNN9lBGaJnk/OQ7LxG+0A8kri83DF 7i51m50GRXr8KXnGEhLnkEi7vC8eUaAPDoqm0kBxGWBoXClzfTzBO0dtaFWmwYudFR VZ7+KlIoAOc0Q== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Sat, 1 Jul 2023 09:28:50 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Sat, 1 Jul 2023 09:28:24 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [RFC PATCH v5 12/17] vsock/loopback: support MSG_ZEROCOPY for transport Date: Sat, 1 Jul 2023 09:23:05 +0300 Message-ID: <20230701062310.3397129-13-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> References: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178380 [Jun 30 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 517 517 b0056c19d8e10afbb16cb7aad7258dedb0179a79, {Tracking_from_domain_doesnt_match_to}, sberdevices.ru:5.0.1,7.1.1;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;127.0.0.199:7.1.2;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/01 00:58:00 #21597365 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770200129146325850?= X-GMAIL-MSGID: =?utf-8?q?1770200129146325850?= Add 'msgzerocopy_allow()' callback for loopback transport. Signed-off-by: Arseniy Krasnov --- Changelog: v4 -> v5: * Move 'msgzerocopy_allow' right after seqpacket callbacks. * Don't use prototype for 'vsock_loopback_msgzerocopy_allow()'. net/vmw_vsock/vsock_loopback.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/net/vmw_vsock/vsock_loopback.c b/net/vmw_vsock/vsock_loopback.c index 5c6360df1f31..048640167411 100644 --- a/net/vmw_vsock/vsock_loopback.c +++ b/net/vmw_vsock/vsock_loopback.c @@ -47,6 +47,10 @@ static int vsock_loopback_cancel_pkt(struct vsock_sock *vsk) } static bool vsock_loopback_seqpacket_allow(u32 remote_cid); +static bool vsock_loopback_msgzerocopy_allow(void) +{ + return true; +} static struct virtio_transport loopback_transport = { .transport = { @@ -79,6 +83,8 @@ static struct virtio_transport loopback_transport = { .seqpacket_allow = vsock_loopback_seqpacket_allow, .seqpacket_has_data = virtio_transport_seqpacket_has_data, + .msgzerocopy_allow = vsock_loopback_msgzerocopy_allow, + .notify_poll_in = virtio_transport_notify_poll_in, .notify_poll_out = virtio_transport_notify_poll_out, .notify_recv_init = virtio_transport_notify_recv_init, From patchwork Sat Jul 1 06:23:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 114976 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp10870636vqr; Fri, 30 Jun 2023 23:52:33 -0700 (PDT) X-Google-Smtp-Source: APBJJlFAvr8CAjhS0/2gnaRwQYVbLhdTb9Exp70ooXzkLM6lNiau+P2NHrBqJYQS/0HejlDHV5wI X-Received: by 2002:a05:6359:d0f:b0:132:f294:77fe with SMTP id gp15-20020a0563590d0f00b00132f29477femr4006193rwb.2.1688194352768; Fri, 30 Jun 2023 23:52:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688194352; cv=none; d=google.com; s=arc-20160816; b=H1A8KRwju0Pu4EU0OLrKIZ/9A5xyxSa5LS/c4Y/09yh5GjuuPtNrjqVvtil6TQX8Q9 dl+WX1X4sAyy4KlU8bqb8qVtzP/F4p71k4JetB+H9MWNBnNfVljDWr/R2Vns20CExrD2 cBzaokpSwBikqvP7Eid2xHT3p6sTEykiZWwMkTSO0k+KYjk28ms8uQXAI9qilfMNkAMG 3tLXALTzPDvsXC0iu3gEYxhLhroECSz1v5ju4TfMp9yWfAffIGfO8ZRgnNbL2UcFVR9E hElLii8QJKHtxxYPJpLlc0Zx+YwIWWiCKegYgsw07l42K46RK9pGcTJoUJAPjD/jH23m mFRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=tL2E1Y90S5bBkA16ZM0oXYsz18KCFk1Zn9JPdTBbPT8=; fh=PzWIi6JmlJNekYqVy23433fKO6OE4+V62ucRx731HUI=; b=W+PeUPQ7QhvZ4Kvm8G7ganAxrbMkl28lSi2zYwfnXgauEUrVwIBzbdhBKnX+xTo1Ry WpAehSEYPj+yZrCv9gCJFkqryWhlCrT8B0KrQeCNIPAxt7pFHYkqg2ts3JuiSUYdQrX6 zQ0gHBqbG9+/6PrVRp6SESVeMgZH5HkiHSvPBiv4rjPQTQc0Esxm+AYW7tnlJiRTIhEs squMnUV6Bb+x+mdseDkZyjdf2sz69GCusOvfrO5L3nyHC9gehFCXcFjxShvS2sf1w7VR wIXTL8QhIt6f8tRhtmV4kMIoYuw1NMBQQA/hpmrPMWM8aGyayv2/eT6hm3J4ZBbuBXzp JdYA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=AONkneIV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q16-20020a170902bd9000b001b674055d72si13341541pls.621.2023.06.30.23.52.17; Fri, 30 Jun 2023 23:52:32 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=AONkneIV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231205AbjGAGkp (ORCPT + 99 others); Sat, 1 Jul 2023 02:40:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40150 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230082AbjGAGjk (ORCPT ); Sat, 1 Jul 2023 02:39:40 -0400 Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AB833448B; Fri, 30 Jun 2023 23:39:34 -0700 (PDT) Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id 493E9120019; Sat, 1 Jul 2023 09:28:54 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru 493E9120019 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1688192934; bh=tL2E1Y90S5bBkA16ZM0oXYsz18KCFk1Zn9JPdTBbPT8=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=AONkneIVY6z7uG6hGz0R3adxk6xqQ8wWG+QmW6Xj4HFpm/W6DayErS2I7yKgSN7uW oPP2OxpUAJrN2L+egYLzMko63eJKcGMb9VWh1V75J183W2u8pUtWoerjGwztUPZ2aY rKj8co3FcC1qPLfVN57NuOL6J7GCtbhFsziy1t3xNKG4BBp9kMKg/Vu5BCsfA6L3pZ jkpVFcGiFugKUUtTnn6YxAtuTW9uk7jgVzRyCmqnUn7WPoNcyk0xKURpq8lT3ced5/ +iWKXh3X0OI2iHu+IAM3A64XxKF59JQMK8905a6QoJsyMs7ObdKVJb6TfHUSptk4jD LokT5swgZvE5g== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Sat, 1 Jul 2023 09:28:53 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Sat, 1 Jul 2023 09:28:26 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [RFC PATCH v5 14/17] docs: net: description of MSG_ZEROCOPY for AF_VSOCK Date: Sat, 1 Jul 2023 09:23:07 +0300 Message-ID: <20230701062310.3397129-15-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> References: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178380 [Jun 30 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 517 517 b0056c19d8e10afbb16cb7aad7258dedb0179a79, {Tracking_from_domain_doesnt_match_to}, sberdevices.ru:5.0.1,7.1.1;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;127.0.0.199:7.1.2;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/01 00:58:00 #21597365 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770200081938010715?= X-GMAIL-MSGID: =?utf-8?q?1770200081938010715?= This adds description of MSG_ZEROCOPY flag support for AF_VSOCK type of socket. Signed-off-by: Arseniy Krasnov --- Documentation/networking/msg_zerocopy.rst | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/Documentation/networking/msg_zerocopy.rst b/Documentation/networking/msg_zerocopy.rst index b3ea96af9b49..34bc7ff411ce 100644 --- a/Documentation/networking/msg_zerocopy.rst +++ b/Documentation/networking/msg_zerocopy.rst @@ -7,7 +7,8 @@ Intro ===== The MSG_ZEROCOPY flag enables copy avoidance for socket send calls. -The feature is currently implemented for TCP and UDP sockets. +The feature is currently implemented for TCP, UDP and VSOCK (with +virtio transport) sockets. Opportunity and Caveats @@ -174,7 +175,7 @@ read_notification() call in the previous snippet. A notification is encoded in the standard error format, sock_extended_err. The level and type fields in the control data are protocol family -specific, IP_RECVERR or IPV6_RECVERR. +specific, IP_RECVERR or IPV6_RECVERR (for TCP or UDP socket). Error origin is the new type SO_EE_ORIGIN_ZEROCOPY. ee_errno is zero, as explained before, to avoid blocking read and write system calls on @@ -201,6 +202,7 @@ undefined, bar for ee_code, as discussed below. printf("completed: %u..%u\n", serr->ee_info, serr->ee_data); +For VSOCK socket, cmsg_level will be SOL_VSOCK and cmsg_type will be 0. Deferred copies ~~~~~~~~~~~~~~~ @@ -235,12 +237,15 @@ Implementation Loopback -------- +For TCP and UDP: Data sent to local sockets can be queued indefinitely if the receive process does not read its socket. Unbound notification latency is not acceptable. For this reason all packets generated with MSG_ZEROCOPY that are looped to a local socket will incur a deferred copy. This includes looping onto packet sockets (e.g., tcpdump) and tun devices. +For VSOCK: +Data path sent to local sockets is the same as for non-local sockets. Testing ======= @@ -254,3 +259,6 @@ instance when run with msg_zerocopy.sh between a veth pair across namespaces, the test will not show any improvement. For testing, the loopback restriction can be temporarily relaxed by making skb_orphan_frags_rx identical to skb_orphan_frags. + +For VSOCK type of socket example can be found in tools/testing/vsock/ +vsock_test_zerocopy.c. From patchwork Sat Jul 1 06:23:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 114982 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp10871120vqr; Fri, 30 Jun 2023 23:53:56 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6gVg+jtnBAx6KMyjm7kXY5TQ3UL0VYY5g4kjyWWf9ETlr+qd4ADJji1+XL/euBfPJ1slM4 X-Received: by 2002:a05:6830:4a3:b0:6b7:40b2:9d9 with SMTP id l3-20020a05683004a300b006b740b209d9mr5254565otd.33.1688194435642; Fri, 30 Jun 2023 23:53:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688194435; cv=none; d=google.com; s=arc-20160816; b=O9IiDWRDahkVQbtcGURWS+jf1JHsjZh8ZK6MD85TRy2X64uGCK/aFxXzX1TohXnHvQ RdFg8fEG/hR04rwPh44Jxuu3NlivRHyEs2rT7C2XF3KOI5Wph3++2hMk8/LNpLPn+25r jqmlMSNFrYOKM9BjQBu2+1by9OvUWGO2eG720cbKe6NZ9tmzy57xaVtK3iXwRmJoP2rD PJfuCNw/u0VUcQqU3S8iX7T0bVuyaPalqRsMQAMHPeiMaNB0mTTHaLBXUiOHQ3ACLiz7 MZogkH+0rASnNyrKaLkh6GQBcYQIa2rxsCc3uEijZZwaJW6LyWKeWdqou+1wxAAOd8th o6dA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=317G4gwJx83ooHHpoPBTb4zLkCxZlZnzd5NTZs0DzNU=; fh=PzWIi6JmlJNekYqVy23433fKO6OE4+V62ucRx731HUI=; b=f33u6veJP6CsnNHV4vHh0SgBg98yirdrnbnqCoQQVwXpEZo8PHGoj5CFmtbz1U4UUk 7KoQYPd6fcp46SREGJ7Vz8ueNrkLT8gl0adGJXeDOoLZ+5fluT1cvmBLnQRMvmqjyWkZ FStk/KO7HImOgjCFIAHT+vhlD2meMwLV/2iGV7PQA4MGYiM3Q/frZsBvwQAYuCxJy5mI DynEoy50foACgsudN942nNbvKelggqs4WogZIhU7ZZTD8r8Yi2bisDnI8yi9Tm1f+nWj VVwWMgbIXx3on1atRTXYUcwMNCTWP5jGtFd3csEGcr3KyDzq7Z7CO8DTyNTDZpbrSDMx Tc7Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=Ozh+8Q5e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 30-20020a17090a001e00b002613720fcd5si16325178pja.35.2023.06.30.23.53.41; Fri, 30 Jun 2023 23:53:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=Ozh+8Q5e; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230306AbjGAGml (ORCPT + 99 others); Sat, 1 Jul 2023 02:42:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40886 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230022AbjGAGlo (ORCPT ); Sat, 1 Jul 2023 02:41:44 -0400 Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 467F349D4; Fri, 30 Jun 2023 23:39:55 -0700 (PDT) Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id 7FEEB12001A; Sat, 1 Jul 2023 09:28:55 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru 7FEEB12001A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1688192935; bh=317G4gwJx83ooHHpoPBTb4zLkCxZlZnzd5NTZs0DzNU=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=Ozh+8Q5etchOAchh6xL9oWuFGVWBU5jmp6NYJXVhzpw80mp5VJG38GmFMPFuJhPDA eB6q/3LHVsyHhg3n24FhBTf2GoQDijEnIIgSa/dfWuU1YhhldQHgfvJLeerkL4wuDl FbG679hvUfQDfur9L7YQGroLKpUNb1kGb0ZWVayqrrsDf9Y1hpGK+Xht2zvN68xY8F dL9yn22hbRpxvg9njwTeRpx7x7pclSMpERmFj0rOd+BBPgx2XmY88qZwP75fWLwab7 wOS07bE/97Ttsqzv00RAoXPWoSHRiZcieLxGMnYFXvlv5HWy03n+n/6e/GD9snDm0u ARqeIWz1dnkzA== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Sat, 1 Jul 2023 09:28:54 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Sat, 1 Jul 2023 09:28:26 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [RFC PATCH v5 15/17] test/vsock: MSG_ZEROCOPY flag tests Date: Sat, 1 Jul 2023 09:23:08 +0300 Message-ID: <20230701062310.3397129-16-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> References: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178380 [Jun 30 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 517 517 b0056c19d8e10afbb16cb7aad7258dedb0179a79, {Tracking_from_domain_doesnt_match_to}, sberdevices.ru:5.0.1,7.1.1;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;127.0.0.199:7.1.2;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/01 00:58:00 #21597365 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770200169007769719?= X-GMAIL-MSGID: =?utf-8?q?1770200169007769719?= This adds three tests for MSG_ZEROCOPY feature: 1) SOCK_STREAM tx with different buffers. 1) SOCK_SEQPACKET tx with different buffers. 1) SOCK_STREAM test to read empty error queue of the socket. Signed-off-by: Arseniy Krasnov --- tools/testing/vsock/Makefile | 2 +- tools/testing/vsock/util.c | 218 +++++++++++++++ tools/testing/vsock/util.h | 18 ++ tools/testing/vsock/vsock_test.c | 16 ++ tools/testing/vsock/vsock_test_zerocopy.c | 312 ++++++++++++++++++++++ tools/testing/vsock/vsock_test_zerocopy.h | 15 ++ 6 files changed, 580 insertions(+), 1 deletion(-) create mode 100644 tools/testing/vsock/vsock_test_zerocopy.c create mode 100644 tools/testing/vsock/vsock_test_zerocopy.h diff --git a/tools/testing/vsock/Makefile b/tools/testing/vsock/Makefile index 43a254f0e14d..0a78787d1d92 100644 --- a/tools/testing/vsock/Makefile +++ b/tools/testing/vsock/Makefile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0-only all: test vsock_perf test: vsock_test vsock_diag_test -vsock_test: vsock_test.o timeout.o control.o util.o +vsock_test: vsock_test.o vsock_test_zerocopy.o timeout.o control.o util.o vsock_diag_test: vsock_diag_test.o timeout.o control.o util.o vsock_perf: vsock_perf.o diff --git a/tools/testing/vsock/util.c b/tools/testing/vsock/util.c index 01b636d3039a..6397bfe661b6 100644 --- a/tools/testing/vsock/util.c +++ b/tools/testing/vsock/util.c @@ -11,15 +11,23 @@ #include #include #include +#include #include #include #include #include +#include +#include +#include #include "timeout.h" #include "control.h" #include "util.h" +#ifndef SOL_VSOCK +#define SOL_VSOCK 287 +#endif + /* Install signal handlers */ void init_signals(void) { @@ -408,3 +416,213 @@ unsigned long hash_djb2(const void *data, size_t len) return hash; } + +void enable_so_zerocopy(int fd) +{ + int val = 1; + + if (setsockopt(fd, SOL_SOCKET, SO_ZEROCOPY, &val, sizeof(val))) { + perror("setsockopt"); + exit(EXIT_FAILURE); + } +} + +static void *mmap_no_fail(size_t bytes) +{ + void *res; + + res = mmap(NULL, bytes, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS | MAP_POPULATE, -1, 0); + if (res == MAP_FAILED) { + perror("mmap"); + exit(EXIT_FAILURE); + } + + return res; +} + +size_t iovec_bytes(const struct iovec *iov, size_t iovnum) +{ + size_t bytes; + int i; + + for (bytes = 0, i = 0; i < iovnum; i++) + bytes += iov[i].iov_len; + + return bytes; +} + +static void iovec_random_init(struct iovec *iov, + const struct vsock_test_data *test_data) +{ + int i; + + for (i = 0; i < test_data->vecs_cnt; i++) { + int j; + + if (test_data->vecs[i].iov_base == MAP_FAILED) + continue; + + for (j = 0; j < iov[i].iov_len; j++) + ((uint8_t *)iov[i].iov_base)[j] = rand() & 0xff; + } +} + +unsigned long iovec_hash_djb2(struct iovec *iov, size_t iovnum) +{ + unsigned long hash; + size_t iov_bytes; + size_t offs; + void *tmp; + int i; + + iov_bytes = iovec_bytes(iov, iovnum); + + tmp = malloc(iov_bytes); + if (!tmp) { + perror("malloc"); + exit(EXIT_FAILURE); + } + + for (offs = 0, i = 0; i < iovnum; i++) { + memcpy(tmp + offs, iov[i].iov_base, iov[i].iov_len); + offs += iov[i].iov_len; + } + + hash = hash_djb2(tmp, iov_bytes); + free(tmp); + + return hash; +} + +struct iovec *iovec_from_test_data(const struct vsock_test_data *test_data) +{ + const struct iovec *test_iovec; + struct iovec *iovec; + int i; + + iovec = malloc(sizeof(*iovec) * test_data->vecs_cnt); + if (!iovec) { + perror("malloc"); + exit(EXIT_FAILURE); + } + + test_iovec = test_data->vecs; + + for (i = 0; i < test_data->vecs_cnt; i++) { + iovec[i].iov_len = test_iovec[i].iov_len; + iovec[i].iov_base = mmap_no_fail(test_iovec[i].iov_len); + + if (test_iovec[i].iov_base != MAP_FAILED && + test_iovec[i].iov_base) + iovec[i].iov_base += (uintptr_t)test_iovec[i].iov_base; + } + + /* Unmap "invalid" elements. */ + for (i = 0; i < test_data->vecs_cnt; i++) { + if (test_iovec[i].iov_base == MAP_FAILED) { + if (munmap(iovec[i].iov_base, iovec[i].iov_len)) { + perror("munmap"); + exit(EXIT_FAILURE); + } + } + } + + iovec_random_init(iovec, test_data); + + return iovec; +} + +void free_iovec_test_data(const struct vsock_test_data *test_data, + struct iovec *iovec) +{ + int i; + + for (i = 0; i < test_data->vecs_cnt; i++) { + if (test_data->vecs[i].iov_base != MAP_FAILED) { + if (test_data->vecs[i].iov_base) + iovec[i].iov_base -= (uintptr_t)test_data->vecs[i].iov_base; + + if (munmap(iovec[i].iov_base, iovec[i].iov_len)) { + perror("munmap"); + exit(EXIT_FAILURE); + } + } + } + + free(iovec); +} + +#define POLL_TIMEOUT_MS 100 +void vsock_recv_completion(int fd, bool zerocopied, bool completion) +{ + struct sock_extended_err *serr; + struct msghdr msg = { 0 }; + struct pollfd fds = { 0 }; + char cmsg_data[128]; + struct cmsghdr *cm; + ssize_t res; + + fds.fd = fd; + fds.events = 0; + + if (poll(&fds, 1, POLL_TIMEOUT_MS) < 0) { + perror("poll"); + exit(EXIT_FAILURE); + } + + if (!(fds.revents & POLLERR)) { + if (completion) { + fprintf(stderr, "POLLERR expected\n"); + exit(EXIT_FAILURE); + } else { + return; + } + } + + msg.msg_control = cmsg_data; + msg.msg_controllen = sizeof(cmsg_data); + + res = recvmsg(fd, &msg, MSG_ERRQUEUE); + if (res) { + fprintf(stderr, "failed to read error queue: %zi\n", res); + exit(EXIT_FAILURE); + } + + cm = CMSG_FIRSTHDR(&msg); + if (!cm) { + fprintf(stderr, "cmsg: no cmsg\n"); + exit(EXIT_FAILURE); + } + + if (cm->cmsg_level != SOL_VSOCK) { + fprintf(stderr, "cmsg: unexpected 'cmsg_level'\n"); + exit(EXIT_FAILURE); + } + + if (cm->cmsg_type != 0) { + fprintf(stderr, "cmsg: unexpected 'cmsg_type'\n"); + exit(EXIT_FAILURE); + } + + serr = (void *)CMSG_DATA(cm); + if (serr->ee_origin != SO_EE_ORIGIN_ZEROCOPY) { + fprintf(stderr, "serr: wrong origin: %u\n", serr->ee_origin); + exit(EXIT_FAILURE); + } + + if (serr->ee_errno) { + fprintf(stderr, "serr: wrong error code: %u\n", serr->ee_errno); + exit(EXIT_FAILURE); + } + + if (zerocopied && (serr->ee_code & SO_EE_CODE_ZEROCOPY_COPIED)) { + fprintf(stderr, "serr: was copy instead of zerocopy\n"); + exit(EXIT_FAILURE); + } + + if (!zerocopied && !(serr->ee_code & SO_EE_CODE_ZEROCOPY_COPIED)) { + fprintf(stderr, "serr: was zerocopy instead of copy\n"); + exit(EXIT_FAILURE); + } +} diff --git a/tools/testing/vsock/util.h b/tools/testing/vsock/util.h index fb99208a95ea..d07c1a9c2e0a 100644 --- a/tools/testing/vsock/util.h +++ b/tools/testing/vsock/util.h @@ -2,6 +2,7 @@ #ifndef UTIL_H #define UTIL_H +#include #include #include @@ -18,6 +19,16 @@ struct test_opts { unsigned int peer_cid; }; +#define VSOCK_TEST_DATA_MAX_IOV 4 + +struct vsock_test_data { + bool zerocopied; /* Data must be zerocopied. */ + bool completion; /* Must dequeue completion. */ + int sendmsg_errno; /* 'errno' after 'sendmsg()'. */ + int vecs_cnt; /* Number of elements in 'vecs'. */ + struct iovec vecs[VSOCK_TEST_DATA_MAX_IOV]; +}; + /* A test case definition. Test functions must print failures to stderr and * terminate with exit(EXIT_FAILURE). */ @@ -50,4 +61,11 @@ void list_tests(const struct test_case *test_cases); void skip_test(struct test_case *test_cases, size_t test_cases_len, const char *test_id_str); unsigned long hash_djb2(const void *data, size_t len); +void enable_so_zerocopy(int fd); +size_t iovec_bytes(const struct iovec *iov, size_t iovnum); +unsigned long iovec_hash_djb2(struct iovec *iov, size_t iovnum); +struct iovec *iovec_from_test_data(const struct vsock_test_data *test_data); +void free_iovec_test_data(const struct vsock_test_data *test_data, + struct iovec *iovec); +void vsock_recv_completion(int fd, bool zerocopied, bool completion); #endif /* UTIL_H */ diff --git a/tools/testing/vsock/vsock_test.c b/tools/testing/vsock/vsock_test.c index ac1bd3ac1533..d576b18bd357 100644 --- a/tools/testing/vsock/vsock_test.c +++ b/tools/testing/vsock/vsock_test.c @@ -20,6 +20,7 @@ #include #include +#include "vsock_test_zerocopy.h" #include "timeout.h" #include "control.h" #include "util.h" @@ -1128,6 +1129,21 @@ static struct test_case test_cases[] = { .run_client = test_stream_virtio_skb_merge_client, .run_server = test_stream_virtio_skb_merge_server, }, + { + .name = "SOCK_STREAM MSG_ZEROCOPY", + .run_client = test_stream_msg_zcopy_client, + .run_server = test_stream_msg_zcopy_server, + }, + { + .name = "SOCK_SEQPACKET MSG_ZEROCOPY", + .run_client = test_seqpacket_msg_zcopy_client, + .run_server = test_seqpacket_msg_zcopy_server, + }, + { + .name = "SOCK_STREAM MSG_ZEROCOPY empty MSG_ERRQUEUE", + .run_client = test_stream_msg_zcopy_empty_errq_client, + .run_server = test_stream_msg_zcopy_empty_errq_server, + }, {}, }; diff --git a/tools/testing/vsock/vsock_test_zerocopy.c b/tools/testing/vsock/vsock_test_zerocopy.c new file mode 100644 index 000000000000..c5539c5dbded --- /dev/null +++ b/tools/testing/vsock/vsock_test_zerocopy.c @@ -0,0 +1,312 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* MSG_ZEROCOPY feature tests for vsock + * + * Copyright (C) 2023 SberDevices. + * + * Author: Arseniy Krasnov + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "control.h" +#include "vsock_test_zerocopy.h" + +#define PAGE_SIZE 4096 + +static struct vsock_test_data test_data_array[] = { + /* Last element has non-page aligned size. */ + { + .zerocopied = true, + .completion = true, + .sendmsg_errno = 0, + .vecs_cnt = 3, + { + { NULL, PAGE_SIZE }, + { NULL, PAGE_SIZE }, + { NULL, 200 } + } + }, + /* All elements have page aligned base and size. */ + { + .zerocopied = true, + .completion = true, + .sendmsg_errno = 0, + .vecs_cnt = 3, + { + { NULL, PAGE_SIZE }, + { NULL, PAGE_SIZE * 2 }, + { NULL, PAGE_SIZE * 3 } + } + }, + /* All elements have page aligned base and size. But + * data length is bigger than 64Kb. + */ + { + .zerocopied = true, + .completion = true, + .sendmsg_errno = 0, + .vecs_cnt = 3, + { + { NULL, PAGE_SIZE * 16 }, + { NULL, PAGE_SIZE * 16 }, + { NULL, PAGE_SIZE * 16 } + } + }, + /* All elements have page aligned base and size. */ + { + .zerocopied = true, + .completion = true, + .sendmsg_errno = 0, + .vecs_cnt = 3, + { + { NULL, PAGE_SIZE }, + { NULL, PAGE_SIZE }, + { NULL, PAGE_SIZE } + } + }, + /* Middle element has non-page aligned size. */ + { + .zerocopied = true, + .completion = true, + .sendmsg_errno = 0, + .vecs_cnt = 3, + { + { NULL, PAGE_SIZE }, + { NULL, 100 }, + { NULL, PAGE_SIZE } + } + }, + /* Middle element has both non-page aligned base and size. */ + { + .zerocopied = true, + .completion = true, + .sendmsg_errno = 0, + .vecs_cnt = 3, + { + { NULL, PAGE_SIZE }, + { (void *)1, 100 }, + { NULL, PAGE_SIZE } + } + }, + /* Middle element is unmapped. */ + { + .zerocopied = false, + .completion = false, + .sendmsg_errno = ENOMEM, + .vecs_cnt = 3, + { + { NULL, PAGE_SIZE }, + { MAP_FAILED, PAGE_SIZE }, + { NULL, PAGE_SIZE } + } + }, + /* Valid data, but SO_ZEROCOPY is off. */ + { + .zerocopied = true, + .completion = false, + .sendmsg_errno = 0, + .vecs_cnt = 1, + { + { NULL, PAGE_SIZE } + } + }, +}; + +static void __test_msg_zerocopy_client(const struct test_opts *opts, + const struct vsock_test_data *test_data, + bool sock_seqpacket) +{ + struct msghdr msg = { 0 }; + ssize_t sendmsg_res; + struct iovec *iovec; + int fd; + + if (sock_seqpacket) + fd = vsock_seqpacket_connect(opts->peer_cid, 1234); + else + fd = vsock_stream_connect(opts->peer_cid, 1234); + + if (fd < 0) { + perror("connect"); + exit(EXIT_FAILURE); + } + + if (test_data->completion) + enable_so_zerocopy(fd); + + iovec = iovec_from_test_data(test_data); + + msg.msg_iov = iovec; + msg.msg_iovlen = test_data->vecs_cnt; + + errno = 0; + + sendmsg_res = sendmsg(fd, &msg, MSG_ZEROCOPY); + if (errno != test_data->sendmsg_errno) { + fprintf(stderr, "expected 'errno' == %i, got %i\n", + test_data->sendmsg_errno, errno); + exit(EXIT_FAILURE); + } + + if (!errno) { + if (sendmsg_res != iovec_bytes(iovec, test_data->vecs_cnt)) { + fprintf(stderr, "expected 'sendmsg()' == %li, got %li\n", + iovec_bytes(iovec, test_data->vecs_cnt), + sendmsg_res); + exit(EXIT_FAILURE); + } + } + + vsock_recv_completion(fd, test_data->zerocopied, test_data->completion); + + if (test_data->sendmsg_errno == 0) + control_writeulong(iovec_hash_djb2(iovec, test_data->vecs_cnt)); + else + control_writeulong(0); + + control_writeln("DONE"); + free_iovec_test_data(test_data, iovec); + close(fd); +} + +void test_stream_msg_zcopy_client(const struct test_opts *opts) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(test_data_array); i++) + __test_msg_zerocopy_client(opts, &test_data_array[i], false); +} + +void test_seqpacket_msg_zcopy_client(const struct test_opts *opts) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(test_data_array); i++) + __test_msg_zerocopy_client(opts, &test_data_array[i], true); +} + +static void __test_stream_server(const struct test_opts *opts, + const struct vsock_test_data *test_data, + bool sock_seqpacket) +{ + unsigned long remote_hash; + unsigned long local_hash; + ssize_t total_bytes_rec; + unsigned char *data; + size_t data_len; + int fd; + + if (sock_seqpacket) + fd = vsock_seqpacket_accept(VMADDR_CID_ANY, 1234, NULL); + else + fd = vsock_stream_accept(VMADDR_CID_ANY, 1234, NULL); + + if (fd < 0) { + perror("accept"); + exit(EXIT_FAILURE); + } + + data_len = iovec_bytes(test_data->vecs, test_data->vecs_cnt); + + data = malloc(data_len); + if (!data) { + perror("malloc"); + exit(EXIT_FAILURE); + } + + total_bytes_rec = 0; + + while (total_bytes_rec != data_len) { + ssize_t bytes_rec; + + bytes_rec = read(fd, data + total_bytes_rec, + data_len - total_bytes_rec); + if (bytes_rec <= 0) + break; + + total_bytes_rec += bytes_rec; + } + + if (test_data->sendmsg_errno == 0) + local_hash = hash_djb2(data, data_len); + else + local_hash = 0; + + free(data); + + /* Waiting for some result. */ + remote_hash = control_readulong(); + if (remote_hash != local_hash) { + fprintf(stderr, "hash mismatch\n"); + exit(EXIT_FAILURE); + } + + control_expectln("DONE"); + close(fd); +} + +void test_stream_msg_zcopy_server(const struct test_opts *opts) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(test_data_array); i++) + __test_stream_server(opts, &test_data_array[i], false); +} + +void test_seqpacket_msg_zcopy_server(const struct test_opts *opts) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(test_data_array); i++) + __test_stream_server(opts, &test_data_array[i], true); +} + +void test_stream_msg_zcopy_empty_errq_client(const struct test_opts *opts) +{ + struct msghdr msg = { 0 }; + char cmsg_data[128]; + ssize_t res; + int fd; + + fd = vsock_stream_connect(opts->peer_cid, 1234); + if (fd < 0) { + perror("connect"); + exit(EXIT_FAILURE); + } + + msg.msg_control = cmsg_data; + msg.msg_controllen = sizeof(cmsg_data); + + res = recvmsg(fd, &msg, MSG_ERRQUEUE); + if (res != -1) { + fprintf(stderr, "expected 'recvmsg(2)' failure, got %zi\n", + res); + exit(EXIT_FAILURE); + } + + control_writeln("DONE"); + close(fd); +} + +void test_stream_msg_zcopy_empty_errq_server(const struct test_opts *opts) +{ + int fd; + + fd = vsock_stream_accept(VMADDR_CID_ANY, 1234, NULL); + if (fd < 0) { + perror("accept"); + exit(EXIT_FAILURE); + } + + control_expectln("DONE"); + close(fd); +} diff --git a/tools/testing/vsock/vsock_test_zerocopy.h b/tools/testing/vsock/vsock_test_zerocopy.h new file mode 100644 index 000000000000..220b4f94f042 --- /dev/null +++ b/tools/testing/vsock/vsock_test_zerocopy.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef VSOCK_TEST_ZEROCOPY_H +#define VSOCK_TEST_ZEROCOPY_H +#include "util.h" + +void test_stream_msg_zcopy_client(const struct test_opts *opts); +void test_stream_msg_zcopy_server(const struct test_opts *opts); + +void test_seqpacket_msg_zcopy_client(const struct test_opts *opts); +void test_seqpacket_msg_zcopy_server(const struct test_opts *opts); + +void test_stream_msg_zcopy_empty_errq_client(const struct test_opts *opts); +void test_stream_msg_zcopy_empty_errq_server(const struct test_opts *opts); + +#endif /* VSOCK_TEST_ZEROCOPY_H */ From patchwork Sat Jul 1 06:23:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 114979 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp10871000vqr; Fri, 30 Jun 2023 23:53:40 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5MgDNNHD7agcMK6BQDKKqO4MgbqLf1RrtrFrNc0T+k242GQbUe56csjRPi4PhtZuHyYeiZ X-Received: by 2002:a17:90a:5a4c:b0:263:3847:e6e6 with SMTP id m12-20020a17090a5a4c00b002633847e6e6mr12919341pji.19.1688194420111; Fri, 30 Jun 2023 23:53:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688194420; cv=none; d=google.com; s=arc-20160816; b=SfxLVT4MqUlCxXrmjJ3T5mlrq2iACD6AuZyDFAa4B76e8SsVuO/eh6IDKXv52H2koq P9dZZZwLTXDIFkNMyVnnuz101O4Rr2rSycjOYEWYcu1rm7XJtz6lPQ4gfEp6/72j7nbh Woc3V6yl4PSBwUu1bXAsigAby8EFSJrKcKiaEj583uYuuqijiB93Mx5gi0gZ5mHBqm6c Ha62myh4dFJqjr3VTCYiyo91Ff8jb36kEIoJwGGJOpcF1IXGk9Di7vxzSo+qAW73s74B DCS4iE4kPdD4uXeCp2BNTsTZbNWozccdDU/v84NFF/EjHOBR6wdKO91jzW0YiBLP9kUo nDVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=FgPTGIGxeBtK5P9H+BLgs5wiavf5GzMitjxHRDrphMA=; fh=PzWIi6JmlJNekYqVy23433fKO6OE4+V62ucRx731HUI=; b=BVTIO/ldoIND10fJXaeEYkUT//k1LrrK7Ohe92O6Ldf9GiimPAGAzz/Q15XIJVBTAK YsliwnHfLr6nRbarJVlkFslQ5ceGgEKKn9tjbXpjjpnKcZL4g4zbD4PTA4B1zLdPVH3d AG/iWRYXofnDnG4jngw+EUY/7bIU8aSuZbwrzcx4fdjHG7L/hg9bjJtQYcW4DLak+VcS 6MSV1PZfE9bJ0LXeSOkCmtAvtNRc61DtEZgEY/z77J8/NWmWSEKuXsoUyQXDRTlULTu5 sEKxtv4OC6STQbBoLvpZV85TEL/5tdwNJXxelEherzQClUDf9HSqMEwDHP81+8M5PeiU SGFg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=qUXKV3FI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q19-20020a17090aa01300b0026301f58ac4si9784611pjp.82.2023.06.30.23.53.25; Fri, 30 Jun 2023 23:53:40 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=qUXKV3FI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231147AbjGAGki (ORCPT + 99 others); Sat, 1 Jul 2023 02:40:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40142 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229863AbjGAGjk (ORCPT ); Sat, 1 Jul 2023 02:39:40 -0400 Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3DEBD422F; Fri, 30 Jun 2023 23:39:33 -0700 (PDT) Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id AB3A412001B; Sat, 1 Jul 2023 09:28:56 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru AB3A412001B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1688192936; bh=FgPTGIGxeBtK5P9H+BLgs5wiavf5GzMitjxHRDrphMA=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=qUXKV3FIGAjaO6uAdwsKQhZPV6nKrY1eJ+xqxGm6RQ0d0fcBkxWzujMiFgigA+jzP hImC+098V1w/hAUBNyDE9y8PpilvZtfjUM015VkjJJgPWrc5QonPneWzLZB+hPozI7 cMvQO6JrquAsKk2K1+uABRbtz/SqOcivtp6aWagPSj+6e+8LGMtjsEEKBzXhxaNJhb eqH1a6cBZTiVUcoUjSWUvXpmWu4pWTu5pFTC+1bTuHaMEe7ewMD1Ys6zUC4+Fk+CAB BaoSXwMmx6YwhSPfdNGvw3NdKNcvVHxu8ZXoSwSXdGHV6m1uHzegV+/UR6b5Rppwxj bqW8bhRqfEKNQ== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Sat, 1 Jul 2023 09:28:55 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Sat, 1 Jul 2023 09:28:27 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [RFC PATCH v5 16/17] test/vsock: MSG_ZEROCOPY support for vsock_perf Date: Sat, 1 Jul 2023 09:23:09 +0300 Message-ID: <20230701062310.3397129-17-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> References: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178380 [Jun 30 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 517 517 b0056c19d8e10afbb16cb7aad7258dedb0179a79, {Tracking_from_domain_doesnt_match_to}, sberdevices.ru:5.0.1,7.1.1;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;127.0.0.199:7.1.2;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/01 00:58:00 #21597365 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770200151956770843?= X-GMAIL-MSGID: =?utf-8?q?1770200151956770843?= To use this option pass '--zc' parameter: ./vsock_perf --zc --sender --port --bytes With this option MSG_ZEROCOPY flag will be passed to the 'send()' call. Signed-off-by: Arseniy Krasnov --- tools/testing/vsock/vsock_perf.c | 139 +++++++++++++++++++++++++++++-- 1 file changed, 130 insertions(+), 9 deletions(-) diff --git a/tools/testing/vsock/vsock_perf.c b/tools/testing/vsock/vsock_perf.c index a72520338f84..7fd76f7a3c16 100644 --- a/tools/testing/vsock/vsock_perf.c +++ b/tools/testing/vsock/vsock_perf.c @@ -18,6 +18,8 @@ #include #include #include +#include +#include #define DEFAULT_BUF_SIZE_BYTES (128 * 1024) #define DEFAULT_TO_SEND_BYTES (64 * 1024) @@ -28,9 +30,14 @@ #define BYTES_PER_GB (1024 * 1024 * 1024ULL) #define NSEC_PER_SEC (1000000000ULL) +#ifndef SOL_VSOCK +#define SOL_VSOCK 287 +#endif + static unsigned int port = DEFAULT_PORT; static unsigned long buf_size_bytes = DEFAULT_BUF_SIZE_BYTES; static unsigned long vsock_buf_bytes = DEFAULT_VSOCK_BUF_BYTES; +static bool zerocopy; static void error(const char *s) { @@ -247,15 +254,76 @@ static void run_receiver(unsigned long rcvlowat_bytes) close(fd); } +static void recv_completion(int fd) +{ + struct sock_extended_err *serr; + char cmsg_data[128]; + struct cmsghdr *cm; + struct msghdr msg = { 0 }; + ssize_t ret; + + msg.msg_control = cmsg_data; + msg.msg_controllen = sizeof(cmsg_data); + + ret = recvmsg(fd, &msg, MSG_ERRQUEUE); + if (ret) { + fprintf(stderr, "recvmsg: failed to read err: %zi\n", ret); + return; + } + + cm = CMSG_FIRSTHDR(&msg); + if (!cm) { + fprintf(stderr, "cmsg: no cmsg\n"); + return; + } + + if (cm->cmsg_level != SOL_VSOCK) { + fprintf(stderr, "cmsg: unexpected 'cmsg_level'\n"); + return; + } + + if (cm->cmsg_type) { + fprintf(stderr, "cmsg: unexpected 'cmsg_type'\n"); + return; + } + + serr = (void *)CMSG_DATA(cm); + if (serr->ee_origin != SO_EE_ORIGIN_ZEROCOPY) { + fprintf(stderr, "serr: wrong origin\n"); + return; + } + + if (serr->ee_errno) { + fprintf(stderr, "serr: wrong error code\n"); + return; + } + + if (zerocopy && (serr->ee_code & SO_EE_CODE_ZEROCOPY_COPIED)) + fprintf(stderr, "warning: copy instead of zerocopy\n"); +} + +static void enable_so_zerocopy(int fd) +{ + int val = 1; + + if (setsockopt(fd, SOL_SOCKET, SO_ZEROCOPY, &val, sizeof(val))) + error("setsockopt(SO_ZEROCOPY)"); +} + static void run_sender(int peer_cid, unsigned long to_send_bytes) { time_t tx_begin_ns; time_t tx_total_ns; size_t total_send; + time_t time_in_send; void *data; int fd; - printf("Run as sender\n"); + if (zerocopy) + printf("Run as sender MSG_ZEROCOPY\n"); + else + printf("Run as sender\n"); + printf("Connect to %i:%u\n", peer_cid, port); printf("Send %lu bytes\n", to_send_bytes); printf("TX buffer %lu bytes\n", buf_size_bytes); @@ -265,38 +333,82 @@ static void run_sender(int peer_cid, unsigned long to_send_bytes) if (fd < 0) exit(EXIT_FAILURE); - data = malloc(buf_size_bytes); + if (zerocopy) { + enable_so_zerocopy(fd); - if (!data) { - fprintf(stderr, "'malloc()' failed\n"); - exit(EXIT_FAILURE); + data = mmap(NULL, buf_size_bytes, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (data == MAP_FAILED) { + perror("mmap"); + exit(EXIT_FAILURE); + } + } else { + data = malloc(buf_size_bytes); + + if (!data) { + fprintf(stderr, "'malloc()' failed\n"); + exit(EXIT_FAILURE); + } } memset(data, 0, buf_size_bytes); total_send = 0; + time_in_send = 0; tx_begin_ns = current_nsec(); while (total_send < to_send_bytes) { ssize_t sent; + size_t rest_bytes; + time_t before; - sent = write(fd, data, buf_size_bytes); + rest_bytes = to_send_bytes - total_send; + + before = current_nsec(); + sent = send(fd, data, (rest_bytes > buf_size_bytes) ? + buf_size_bytes : rest_bytes, + zerocopy ? MSG_ZEROCOPY : 0); + time_in_send += (current_nsec() - before); if (sent <= 0) error("write"); total_send += sent; + + if (zerocopy) { + struct pollfd fds = { 0 }; + + fds.fd = fd; + + if (poll(&fds, 1, -1) < 0) { + perror("poll"); + exit(EXIT_FAILURE); + } + + if (!(fds.revents & POLLERR)) { + fprintf(stderr, "POLLERR expected\n"); + exit(EXIT_FAILURE); + } + + recv_completion(fd); + } } tx_total_ns = current_nsec() - tx_begin_ns; printf("total bytes sent: %zu\n", total_send); printf("tx performance: %f Gbits/s\n", - get_gbps(total_send * 8, tx_total_ns)); - printf("total time in 'write()': %f sec\n", + get_gbps(total_send * 8, time_in_send)); + printf("total time in tx loop: %f sec\n", (float)tx_total_ns / NSEC_PER_SEC); + printf("time in 'send()': %f sec\n", + (float)time_in_send / NSEC_PER_SEC); close(fd); - free(data); + + if (zerocopy) + munmap(data, buf_size_bytes); + else + free(data); } static const char optstring[] = ""; @@ -336,6 +448,11 @@ static const struct option longopts[] = { .has_arg = required_argument, .val = 'R', }, + { + .name = "zc", + .has_arg = no_argument, + .val = 'Z', + }, {}, }; @@ -351,6 +468,7 @@ static void usage(void) " --help This message\n" " --sender Sender mode (receiver default)\n" " of the receiver to connect to\n" + " --zc Enable zerocopy\n" " --port Port (default %d)\n" " --bytes KMG Bytes to send (default %d)\n" " --buf-size KMG Data buffer size (default %d). In sender mode\n" @@ -413,6 +531,9 @@ int main(int argc, char **argv) case 'H': /* Help. */ usage(); break; + case 'Z': /* Zerocopy. */ + zerocopy = true; + break; default: usage(); } From patchwork Sat Jul 1 06:23:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 114969 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:994d:0:b0:3d9:f83d:47d9 with SMTP id k13csp10866784vqr; Fri, 30 Jun 2023 23:40:34 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5f0MRWVAPQg8iX9qtudh3WznpObmigWXAvKi9uFSbFhSlcuoyE98RutWB1/HqFzYp18FCM X-Received: by 2002:a05:6808:1525:b0:3a3:6382:b67d with SMTP id u37-20020a056808152500b003a36382b67dmr6228967oiw.41.1688193634481; Fri, 30 Jun 2023 23:40:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688193634; cv=none; d=google.com; s=arc-20160816; b=cFAzBoNH8aOiGdkQ5JA/0oD12ea3XKMRERLvkZzQ4iQesLbfpaOQwJYVDl9SfOnpAF 80p3j/md2ps5bHSoFrUUd9gQoUCM9FBqiTeruKfwNDynJhd0fnhA2I70Kzpip6wmai0R cHRu/lEzs0Czk8pe4dbQXQNUlrj21d/KrA9bcdMJWe7zXyUdMe6iIqsZO74R4dqhhi6K GMbG9AyhdSzaShwVGwwZ+l10iYDfgCrjAsszAGTYjy4ZF6Q+C7MaA++3bG1EKd2EBeUJ ETyvuxI9LnychYeZ9cmtGKo7JZMjQ6ZXXfLY0GcOll7k0Lg90SOW2jXzaRyHQ5ayRDDF bajQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=XeBcocw7Ow0J3zl/HOChhEU6WoykzJ9fUhXcW0cNTyA=; fh=PzWIi6JmlJNekYqVy23433fKO6OE4+V62ucRx731HUI=; b=mt0AcrTxPRrg/x0XU5XPWQUHOq5QEO+trpWTd3XdsmqozP8sp7mLL8Xc8nwlVxHYfL 86NXaswqp7xrLn2uvwg8QxKa4LrgY6BH56PLFY+zPWN61GwXQggLpEQ3qoRkHRIuf3lN DXtjdL8YoSezcELiwlHazJpo6emjFlKFE1Eo6GxNHJjiRz+AXck4U5ihxdRYCC96xFt6 1/mtYOnehvcN6WXVp7uZ/Pux3tl/5vKx88nq2pQ4JUQ9VqVbdQ/Z9cF1m88hBipDIb8z vzj1KEusja4V54ba9YXdH9UYZogeT12/jdya4UqJopXjc2DQtNxnMjx0h9BB8ykcCPNs AdEA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=mVXmgdr0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g18-20020aa796b2000000b00682689dbdd1si1880858pfk.278.2023.06.30.23.40.18; Fri, 30 Jun 2023 23:40:34 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=mVXmgdr0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230054AbjGAGjf (ORCPT + 99 others); Sat, 1 Jul 2023 02:39:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39938 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229886AbjGAGjW (ORCPT ); Sat, 1 Jul 2023 02:39:22 -0400 Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9AE7F4232; Fri, 30 Jun 2023 23:39:13 -0700 (PDT) Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id D58BA12001C; Sat, 1 Jul 2023 09:28:57 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru D58BA12001C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1688192937; bh=XeBcocw7Ow0J3zl/HOChhEU6WoykzJ9fUhXcW0cNTyA=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=mVXmgdr08QHAcQrRIidTkR/jetkcm2aJF6WBZNlNKFvbaj9kjI56otI6J9CyWIGPQ OefMAyOvIFZNaXDQN6LGVIwXQ3NxhdNnHGGL95y+o3rXtoyxgbnGmX4Ph4wIyD1yDr I3iFlw9IjmDIeY/3MAGuKFd35WlONHfp+LrJNx1+ToJw8YzDGQ6A+ioAH55j14rBgq m4YDS9YoZUI5i5pEvyUCaALXsz1CbOpy6RqInz/7U5l3Z1YEEfnWb0gQlezEVCpTCX /MKVv20QbVHWMg23aLSZkU9mKv1GROE9SDO6wQ/gPLESXy9tC//Jqi/0YZUlG2RlcF lfn8EncLAWnWQ== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Sat, 1 Jul 2023 09:28:56 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.30; Sat, 1 Jul 2023 09:28:28 +0300 From: Arseniy Krasnov To: Stefan Hajnoczi , Stefano Garzarella , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "Michael S. Tsirkin" , Jason Wang , Bobby Eshleman CC: , , , , , , , Arseniy Krasnov Subject: [RFC PATCH v5 17/17] test/vsock: io_uring rx/tx tests Date: Sat, 1 Jul 2023 09:23:10 +0300 Message-ID: <20230701062310.3397129-18-AVKrasnov@sberdevices.ru> X-Mailer: git-send-email 2.35.0 In-Reply-To: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> References: <20230701062310.3397129-1-AVKrasnov@sberdevices.ru> MIME-Version: 1.0 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 178380 [Jun 30 2023] X-KSMG-AntiSpam-Version: 5.9.59.0 X-KSMG-AntiSpam-Envelope-From: AVKrasnov@sberdevices.ru X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 517 517 b0056c19d8e10afbb16cb7aad7258dedb0179a79, {Tracking_from_domain_doesnt_match_to}, sberdevices.ru:5.0.1,7.1.1;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;127.0.0.199:7.1.2;100.64.160.123:7.1.2, FromAlignment: s, {Tracking_white_helo}, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/07/01 00:58:00 #21597365 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770199328776801360?= X-GMAIL-MSGID: =?utf-8?q?1770199328776801360?= This adds set of tests which use io_uring for rx/tx. This test suite is implemented as separated util like 'vsock_test' and has the same set of input arguments as 'vsock_test'. These tests only cover cases of data transmission (no connect/bind/accept etc). Signed-off-by: Arseniy Krasnov --- tools/testing/vsock/Makefile | 7 +- tools/testing/vsock/vsock_uring_test.c | 321 +++++++++++++++++++++++++ 2 files changed, 327 insertions(+), 1 deletion(-) create mode 100644 tools/testing/vsock/vsock_uring_test.c diff --git a/tools/testing/vsock/Makefile b/tools/testing/vsock/Makefile index 0a78787d1d92..8621ae73051d 100644 --- a/tools/testing/vsock/Makefile +++ b/tools/testing/vsock/Makefile @@ -1,12 +1,17 @@ # SPDX-License-Identifier: GPL-2.0-only +ifeq ($(MAKECMDGOALS),vsock_uring_test) +LDFLAGS = -luring +endif + all: test vsock_perf test: vsock_test vsock_diag_test vsock_test: vsock_test.o vsock_test_zerocopy.o timeout.o control.o util.o vsock_diag_test: vsock_diag_test.o timeout.o control.o util.o vsock_perf: vsock_perf.o +vsock_uring_test: control.o util.o vsock_uring_test.o timeout.o $(LDFLAGS) CFLAGS += -g -O2 -Werror -Wall -I. -I../../include -I../../../usr/include -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -D_GNU_SOURCE .PHONY: all test clean clean: - ${RM} *.o *.d vsock_test vsock_diag_test + ${RM} *.o *.d vsock_test vsock_diag_test vsock_uring_test -include *.d diff --git a/tools/testing/vsock/vsock_uring_test.c b/tools/testing/vsock/vsock_uring_test.c new file mode 100644 index 000000000000..7637ff510490 --- /dev/null +++ b/tools/testing/vsock/vsock_uring_test.c @@ -0,0 +1,321 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* io_uring tests for vsock + * + * Copyright (C) 2023 SberDevices. + * + * Author: Arseniy Krasnov + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "util.h" +#include "control.h" + +#define PAGE_SIZE 4096 +#define RING_ENTRIES_NUM 4 + +static struct vsock_test_data test_data_array[] = { + /* All elements have page aligned base and size. */ + { + .vecs_cnt = 3, + { + { NULL, PAGE_SIZE }, + { NULL, 2 * PAGE_SIZE }, + { NULL, 3 * PAGE_SIZE }, + } + }, + /* Middle element has both non-page aligned base and size. */ + { + .vecs_cnt = 3, + { + { NULL, PAGE_SIZE }, + { (void *)1, 200 }, + { NULL, 3 * PAGE_SIZE }, + } + } +}; + +static void vsock_io_uring_client(const struct test_opts *opts, + const struct vsock_test_data *test_data, + bool msg_zerocopy) +{ + struct io_uring_sqe *sqe; + struct io_uring_cqe *cqe; + struct io_uring ring; + struct iovec *iovec; + struct msghdr msg; + int fd; + + fd = vsock_stream_connect(opts->peer_cid, 1234); + if (fd < 0) { + perror("connect"); + exit(EXIT_FAILURE); + } + + if (msg_zerocopy) + enable_so_zerocopy(fd); + + iovec = iovec_from_test_data(test_data); + + if (io_uring_queue_init(RING_ENTRIES_NUM, &ring, 0)) + error(1, errno, "io_uring_queue_init"); + + if (io_uring_register_buffers(&ring, iovec, test_data->vecs_cnt)) + error(1, errno, "io_uring_register_buffers"); + + memset(&msg, 0, sizeof(msg)); + msg.msg_iov = iovec; + msg.msg_iovlen = test_data->vecs_cnt; + sqe = io_uring_get_sqe(&ring); + + if (msg_zerocopy) + io_uring_prep_sendmsg_zc(sqe, fd, &msg, 0); + else + io_uring_prep_sendmsg(sqe, fd, &msg, 0); + + if (io_uring_submit(&ring) != 1) + error(1, errno, "io_uring_submit"); + + if (io_uring_wait_cqe(&ring, &cqe)) + error(1, errno, "io_uring_wait_cqe"); + + io_uring_cqe_seen(&ring, cqe); + + control_writeulong(iovec_hash_djb2(iovec, test_data->vecs_cnt)); + + control_writeln("DONE"); + io_uring_queue_exit(&ring); + free_iovec_test_data(test_data, iovec); + close(fd); +} + +static void vsock_io_uring_server(const struct test_opts *opts, + const struct vsock_test_data *test_data) +{ + unsigned long remote_hash; + unsigned long local_hash; + struct io_uring_sqe *sqe; + struct io_uring_cqe *cqe; + struct io_uring ring; + struct iovec iovec; + size_t data_len; + void *data; + int fd; + + fd = vsock_stream_accept(VMADDR_CID_ANY, 1234, NULL); + if (fd < 0) { + perror("accept"); + exit(EXIT_FAILURE); + } + + data_len = iovec_bytes(test_data->vecs, test_data->vecs_cnt); + + data = malloc(data_len); + if (!data) { + perror("malloc"); + exit(EXIT_FAILURE); + } + + if (io_uring_queue_init(RING_ENTRIES_NUM, &ring, 0)) + error(1, errno, "io_uring_queue_init"); + + sqe = io_uring_get_sqe(&ring); + iovec.iov_base = data; + iovec.iov_len = data_len; + + io_uring_prep_readv(sqe, fd, &iovec, 1, 0); + + if (io_uring_submit(&ring) != 1) + error(1, errno, "io_uring_submit"); + + if (io_uring_wait_cqe(&ring, &cqe)) + error(1, errno, "io_uring_wait_cqe"); + + if (cqe->res != data_len) { + fprintf(stderr, "expected %zu, got %u\n", data_len, + cqe->res); + exit(EXIT_FAILURE); + } + + local_hash = hash_djb2(data, data_len); + + remote_hash = control_readulong(); + if (remote_hash != local_hash) { + fprintf(stderr, "hash mismatch\n"); + exit(EXIT_FAILURE); + } + + control_expectln("DONE"); + io_uring_queue_exit(&ring); + free(data); +} + +void test_stream_uring_server(const struct test_opts *opts) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(test_data_array); i++) + vsock_io_uring_server(opts, &test_data_array[i]); +} + +void test_stream_uring_client(const struct test_opts *opts) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(test_data_array); i++) + vsock_io_uring_client(opts, &test_data_array[i], false); +} + +void test_stream_uring_msg_zc_server(const struct test_opts *opts) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(test_data_array); i++) + vsock_io_uring_server(opts, &test_data_array[i]); +} + +void test_stream_uring_msg_zc_client(const struct test_opts *opts) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(test_data_array); i++) + vsock_io_uring_client(opts, &test_data_array[i], true); +} + +static struct test_case test_cases[] = { + { + .name = "SOCK_STREAM io_uring test", + .run_server = test_stream_uring_server, + .run_client = test_stream_uring_client, + }, + { + .name = "SOCK_STREAM io_uring MSG_ZEROCOPY test", + .run_server = test_stream_uring_msg_zc_server, + .run_client = test_stream_uring_msg_zc_client, + }, + {}, +}; + +static const char optstring[] = ""; +static const struct option longopts[] = { + { + .name = "control-host", + .has_arg = required_argument, + .val = 'H', + }, + { + .name = "control-port", + .has_arg = required_argument, + .val = 'P', + }, + { + .name = "mode", + .has_arg = required_argument, + .val = 'm', + }, + { + .name = "peer-cid", + .has_arg = required_argument, + .val = 'p', + }, + { + .name = "help", + .has_arg = no_argument, + .val = '?', + }, + {}, +}; + +static void usage(void) +{ + fprintf(stderr, "Usage: vsock_uring_test [--help] [--control-host=] --control-port= --mode=client|server --peer-cid=\n" + "\n" + " Server: vsock_uring_test --control-port=1234 --mode=server --peer-cid=3\n" + " Client: vsock_uring_test --control-host=192.168.0.1 --control-port=1234 --mode=client --peer-cid=2\n" + "\n" + "Run transmission tests using io_uring. Usage is the same as\n" + "in ./vsock_test\n" + "\n" + "Options:\n" + " --help This help message\n" + " --control-host Server IP address to connect to\n" + " --control-port Server port to listen on/connect to\n" + " --mode client|server Server or client mode\n" + " --peer-cid CID of the other side\n" + ); + exit(EXIT_FAILURE); +} + +int main(int argc, char **argv) +{ + const char *control_host = NULL; + const char *control_port = NULL; + struct test_opts opts = { + .mode = TEST_MODE_UNSET, + .peer_cid = VMADDR_CID_ANY, + }; + + init_signals(); + + for (;;) { + int opt = getopt_long(argc, argv, optstring, longopts, NULL); + + if (opt == -1) + break; + + switch (opt) { + case 'H': + control_host = optarg; + break; + case 'm': + if (strcmp(optarg, "client") == 0) { + opts.mode = TEST_MODE_CLIENT; + } else if (strcmp(optarg, "server") == 0) { + opts.mode = TEST_MODE_SERVER; + } else { + fprintf(stderr, "--mode must be \"client\" or \"server\"\n"); + return EXIT_FAILURE; + } + break; + case 'p': + opts.peer_cid = parse_cid(optarg); + break; + case 'P': + control_port = optarg; + break; + case '?': + default: + usage(); + } + } + + if (!control_port) + usage(); + if (opts.mode == TEST_MODE_UNSET) + usage(); + if (opts.peer_cid == VMADDR_CID_ANY) + usage(); + + if (!control_host) { + if (opts.mode != TEST_MODE_SERVER) + usage(); + control_host = "0.0.0.0"; + } + + control_init(control_host, control_port, + opts.mode == TEST_MODE_SERVER); + + run_tests(test_cases, &opts); + + control_cleanup(); + + return 0; +}