From patchwork Sun Dec 4 19:24:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arseniy Krasnov X-Patchwork-Id: 29457 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp1919331wrr; Sun, 4 Dec 2022 11:54:49 -0800 (PST) X-Google-Smtp-Source: AA0mqf5tHsiWcs52HZlZZ1yplAo6wT+uhJhdkYBX1lGO1Ab9+1VwoPjz5Mrwp+bOyBIrIaPKlobR X-Received: by 2002:a50:ff04:0:b0:46b:19aa:cfc4 with SMTP id a4-20020a50ff04000000b0046b19aacfc4mr29753808edu.384.1670183689817; Sun, 04 Dec 2022 11:54:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670183689; cv=none; d=google.com; s=arc-20160816; b=pU9MzVm2jKF08g+uUMR1TeuYIG0sVNbdHVg4Jqa3wrQvqQZXdHskWAPwx+uR5CHOGA Q8H7YYn/wmmhC6iCLjALXKa5PlNPwRr1vvq6R/yAxZSxpkwMvHR3Bou1FuKE5zSebORf t7lpEOkKwr5JbI/lxIGsvm5/l9TIAF1V8JtfaL68wEA9HgzDD51WNgFmcqn1WtxtMhpp hl2SROaSCqCruHZLZkmKwdihZJ+zFoDuDN9t6DlWA94GKU4f/v7ThYqfMSpVoVrrFDfn A7HbX9JMeJFEFw6RSPrAx9xtL0G8TVMcCaopZs1Cu9DstBTgy1zYrpEhDwK69umaQHBb kW5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:content-transfer-encoding :content-id:content-language:accept-language:in-reply-to:message-id :date:thread-index:thread-topic:subject:cc:to:from:dkim-signature; bh=CoowoPIuXbSDgd8CXW+eOK6R1DPBai/bTdt2wwkOrpE=; b=C7doidsdKzB6h3iSHiGADJdEy8igJhVjDH1HJBku9osYsHVA8zOqwB/4yBjCaXCm7G soDg6IK9yqhqF5ReHxzbiI15dAmYL7bothfZ2ZMJD8TU4KL0yAG8uqCmXiGXG/n6bH5Z 4b9WtOW7QctLVdt+AjXDAVpARr7q56tm+iMuRjF+kXsO4jyds/0TlBGiXYwQ7FdRFnxX 3KFc3/qrytNO2XR4fPTYZ6fjfbH1eLqBL3JPW5wH0sqSRISX4bxJb/1jATOEL0vEH7jp yZ3xhdfF0qP3yf/KadJkVDvz2nkhtoH8dDfc3osC2Ow8DJ4duguVvYYdAIA/G0uzBOXE ajDQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=d2AQbL44; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id cr21-20020a170906d55500b0073d5a794b43si10882475ejc.985.2022.12.04.11.54.26; Sun, 04 Dec 2022 11:54:49 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@sberdevices.ru header.s=mail header.b=d2AQbL44; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=sberdevices.ru Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230370AbiLDTYj (ORCPT + 99 others); Sun, 4 Dec 2022 14:24:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44472 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229949AbiLDTYi (ORCPT ); Sun, 4 Dec 2022 14:24:38 -0500 Received: from mx.sberdevices.ru (mx.sberdevices.ru [45.89.227.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 13AD910B48; Sun, 4 Dec 2022 11:24:36 -0800 (PST) Received: from s-lin-edge02.sberdevices.ru (localhost [127.0.0.1]) by mx.sberdevices.ru (Postfix) with ESMTP id 66CEB5FD03; Sun, 4 Dec 2022 22:24:34 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sberdevices.ru; s=mail; t=1670181874; bh=CoowoPIuXbSDgd8CXW+eOK6R1DPBai/bTdt2wwkOrpE=; h=From:To:Subject:Date:Message-ID:Content-Type:MIME-Version; b=d2AQbL44MFn6f5yJLLJkUJwHJOVGqFHDGx67jGRUHxMvMf4ehIj00MBfM0b5DEn2a Iwn68siSkkubA4jIF7AdhwnQbqa6J9W9dmzKUBANpdh4rGlAFwDGfEf8CvgQ24QhgP OKPok5W1r21hM0QZPosIO2vWwNtE+Ygcr+NuV7CLT6S6eQrZtOV0R+LZqS9bx/+Mca 2PNOEXhjNbN64L5rX2yPnqXbezpQvHRWNX0jlCLHFxklQuaQa2FB7U/HYRdU60eNpm PVo+Mp81uncrlmDEfyo5Uq4bzk2YEd3lgorOnNcqJy9DJNwppcxdlId9fluQOWJygK 6yUE9GVfsVBCQ== Received: from S-MS-EXCH02.sberdevices.ru (S-MS-EXCH02.sberdevices.ru [172.16.1.5]) by mx.sberdevices.ru (Postfix) with ESMTP; Sun, 4 Dec 2022 22:24:34 +0300 (MSK) From: Arseniy Krasnov To: Stefano Garzarella , "edumazet@google.com" , Jakub Kicinski , Paolo Abeni , "David S. Miller" CC: Bobby Eshleman , Krasnov Arseniy , Arseniy Krasnov , "linux-kernel@vger.kernel.org" , "virtualization@lists.linux-foundation.org" , "netdev@vger.kernel.org" , kernel Subject: [RFC PATCH v3 4/4] test/vsock: vsock_perf utility Thread-Topic: [RFC PATCH v3 4/4] test/vsock: vsock_perf utility Thread-Index: AQHZCBYK/cvwOp5740W2DR5unQsb0A== Date: Sun, 4 Dec 2022 19:24:33 +0000 Message-ID: <9cc7610f-bcf5-6f77-96fe-4239efb67ca7@sberdevices.ru> In-Reply-To: <6bd77692-8388-8693-f15f-833df1fa6afd@sberdevices.ru> Accept-Language: en-US, ru-RU Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.16.1.12] Content-ID: <4AC60EEB2284A14EBE3C65D2F0456A84@sberdevices.ru> MIME-Version: 1.0 X-KSMG-Rule-ID: 4 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Status: not scanned, disabled by settings X-KSMG-AntiSpam-Interceptor-Info: not scanned X-KSMG-AntiPhishing: not scanned, disabled by settings X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 1.1.2.30, bases: 2022/12/04 13:44:00 #20647742 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751314533008021845?= X-GMAIL-MSGID: =?utf-8?q?1751314533008021845?= This adds utility to check vsock rx/tx performance. Usage as sender: ./vsock_perf --port --mb --port --so_rcvlowat Signed-off-by: Arseniy Krasnov --- tools/testing/vsock/Makefile | 1 + tools/testing/vsock/README | 34 +++ tools/testing/vsock/vsock_perf.c | 449 +++++++++++++++++++++++++++++++ 3 files changed, 484 insertions(+) create mode 100644 tools/testing/vsock/vsock_perf.c -- 2.25.1 diff --git a/tools/testing/vsock/Makefile b/tools/testing/vsock/Makefile index f8293c6910c9..d36fdd59fe2e 100644 --- a/tools/testing/vsock/Makefile +++ b/tools/testing/vsock/Makefile @@ -3,6 +3,7 @@ all: test test: vsock_test vsock_diag_test vsock_test: vsock_test.o timeout.o control.o util.o vsock_diag_test: vsock_diag_test.o timeout.o control.o util.o +vsock_perf: vsock_perf.o CFLAGS += -g -O2 -Werror -Wall -I. -I../../include -I../../../usr/include -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -D_GNU_SOURCE .PHONY: all test clean diff --git a/tools/testing/vsock/README b/tools/testing/vsock/README index 4d5045e7d2c3..64c5757b7ecc 100644 --- a/tools/testing/vsock/README +++ b/tools/testing/vsock/README @@ -35,3 +35,37 @@ Invoke test binaries in both directions as follows: --control-port=$GUEST_IP \ --control-port=1234 \ --peer-cid=3 + +vsock_perf utility +------------------- +'vsock_perf' is a simple tool to measure vsock performance. It works in +sender/receiver modes: sender waits for connection at specified port, +and after accepting it, starts data transmission to the receiver. After data +processing is done, it prints several metrics(see below). + +Usage: +# run as sender +# listen port 1234, tx buffer size is 1MB, send of 1G data +./vsock_perf --sender --port 1234 --buf-size 1MB --mb 1G + +Output: +tx performance: A Gb/s + +Output explanation: +A is calculated as "number of bytes to send" / "time in tx loop" + +# run as receiver +# connect to CID 2, port 1234, rx buffer size is 1MB, peer buf is 1G, SO_RCVLOWAT is 65536 +./vsock_perf --cid 2 --port 1234 --buf-size 1MB --vsk-size 1G -so_rcvlowat 65536 + +Output: +rx performance: A Gb/s +total in 'read()': B sec +POLLIN wakeups: C +average in 'read()': D ns + +Output explanation: +A is calculated as "number of received bytes" / "time in rx loop". +B is time, spent in 'read()' system call(excluding 'poll()') +C is number of 'poll()' wake ups with POLLIN bit set. +D is B / C, e.g. average amount of time, spent in single 'read()'. diff --git a/tools/testing/vsock/vsock_perf.c b/tools/testing/vsock/vsock_perf.c new file mode 100644 index 000000000000..69e3b24868d7 --- /dev/null +++ b/tools/testing/vsock/vsock_perf.c @@ -0,0 +1,449 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * vsock_perf - benchmark utility for vsock. + * + * Copyright (C) 2022 SberDevices. + * + * Author: Arseniy Krasnov + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define DEFAULT_BUF_SIZE_BYTES (128 * 1024) +#define DEFAULT_TO_SEND_BYTES (64 * 1024) +#define DEFAULT_VSOCK_BUF_BYTES (256 * 1024) +#define DEFAULT_RCVLOWAT_BYTES 1 +#define DEFAULT_PORT 1234 +#define DEFAULT_CID 2 + +#define BYTES_PER_GB (1024 * 1024 * 1024ULL) +#define NSEC_PER_SEC (1000000000ULL) + +static unsigned int port = DEFAULT_PORT; +static unsigned long buf_size_bytes = DEFAULT_BUF_SIZE_BYTES; +static unsigned long vsock_buf_bytes = DEFAULT_VSOCK_BUF_BYTES; + +static inline time_t current_nsec(void) +{ + struct timespec ts; + + if (clock_gettime(CLOCK_REALTIME, &ts)) { + perror("clock_gettime"); + exit(EXIT_FAILURE); + } + + return (ts.tv_sec * NSEC_PER_SEC) + ts.tv_nsec; +} + +/* From lib/cmdline.c. */ +static unsigned long memparse(const char *ptr) +{ + char *endptr; + + unsigned long long ret = strtoull(ptr, &endptr, 0); + + switch (*endptr) { + case 'E': + case 'e': + ret <<= 10; + case 'P': + case 'p': + ret <<= 10; + case 'T': + case 't': + ret <<= 10; + case 'G': + case 'g': + ret <<= 10; + case 'M': + case 'm': + ret <<= 10; + case 'K': + case 'k': + ret <<= 10; + endptr++; + default: + break; + } + + return ret; +} + +static void vsock_increase_buf_size(int fd) +{ + if (setsockopt(fd, AF_VSOCK, SO_VM_SOCKETS_BUFFER_MAX_SIZE, + &vsock_buf_bytes, sizeof(vsock_buf_bytes))) { + perror("setsockopt"); + exit(EXIT_FAILURE); + } + + if (setsockopt(fd, AF_VSOCK, SO_VM_SOCKETS_BUFFER_SIZE, + &vsock_buf_bytes, sizeof(vsock_buf_bytes))) { + perror("setsockopt"); + exit(EXIT_FAILURE); + } +} + +static int vsock_connect(unsigned int cid, unsigned int port) +{ + union { + struct sockaddr sa; + struct sockaddr_vm svm; + } addr = { + .svm = { + .svm_family = AF_VSOCK, + .svm_port = port, + .svm_cid = cid, + }, + }; + int fd; + + fd = socket(AF_VSOCK, SOCK_STREAM, 0); + + if (fd < 0) + return -1; + + vsock_increase_buf_size(fd); + + if (connect(fd, &addr.sa, sizeof(addr.svm)) < 0) { + close(fd); + return -1; + } + + return fd; +} + +static float get_gbps(unsigned long bytes, time_t ns_delta) +{ + return ((float)bytes / BYTES_PER_GB) / + ((float)ns_delta / NSEC_PER_SEC); +} + +static void run_sender(unsigned long to_send_bytes) +{ + time_t tx_begin_ns; + size_t total_send; + int client_fd; + char *data; + int fd; + union { + struct sockaddr sa; + struct sockaddr_vm svm; + } addr = { + .svm = { + .svm_family = AF_VSOCK, + .svm_port = port, + .svm_cid = VMADDR_CID_ANY, + }, + }; + union { + struct sockaddr sa; + struct sockaddr_vm svm; + } clientaddr; + + socklen_t clientaddr_len = sizeof(clientaddr.svm); + + printf("Run as sender\n"); + printf("Listen port %u\n", port); + printf("Send %lu bytes\n", to_send_bytes); + printf("TX buffer %lu bytes\n", buf_size_bytes); + printf("Peer buffer %lu bytes\n", vsock_buf_bytes); + + fd = socket(AF_VSOCK, SOCK_STREAM, 0); + + if (fd < 0) { + perror("socket"); + exit(EXIT_FAILURE); + } + + if (bind(fd, &addr.sa, sizeof(addr.svm)) < 0) { + perror("bind"); + exit(EXIT_FAILURE); + } + + if (listen(fd, 1) < 0) { + perror("listen"); + exit(EXIT_FAILURE); + } + + client_fd = accept(fd, &clientaddr.sa, &clientaddr_len); + + if (client_fd < 0) { + perror("accept"); + exit(EXIT_FAILURE); + } + + vsock_increase_buf_size(client_fd); + + data = malloc(buf_size_bytes); + + if (!data) { + printf("malloc failed\n"); + exit(EXIT_FAILURE); + } + + memset(data, 0, buf_size_bytes); + total_send = 0; + tx_begin_ns = current_nsec(); + + while (total_send < to_send_bytes) { + ssize_t sent; + + sent = write(client_fd, data, buf_size_bytes); + + if (sent <= 0) { + perror("write"); + exit(EXIT_FAILURE); + } + + total_send += sent; + } + + printf("tx performance: %f Gb/s\n", + get_gbps(total_send, current_nsec() - tx_begin_ns)); + + close(client_fd); + close(fd); + + free(data); +} + +static void run_receiver(int peer_cid, unsigned long rcvlowat_bytes) +{ + unsigned int read_cnt; + time_t rx_begin_ns; + time_t in_read_ns; + size_t total_recv; + void *data; + int fd; + + printf("Run as receiver\n"); + printf("Connect to %i:%u\n", peer_cid, port); + printf("RX buffer %lu bytes\n", buf_size_bytes); + printf("Peer buffer %lu bytes\n", vsock_buf_bytes); + printf("SO_RCVLOWAT %lu bytes\n", rcvlowat_bytes); + + fd = vsock_connect(peer_cid, port); + + if (fd < 0) { + perror("socket"); + exit(EXIT_FAILURE); + } + + if (setsockopt(fd, SOL_SOCKET, SO_RCVLOWAT, + &rcvlowat_bytes, + sizeof(rcvlowat_bytes))) { + perror("setsockopt"); + exit(EXIT_FAILURE); + } + + data = mmap(NULL, buf_size_bytes, PROT_READ | PROT_WRITE, + MAP_ANONYMOUS | MAP_PRIVATE | MAP_POPULATE, -1, 0); + + if (data == MAP_FAILED) { + perror("mmap"); + exit(EXIT_FAILURE); + } + + read_cnt = 0; + in_read_ns = 0; + total_recv = 0; + rx_begin_ns = current_nsec(); + + while (1) { + struct pollfd fds = { 0 }; + + fds.fd = fd; + fds.events = POLLIN | POLLERR | POLLHUP | + POLLRDHUP | POLLNVAL; + + if (poll(&fds, 1, -1) < 0) { + perror("poll"); + exit(EXIT_FAILURE); + } + + if (fds.revents & POLLERR) { + printf("'poll()' error\n"); + exit(EXIT_FAILURE); + } + + if (fds.revents & POLLIN) { + ssize_t bytes_read; + time_t t; + + t = current_nsec(); + bytes_read = read(fd, data, buf_size_bytes); + in_read_ns += (current_nsec() - t); + read_cnt++; + + if (!bytes_read) + break; + + if (bytes_read < 0) { + perror("recv"); + exit(EXIT_FAILURE); + } + + total_recv += bytes_read; + } + + if (fds.revents & (POLLHUP | POLLRDHUP)) + break; + } + + printf("rx performance: %f Gb/s\n", + get_gbps(total_recv, current_nsec() - rx_begin_ns)); + printf("total time in 'read()': %f sec\n", (float)in_read_ns / NSEC_PER_SEC); + printf("POLLIN wakeups: %i\n", read_cnt); + printf("average time in 'read()': %f ns\n", (float)in_read_ns / read_cnt); + + munmap(data, buf_size_bytes); + close(fd); +} + +static const char optstring[] = ""; +static const struct option longopts[] = { + { + .name = "help", + .has_arg = no_argument, + .val = 'H', + }, + { + .name = "sender", + .has_arg = no_argument, + .val = 'S', + }, + { + .name = "port", + .has_arg = required_argument, + .val = 'P', + }, + { + .name = "cid", + .has_arg = required_argument, + .val = 'C', + }, + { + .name = "mb", + .has_arg = required_argument, + .val = 'M', + }, + { + .name = "buf-size", + .has_arg = required_argument, + .val = 'B', + }, + { + .name = "vsk-size", + .has_arg = required_argument, + .val = 'V', + }, + { + .name = "so_rcvlowat", + .has_arg = required_argument, + .val = 'R', + }, + {}, +}; + +static void usage(void) +{ + printf("Help:\n" + "\n" + "This is benchmarking utility, to test vsock performance.\n" + "It runs in two modes: sender or receiver. In sender mode, it waits\n" + "connection from receiver, and when established, sender starts data\n" + "transmission.\n" + "\n" + "Options:\n" + " --help This help message\n" + " --sender Sender mode(receiver default)\n" + " --port Port (%d)\n" + " --cid CID of the peer (%d)\n" + " --mb Bytes to send (%d)\n" + " --buf-size Rx/Tx buffer size (%d). In sender mode\n" + " it is size of buffer passed to 'write()'.\n" + " In receiver mode it is size of buffer passed\n" + " to 'read()'.\n" + " --vsk-size Socket buffer size (%d)\n" + " --so_rcvlowat SO_RCVLOWAT (%d)\n" + "\n", DEFAULT_PORT, DEFAULT_CID, DEFAULT_TO_SEND_BYTES, + DEFAULT_BUF_SIZE_BYTES, DEFAULT_VSOCK_BUF_BYTES, + DEFAULT_RCVLOWAT_BYTES); + exit(EXIT_FAILURE); +} + +static long strtolx(const char *arg) +{ + long value; + char *end; + + value = strtol(arg, &end, 10); + + if (end != arg + strlen(arg)) + usage(); + + return value; +} + +int main(int argc, char **argv) +{ + unsigned long to_send_bytes = DEFAULT_TO_SEND_BYTES; + unsigned long rcvlowat_bytes = DEFAULT_RCVLOWAT_BYTES; + bool receiver_mode = true; + int peer_cid = DEFAULT_CID; + + while (1) { + int opt = getopt_long(argc, argv, optstring, longopts, NULL); + + if (opt == -1) + break; + + switch (opt) { + case 'V': /* Peer buffer size. */ + vsock_buf_bytes = memparse(optarg); + break; + case 'R': /* SO_RCVLOWAT value. */ + rcvlowat_bytes = memparse(optarg); + break; + case 'C': /* CID to connect to. */ + peer_cid = strtolx(optarg); + break; + case 'P': /* Port to connect to. */ + port = strtolx(optarg); + break; + case 'M': /* Bytes to send. */ + to_send_bytes = memparse(optarg); + break; + case 'B': /* Size of rx/tx buffer. */ + buf_size_bytes = memparse(optarg); + break; + case 'S': /* Sender mode. */ + receiver_mode = false; + break; + case 'H': /* Help. */ + usage(); + break; + default: + usage(); + } + } + + if (receiver_mode) + run_receiver(peer_cid, rcvlowat_bytes); + else + run_sender(to_send_bytes); + + return 0; +}