Message ID | 20231214091947.395892-1-avkrasnov@salutedevices.com |
---|---|
Headers |
Return-Path: <linux-kernel-owner@vger.kernel.org> Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:3b04:b0:fb:cd0c:d3e with SMTP id c4csp8418872dys; Thu, 14 Dec 2023 01:28:35 -0800 (PST) X-Google-Smtp-Source: AGHT+IGumvt2LA5Ff7BYntB+r8Yu6IspmT6mcHJ8zjJBo5fXnfaJn/5l7wk0mSWDDzutTwb0aaSA X-Received: by 2002:a05:6a20:918b:b0:18f:97c:b9fa with SMTP id v11-20020a056a20918b00b0018f097cb9famr5978305pzd.84.1702546115194; Thu, 14 Dec 2023 01:28:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702546115; cv=none; d=google.com; s=arc-20160816; b=r28i0+mZKKInliBqGu4ShhC3NUoyZ5fBT1F1AeW03tC8/xKDWvVeQKLLRCvBE5FKur aC/9183QgvzsdKNLKicoHSDCKxkGWjLSEkG6J5EulKETdxMFwNIgbFCz77CyepKpfCnM pIdU/1Ozort2Bk1p74jvyFBHW+Plu3m2rBQ1NCvHv6RcFMv6iS4YAENSUyWrtSmGdAw6 ++p88RQzYPTNmCe2HSjZOkPOUR5mnYiDzxGk7yHxwIluqrgaRU+Y9oQjBb3+UcIuQQYB oIQjM68i1c/6GFx51Ke2Brdp7u2IzNayH97EyxcnketxruyAl2Dvm4O1nxc4NDPFE01P P34g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature:dkim-filter; bh=9LfT07pIk0/ilfQ5AIY8whpQsnw26U6S9QEMl0MdPM0=; fh=wIHqZpOIuzcidDZ82yQbOZuyHJty7uvaWDEh/efoVzA=; b=t4ygoXZNRND3D6MN1tQ8n3I01tFU6iy+smlRnT1XzBL0KW+0EjzqdpOFRBGftgCyvW 4dgjEpQYbnZUf2e9SbjTKyEo0RNY4zG6OmcxzS18pWGeNIIdWa1w5XNEzr3ihHsWVgRy yCItt18R2xS10AC+vWpmFcuCQ4y6bFa8LI41Pj3jI8YKQTH4uJsXzkJM1XRDKBi9JurK idVamhDlIsdv/46iviTkpT4V9dVSJaWLGXos1BL+urqQED1u+Hh08mLClCX8+4TvOfMn dUeseyxa7jmYGYmyhTgs4HR2zjO/MqUET+ETNt92ghVEwFytJ+62ztRTAeSUFqBllQ4g BSNw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@salutedevices.com header.s=mail header.b=Ty2dLg49; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=salutedevices.com Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id cl14-20020a17090af68e00b00279866aa14csi10994002pjb.16.2023.12.14.01.28.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Dec 2023 01:28:35 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; dkim=pass header.i=@salutedevices.com header.s=mail header.b=Ty2dLg49; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=salutedevices.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 8F34980F5F1F; Thu, 14 Dec 2023 01:28:29 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235533AbjLNJ2U (ORCPT <rfc822;dexuan.linux@gmail.com> + 99 others); Thu, 14 Dec 2023 04:28:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33346 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234413AbjLNJ2T (ORCPT <rfc822;linux-kernel@vger.kernel.org>); Thu, 14 Dec 2023 04:28:19 -0500 Received: from mx1.sberdevices.ru (mx1.sberdevices.ru [37.18.73.165]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4CAA3114; Thu, 14 Dec 2023 01:28:24 -0800 (PST) Received: from p-infra-ksmg-sc-msk01 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id A4E5610006A; Thu, 14 Dec 2023 12:28:22 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru A4E5610006A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=salutedevices.com; s=mail; t=1702546102; bh=9LfT07pIk0/ilfQ5AIY8whpQsnw26U6S9QEMl0MdPM0=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=Ty2dLg49aJAyeKr2yL3xdygjDKYDQfVUQIxidrE1y9su8X7Rrzfy56EBXPkwCo42u RUBkkXIdHHzndrKaLVhNmgmzAx3Q51CaNrb6ot9hVD3RTFyE1ZwBzuZNDYxCr1ny0/ o7dd/FxwgRe9SbSHmRfjaFLSbf07qyuTbhaeLvQd+eqiFTypgfF+PaI+xK6r1kl2zK f3+eBHlXJ/F7KEH/lKu/fmBWbfig/azlNWUYscovNrfn9a8nQuj68QrAXhzklMrZcP VdCHm7igbtZmQ0Hbf9tcUVNAizZYhthyKklXxYIJFP5SwS2hstnnmN8E0ZUYMMa5vQ /EjneEwlVGrwA== Received: from smtp.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Thu, 14 Dec 2023 12:28:21 +0300 (MSK) Received: from localhost.localdomain (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Thu, 14 Dec 2023 12:28:21 +0300 From: Arseniy Krasnov <avkrasnov@salutedevices.com> To: Stefan Hajnoczi <stefanha@redhat.com>, Stefano Garzarella <sgarzare@redhat.com>, "David S. Miller" <davem@davemloft.net>, Eric Dumazet <edumazet@google.com>, Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>, "Michael S. Tsirkin" <mst@redhat.com>, Jason Wang <jasowang@redhat.com>, Bobby Eshleman <bobby.eshleman@bytedance.com> CC: <kvm@vger.kernel.org>, <virtualization@lists.linux-foundation.org>, <netdev@vger.kernel.org>, <linux-kernel@vger.kernel.org>, <kernel@sberdevices.ru>, <oxffffaa@gmail.com>, <avkrasnov@salutedevices.com> Subject: [PATCH net-next v9 0/4] send credit update during setting SO_RCVLOWAT Date: Thu, 14 Dec 2023 12:19:43 +0300 Message-ID: <20231214091947.395892-1-avkrasnov@salutedevices.com> X-Mailer: git-send-email 2.35.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m02.sberdevices.ru (172.16.192.103) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 182105 [Dec 14 2023] X-KSMG-AntiSpam-Version: 6.1.0.3 X-KSMG-AntiSpam-Envelope-From: avkrasnov@salutedevices.com X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 7 0.3.7 6d6bf5bd8eea7373134f756a2fd73e9456bb7d1a, {Tracking_uf_ne_domains}, {Tracking_from_domain_doesnt_match_to}, git.kernel.org:7.1.1;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;smtp.sberdevices.ru:5.0.1,7.1.1;100.64.160.123:7.1.2;salutedevices.com:7.1.1;lore.kernel.org:7.1.1;127.0.0.199:7.1.2, FromAlignment: s, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean, bases: 2023/12/14 08:01:00 X-KSMG-LinksScanning: Clean, bases: 2023/12/14 08:01:00 X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/12/14 08:33:00 #22688916 X-KSMG-AntiVirus-Status: Clean, skipped X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: <linux-kernel.vger.kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Thu, 14 Dec 2023 01:28:29 -0800 (PST) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785248995399385358 X-GMAIL-MSGID: 1785248995399385358 |
Series |
send credit update during setting SO_RCVLOWAT
|
|
Message
Arseniy Krasnov
Dec. 14, 2023, 9:19 a.m. UTC
Hello, DESCRIPTION This patchset fixes old problem with hungup of both rx/tx sides and adds test for it. This happens due to non-default SO_RCVLOWAT value and deferred credit update in virtio/vsock. Link to previous old patchset: https://lore.kernel.org/netdev/39b2e9fd-601b-189d-39a9-914e5574524c@sberdevices.ru/ Here is what happens step by step: TEST INITIAL CONDITIONS 1) Vsock buffer size is 128KB. 2) Maximum packet size is also 64KB as defined in header (yes it is hardcoded, just to remind about that value). 3) SO_RCVLOWAT is default, e.g. 1 byte. STEPS SENDER RECEIVER 1) sends 128KB + 1 byte in a single buffer. 128KB will be sent, but for 1 byte sender will wait for free space at peer. Sender goes to sleep. 2) reads 64KB, credit update not sent 3) sets SO_RCVLOWAT to 64KB + 1 4) poll() -> wait forever, there is only 64KB available to read. So in step 4) receiver also goes to sleep, waiting for enough data or connection shutdown message from the sender. Idea to fix it is that rx kicks tx side to continue transmission (and may be close connection) when rx changes number of bytes to be woken up (e.g. SO_RCVLOWAT) and this value is bigger than number of available bytes to read. I've added small test for this, but not sure as it uses hardcoded value for maximum packet length, this value is defined in kernel header and used to control deferred credit update. And as this is not available to userspace, I can't control test parameters correctly (if one day this define will be changed - test may become useless). Head for this patchset is: https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=9bab51bd662be4c3ebb18a28879981d69f3ef15a Link to v1: https://lore.kernel.org/netdev/20231108072004.1045669-1-avkrasnov@salutedevices.com/ Link to v2: https://lore.kernel.org/netdev/20231119204922.2251912-1-avkrasnov@salutedevices.com/ Link to v3: https://lore.kernel.org/netdev/20231122180510.2297075-1-avkrasnov@salutedevices.com/ Link to v4: https://lore.kernel.org/netdev/20231129212519.2938875-1-avkrasnov@salutedevices.com/ Link to v5: https://lore.kernel.org/netdev/20231130130840.253733-1-avkrasnov@salutedevices.com/ Link to v6: https://lore.kernel.org/netdev/20231205064806.2851305-1-avkrasnov@salutedevices.com/ Link to v7: https://lore.kernel.org/netdev/20231206211849.2707151-1-avkrasnov@salutedevices.com/ Link to v8: https://lore.kernel.org/netdev/20231211211658.2904268-1-avkrasnov@salutedevices.com/ Changelog: v1 -> v2: * Patchset rebased and tested on new HEAD of net-next (see hash above). * New patch is added as 0001 - it removes return from SO_RCVLOWAT set callback in 'af_vsock.c' when transport callback is set - with that we can set 'sk_rcvlowat' only once in 'af_vsock.c' and in future do not copy-paste it to every transport. It was discussed in v1. * See per-patch changelog after ---. v2 -> v3: * See changelog after --- in 0003 only (0001 and 0002 still same). v3 -> v4: * Patchset rebased and tested on new HEAD of net-next (see hash above). * See per-patch changelog after ---. v4 -> v5: * Change patchset tag 'RFC' -> 'net-next'. * See per-patch changelog after ---. v5 -> v6: * New patch 0003 which sends credit update during reading bytes from socket. * See per-patch changelog after ---. v6 -> v7: * Patchset rebased and tested on new HEAD of net-next (see hash above). * See per-patch changelog after ---. v7 -> v8: * See per-patch changelog after ---. v8 -> v9: * Patchset rebased and tested on new HEAD of net-next (see hash above). * Add 'Fixes' tag for the current 0002. * Reorder patches by moving two fixes first. Arseniy Krasnov (4): virtio/vsock: fix logic which reduces credit update messages virtio/vsock: send credit update during setting SO_RCVLOWAT vsock: update SO_RCVLOWAT setting callback vsock/test: two tests to check credit update logic drivers/vhost/vsock.c | 1 + include/linux/virtio_vsock.h | 1 + include/net/af_vsock.h | 2 +- net/vmw_vsock/af_vsock.c | 9 +- net/vmw_vsock/hyperv_transport.c | 4 +- net/vmw_vsock/virtio_transport.c | 1 + net/vmw_vsock/virtio_transport_common.c | 43 +++++- net/vmw_vsock/vsock_loopback.c | 1 + tools/testing/vsock/vsock_test.c | 175 ++++++++++++++++++++++++ 9 files changed, 229 insertions(+), 8 deletions(-)
Comments
On Thu, Dec 14, 2023 at 12:19:43PM +0300, Arseniy Krasnov wrote: >Hello, > > DESCRIPTION > >This patchset fixes old problem with hungup of both rx/tx sides and adds >test for it. This happens due to non-default SO_RCVLOWAT value and >deferred credit update in virtio/vsock. Link to previous old patchset: >https://lore.kernel.org/netdev/39b2e9fd-601b-189d-39a9-914e5574524c@sberdevices.ru/ > >Here is what happens step by step: > > TEST > > INITIAL CONDITIONS > >1) Vsock buffer size is 128KB. >2) Maximum packet size is also 64KB as defined in header (yes it is > hardcoded, just to remind about that value). >3) SO_RCVLOWAT is default, e.g. 1 byte. > > > STEPS > > SENDER RECEIVER >1) sends 128KB + 1 byte in a > single buffer. 128KB will > be sent, but for 1 byte > sender will wait for free > space at peer. Sender goes > to sleep. > > >2) reads 64KB, credit update not sent >3) sets SO_RCVLOWAT to 64KB + 1 >4) poll() -> wait forever, there is > only 64KB available to read. > >So in step 4) receiver also goes to sleep, waiting for enough data or >connection shutdown message from the sender. Idea to fix it is that rx >kicks tx side to continue transmission (and may be close connection) >when rx changes number of bytes to be woken up (e.g. SO_RCVLOWAT) and >this value is bigger than number of available bytes to read. > >I've added small test for this, but not sure as it uses hardcoded value >for maximum packet length, this value is defined in kernel header and >used to control deferred credit update. And as this is not available to >userspace, I can't control test parameters correctly (if one day this >define will be changed - test may become useless). > >Head for this patchset is: >https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=9bab51bd662be4c3ebb18a28879981d69f3ef15a > >Link to v1: >https://lore.kernel.org/netdev/20231108072004.1045669-1-avkrasnov@salutedevices.com/ >Link to v2: >https://lore.kernel.org/netdev/20231119204922.2251912-1-avkrasnov@salutedevices.com/ >Link to v3: >https://lore.kernel.org/netdev/20231122180510.2297075-1-avkrasnov@salutedevices.com/ >Link to v4: >https://lore.kernel.org/netdev/20231129212519.2938875-1-avkrasnov@salutedevices.com/ >Link to v5: >https://lore.kernel.org/netdev/20231130130840.253733-1-avkrasnov@salutedevices.com/ >Link to v6: >https://lore.kernel.org/netdev/20231205064806.2851305-1-avkrasnov@salutedevices.com/ >Link to v7: >https://lore.kernel.org/netdev/20231206211849.2707151-1-avkrasnov@salutedevices.com/ >Link to v8: >https://lore.kernel.org/netdev/20231211211658.2904268-1-avkrasnov@salutedevices.com/ > >Changelog: >v1 -> v2: > * Patchset rebased and tested on new HEAD of net-next (see hash above). > * New patch is added as 0001 - it removes return from SO_RCVLOWAT set > callback in 'af_vsock.c' when transport callback is set - with that > we can set 'sk_rcvlowat' only once in 'af_vsock.c' and in future do > not copy-paste it to every transport. It was discussed in v1. > * See per-patch changelog after ---. >v2 -> v3: > * See changelog after --- in 0003 only (0001 and 0002 still same). >v3 -> v4: > * Patchset rebased and tested on new HEAD of net-next (see hash above). > * See per-patch changelog after ---. >v4 -> v5: > * Change patchset tag 'RFC' -> 'net-next'. > * See per-patch changelog after ---. >v5 -> v6: > * New patch 0003 which sends credit update during reading bytes from > socket. > * See per-patch changelog after ---. >v6 -> v7: > * Patchset rebased and tested on new HEAD of net-next (see hash above). > * See per-patch changelog after ---. >v7 -> v8: > * See per-patch changelog after ---. >v8 -> v9: > * Patchset rebased and tested on new HEAD of net-next (see hash above). > * Add 'Fixes' tag for the current 0002. > * Reorder patches by moving two fixes first. > >Arseniy Krasnov (4): > virtio/vsock: fix logic which reduces credit update messages > virtio/vsock: send credit update during setting SO_RCVLOWAT > vsock: update SO_RCVLOWAT setting callback > vsock/test: two tests to check credit update logic This order will break the bisectability, since now patch 2 will not build if patch 3 is not applied. So you need to implement in patch 2 `set_rcvlowat` and in patch 3 updated it to `notify_set_rcvlowat`, otherwise we always need to backport patch 3 in stable branches, that should be applied before patch 2. You have 2 options: a. move patch 3 before patch 2 without changing the code b. change patch 2 to use `set_rcvlowat` and updated that code in patch 3 I don't have a strong opinion, but I slightly prefer option a. BTW that forces us to backport more patches on stable branches, so I'm fine with option b as well. That said: Nacked-by: Stefano Garzarella <sgarzare@redhat.com>