From patchwork Mon Apr 3 21:34:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Safonov X-Patchwork-Id: 78765 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp2601415vqo; Mon, 3 Apr 2023 14:44:27 -0700 (PDT) X-Google-Smtp-Source: AKy350aNUmGA4fVcrQZyvSoN4tlW4IGRWJRuZwK2FjxslN93CHZHPJ97LM52440pKMH7UiI9NwSg X-Received: by 2002:a05:6a20:6596:b0:db:a5b8:1c5b with SMTP id p22-20020a056a20659600b000dba5b81c5bmr251011pzh.1.1680558267549; Mon, 03 Apr 2023 14:44:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680558267; cv=none; d=google.com; s=arc-20160816; b=iY/LNoUOjPzszoVD3e7KxOktcab+FydhpaK0iyzHKgPyrvEALcA+0UJLKj95wwelZs APnSMW+sVR4OjQiraUqiCrpCRxvihFOc4Ps6Ma2lMM79Sat+heg8+6Jk2Orw0Wb76xpf GiYrDe+ypC6qk5Zu4c3ahfUGHPf34SRcfqMJR460flu7l1+bFLp3dJsRes4hwXjcjQx1 WeUkvGSHy7fnlJ5rl2QN53T5VJqlrbUaoZIpclMG0BuoxkKnlKugBcUoLmsSoClO1idZ LATMreDVgg7n99tE59TlW4AovBF0QOUv2beNko8MfA2DsiLLneF2Z8A+akMhpCcGyzCc s2eA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=mTUCL8/BOTl0j0dN4r+g/wrVq5oby7gw608S+52UjfU=; b=mT8azOQWcVGr2/D7T2wYn9s/u/OI0WtA4nVXjUUiKjjG2+aJ8fyVWzSkYwXNjRUiev lF9VwKnZNqe+BUcH0D4cbbAfjBTHBMaw2H4ESaJb8fi/YTMgKwCr1SUpXdf34oPOsbLy vig0zNzZQNnGG7aYMTxyYd+8na3iBCfw1LYyIJ8ckwywMzq+V7OoiJJukqN7WDvQFzR4 GxqFu9p/hGPg2hrMdJElHzUn5djgdGb48iOod1/lxewZUNNl57fEQ8aK7Vee2NBnFp+Z u60NJ+G3quIbXyK511hZuIitn/+79DQBNizI5s9cn+TJfJUGyf9+ibLqd6PaJ2Qlovo5 NR/A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@arista.com header.s=google header.b=bgCCdv2K; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=arista.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id s11-20020a63f04b000000b0050bf8a41c8asi1520454pgj.744.2023.04.03.14.44.15; Mon, 03 Apr 2023 14:44:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@arista.com header.s=google header.b=bgCCdv2K; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=arista.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233674AbjDCVhS (ORCPT + 99 others); Mon, 3 Apr 2023 17:37:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53190 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233173AbjDCVgP (ORCPT ); Mon, 3 Apr 2023 17:36:15 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 006424EFC for ; Mon, 3 Apr 2023 14:34:54 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id e18so30775828wra.9 for ; Mon, 03 Apr 2023 14:34:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arista.com; s=google; t=1680557694; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mTUCL8/BOTl0j0dN4r+g/wrVq5oby7gw608S+52UjfU=; b=bgCCdv2KIarB9LyhaXvi8Lf/Bo577EhjYgtMsyaAdV1K0/6a0dwSu9fdhn4nzuYhpG mDHWJhX7d/42jlSawQNSKmaAlwClOGi3CW8YRBCAJ1T9W6441qGzjGyGdEDxutsy4Fgz zVVx6gp2VrXxY4xXVOD7AxHaphr2IKVVcJracnT+ENerI6blkesWkUHI+cjE0TRryJeH m9oI5u2BR0eW2kgTX3IqBcwG5q8PsZsupWSufojRyJAfFrZRnFRJC+b3wa8wBuuVAFRt MhDNRGENdJUGYmvM1XsW5qD+uTw0xEP6iR+FZaUFr2gkxbCfPBgZN2Fy6b2ayLK/q6KE DuJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680557694; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mTUCL8/BOTl0j0dN4r+g/wrVq5oby7gw608S+52UjfU=; b=iiv5mvcLn/d9KQB7aJ3WpjQx359pSsEFfGgdpiKhQ4C9kzh+5l7sLWm872q91JoC2l TcK0KSRwyyGXSFw1UALOShCWs1zytj+9+8oIb/rROwtUsV1r9o5tekJBe+Ye4vNEsG77 TbTvj4Hf9sUKUZzdGZWo+t1h1sX0g89SJbmdIjMjaTpBRaLY1MnPS4XcF67CnxdN6SE3 AhshWxPM/XyiLY/shVI3s1CckRN/0XZVIrwz3jWTx+erfCyoOwbfPjzqXjsQlozgL8kn QQm3du782DVu6RAdrLQLFWDOlVOdSRTEJXCdXdiVSvXjuAI6WfMZxw6KU2uO9JDle9bd lWfA== X-Gm-Message-State: AAQBX9fcY6EfH2mryHGMKtrds2ncefobxJMuMZaDCxCBUtOePJktAi/y 5HebfMgI7iMLXzTLEKI7IKvzuczmhun8Zgsdm+U= X-Received: by 2002:adf:e951:0:b0:2e4:b9a3:4419 with SMTP id m17-20020adfe951000000b002e4b9a34419mr10980235wrn.51.1680557693733; Mon, 03 Apr 2023 14:34:53 -0700 (PDT) Received: from Mindolluin.ire.aristanetworks.com ([217.173.96.166]) by smtp.gmail.com with ESMTPSA id o5-20020a5d4a85000000b002c3f9404c45sm10682740wrq.7.2023.04.03.14.34.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Apr 2023 14:34:53 -0700 (PDT) From: Dmitry Safonov To: linux-kernel@vger.kernel.org, David Ahern , Eric Dumazet , Paolo Abeni , Jakub Kicinski , "David S. Miller" Cc: Dmitry Safonov , Andy Lutomirski , Ard Biesheuvel , Bob Gilligan , Dan Carpenter , David Laight , Dmitry Safonov <0x7f454c46@gmail.com>, Eric Biggers , "Eric W. Biederman" , Francesco Ruggeri , Herbert Xu , Hideaki YOSHIFUJI , Ivan Delalande , Leonard Crestez , Salam Noureddine , netdev@vger.kernel.org, Francesco Ruggeri Subject: [PATCH v5 19/21] net/tcp: Allow asynchronous delete for TCP-AO keys (MKTs) Date: Mon, 3 Apr 2023 22:34:18 +0100 Message-Id: <20230403213420.1576559-20-dima@arista.com> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230403213420.1576559-1-dima@arista.com> References: <20230403213420.1576559-1-dima@arista.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.2 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1762193065802768585?= X-GMAIL-MSGID: =?utf-8?q?1762193065802768585?= Delete becomes very, very fast - almost free, but after setsockopt() syscall returns, the key is still alive until next RCU grace period. Which is fine for listen sockets as userspace needs to be aware of setsockopt(TCP_AO) and accept() race and resolve it with verification by getsockopt() after TCP connection was accepted. The benchmark results (on non-loaded box, worse with more RCU work pending): > ok 33 Worst case delete 16384 keys: min=5ms max=10ms mean=6.93904ms stddev=0.263421 > ok 34 Add a new key 16384 keys: min=1ms max=4ms mean=2.17751ms stddev=0.147564 > ok 35 Remove random-search 16384 keys: min=5ms max=10ms mean=6.50243ms stddev=0.254999 > ok 36 Remove async 16384 keys: min=0ms max=0ms mean=0.0296107ms stddev=0.0172078 Co-developed-by: Francesco Ruggeri Signed-off-by: Francesco Ruggeri Co-developed-by: Salam Noureddine Signed-off-by: Salam Noureddine Signed-off-by: Dmitry Safonov --- include/uapi/linux/tcp.h | 3 ++- net/ipv4/tcp_ao.c | 21 ++++++++++++++++++--- 2 files changed, 20 insertions(+), 4 deletions(-) diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h index 1109093bbb24..979ff960fddb 100644 --- a/include/uapi/linux/tcp.h +++ b/include/uapi/linux/tcp.h @@ -383,7 +383,8 @@ struct tcp_ao_del { /* setsockopt(TCP_AO_DEL_KEY) */ __s32 ifindex; /* L3 dev index for VRF */ __u32 set_current :1, /* corresponding ::current_key */ set_rnext :1, /* corresponding ::rnext */ - reserved :30; /* must be 0 */ + del_async :1, /* only valid for listen sockets */ + reserved :29; /* must be 0 */ __u16 reserved2; /* padding, must be 0 */ __u8 prefix; /* peer's address prefix */ __u8 sndid; /* SendID for outgoing segments */ diff --git a/net/ipv4/tcp_ao.c b/net/ipv4/tcp_ao.c index 21242ba2d237..d9a4b9bb9872 100644 --- a/net/ipv4/tcp_ao.c +++ b/net/ipv4/tcp_ao.c @@ -1464,7 +1464,7 @@ static int tcp_ao_add_cmd(struct sock *sk, unsigned short int family, } static int tcp_ao_delete_key(struct sock *sk, struct tcp_ao_info *ao_info, - struct tcp_ao_key *key, + bool del_async, struct tcp_ao_key *key, struct tcp_ao_key *new_current, struct tcp_ao_key *new_rnext) { @@ -1472,11 +1472,24 @@ static int tcp_ao_delete_key(struct sock *sk, struct tcp_ao_info *ao_info, hlist_del_rcu(&key->node); + /* Support for async delete on listening sockets: as they don't + * need current_key/rnext_key maintaining, we don't need to check + * them and we can just free all resources in RCU fashion. + */ + if (del_async) { + atomic_sub(tcp_ao_sizeof_key(key), &sk->sk_omem_alloc); + call_rcu(&key->rcu, tcp_ao_key_free_rcu); + return 0; + } + /* At this moment another CPU could have looked this key up * while it was unlinked from the list. Wait for RCU grace period, * after which the key is off-list and can't be looked up again; * the rx path [just before RCU came] might have used it and set it * as current_key (very unlikely). + * Free the key with next RCU grace period (in case it was + * current_key before tcp_ao_current_rnext() might have + * changed it in forced-delete). */ synchronize_rcu(); if (new_current) @@ -1546,6 +1559,8 @@ static int tcp_ao_del_cmd(struct sock *sk, unsigned short int family, if (!new_rnext) return -ENOENT; } + if (cmd.del_async && sk->sk_state != TCP_LISTEN) + return -EINVAL; if (family == AF_INET) { struct sockaddr_in *sin = (struct sockaddr_in *)&cmd.addr; @@ -1590,8 +1605,8 @@ static int tcp_ao_del_cmd(struct sock *sk, unsigned short int family, if (key == new_current || key == new_rnext) continue; - return tcp_ao_delete_key(sk, ao_info, key, - new_current, new_rnext); + return tcp_ao_delete_key(sk, ao_info, cmd.del_async, key, + new_current, new_rnext); } return -ENOENT; }