From patchwork Fri May 12 20:23:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dmitry Safonov X-Patchwork-Id: 93354 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:b0ea:0:b0:3b6:4342:cba0 with SMTP id b10csp5376504vqo; Fri, 12 May 2023 13:35:12 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7oSbzxCqQ3Jp570VHP8GiBcBVjpruC9sN95qKDcVEHwMyNYtbZPOHfEZcL5NfSpC14b+0n X-Received: by 2002:a17:90b:4f46:b0:252:aa5e:e54f with SMTP id pj6-20020a17090b4f4600b00252aa5ee54fmr4525656pjb.13.1683923712162; Fri, 12 May 2023 13:35:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1683923712; cv=none; d=google.com; s=arc-20160816; b=c3AvYqMjaRME6e0cvWSx3cVP+r3N3tyuw7F53WGrEg+BJJFj6eKcfZf3RK6RM7De0M J/tKxgxULGnVr7Sw3Ch6mDFhjaybRIry6d1Eaa8iaJyBR0g207uO05/uYmTZja/jpCIV 1XAjhvLdT3AieOra/is3qh7HHncEF283g/IHyT5zegjUVH+r2OVPamhlCU0J0qCmpD4d W1NeZ539U7/0555KpzBXEDtoRYGv+cHHsBq/lsdexppX688OxRduXO+1HLtw1Vh5SLDa U5w+cNrSPTxJFvFKiI0qfwybwFgIj0GOPYMcLHj8GrYX2bOqlkrQjCQffheb2zSvFwi7 22DA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=EmFlDq9FRQjcXdrcJghItvS8UjQ6n89V7Tl5zTXj8lE=; b=DW3H2LGqHHjyol7YPXCl2MD4EBM1AcBROOkX+AyGnU8VLGiJzhfJq2uXnM9WknrJGI SJ+JmAMInlcCtV6RK5TWqVnSCEtWMTzf3FnhWbR0rWfSB101ZsSoOdTrlC4LW/W5KHqc Ddzj1Gr2I9I1Qh7y/6933uHZ6BTqlmrab7upO2MRcST1gHjep9/siQAGBD0rBuy54rof vIK+u2Sp6ciFWMAoUD3Rgof0i5Ahf192roRUF1/nymgghak0F+9OFWTmuiokETqUw7LG HvO3tqh8qYkRswUbd3C9t05+LOtk1WOrAVFQpODNeeVeopFcsAX9hDnLMGL6CuQ/eZRd xH1Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@arista.com header.s=google header.b=ba0B7lCB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=arista.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u10-20020a17090282ca00b001ac94b7f2f0si9309348plz.523.2023.05.12.13.34.56; Fri, 12 May 2023 13:35:12 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@arista.com header.s=google header.b=ba0B7lCB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=arista.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239502AbjELUZG (ORCPT + 99 others); Fri, 12 May 2023 16:25:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35602 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239397AbjELUYl (ORCPT ); Fri, 12 May 2023 16:24:41 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 45DDB65B4 for ; Fri, 12 May 2023 13:23:46 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id 5b1f17b1804b1-3f423ac6e2dso48357845e9.2 for ; Fri, 12 May 2023 13:23:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arista.com; s=google; t=1683923025; x=1686515025; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EmFlDq9FRQjcXdrcJghItvS8UjQ6n89V7Tl5zTXj8lE=; b=ba0B7lCBiq/kgRv3Ps9O9t4NLADUDoA9QxWB/bP45CEhojhsofcbeTWIC8wj8R5pwC 2VVihsm6DDri/17m/RPizanHXFV2PMLRMLqHH1Avsr4rOV2LRtIvw0HvHym7Aab3xaqZ R1vOqHn0IruYopxjqIR1I05+T7F/bOuwx22Jmg5eu4yYIr66Quq76mRJ+IXLJLP/Ff7W OGCvHydnD3WpwATrapiOZrH9TNPY5EsU1/yuACMP4XNFYAMqLwGOszeOBaNTKGU/80gZ 1wx0pmTpaowGcPJT9l/E/3tusfmCTouRTIAn0qMLWDx9X2bcDeqwb1yeZLRc0G2h4rdm zHjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683923025; x=1686515025; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EmFlDq9FRQjcXdrcJghItvS8UjQ6n89V7Tl5zTXj8lE=; b=OhxzXKGCZLaSxbEQGcUiPyz0azZ6dL6zDbBKOmVXk6aKNSM9lj7Eq3CuiPJntpDLeD tCk/eXrUFP8LkxmywRrU2F9dAYQ59SRkekjgSkOFxMCRgE9B8myXbVtGlfyU61o6Tvp3 dK69DsvyfPm+3zuzn15nnJR+jPP1CZHXQ0jM6X+G/+yyRT5qFi3FgIgWOWyNu3jY7JDk gelzjlLjqLHE9ljzMorHQwnpyPqOBz4S9A6+ip+evY0IVHfSwKiUKWA4SWZCp0MjBDUs Chv1hj9NRDf1WbPDB9U5nKgpi6KkPePR0N9rB/zPfnjfYcrsKesowfwtyoqqw0u10wc0 FFCQ== X-Gm-Message-State: AC+VfDx+V2X2d2pnY1HinMqDNUX/XbwHlHqqVfGklzYw9mw+SiwasdSW /l0wZFbAH40aHSl0jjEuaknXjYPvYUHix/2jBdY= X-Received: by 2002:a7b:c3cf:0:b0:3f4:21cf:b4a4 with SMTP id t15-20020a7bc3cf000000b003f421cfb4a4mr13780134wmj.20.1683923025478; Fri, 12 May 2023 13:23:45 -0700 (PDT) Received: from Mindolluin.ire.aristanetworks.com ([217.173.96.166]) by smtp.gmail.com with ESMTPSA id n9-20020a05600c294900b003f423508c6bsm17304527wmd.44.2023.05.12.13.23.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 May 2023 13:23:45 -0700 (PDT) From: Dmitry Safonov To: linux-kernel@vger.kernel.org, David Ahern , Eric Dumazet , Paolo Abeni , Jakub Kicinski , "David S. Miller" Cc: Dmitry Safonov , Andy Lutomirski , Ard Biesheuvel , Bob Gilligan , Dan Carpenter , David Laight , Dmitry Safonov <0x7f454c46@gmail.com>, Eric Biggers , "Eric W. Biederman" , Francesco Ruggeri , Herbert Xu , Hideaki YOSHIFUJI , Ivan Delalande , Leonard Crestez , Salam Noureddine , netdev@vger.kernel.org Subject: [PATCH v6 19/21] net/tcp: Allow asynchronous delete for TCP-AO keys (MKTs) Date: Fri, 12 May 2023 21:23:09 +0100 Message-Id: <20230512202311.2845526-20-dima@arista.com> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230512202311.2845526-1-dima@arista.com> References: <20230512202311.2845526-1-dima@arista.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1765721990448096013?= X-GMAIL-MSGID: =?utf-8?q?1765721990448096013?= Delete becomes very, very fast - almost free, but after setsockopt() syscall returns, the key is still alive until next RCU grace period. Which is fine for listen sockets as userspace needs to be aware of setsockopt(TCP_AO) and accept() race and resolve it with verification by getsockopt() after TCP connection was accepted. The benchmark results (on non-loaded box, worse with more RCU work pending): > ok 33 Worst case delete 16384 keys: min=5ms max=10ms mean=6.93904ms stddev=0.263421 > ok 34 Add a new key 16384 keys: min=1ms max=4ms mean=2.17751ms stddev=0.147564 > ok 35 Remove random-search 16384 keys: min=5ms max=10ms mean=6.50243ms stddev=0.254999 > ok 36 Remove async 16384 keys: min=0ms max=0ms mean=0.0296107ms stddev=0.0172078 Co-developed-by: Francesco Ruggeri Signed-off-by: Francesco Ruggeri Co-developed-by: Salam Noureddine Signed-off-by: Salam Noureddine Signed-off-by: Dmitry Safonov --- include/uapi/linux/tcp.h | 3 ++- net/ipv4/tcp_ao.c | 21 ++++++++++++++++++--- 2 files changed, 20 insertions(+), 4 deletions(-) diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h index 1109093bbb24..979ff960fddb 100644 --- a/include/uapi/linux/tcp.h +++ b/include/uapi/linux/tcp.h @@ -383,7 +383,8 @@ struct tcp_ao_del { /* setsockopt(TCP_AO_DEL_KEY) */ __s32 ifindex; /* L3 dev index for VRF */ __u32 set_current :1, /* corresponding ::current_key */ set_rnext :1, /* corresponding ::rnext */ - reserved :30; /* must be 0 */ + del_async :1, /* only valid for listen sockets */ + reserved :29; /* must be 0 */ __u16 reserved2; /* padding, must be 0 */ __u8 prefix; /* peer's address prefix */ __u8 sndid; /* SendID for outgoing segments */ diff --git a/net/ipv4/tcp_ao.c b/net/ipv4/tcp_ao.c index b71f69afd409..40b13fcb0723 100644 --- a/net/ipv4/tcp_ao.c +++ b/net/ipv4/tcp_ao.c @@ -1463,7 +1463,7 @@ static int tcp_ao_add_cmd(struct sock *sk, unsigned short int family, } static int tcp_ao_delete_key(struct sock *sk, struct tcp_ao_info *ao_info, - struct tcp_ao_key *key, + bool del_async, struct tcp_ao_key *key, struct tcp_ao_key *new_current, struct tcp_ao_key *new_rnext) { @@ -1471,11 +1471,24 @@ static int tcp_ao_delete_key(struct sock *sk, struct tcp_ao_info *ao_info, hlist_del_rcu(&key->node); + /* Support for async delete on listening sockets: as they don't + * need current_key/rnext_key maintaining, we don't need to check + * them and we can just free all resources in RCU fashion. + */ + if (del_async) { + atomic_sub(tcp_ao_sizeof_key(key), &sk->sk_omem_alloc); + call_rcu(&key->rcu, tcp_ao_key_free_rcu); + return 0; + } + /* At this moment another CPU could have looked this key up * while it was unlinked from the list. Wait for RCU grace period, * after which the key is off-list and can't be looked up again; * the rx path [just before RCU came] might have used it and set it * as current_key (very unlikely). + * Free the key with next RCU grace period (in case it was + * current_key before tcp_ao_current_rnext() might have + * changed it in forced-delete). */ synchronize_rcu(); if (new_current) @@ -1545,6 +1558,8 @@ static int tcp_ao_del_cmd(struct sock *sk, unsigned short int family, if (!new_rnext) return -ENOENT; } + if (cmd.del_async && sk->sk_state != TCP_LISTEN) + return -EINVAL; if (family == AF_INET) { struct sockaddr_in *sin = (struct sockaddr_in *)&cmd.addr; @@ -1589,8 +1604,8 @@ static int tcp_ao_del_cmd(struct sock *sk, unsigned short int family, if (key == new_current || key == new_rnext) continue; - return tcp_ao_delete_key(sk, ao_info, key, - new_current, new_rnext); + return tcp_ao_delete_key(sk, ao_info, cmd.del_async, key, + new_current, new_rnext); } return -ENOENT; }