From patchwork Mon Dec 18 02:40:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mina Almasry X-Patchwork-Id: 180115 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:24d3:b0:fb:cd0c:d3e with SMTP id r19csp993920dyi; Sun, 17 Dec 2023 18:47:16 -0800 (PST) X-Google-Smtp-Source: AGHT+IEDUfuh4YEc4dcvOnJIibPMZAB53okY0OMtz3lBCVUcJGeVIFYmYDH55R2Mx1WSQiBmunsh X-Received: by 2002:a05:6358:9047:b0:172:98ae:cd35 with SMTP id f7-20020a056358904700b0017298aecd35mr8887229rwf.29.1702867636214; Sun, 17 Dec 2023 18:47:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702867636; cv=none; d=google.com; s=arc-20160816; b=gaRCYT/5wJCuFSo4DFoxY0wlbsQDpnFHwaOhiE2ORIh5t9P2hPeE4HhXyT1SBVqctc OQZbmN3d3eafCdwPFwhrKCorLBaAFKyIaxPeaRo5AO0O2IUac2fPIESgb8SkUojGwxYw 2RHdjUkco6EQZZNZ8OLI99H2/gGEcNqQcXVMj2omYzCfupCB3kVSB9UEsHRNBynMWEcu bvWlrxDopqW2TUeMvtwkvUSruNz51PNhZBIfP73sBWpsIS20otgmNXnJacO5zt9dCrbO VEM9sQjBT8KpdKgMydskUXw0LhvUILaPwgtsekPcOd/Ap+A17ateFW7T6iV5SnUEitN9 8bIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=NsLvXzxzNOPiHQjobkRux+OpGRSNf4fEva+dH69h4SM=; fh=12Y/+q4Q330f164yAx6QvqXGU1NVew2yR8fswrq0I10=; b=retFdxgAwIu+ETyVvBbCvGjyh0khA0/o5PcSkgMASsqK4AkNBvQuzn/m/eQq9TJ8Qd KRZmCNsfHCmP6d5TnbCwkD1sIsnHYlO2X/SXAG5fSnbQ2xFCVzBHpShtiQb+vag53xtp RcpUuXnW4ad6PjG/BIUSJTWLfzU0r8dFEGKKI4GDrJ7FekqBtwaxssMh5pX/1lXZxUeF TOoK5ESFnHOxGABbO/afIDBAECe9NdYgvQd+sLAUYALf+Ry6+XYslFWA2Sg85TuJ6Wos oSjm/Fno/zy13vxbCYv+lEeCjMnUp8U+FiMgPmLlgwT8CacYwwujKZ1Qp7fm3Vnjn52D Q7Aw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=E1dqV7YL; spf=pass (google.com: domain of linux-kernel+bounces-2936-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-2936-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id fr7-20020a17090ae2c700b0028aec84d78dsi8022655pjb.67.2023.12.17.18.47.16 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 17 Dec 2023 18:47:16 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-2936-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=E1dqV7YL; spf=pass (google.com: domain of linux-kernel+bounces-2936-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-2936-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id EB7FA282A2F for ; Mon, 18 Dec 2023 02:47:15 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7C5C2182C7; Mon, 18 Dec 2023 02:41:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="E1dqV7YL" X-Original-To: linux-kernel@vger.kernel.org Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB1D11426F for ; Mon, 18 Dec 2023 02:40:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--almasrymina.bounces.google.com Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-5e4c255846aso20986617b3.1 for ; Sun, 17 Dec 2023 18:40:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1702867254; x=1703472054; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=NsLvXzxzNOPiHQjobkRux+OpGRSNf4fEva+dH69h4SM=; b=E1dqV7YLEzfEJRTb6Wf4tZln4+tPViZeLB2O4uKhzttyD1JCJAbvkrfMzwYnTFUIhy Zdqdb/LoyUFj84ePjxCPAh412F/G3wy4bGvCneM6o742c6C79akivah2TQNUkb812GGB Tw2YxikP9Xx1U0dgu/GcQdr3blZ+2rWTeZyPgWDX33nO3dbzxG0iUVuOGpphvynwZ78n hMs/+Boh82kfRIK2thGM377d0QiRcm1+2A3dBJgb+rybbUiEqzDWy+1IhitO8N2vrGXK xzvrRgg1Ir8e9l6nk50LP/Uev89DxdCvvyMwAgS8H5TXF/2dmfiuBN08m6gd+aemMUGn 1t8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702867254; x=1703472054; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NsLvXzxzNOPiHQjobkRux+OpGRSNf4fEva+dH69h4SM=; b=WgOzlL5wKn57HY/yjruVOO7OPNt5TKQr/+PK8t+G1nO+gvHG1EtbKR9IT6+9hElBqa fud8d1pupCK9m+EZajzBqpLw3ARpLR26co3aEwYvXhwWMT3Nx7kjMBTpkubYQ0ve02YG Kc95Su6NNSMWLnF03i03LCpNSwFS2buYvShzuEKqWjObRhEwZKsgTPQP23bA9sPng8pD afx6UYXAZTTkPpr/TZCJ9dMXfUYnvSRI6mHNVJ+a3VYy8buTaN2EdHFhPSEqLalqV2RP OmfY12nHMWG44kSUrEHuJV0kJgbvU0NdnS7WNSQ1UdyygKVasgF76y4ahvT6TJSs9toK 3onQ== X-Gm-Message-State: AOJu0YwUAtBvXpTKgMQsTckmPEQHsC7cFsjRqwwRhFAFpOSuermeNro9 on2apzzPdxb0LTznB+n4h/ufhAJ1UFYrysHdGg== X-Received: from almasrymina.svl.corp.google.com ([2620:15c:2c4:200:5cbf:3534:fb34:758e]) (user=almasrymina job=sendgmr) by 2002:a05:690c:b1d:b0:5d8:394d:a8f4 with SMTP id cj29-20020a05690c0b1d00b005d8394da8f4mr1615182ywb.10.1702867254746; Sun, 17 Dec 2023 18:40:54 -0800 (PST) Date: Sun, 17 Dec 2023 18:40:19 -0800 In-Reply-To: <20231218024024.3516870-1-almasrymina@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20231218024024.3516870-1-almasrymina@google.com> X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20231218024024.3516870-13-almasrymina@google.com> Subject: [RFC PATCH net-next v5 12/14] net: add SO_DEVMEM_DONTNEED setsockopt to release RX frags From: Mina Almasry To: Mathieu Desnoyers , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-alpha@vger.kernel.org, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, sparclinux@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-arch@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org Cc: Mina Almasry , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Jonathan Corbet , Richard Henderson , Ivan Kokshaysky , Matt Turner , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Jesper Dangaard Brouer , Ilias Apalodimas , Steven Rostedt , Masami Hiramatsu , Arnd Bergmann , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , David Ahern , Willem de Bruijn , Shuah Khan , Sumit Semwal , " =?utf-8?q?Christian_K=C3=B6nig?= " , Pavel Begunkov , David Wei , Jason Gunthorpe , Yunsheng Lin , Shailend Chand , Harshitha Ramamurthy , Shakeel Butt , Jeroen de Borst , Praveen Kaligineedi , Willem de Bruijn , Kaiyuan Zhang X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785586134009752818 X-GMAIL-MSGID: 1785586134009752818 Add an interface for the user to notify the kernel that it is done reading the devmem dmabuf frags returned as cmsg. The kernel will drop the reference on the frags to make them available for re-use. Signed-off-by: Willem de Bruijn Signed-off-by: Kaiyuan Zhang Signed-off-by: Mina Almasry --- Changes in v1: - devmemtoken -> dmabuf_token (David). - Use napi_pp_put_page() for refcounting (Yunsheng). - Fix build error with missing socket options on other asms. --- arch/alpha/include/uapi/asm/socket.h | 8 ++++- arch/mips/include/uapi/asm/socket.h | 6 ++++ arch/parisc/include/uapi/asm/socket.h | 6 ++++ arch/sparc/include/uapi/asm/socket.h | 6 ++++ include/uapi/asm-generic/socket.h | 1 + include/uapi/linux/uio.h | 4 +++ net/core/sock.c | 45 +++++++++++++++++++++++++++ 7 files changed, 75 insertions(+), 1 deletion(-) diff --git a/arch/alpha/include/uapi/asm/socket.h b/arch/alpha/include/uapi/asm/socket.h index e94f621903fe..00146c243037 100644 --- a/arch/alpha/include/uapi/asm/socket.h +++ b/arch/alpha/include/uapi/asm/socket.h @@ -10,7 +10,7 @@ * Note: we only bother about making the SOL_SOCKET options * same as OSF/1, as that's all that "normal" programs are * likely to set. We don't necessarily want to be binary - * compatible with _everything_. + * compatible with _everything_. */ #define SOL_SOCKET 0xffff @@ -140,6 +140,12 @@ #define SO_PASSPIDFD 76 #define SO_PEERPIDFD 77 +#define SO_DEVMEM_DONTNEED 78 +#define SO_DEVMEM_LINEAR 79 +#define SCM_DEVMEM_LINEAR SO_DEVMEM_LINEAR +#define SO_DEVMEM_DMABUF 80 +#define SCM_DEVMEM_DMABUF SO_DEVMEM_DMABUF + #if !defined(__KERNEL__) #if __BITS_PER_LONG == 64 diff --git a/arch/mips/include/uapi/asm/socket.h b/arch/mips/include/uapi/asm/socket.h index 60ebaed28a4c..9a71ee8f36db 100644 --- a/arch/mips/include/uapi/asm/socket.h +++ b/arch/mips/include/uapi/asm/socket.h @@ -151,6 +151,12 @@ #define SO_PASSPIDFD 76 #define SO_PEERPIDFD 77 +#define SO_DEVMEM_DONTNEED 78 +#define SO_DEVMEM_LINEAR 79 +#define SCM_DEVMEM_LINEAR SO_DEVMEM_LINEAR +#define SO_DEVMEM_DMABUF 80 +#define SCM_DEVMEM_DMABUF SO_DEVMEM_DMABUF + #if !defined(__KERNEL__) #if __BITS_PER_LONG == 64 diff --git a/arch/parisc/include/uapi/asm/socket.h b/arch/parisc/include/uapi/asm/socket.h index be264c2b1a11..6b8674399363 100644 --- a/arch/parisc/include/uapi/asm/socket.h +++ b/arch/parisc/include/uapi/asm/socket.h @@ -132,6 +132,12 @@ #define SO_PASSPIDFD 0x404A #define SO_PEERPIDFD 0x404B +#define SO_DEVMEM_DONTNEED 0x404C +#define SO_DEVMEM_LINEAR 0x404D +#define SCM_DEVMEM_LINEAR SO_DEVMEM_LINEAR +#define SO_DEVMEM_DMABUF 0x404E +#define SCM_DEVMEM_DMABUF SO_DEVMEM_DMABUF + #if !defined(__KERNEL__) #if __BITS_PER_LONG == 64 diff --git a/arch/sparc/include/uapi/asm/socket.h b/arch/sparc/include/uapi/asm/socket.h index 682da3714686..ecfc8bfa9fe0 100644 --- a/arch/sparc/include/uapi/asm/socket.h +++ b/arch/sparc/include/uapi/asm/socket.h @@ -133,6 +133,12 @@ #define SO_PASSPIDFD 0x0055 #define SO_PEERPIDFD 0x0056 +#define SO_DEVMEM_DONTNEED 0x0057 +#define SO_DEVMEM_LINEAR 0x0058 +#define SCM_DEVMEM_LINEAR SO_DEVMEM_LINEAR +#define SO_DEVMEM_DMABUF 0x0059 +#define SCM_DEVMEM_DMABUF SO_DEVMEM_DMABUF + #if !defined(__KERNEL__) diff --git a/include/uapi/asm-generic/socket.h b/include/uapi/asm-generic/socket.h index 25a2f5255f52..1acb77780f10 100644 --- a/include/uapi/asm-generic/socket.h +++ b/include/uapi/asm-generic/socket.h @@ -135,6 +135,7 @@ #define SO_PASSPIDFD 76 #define SO_PEERPIDFD 77 +#define SO_DEVMEM_DONTNEED 97 #define SO_DEVMEM_LINEAR 98 #define SCM_DEVMEM_LINEAR SO_DEVMEM_LINEAR #define SO_DEVMEM_DMABUF 99 diff --git a/include/uapi/linux/uio.h b/include/uapi/linux/uio.h index ad92e37699da..65f33178a601 100644 --- a/include/uapi/linux/uio.h +++ b/include/uapi/linux/uio.h @@ -30,6 +30,10 @@ struct dmabuf_cmsg { __u32 dmabuf_id; /* dmabuf id this frag belongs to. */ }; +struct dmabuf_token { + __u32 token_start; + __u32 token_count; +}; /* * UIO_MAXIOV shall be at least 16 1003.1g (5.4.1.1) */ diff --git a/net/core/sock.c b/net/core/sock.c index 446e945f736b..77497dbb9022 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -1047,6 +1047,46 @@ static int sock_reserve_memory(struct sock *sk, int bytes) return 0; } +#ifdef CONFIG_PAGE_POOL +static noinline_for_stack int +sock_devmem_dontneed(struct sock *sk, sockptr_t optval, unsigned int optlen) +{ + unsigned int num_tokens, i, j; + struct dmabuf_token *tokens; + int ret; + + if (sk->sk_type != SOCK_STREAM || sk->sk_protocol != IPPROTO_TCP) + return -EBADF; + + if (optlen % sizeof(struct dmabuf_token) || optlen > sizeof(*tokens) * 128) + return -EINVAL; + + tokens = kvmalloc_array(128, sizeof(*tokens), GFP_KERNEL); + if (!tokens) + return -ENOMEM; + + num_tokens = optlen / sizeof(struct dmabuf_token); + if (copy_from_sockptr(tokens, optval, optlen)) + return -EFAULT; + + ret = 0; + for (i = 0; i < num_tokens; i++) { + for (j = 0; j < tokens[i].token_count; j++) { + struct netmem *netmem = xa_erase(&sk->sk_user_frags, + tokens[i].token_start + j); + + if (netmem && !WARN_ON_ONCE(!netmem_is_net_iov(netmem))) { + WARN_ON_ONCE(!napi_pp_put_page(netmem, false)); + ret++; + } + } + } + + kvfree(tokens); + return ret; +} +#endif + void sockopt_lock_sock(struct sock *sk) { /* When current->bpf_ctx is set, the setsockopt is called from @@ -1534,6 +1574,11 @@ int sk_setsockopt(struct sock *sk, int level, int optname, break; } +#ifdef CONFIG_PAGE_POOL + case SO_DEVMEM_DONTNEED: + ret = sock_devmem_dontneed(sk, optval, optlen); + break; +#endif default: ret = -ENOPROTOOPT; break;