From patchwork Tue Jan 9 01:14:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mina Almasry X-Patchwork-Id: 186185 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:37c1:b0:101:2151:f287 with SMTP id y1csp1378407dyq; Mon, 8 Jan 2024 17:15:55 -0800 (PST) X-Google-Smtp-Source: AGHT+IEIKsXXYyndCbKZXQRwhB8++pJZc181bLFxFSSHTBQVORoO0GaST3d4rXq+jyxpdqtCwOEE X-Received: by 2002:a05:600c:35cc:b0:40e:3d34:ec58 with SMTP id r12-20020a05600c35cc00b0040e3d34ec58mr1191206wmq.68.1704762955140; Mon, 08 Jan 2024 17:15:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704762955; cv=none; d=google.com; s=arc-20160816; b=pbb512DzNm71W4pANkG/mGYutuUfBU5L0jpy4uDBr9t9gEYyCAgjsFZld7QNCDjDro CzgpfkDeZlSXIki9Ki/GuU3BrqiOrOWCDWUDMo5BQ/dLsypXA+jJgqeQU6ySgiJ09dhB X/O+KaFsEoi6CIPIFm8rJi8hT+4LUNBV0yP7OAncKy0gYDiky1BnfugtwQvNjoNb9RVh 2ZHrE9tdJv6aDSFeMEFSvSXJfgXyA5TUe/UkMyJW+BKS9wLmgr7Edf7+N/+ZCO9/TC3d 1aKoYTrcGt23My1devTP+nVkjHh7UqUJw2QcYg3ebTewpTCTRMA7crsr3X8wl2wq9c7C kyRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=iiR5N2IDxIjhT8/ImfsmGNpE4EHm5swVTh/rfapPQuI=; fh=3zG4NHuhPxuhQ6Shw7Nt9BiRVbWd7elghbVYpfpS9LI=; b=r1Ye8nQr2jB7Lg3Gdpjb16MqW02zX2f8eSjCDEx5cw2aimmN9cB0uai6jWXC8j0C5a s/7C7DjIvhdb7Az/bPv4BAWfG1asyKR1d/iNS/rH75pP6t4doqoOjiLTSMinvlR4n9NA 2tUaWN0cu43Jwm1X5s33kL1LSidTLq/A3FsVIskLGe6Fg0ac0sDPxtfsX8+B6E9GUvV8 RxtrER/g9oOORKE6mrhaV2I8mn/jYg6xBSVNipskMokiIDmt7qK8FCVTTY6tT3sy9EmG O4aPX14qPUl+XPFDTuIh7fHSeIghr0Rcqbxcn77nrEAjo6Unstvvy1E7IrEusBX12zrG 4gug== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=KxrhJOEC; spf=pass (google.com: domain of linux-kernel+bounces-20250-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-20250-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id u9-20020a17090663c900b00a2791b24dd1si370959ejk.967.2024.01.08.17.15.54 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jan 2024 17:15:55 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-20250-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=KxrhJOEC; spf=pass (google.com: domain of linux-kernel+bounces-20250-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-20250-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id F3BE51F238BE for ; Tue, 9 Jan 2024 01:15:53 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2B0A86124; Tue, 9 Jan 2024 01:15:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="KxrhJOEC" Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 021AC15A4 for ; Tue, 9 Jan 2024 01:15:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--almasrymina.bounces.google.com Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-dbeac1f5045so3098944276.1 for ; Mon, 08 Jan 2024 17:15:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1704762900; x=1705367700; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=iiR5N2IDxIjhT8/ImfsmGNpE4EHm5swVTh/rfapPQuI=; b=KxrhJOECZVq0swda5+Awr1AC+TOrrHUgwnHAmKXYyPk1IpO4I4qda3IisDOwDaVNlj 0YNJ2iD5w7uN/kyOB8r0P7ePgADInBANgbvaoIFr8qcgs2s10ciq+06x4pZmVS6vVN45 QpBvYjZ0Km6rD4Zs8umqodFSPJ/JMB0Gw2lw3nWGDJ7jc6M/4HNXw5c6AJc/3zcX/gSF w5uJyBmKspQL2qLc2Y9nE1jcztfoaZIK5YPl00GR8qGJQzz4KbqDG8iTXy5JyGVckbKp J06uplp/yiXJWBQKttJ/F5cQHujoh3Ik++CZMymVmVo0obZTPZQg/FKTpPkoeBFQt2nw FqQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704762900; x=1705367700; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=iiR5N2IDxIjhT8/ImfsmGNpE4EHm5swVTh/rfapPQuI=; b=tViCUELeF1DXizJtMO5UFnKmizvMufbC61qIrnw82akhks3XRwrbZEM/i/d4rHJc+C 9ULk277V49Gbblj6WA5zA08FT36FO9VkCO3C9qv3ywdHAxOlQcAs2VHia+mGcTzfo6FD ZncRL3I7gqKS4wgBgp1cSYBb2AaHJLOMDZAC0kPLQB+/AY9zvMammiaFdTAKac6Us8v/ 0yZivKJ6XFLwDt2pMdPepo7pfz53octHnz0vuxr9pq1KyDKrGxNLMy9fZwLGgIShwV1S UPIMLHx+qEKrPfHFb4PuES0RUyUXuvUbuRkaJzTPFK/ZPGEvh/MmqVo4SYqXbwh1ZJEC GHbg== X-Gm-Message-State: AOJu0YyibXDXQruptyfoCZ11nuMHMbyZLq25Zqs/dMNol8bzpv0qxFZD /FEyxE5dRA+Gz/YOzeQ7+6iiVonWkfERTInPkFB/Jf62zEhv4s8wJEropou6YT8Ytj2nriBnwAc XjPuSdDIbv9psdBgnhPV/xnMuKNIBKj3atWYCqj6G4x4jRhPv+ejHdqTqD2hAeZN/n2fL7v4K8n qpHzkgOioACrq1zg== X-Received: from almasrymina.svl.corp.google.com ([2620:15c:2c4:200:23bb:bfcc:9879:32e2]) (user=almasrymina job=sendgmr) by 2002:a25:b281:0:b0:dbd:c21f:eb89 with SMTP id k1-20020a25b281000000b00dbdc21feb89mr221517ybj.1.1704762899931; Mon, 08 Jan 2024 17:14:59 -0800 (PST) Date: Mon, 8 Jan 2024 17:14:51 -0800 In-Reply-To: <20240109011455.1061529-1-almasrymina@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240109011455.1061529-1-almasrymina@google.com> X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20240109011455.1061529-2-almasrymina@google.com> Subject: [RFC PATCH net-next v5 1/2] net: introduce abstraction for network memory From: Mina Almasry To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: Mina Almasry , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Jason Gunthorpe , " =?utf-8?q?Christian_K=C3=B6nig?= " , Shakeel Butt , Yunsheng Lin , Willem de Bruijn X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1787573520065223444 X-GMAIL-MSGID: 1787573520065223444 Add the netmem_ref type, an abstraction for network memory. To add support for new memory types to the net stack, we must first abstract the current memory type. Currently parts of the net stack use struct page directly: - page_pool - drivers - skb_frag_t Originally the plan was to reuse struct page* for the new memory types, and to set the LSB on the page* to indicate it's not really a page. However, for compiler type checking we need to introduce a new type. netmem_ref is introduced to abstract the underlying memory type. Currently it's a no-op abstraction that is always a struct page underneath. In parallel there is an undergoing effort to add support for devmem to the net stack: https://lore.kernel.org/netdev/20231208005250.2910004-1-almasrymina@google.com/ netmem_ref can be pointers to different underlying memory types, and the low bits are set to indicate the memory type. Helpers are provided to convert netmem pointers to the underlying memory type (currently only struct page). In the devmem series helpers are provided so that calling code can use netmem without worrying about the underlying memory type unless absolutely necessary. Signed-off-by: Mina Almasry Reviewed-by: Shakeel Butt --- rfc v5: - RFC due to merge window. - Change to 'typedef unsigned long __bitwise netmem_ref;' - Fixed commit message (Shakeel). - Did not apply Shakeel's reviewed-by since the code changed significantly. v4: - use 'struct netmem;' instead of 'typedef void *__bitwise netmem_ref;' Using __bitwise with a non-integer type was wrong and triggered many patchwork bot errors/warnings. Using an integer type causes the compiler to warn when casting NULL to the integer type. Attempt to use an empty struct for our opaque network memory. v3: - Modify struct netmem from a union of struct page + new types to an opaque netmem_ref type. I went with: +typedef void *__bitwise netmem_ref; rather than this that Jakub recommended: +typedef unsigned long __bitwise netmem_ref; Because with the latter the compiler issues warnings to cast NULL to netmem_ref. I hope that's ok. - Add some function docs. v2: - Use container_of instead of a type cast (David). --- include/net/netmem.h | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 include/net/netmem.h diff --git a/include/net/netmem.h b/include/net/netmem.h new file mode 100644 index 000000000000..9f327d964782 --- /dev/null +++ b/include/net/netmem.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: GPL-2.0 + * + * Network memory + * + * Author: Mina Almasry + */ + +#ifndef _NET_NETMEM_H +#define _NET_NETMEM_H + +/** + * netmem_ref - a nonexistent type marking a reference to generic network + * memory. + * + * A netmem_ref currently is always a reference to a struct page. This + * abstraction is introduced so support for new memory types can be added. + * + * Use the supplied helpers to obtain the underlying memory pointer and fields. + */ +typedef unsigned long __bitwise netmem_ref; + +/* This conversion fails (returns NULL) if the netmem_ref is not struct page + * backed. + * + * Currently struct page is the only possible netmem, and this helper never + * fails. + */ +static inline struct page *netmem_to_page(netmem_ref netmem) +{ + return (__force struct page *)netmem; +} + +/* Converting from page to netmem is always safe, because a page can always be + * a netmem. + */ +static inline netmem_ref page_to_netmem(struct page *page) +{ + return (__force netmem_ref)page; +} + +#endif /* _NET_NETMEM_H */ From patchwork Tue Jan 9 01:14:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mina Almasry X-Patchwork-Id: 186186 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:37c1:b0:101:2151:f287 with SMTP id y1csp1378458dyq; Mon, 8 Jan 2024 17:16:06 -0800 (PST) X-Google-Smtp-Source: AGHT+IHSXX8l3VXa2j7NQZZZ2WZifL06DrOgMaA/X5mLl4XUjIAYoAzLyFIwLTnu1ntrZuFi5NM1 X-Received: by 2002:a05:6512:3054:b0:50e:8d0c:5eeb with SMTP id b20-20020a056512305400b0050e8d0c5eebmr2516521lfb.85.1704762966489; Mon, 08 Jan 2024 17:16:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704762966; cv=none; d=google.com; s=arc-20160816; b=V7nN8iCs2wyJkIvYNVgFVaXDiDeEmdNGruN/rn919Vxxu2yKAankdQXlS15UR4qBNR J8Yn6YiCvd/8WJRDmbrBj6Y0JeuQ+6gXul3cqusdz8NZ52k7SHcWVGMoNBfRzTnDVmrY HEHVYOfEGCavML1r/ETS2DV/j3MNdXcE1aTzX5BryRv+n0llEE6sqZhtFGiP4A1K2s+f kpfIm5pkAAw2LqFR464l3RADMWIAwxdsz2E/6MiEIA8oXk/2olSpK3AU80V9+otSpyHD uenyMihg7qWVv5zCVGC44pno+btFOQKoLenLZUOOq75tyoQxiM/ziZpIcKz7oKIjdJ5b KDhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=uOi1F+FRmn+fSiu8UKXKbDCli0oscgvxljmfydyauXs=; fh=3zG4NHuhPxuhQ6Shw7Nt9BiRVbWd7elghbVYpfpS9LI=; b=n/IT6RP/mFwxpKLnVsvKxFA1Tp+lf+AC7xl0/ygVtX2tBknH3fS9wZk91gjLaT0nZL I/4XTPhmtMcAagFiCO2L+6Kvg8vKhlic1oAq8+ZE1tqA00HRofYwrfOGp6sP/Pv+kRAl y8uhsjn/NAKiLw/swWA7FMcWQMWigMTipt1g55Z2ZzxvEHEptg62i+OQYXbSFJeBbTQp JwlLQVz3TGZti0e9U+OkvYEMheLNqw6venx+sZfY9tOP+1LEa3AIQ7++mwOlGW3k4PrA F1RUh0h1eLbkpOE3RiXf3yRrXOzWftz1ffAy61KNQYusCwWbCrAhDl1I5T4NIkM2Yn3F +PUQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=Hh1J1+gU; spf=pass (google.com: domain of linux-kernel+bounces-20251-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-20251-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id cf15-20020a170906b2cf00b00a2af02c94f4si397816ejb.259.2024.01.08.17.16.06 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jan 2024 17:16:06 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-20251-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=Hh1J1+gU; spf=pass (google.com: domain of linux-kernel+bounces-20251-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-20251-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id E26701F22624 for ; Tue, 9 Jan 2024 01:16:05 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C8B968801; Tue, 9 Jan 2024 01:15:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Hh1J1+gU" Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0449B28FD for ; Tue, 9 Jan 2024 01:15:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--almasrymina.bounces.google.com Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-5e8d966fbc0so29327337b3.1 for ; Mon, 08 Jan 2024 17:15:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1704762902; x=1705367702; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=uOi1F+FRmn+fSiu8UKXKbDCli0oscgvxljmfydyauXs=; b=Hh1J1+gUwJI4GeRDGZz8duGJP4QESWX/S0tTS7TIrGnUpjqZvjRBxWskoD+a8yFOHu 86iq8YcUCm/QZhhlZr2eOiFv6BIexC6mvs+nPrSuAicHg7FzzpfewLSzo5oU771F0MCz GlHgHdwEuofE+VfHHVZagfoVEqwVYYVDwZfdI8jIgVM8eQv+BSdyV2gJ6egnsqCijpSA Yg4gQbzYiZBbcCve4naQQN3h+3zE5lLawWMQzJYripMKRqrvpZUuxnaeB3ZCBt2xX/BH 4ubNu6Qa5VOattoGuMWdeQuTkFPu4+UpdNR9Xz+Bh7voZLizRL5ywYucXU2SCE9egsuk fkYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704762902; x=1705367702; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=uOi1F+FRmn+fSiu8UKXKbDCli0oscgvxljmfydyauXs=; b=aEyK0vxspPqZYDFxxJu5coHEG0RiMNAIAchh3zRzFhIjZaaXmyiM1KW9i5++PgEKcv 1wspOnoP/JpBfqlv5CQv1ThOXeyhQVml0nR5Bcewsy1uc2QwAueKME2WN6p+ysmzMRK3 Cb1rg1FqclCcpxTr1loaSuyjQy/8jzl6RgHY44suQtsx0XjkOrdSPVe3BrozxLASZgWS gqFSGxQcI2fWuM97suxkcnSzqJ5HqSotKs3BmdB2fh1q0gJIdvbf/A9Rs+g65957NkJd c3EMEym6CZPX12I8B45l3sovNf4b40Jz7/5FYUcckdUFZtRsTScbuVhFsMHqbeudWbYy JA9A== X-Gm-Message-State: AOJu0Yxh0cDu573LR76MxkJqtNSeftHaQmVRauJqKCRTbTme4OMfKDCv /pet1gmiZ8MzyvsqPCX9MDG+OzwbnhBIFw81aQ1xoEeFgIpWz+FiJH2JIjIOtyz6p82HkXx2CbT IBVeQqwtY1KYuW/sUHUG0w1uT602ArB7qdRv7QufWOVYnst8VNvOilEPQFUl5+UnTyTJCF2cEwR OehgvOGrv+ga+psw== X-Received: from almasrymina.svl.corp.google.com ([2620:15c:2c4:200:23bb:bfcc:9879:32e2]) (user=almasrymina job=sendgmr) by 2002:a05:690c:338d:b0:5d6:f1d2:2e5e with SMTP id fl13-20020a05690c338d00b005d6f1d22e5emr349161ywb.0.1704762901922; Mon, 08 Jan 2024 17:15:01 -0800 (PST) Date: Mon, 8 Jan 2024 17:14:52 -0800 In-Reply-To: <20240109011455.1061529-1-almasrymina@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240109011455.1061529-1-almasrymina@google.com> X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20240109011455.1061529-3-almasrymina@google.com> Subject: [RFC PATCH net-next v5 2/2] net: add netmem to skb_frag_t From: Mina Almasry To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: Mina Almasry , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Jason Gunthorpe , " =?utf-8?q?Christian_K=C3=B6nig?= " , Shakeel Butt , Yunsheng Lin , Willem de Bruijn X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1787573531873530093 X-GMAIL-MSGID: 1787573531873530093 Use struct netmem* instead of page in skb_frag_t. Currently struct netmem* is always a struct page underneath, but the abstraction allows efforts to add support for skb frags not backed by pages. There is unfortunately 1 instance where the skb_frag_t is assumed to be a exactly a bio_vec in kcm. For this case, WARN_ON_ONCE and return error before doing a cast. Add skb[_frag]_fill_netmem_*() and skb_add_rx_frag_netmem() helpers so that the API can be used to create netmem skbs. Signed-off-by: Mina Almasry --- v4: - Handle error in kcm_write_msgs() instead of only warning (Willem) v3: - Renamed the fields in skb_frag_t. v2: - Add skb frag filling helpers. --- include/linux/skbuff.h | 90 +++++++++++++++++++++++++++++------------- net/core/skbuff.c | 22 ++++++++--- net/kcm/kcmsock.c | 9 ++++- 3 files changed, 86 insertions(+), 35 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index a5ae952454c8..e59f76151628 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -37,6 +37,7 @@ #endif #include #include +#include /** * DOC: skb checksums @@ -359,7 +360,11 @@ extern int sysctl_max_skb_frags; */ #define GSO_BY_FRAGS 0xFFFF -typedef struct bio_vec skb_frag_t; +typedef struct skb_frag { + netmem_ref netmem; + unsigned int len; + unsigned int offset; +} skb_frag_t; /** * skb_frag_size() - Returns the size of a skb fragment @@ -367,7 +372,7 @@ typedef struct bio_vec skb_frag_t; */ static inline unsigned int skb_frag_size(const skb_frag_t *frag) { - return frag->bv_len; + return frag->len; } /** @@ -377,7 +382,7 @@ static inline unsigned int skb_frag_size(const skb_frag_t *frag) */ static inline void skb_frag_size_set(skb_frag_t *frag, unsigned int size) { - frag->bv_len = size; + frag->len = size; } /** @@ -387,7 +392,7 @@ static inline void skb_frag_size_set(skb_frag_t *frag, unsigned int size) */ static inline void skb_frag_size_add(skb_frag_t *frag, int delta) { - frag->bv_len += delta; + frag->len += delta; } /** @@ -397,7 +402,7 @@ static inline void skb_frag_size_add(skb_frag_t *frag, int delta) */ static inline void skb_frag_size_sub(skb_frag_t *frag, int delta) { - frag->bv_len -= delta; + frag->len -= delta; } /** @@ -417,7 +422,7 @@ static inline bool skb_frag_must_loop(struct page *p) * skb_frag_foreach_page - loop over pages in a fragment * * @f: skb frag to operate on - * @f_off: offset from start of f->bv_page + * @f_off: offset from start of f->netmem * @f_len: length from f_off to loop over * @p: (temp var) current page * @p_off: (temp var) offset from start of current page, @@ -2429,22 +2434,37 @@ static inline unsigned int skb_pagelen(const struct sk_buff *skb) return skb_headlen(skb) + __skb_pagelen(skb); } +static inline void skb_frag_fill_netmem_desc(skb_frag_t *frag, + netmem_ref netmem, int off, + int size) +{ + frag->netmem = netmem; + frag->offset = off; + skb_frag_size_set(frag, size); +} + static inline void skb_frag_fill_page_desc(skb_frag_t *frag, struct page *page, int off, int size) { - frag->bv_page = page; - frag->bv_offset = off; - skb_frag_size_set(frag, size); + skb_frag_fill_netmem_desc(frag, page_to_netmem(page), off, size); +} + +static inline void __skb_fill_netmem_desc_noacc(struct skb_shared_info *shinfo, + int i, netmem_ref netmem, + int off, int size) +{ + skb_frag_t *frag = &shinfo->frags[i]; + + skb_frag_fill_netmem_desc(frag, netmem, off, size); } static inline void __skb_fill_page_desc_noacc(struct skb_shared_info *shinfo, int i, struct page *page, int off, int size) { - skb_frag_t *frag = &shinfo->frags[i]; - - skb_frag_fill_page_desc(frag, page, off, size); + __skb_fill_netmem_desc_noacc(shinfo, i, page_to_netmem(page), off, + size); } /** @@ -2460,10 +2480,10 @@ static inline void skb_len_add(struct sk_buff *skb, int delta) } /** - * __skb_fill_page_desc - initialise a paged fragment in an skb + * __skb_fill_netmem_desc - initialise a fragment in an skb * @skb: buffer containing fragment to be initialised - * @i: paged fragment index to initialise - * @page: the page to use for this fragment + * @i: fragment index to initialise + * @netmem: the netmem to use for this fragment * @off: the offset to the data with @page * @size: the length of the data * @@ -2472,10 +2492,12 @@ static inline void skb_len_add(struct sk_buff *skb, int delta) * * Does not take any additional reference on the fragment. */ -static inline void __skb_fill_page_desc(struct sk_buff *skb, int i, - struct page *page, int off, int size) +static inline void __skb_fill_netmem_desc(struct sk_buff *skb, int i, + netmem_ref netmem, int off, int size) { - __skb_fill_page_desc_noacc(skb_shinfo(skb), i, page, off, size); + struct page *page = netmem_to_page(netmem); + + __skb_fill_netmem_desc_noacc(skb_shinfo(skb), i, netmem, off, size); /* Propagate page pfmemalloc to the skb if we can. The problem is * that not all callers have unique ownership of the page but rely @@ -2483,7 +2505,20 @@ static inline void __skb_fill_page_desc(struct sk_buff *skb, int i, */ page = compound_head(page); if (page_is_pfmemalloc(page)) - skb->pfmemalloc = true; + skb->pfmemalloc = true; +} + +static inline void __skb_fill_page_desc(struct sk_buff *skb, int i, + struct page *page, int off, int size) +{ + __skb_fill_netmem_desc(skb, i, page_to_netmem(page), off, size); +} + +static inline void skb_fill_netmem_desc(struct sk_buff *skb, int i, + netmem_ref netmem, int off, int size) +{ + __skb_fill_netmem_desc(skb, i, netmem, off, size); + skb_shinfo(skb)->nr_frags = i + 1; } /** @@ -2503,8 +2538,7 @@ static inline void __skb_fill_page_desc(struct sk_buff *skb, int i, static inline void skb_fill_page_desc(struct sk_buff *skb, int i, struct page *page, int off, int size) { - __skb_fill_page_desc(skb, i, page, off, size); - skb_shinfo(skb)->nr_frags = i + 1; + skb_fill_netmem_desc(skb, i, page_to_netmem(page), off, size); } /** @@ -2530,6 +2564,8 @@ static inline void skb_fill_page_desc_noacc(struct sk_buff *skb, int i, void skb_add_rx_frag(struct sk_buff *skb, int i, struct page *page, int off, int size, unsigned int truesize); +void skb_add_rx_frag_netmem(struct sk_buff *skb, int i, netmem_ref netmem, + int off, int size, unsigned int truesize); void skb_coalesce_rx_frag(struct sk_buff *skb, int i, int size, unsigned int truesize); @@ -3378,7 +3414,7 @@ static inline void skb_propagate_pfmemalloc(const struct page *page, */ static inline unsigned int skb_frag_off(const skb_frag_t *frag) { - return frag->bv_offset; + return frag->offset; } /** @@ -3388,7 +3424,7 @@ static inline unsigned int skb_frag_off(const skb_frag_t *frag) */ static inline void skb_frag_off_add(skb_frag_t *frag, int delta) { - frag->bv_offset += delta; + frag->offset += delta; } /** @@ -3398,7 +3434,7 @@ static inline void skb_frag_off_add(skb_frag_t *frag, int delta) */ static inline void skb_frag_off_set(skb_frag_t *frag, unsigned int offset) { - frag->bv_offset = offset; + frag->offset = offset; } /** @@ -3409,7 +3445,7 @@ static inline void skb_frag_off_set(skb_frag_t *frag, unsigned int offset) static inline void skb_frag_off_copy(skb_frag_t *fragto, const skb_frag_t *fragfrom) { - fragto->bv_offset = fragfrom->bv_offset; + fragto->offset = fragfrom->offset; } /** @@ -3420,7 +3456,7 @@ static inline void skb_frag_off_copy(skb_frag_t *fragto, */ static inline struct page *skb_frag_page(const skb_frag_t *frag) { - return frag->bv_page; + return netmem_to_page(frag->netmem); } /** @@ -3524,7 +3560,7 @@ static inline void *skb_frag_address_safe(const skb_frag_t *frag) static inline void skb_frag_page_copy(skb_frag_t *fragto, const skb_frag_t *fragfrom) { - fragto->bv_page = fragfrom->bv_page; + fragto->netmem = fragfrom->netmem; } bool skb_page_frag_refill(unsigned int sz, struct page_frag *pfrag, gfp_t prio); diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 12d22c0b8551..4fdc33c81969 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -845,16 +845,24 @@ struct sk_buff *__napi_alloc_skb(struct napi_struct *napi, unsigned int len, } EXPORT_SYMBOL(__napi_alloc_skb); -void skb_add_rx_frag(struct sk_buff *skb, int i, struct page *page, int off, - int size, unsigned int truesize) +void skb_add_rx_frag_netmem(struct sk_buff *skb, int i, netmem_ref netmem, + int off, int size, unsigned int truesize) { DEBUG_NET_WARN_ON_ONCE(size > truesize); - skb_fill_page_desc(skb, i, page, off, size); + skb_fill_netmem_desc(skb, i, netmem, off, size); skb->len += size; skb->data_len += size; skb->truesize += truesize; } +EXPORT_SYMBOL(skb_add_rx_frag_netmem); + +void skb_add_rx_frag(struct sk_buff *skb, int i, struct page *page, int off, + int size, unsigned int truesize) +{ + skb_add_rx_frag_netmem(skb, i, page_to_netmem(page), off, size, + truesize); +} EXPORT_SYMBOL(skb_add_rx_frag); void skb_coalesce_rx_frag(struct sk_buff *skb, int i, int size, @@ -1904,10 +1912,11 @@ int skb_copy_ubufs(struct sk_buff *skb, gfp_t gfp_mask) /* skb frags point to kernel buffers */ for (i = 0; i < new_frags - 1; i++) { - __skb_fill_page_desc(skb, i, head, 0, psize); + __skb_fill_netmem_desc(skb, i, page_to_netmem(head), 0, psize); head = (struct page *)page_private(head); } - __skb_fill_page_desc(skb, new_frags - 1, head, 0, d_off); + __skb_fill_netmem_desc(skb, new_frags - 1, page_to_netmem(head), 0, + d_off); skb_shinfo(skb)->nr_frags = new_frags; release: @@ -3645,7 +3654,8 @@ skb_zerocopy(struct sk_buff *to, struct sk_buff *from, int len, int hlen) if (plen) { page = virt_to_head_page(from->head); offset = from->data - (unsigned char *)page_address(page); - __skb_fill_page_desc(to, 0, page, offset, plen); + __skb_fill_netmem_desc(to, 0, page_to_netmem(page), + offset, plen); get_page(page); j = 1; len -= plen; diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c index 1184d40167b8..145ef22b2b35 100644 --- a/net/kcm/kcmsock.c +++ b/net/kcm/kcmsock.c @@ -636,9 +636,14 @@ static int kcm_write_msgs(struct kcm_sock *kcm) for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) msize += skb_frag_size(&skb_shinfo(skb)->frags[i]); + if (WARN_ON_ONCE(!skb_frag_page(&skb_shinfo(skb)->frags[0]))) { + ret = -EINVAL; + goto out; + } + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, - skb_shinfo(skb)->frags, skb_shinfo(skb)->nr_frags, - msize); + (const struct bio_vec *)skb_shinfo(skb)->frags, + skb_shinfo(skb)->nr_frags, msize); iov_iter_advance(&msg.msg_iter, txm->frag_offset); do {