From patchwork Mon Mar 13 21:55:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 69159 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1429915wrd; Mon, 13 Mar 2023 15:03:11 -0700 (PDT) X-Google-Smtp-Source: AK7set9wawYKu3qzWgx+fr39Mh6d7+iKtA+lfzxmM1vT/nzpAWAAmeV4f1ztSUY7csHU/so+LduB X-Received: by 2002:a17:902:d486:b0:19e:2495:20ea with SMTP id c6-20020a170902d48600b0019e249520eamr42799169plg.28.1678744991132; Mon, 13 Mar 2023 15:03:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678744991; cv=none; d=google.com; s=arc-20160816; b=btBJIspFMnqSyvByGz0eCneYjl54ovIgzxOFneQE5kd7pwV0Ixze/TV9di9BVremuE na39O9uMuhMTlBhDfJe5Q5C/PuPwqnvi7Q5LQD7bJvDEW+WDvoZ/dTcT4YefVZ1tbggo NLNBGD0IKxExlyF1MJhkk4hvVBSi9yAcXshZ4+bkj9lALZdqz/rKWg8vL0eAbY986Had 9y/3nvwDY7/wl+lDgEBi2AZRoe0TL6mk+K2MRCvD3t2hBD8ScQ10hs5buEZuzXP/bK0F ElD8KESlw/KErY5QD8qZEkIfbwtKfuYs1cnEmoByEdLJPLldX/5PgNdOf+dDC/57Elyx vlJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=dzV+7RcR4Q2XTTQGKkT0UEE7wtsOiXf8HBmzqjfNRfI=; b=HuQGspPStY+wukul61dePCOpO9GlOOk76CVufqdl3ck5I6PcC1mZVytNasWw0Ubczg C+WhTA85W3BFeTF9G3oYaE9meV2XcIjanY3c9C/HAxdrK8dZDm7a+iZ0zuhpktv0OPTE 6B1xMoZA0vhHeaUmWHJMaHVdIV0mCaWcd+Hl8R3hUZLRCSPq+xwxmrBy4Tb5kVqawwmW jT1n5v1m87ttIrj9fruiCqNDBDeTKIzLVi357n50Xjwc9zTXH/H9Ectj/fp/pe34z95Q vpWRPu2G0lhhG0DitNV+nuHiRtU0KcPQKFxJ2xAAdoCd9L9T9AQA1lWWG2Ng3FJvMk+D 2PJA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=DUclESgi; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id lh14-20020a170903290e00b00198aa530a7dsi754236plb.97.2023.03.13.15.02.53; Mon, 13 Mar 2023 15:03:11 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=DUclESgi; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230022AbjCMV5E (ORCPT + 99 others); Mon, 13 Mar 2023 17:57:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51894 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229832AbjCMV5A (ORCPT ); Mon, 13 Mar 2023 17:57:00 -0400 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4231C8C976; Mon, 13 Mar 2023 14:56:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678744619; x=1710280619; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=q3tc4GYyiOU2uNgooS6P06z+EYUuo4vPnkgkpRpdBNI=; b=DUclESgiuLTHQw7iy0unlFUfdi5Jh6B3/z+y9D7Yo/eP3OcbU2SSsUi7 gFcb1Jit9A6im0JWKEicjniCWmc7RuVuTrhvcr4QepGPV7OZwv3Yfw6bK 38YE7UZxxj4CNPG2WnEURM5MRmjQ1TcC3oMa970x2hfYSTDEn4jXJ82j3 Kc8d+QpMKPyPAosY3evjf8X0Q39iok/5CCV2Tnd7i36VCyRi2CLe8+R26 dIbvOrzZWI9NmRMzj7EHfTXEdmFfOk2DGqMmAWVkPrPgwgS8BwOuspbig klfWNZYbAlbdvh4xLgBfxVIMGGddJewQKsiEDw/JzWv3xBe2xcfBKh8Rh w==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="364928618" X-IronPort-AV: E=Sophos;i="5.98,258,1673942400"; d="scan'208";a="364928618" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 14:56:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="747750970" X-IronPort-AV: E=Sophos;i="5.98,258,1673942400"; d="scan'208";a="747750970" Received: from newjersey.igk.intel.com ([10.102.20.203]) by fmsmga004.fm.intel.com with ESMTP; 13 Mar 2023 14:56:54 -0700 From: Alexander Lobakin To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau Cc: Alexander Lobakin , Maciej Fijalkowski , Larysa Zaremba , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Song Liu , Jesper Dangaard Brouer , John Fastabend , Menglong Dong , Mykola Lysenko , "David S. Miller" , Jakub Kicinski , Eric Dumazet , Paolo Abeni , bpf@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v3 1/4] selftests/bpf: robustify test_xdp_do_redirect with more payload magics Date: Mon, 13 Mar 2023 22:55:50 +0100 Message-Id: <20230313215553.1045175-2-aleksander.lobakin@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230313215553.1045175-1-aleksander.lobakin@intel.com> References: <20230313215553.1045175-1-aleksander.lobakin@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760291707638908998?= X-GMAIL-MSGID: =?utf-8?q?1760291707638908998?= Currently, the test relies on that only dropped ("xmitted") frames will be recycled and if a frame became an skb, it will be freed later by the stack and never come back to its page_pool. So, it easily gets broken by trying to recycle skbs[0]: test_xdp_do_redirect:PASS:pkt_count_xdp 0 nsec test_xdp_do_redirect:FAIL:pkt_count_zero unexpected pkt_count_zero: actual 9936 != expected 2 test_xdp_do_redirect:PASS:pkt_count_tc 0 nsec That huge mismatch happened because after the TC ingress hook zeroes the magic, the page gets recycled when skb is freed, not returned to the MM layer. "Live frames" mode initializes only new pages and keeps the recycled ones as is by design, so they appear with zeroed magic on the Rx path again. Expand the possible magic values from two: 0 (was "xmitted"/dropped or did hit the TC hook) and 0x42 (hit the input XDP prog) to three: the new one will mark frames hit the TC hook, so that they will elide both @pkt_count_zero and @pkt_count_xdp. They can then be recycled to their page_pool or returned to the page allocator, this won't affect the counters anyhow. Just make sure to mark them as "input" (0x42) when they appear on the Rx path again. Also make an enum from those magics, so that they will be always visible and can be changed in just one place anytime. This also eases adding any new marks later on. Link: https://github.com/kernel-patches/bpf/actions/runs/4386538411/jobs/7681081789 Signed-off-by: Alexander Lobakin --- .../bpf/progs/test_xdp_do_redirect.c | 36 +++++++++++++------ 1 file changed, 26 insertions(+), 10 deletions(-) diff --git a/tools/testing/selftests/bpf/progs/test_xdp_do_redirect.c b/tools/testing/selftests/bpf/progs/test_xdp_do_redirect.c index 77a123071940..cd2d4e3258b8 100644 --- a/tools/testing/selftests/bpf/progs/test_xdp_do_redirect.c +++ b/tools/testing/selftests/bpf/progs/test_xdp_do_redirect.c @@ -4,6 +4,19 @@ #define ETH_ALEN 6 #define HDR_SZ (sizeof(struct ethhdr) + sizeof(struct ipv6hdr) + sizeof(struct udphdr)) + +/** + * enum frame_mark - magics to distinguish page/packet paths + * @MARK_XMIT: page was recycled due to the frame being "xmitted" by the NIC. + * @MARK_IN: frame is being processed by the input XDP prog. + * @MARK_SKB: frame did hit the TC ingress hook as an skb. + */ +enum frame_mark { + MARK_XMIT = 0U, + MARK_IN = 0x42, + MARK_SKB = 0x45, +}; + const volatile int ifindex_out; const volatile int ifindex_in; const volatile __u8 expect_dst[ETH_ALEN]; @@ -34,10 +47,10 @@ int xdp_redirect(struct xdp_md *xdp) if (*metadata != 0x42) return XDP_ABORTED; - if (*payload == 0) { - *payload = 0x42; + if (*payload == MARK_XMIT) pkts_seen_zero++; - } + + *payload = MARK_IN; if (bpf_xdp_adjust_meta(xdp, 4)) return XDP_ABORTED; @@ -51,7 +64,7 @@ int xdp_redirect(struct xdp_md *xdp) return ret; } -static bool check_pkt(void *data, void *data_end) +static bool check_pkt(void *data, void *data_end, const __u32 mark) { struct ipv6hdr *iph = data + sizeof(struct ethhdr); __u8 *payload = data + HDR_SZ; @@ -59,13 +72,13 @@ static bool check_pkt(void *data, void *data_end) if (payload + 1 > data_end) return false; - if (iph->nexthdr != IPPROTO_UDP || *payload != 0x42) + if (iph->nexthdr != IPPROTO_UDP || *payload != MARK_IN) return false; /* reset the payload so the same packet doesn't get counted twice when * it cycles back through the kernel path and out the dst veth */ - *payload = 0; + *payload = mark; return true; } @@ -75,11 +88,11 @@ int xdp_count_pkts(struct xdp_md *xdp) void *data = (void *)(long)xdp->data; void *data_end = (void *)(long)xdp->data_end; - if (check_pkt(data, data_end)) + if (check_pkt(data, data_end, MARK_XMIT)) pkts_seen_xdp++; - /* Return XDP_DROP to make sure the data page is recycled, like when it - * exits a physical NIC. Recycled pages will be counted in the + /* Return %XDP_DROP to recycle the data page with %MARK_XMIT, like + * it exited a physical NIC. Those pages will be counted in the * pkts_seen_zero counter above. */ return XDP_DROP; @@ -91,9 +104,12 @@ int tc_count_pkts(struct __sk_buff *skb) void *data = (void *)(long)skb->data; void *data_end = (void *)(long)skb->data_end; - if (check_pkt(data, data_end)) + if (check_pkt(data, data_end, MARK_SKB)) pkts_seen_tc++; + /* Will be either recycled or freed, %MARK_SKB makes sure it won't + * hit any of the counters above. + */ return 0; } From patchwork Mon Mar 13 21:55:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 69161 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1435523wrd; Mon, 13 Mar 2023 15:16:10 -0700 (PDT) X-Google-Smtp-Source: AK7set+rFE5YS/QXZIqOaAf1hH4IpuLFA9nO4a2g+MjBiHeqDhDF2afmeEl5M/CSrTgseZ9P/ySb X-Received: by 2002:a05:6a20:9390:b0:c6:ff46:c713 with SMTP id x16-20020a056a20939000b000c6ff46c713mr44476701pzh.57.1678745770462; Mon, 13 Mar 2023 15:16:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678745770; cv=none; d=google.com; s=arc-20160816; b=NDYccsPbyZb9HZRYW1P8jpT4jNd2+/qgVqTkNPaNnxdxcJxZkfdK99FDMd+5hsV9xv 9w2xuatfehpK9+YOeSWK6V6BWXx7STemT+zSo31wum6nY79irZuGVLG9XkoxPIubvKOr 7HK+SVajVoFe7DK/ZV8gknhqQXcsva+9ZYUzAUS1P7fdmB0wBc4kgI16vFp7OSpcHXuB /maVGOAkXBeHbQMB4UUMBUNFfvAnZUbS61dW+Vz8LDc041676o/yfDyMdIL/gtLVnCsv p5wBiayHBavJwIfHjbv4syiRPDDd4NXwo9rgo8NkXIL0gEWXhLkmVvkpqso2RwF+ULHQ 4uyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=CErF05Pxkbg8TlBX5rNA+1nEY6eFFfwq5DTottbupX8=; b=BwUOXeUgmB+MVz2Y+z8s4K2PcPni8fR5zBULoWb5lxC+O6j30X6Cmw/cMQlgA+M3vl CBvmT6oz8Ajm4zBmc2fU/y/7gpCrliklyPwy60oqng18INkjCkOk4wSDK8eQ01rjnS8v icLXsWnMNVyZMK/Wp7Eb4w2ZNP8/UMv8bRJyCbxJGX2bvxLNyFaqKdGq5OWARaemZl4s F6ufk9Yao4rc/N8pI/Tmjs6e8I3DEMi6qjuJXv2HOJVkRrhDIH8gSA1ktt15e3BQL6yG 5Ttf8jNfv6vWt7wQ/dNS7nmULk60RP2xKEFZCDDcyN0swRMPWSndPCm0xLSgFqZgnjWw Zc5g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=leclfKGU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id p29-20020a635b1d000000b00502d861f9e5si562754pgb.322.2023.03.13.15.15.56; Mon, 13 Mar 2023 15:16:10 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=leclfKGU; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230135AbjCMV5S (ORCPT + 99 others); Mon, 13 Mar 2023 17:57:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52154 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230079AbjCMV5I (ORCPT ); Mon, 13 Mar 2023 17:57:08 -0400 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 725768F53C; Mon, 13 Mar 2023 14:57:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678744623; x=1710280623; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=g5nPL1hfwFccNcFtYw7ZevITqww5GF47MwYYKvo0Lf4=; b=leclfKGU2+PbK7GAisLWnByA9dvlwT3kmVFeHOeMv0F2TD2W/YSWe5aG 8TjregJ/9oxyF7cz8UrX9FFOHtRSppFQSzpARKYF94a0fUTZRVeBZ7a9y JLJZCYpAd1ONg/wpiClvEq+q7bbs/WCjx0UBZqDLmwaWigEqCFbdOD/SJ AiHn3z8Jhj2EvODvinf1tXUapUkUe7Mi18g68w/34iXs08RVYf9m4Gkpu sk8Z2Vwekz8DFQkRp4wHU5TQzJa8RNatbO5pLj5rgr3RC4QL+CiytMHsV zyUpWXV2R4Z4QyCAmext4xQvphplSFBUrrp/0Nvc4wYfrxzZd1FcpFeaO w==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="364928635" X-IronPort-AV: E=Sophos;i="5.98,258,1673942400"; d="scan'208";a="364928635" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 14:57:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="747750975" X-IronPort-AV: E=Sophos;i="5.98,258,1673942400"; d="scan'208";a="747750975" Received: from newjersey.igk.intel.com ([10.102.20.203]) by fmsmga004.fm.intel.com with ESMTP; 13 Mar 2023 14:56:59 -0700 From: Alexander Lobakin To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau Cc: Alexander Lobakin , Maciej Fijalkowski , Larysa Zaremba , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Song Liu , Jesper Dangaard Brouer , John Fastabend , Menglong Dong , Mykola Lysenko , "David S. Miller" , Jakub Kicinski , Eric Dumazet , Paolo Abeni , bpf@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, kernel test robot Subject: [PATCH bpf-next v3 2/4] net: page_pool, skbuff: make skb_mark_for_recycle() always available Date: Mon, 13 Mar 2023 22:55:51 +0100 Message-Id: <20230313215553.1045175-3-aleksander.lobakin@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230313215553.1045175-1-aleksander.lobakin@intel.com> References: <20230313215553.1045175-1-aleksander.lobakin@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760292524520853666?= X-GMAIL-MSGID: =?utf-8?q?1760292524520853666?= skb_mark_for_recycle() is guarded with CONFIG_PAGE_POOL, this creates unneeded complication when using it in the generic code. For now, it's only used in the drivers always selecting Page Pool, so this works. Move the guards so that preprocessor will cut out only the operation itself and the function will still be a noop on !PAGE_POOL systems, but available there as well. No functional changes. Reported-by: kernel test robot Link: https://lore.kernel.org/oe-kbuild-all/202303020342.Wi2PRFFH-lkp@intel.com Signed-off-by: Alexander Lobakin --- include/linux/skbuff.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index fe661011644b..3f3a2a82a86b 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -5069,12 +5069,12 @@ static inline u64 skb_get_kcov_handle(struct sk_buff *skb) #endif } -#ifdef CONFIG_PAGE_POOL static inline void skb_mark_for_recycle(struct sk_buff *skb) { +#ifdef CONFIG_PAGE_POOL skb->pp_recycle = 1; -} #endif +} #endif /* __KERNEL__ */ #endif /* _LINUX_SKBUFF_H */ From patchwork Mon Mar 13 21:55:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 69163 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1436521wrd; Mon, 13 Mar 2023 15:18:52 -0700 (PDT) X-Google-Smtp-Source: AK7set8M4w4+4cfrfh2VqPnrOAvES6CTfug7ZOnCPgvZeY55h8ULJQKyOfBiQTailhKQUnJflHlD X-Received: by 2002:a05:6a20:3cab:b0:cd:3bca:cfa2 with SMTP id b43-20020a056a203cab00b000cd3bcacfa2mr38158166pzj.23.1678745932062; Mon, 13 Mar 2023 15:18:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678745932; cv=none; d=google.com; s=arc-20160816; b=b//DMfr4wBFmjkkOA3u7LN1Xzey2Ea+BfVj0zKKL3hXm857boiJXT08q6Sgo+om5hr ZNZEN1NxYdNIYqGOMXRpSsywjA4j8G9eLjL6w+QKxfFdtrEH9vWakmmVbIPgVSgNqL0H d6R7S8DaWZSs/6/uB2JWDyHpjMSbgjYYZTBdOCwjEvLAhqUO1sHuyI+qNSYae5X505kd PxwAqW48iX6zo6S3OUr32kjqf6KR1W0CH8lfnZC9pPx3sSXPVHLUj1xFcZfDzfLeK4Fe LcYuWZSyD5d6RkT7BbxkWhgG+KFpomKWtc1vqc1/+BDvWiw+Z1GGnE49wq14ErrNq0St yD5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=2DU+rl9zRBk1Nx+b+CkkzoF0vJbv2MVMdHq1+gT7Id8=; b=rph1EKntBtqJsG6jBeNgAG6puXFKnrt3KwXgcN3ZyPutlRJweVdsoKt94o1z/A34Xq 27Ngpt5bYuqcY5JBPRf/t0hh1m3PLvwvTSH09Vmc8m2QEvMAZoT6baCSnYpYzdUi/hMF NHChsGPwYqiBdPnWVN7d1RV9N41u0YFu0OxZCNF+S9kvI6L/WNlCliLaVFTvYVgcVyHs tNiCHfOJNVn8z6ucZIJag1n3BVWo/J5AeL84ZBF3ipjSGYbBcfq3TRjWvtFDjst5ujRV xBfE/yBHbWmuZgNamAko/rdVZRNCKOjcS6DljOLQQd+E43Ci/tfoIXG0xEAmhsoT8neI DGGQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=MyGi+7i6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id v28-20020a63481c000000b00509461bbf1csi562624pga.79.2023.03.13.15.18.36; Mon, 13 Mar 2023 15:18:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=MyGi+7i6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230097AbjCMV50 (ORCPT + 99 others); Mon, 13 Mar 2023 17:57:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52426 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229743AbjCMV5R (ORCPT ); Mon, 13 Mar 2023 17:57:17 -0400 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F344B8F530; Mon, 13 Mar 2023 14:57:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678744627; x=1710280627; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WV2QgxuHT3p8j+KGI77slUqYCqNGD1cnP+Xz71qqoTc=; b=MyGi+7i6tdmk2MU1rQoilOIcb3i+ftu+R7cev1k6nvB9N8j/eiX/7ikb TiPcIU7Ig5DuB++0TdD9NV3UN6ccz22Wb8klZRgFHW6YcEGV+OmLLLH40 2l785/t4u6TOCdwYywj1pqhFuvMVovd5kBqEcTgw01qRx0ZiHsJtco/9M ZF4PnY9aMGL/ezotL0/DNKOrTlMThy0MXAQZEpJy6cOuYW9sB6r/H6/VE MdtqaIoF0PI9XBylFd5uazqtjXDQL0nWu1VhZxC2fGqQ/YCd8YLoj5jVg vmszJNbLngkz84v1cFJJgQGRiSv7HHkxKyN0Ku2eY7oy7cGAS+L3DmFsj w==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="364928655" X-IronPort-AV: E=Sophos;i="5.98,258,1673942400"; d="scan'208";a="364928655" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 14:57:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="747750981" X-IronPort-AV: E=Sophos;i="5.98,258,1673942400"; d="scan'208";a="747750981" Received: from newjersey.igk.intel.com ([10.102.20.203]) by fmsmga004.fm.intel.com with ESMTP; 13 Mar 2023 14:57:03 -0700 From: Alexander Lobakin To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau Cc: Alexander Lobakin , Maciej Fijalkowski , Larysa Zaremba , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Song Liu , Jesper Dangaard Brouer , John Fastabend , Menglong Dong , Mykola Lysenko , "David S. Miller" , Jakub Kicinski , Eric Dumazet , Paolo Abeni , bpf@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v3 3/4] xdp: recycle Page Pool backed skbs built from XDP frames Date: Mon, 13 Mar 2023 22:55:52 +0100 Message-Id: <20230313215553.1045175-4-aleksander.lobakin@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230313215553.1045175-1-aleksander.lobakin@intel.com> References: <20230313215553.1045175-1-aleksander.lobakin@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760292137802948721?= X-GMAIL-MSGID: =?utf-8?q?1760292694286087498?= __xdp_build_skb_from_frame() state(d): /* Until page_pool get SKB return path, release DMA here */ Page Pool got skb pages recycling in April 2021, but missed this function. xdp_release_frame() is relevant only for Page Pool backed frames and it detaches the page from the corresponding page_pool in order to make it freeable via page_frag_free(). It can instead just mark the output skb as eligible for recycling if the frame is backed by a pp. No change for other memory model types (the same condition check as before). cpumap redirect and veth on Page Pool drivers now become zero-alloc (or almost). Signed-off-by: Alexander Lobakin --- net/core/xdp.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/net/core/xdp.c b/net/core/xdp.c index 8c92fc553317..a2237cfca8e9 100644 --- a/net/core/xdp.c +++ b/net/core/xdp.c @@ -658,8 +658,8 @@ struct sk_buff *__xdp_build_skb_from_frame(struct xdp_frame *xdpf, * - RX ring dev queue index (skb_record_rx_queue) */ - /* Until page_pool get SKB return path, release DMA here */ - xdp_release_frame(xdpf); + if (xdpf->mem.type == MEM_TYPE_PAGE_POOL) + skb_mark_for_recycle(skb); /* Allow SKB to reuse area used by xdp_frame */ xdp_scrub_frame(xdpf); From patchwork Mon Mar 13 21:55:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Lobakin X-Patchwork-Id: 69162 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:5915:0:0:0:0:0 with SMTP id v21csp1435588wrd; Mon, 13 Mar 2023 15:16:19 -0700 (PDT) X-Google-Smtp-Source: AK7set/5/5WwVDiyTzRdecjXFKybZK0875OaJR2J6rqGjcqsxHzkJp+3EfPMOV9Oac3tcfpAlRau X-Received: by 2002:a17:902:da90:b0:19a:a0d0:10f0 with SMTP id j16-20020a170902da9000b0019aa0d010f0mr43846746plx.23.1678745779313; Mon, 13 Mar 2023 15:16:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1678745779; cv=none; d=google.com; s=arc-20160816; b=Iwittbs6qEKndRtn0O/1zBizXn6m2EUUgOPy6y4pyZZEuegbkl6N7Em6Ef6O/bz6OQ 1GXS1ehA4qMdZjMWxUOUKv+NWgBvcTCbilfVMboykjO9f3Hoyk7eDt9iEZdfSIFdKNRE 7+yArhAILxBBhIuynH0UB/7yRrTkX22QIXKnQPMIPzGtcmofoa3KY9lT7ZYo4/oqQVux AEtu1oqklBTrI/0hPof3ze2zntbE9IuF/lp5QlZJfRNDrgR54WR9JwXN6DulnuwpkG/N VorZNeuVfQIAkdKeHlBic/3J1LlCb0xgI1Bekf0jC0AgW7xmHOjffyerIJV3yumqSLHe gfIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=3vMpschW76t6x2KYSHQedx2TNHxOVXCfx58osMRhQvI=; b=r0jB6zVmsT3T2VZls4KLu17pWRB5A5VfQn1wxO1XPJg3i67wgbgVxpa12lq4jx6TKk k//e/SNG+dMYI/ukpVZ+OTp/HcYQmjrpIjTV8xhbwfWzczliJNDy2WeFSgOT6TLKQy+B v+JuKXdX51V5yRESScCQyIuQ3P/wT0xxsCR+ukXywwXzNkfzPuL+LWvm39BbvQTuwgX+ gIIFshfEK/gG4yeFHh1x5Vr20AKZoq0l+/JJpArh9VCUD76GCp24bAfTd0KlZ+mJkA11 5HmXC0noYKP634VNfZB9IdTAyZkX7N6nupQAf2GCnm/kIYIuENvHCDo0I0nDJrRJjRXO kofA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="G3oUjM/j"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id jg22-20020a17090326d600b0019e3728f17csi808441plb.209.2023.03.13.15.16.04; Mon, 13 Mar 2023 15:16:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b="G3oUjM/j"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230111AbjCMV5v (ORCPT + 99 others); Mon, 13 Mar 2023 17:57:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52978 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230176AbjCMV5l (ORCPT ); Mon, 13 Mar 2023 17:57:41 -0400 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7A82A9009A; Mon, 13 Mar 2023 14:57:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678744633; x=1710280633; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZFpBMYKp7Vej9uaBP1GUTHm12MmkAD4Atk8fjSWwcN0=; b=G3oUjM/jERp3N2bTKOWoilQ2IctW14Ps+K4Mcv1YLoDbbTIJANpA6zXc RLvy2YcMFSHDzyoW8J3FF/fGK2wCh6Jy9K/DYYABPFfWM+GzZ2pZqX/B9 g+ffd26W2112hUHd8vSLt+Ney4AhPhgsEVlojSFfHH4TWyzHZCikyhTAU rIU3Gt4vAc3gfrSlSJ58xoCOwHSwdqvl3x1WPSgKKpTC3XdTbtF9avD4Z WcCMKaPjuKrmYj14P+gUzQxUsQ76gw+UT3Lk/zpJdyWHv6VFhg9cMeI7/ /k42A/NbWzlB1clUGuvNEJTNdEM01edRRPalrdR8OdlkGRlfhZ1McC6oP Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="364928677" X-IronPort-AV: E=Sophos;i="5.98,258,1673942400"; d="scan'208";a="364928677" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 14:57:11 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10648"; a="747750987" X-IronPort-AV: E=Sophos;i="5.98,258,1673942400"; d="scan'208";a="747750987" Received: from newjersey.igk.intel.com ([10.102.20.203]) by fmsmga004.fm.intel.com with ESMTP; 13 Mar 2023 14:57:07 -0700 From: Alexander Lobakin To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau Cc: Alexander Lobakin , Maciej Fijalkowski , Larysa Zaremba , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , Song Liu , Jesper Dangaard Brouer , John Fastabend , Menglong Dong , Mykola Lysenko , "David S. Miller" , Jakub Kicinski , Eric Dumazet , Paolo Abeni , bpf@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next v3 4/4] xdp: remove unused {__,}xdp_release_frame() Date: Mon, 13 Mar 2023 22:55:53 +0100 Message-Id: <20230313215553.1045175-5-aleksander.lobakin@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230313215553.1045175-1-aleksander.lobakin@intel.com> References: <20230313215553.1045175-1-aleksander.lobakin@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1760292534517790846?= X-GMAIL-MSGID: =?utf-8?q?1760292534517790846?= __xdp_build_skb_from_frame() was the last user of {__,}xdp_release_frame(), which detaches pages from the page_pool. All the consumers now recycle Page Pool skbs and page, except mlx5, stmmac and tsnep drivers, which use page_pool_release_page() directly (might change one day). It's safe to assume this functionality is not needed anymore and can be removed (in favor of recycling). Signed-off-by: Alexander Lobakin --- include/net/xdp.h | 29 ----------------------------- net/core/xdp.c | 15 --------------- 2 files changed, 44 deletions(-) diff --git a/include/net/xdp.h b/include/net/xdp.h index d517bfac937b..5393b3ebe56e 100644 --- a/include/net/xdp.h +++ b/include/net/xdp.h @@ -317,35 +317,6 @@ void xdp_flush_frame_bulk(struct xdp_frame_bulk *bq); void xdp_return_frame_bulk(struct xdp_frame *xdpf, struct xdp_frame_bulk *bq); -/* When sending xdp_frame into the network stack, then there is no - * return point callback, which is needed to release e.g. DMA-mapping - * resources with page_pool. Thus, have explicit function to release - * frame resources. - */ -void __xdp_release_frame(void *data, struct xdp_mem_info *mem); -static inline void xdp_release_frame(struct xdp_frame *xdpf) -{ - struct xdp_mem_info *mem = &xdpf->mem; - struct skb_shared_info *sinfo; - int i; - - /* Curr only page_pool needs this */ - if (mem->type != MEM_TYPE_PAGE_POOL) - return; - - if (likely(!xdp_frame_has_frags(xdpf))) - goto out; - - sinfo = xdp_get_shared_info_from_frame(xdpf); - for (i = 0; i < sinfo->nr_frags; i++) { - struct page *page = skb_frag_page(&sinfo->frags[i]); - - __xdp_release_frame(page_address(page), mem); - } -out: - __xdp_release_frame(xdpf->data, mem); -} - static __always_inline unsigned int xdp_get_frame_len(struct xdp_frame *xdpf) { struct skb_shared_info *sinfo; diff --git a/net/core/xdp.c b/net/core/xdp.c index a2237cfca8e9..8d3ad315f18d 100644 --- a/net/core/xdp.c +++ b/net/core/xdp.c @@ -531,21 +531,6 @@ void xdp_return_buff(struct xdp_buff *xdp) } EXPORT_SYMBOL_GPL(xdp_return_buff); -/* Only called for MEM_TYPE_PAGE_POOL see xdp.h */ -void __xdp_release_frame(void *data, struct xdp_mem_info *mem) -{ - struct xdp_mem_allocator *xa; - struct page *page; - - rcu_read_lock(); - xa = rhashtable_lookup(mem_id_ht, &mem->id, mem_id_rht_params); - page = virt_to_head_page(data); - if (xa) - page_pool_release_page(xa->page_pool, page); - rcu_read_unlock(); -} -EXPORT_SYMBOL_GPL(__xdp_release_frame); - void xdp_attachment_setup(struct xdp_attachment_info *info, struct netdev_bpf *bpf) {