From patchwork Thu Jul 6 08:10:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Fang X-Patchwork-Id: 116575 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f45:0:b0:3ea:f831:8777 with SMTP id v5csp2420623vqx; Thu, 6 Jul 2023 02:16:49 -0700 (PDT) X-Google-Smtp-Source: APBJJlG6MPWlUOVP9Cc+MoJSfRaQ9aYtvhyfbCgYHweuWhF2QeNdKiyy6hJKM0oPFSeD0QM9s84u X-Received: by 2002:a17:902:daca:b0:1b8:902c:f8e8 with SMTP id q10-20020a170902daca00b001b8902cf8e8mr1030870plx.60.1688635009263; Thu, 06 Jul 2023 02:16:49 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1688635009; cv=pass; d=google.com; s=arc-20160816; b=qqLWmdeMNEpbvX3KQ4Nmlm2OJzkaXV9hNivobo02djNwjc07v8YCaNRXBPXVveLjY4 K8cA1iht+VCh/h0wmB6zPw2eUgg0j+7RYvMOkqODu7+/Xy7ONavM3wxv8Nfi92XV3RKB ItPb7GDeijGPw8C6c6o6K+Znpc0JZpqQ55UqxG3PV7tlR6wEZrwKUcdZm8xdVbgtUr8G gQ6zQcox4p2AR3R3v9Q1aWJScQ/hUZHfyazGKrPtQKdvz3WsuyTSlJRiOKvDrhwmpIv4 IYf31QRu0HdDFB7ilJ9S0uWd3Hw0NKUZbaonr4FGLEgCNkL21lN0SGm77PukBN+kyeYd T8tw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:content-transfer-encoding :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ALHnrNUVR0r6vkOh6Qg2LeUMUThQ/qrbFVs+hSXhMBQ=; fh=AE1PERU6bhr9O8eJNdZsAjdBxzfHJxzlkxEKIMP4fEw=; b=PaW9GaXh4p20jMYab8Oa2HDociaROKFwRmnF4fFPGLFblijPhl9eXmQW9G/gEQBBkU BMHSAglTMe+2ihQQoFnTKnJlPLpH8eBpjA+XgTOUtec1n40VejtVyRXfz34bYwpFaFXz Ox1GgXVz2lYtHeX6cZvq+p1buKHBnUApUDuuThJv8H4GX6n0xBQaMMEfvgYOFfz7a9x8 5uX3kUBXSHGhEXMQfO1poJShJCY+lbYxMncVsyR3tqbbo8iYkGCOjVLEX3vtLC/mzAz9 KLzwDX/czbEh8VGEU3xjOxPqenPdHuJZWs21qjYSunWWokVmY3uHQhLJF8tAMQn5sVKl owtg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@nxp.com header.s=selector2 header.b="jPKl58/j"; arc=pass (i=1 spf=pass spfdomain=nxp.com dkim=pass dkdomain=nxp.com dmarc=pass fromdomain=nxp.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nxp.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b14-20020a170902650e00b001b872e20955si949484plk.30.2023.07.06.02.16.33; Thu, 06 Jul 2023 02:16:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@nxp.com header.s=selector2 header.b="jPKl58/j"; arc=pass (i=1 spf=pass spfdomain=nxp.com dkim=pass dkdomain=nxp.com dmarc=pass fromdomain=nxp.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nxp.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233921AbjGFIRD (ORCPT + 99 others); Thu, 6 Jul 2023 04:17:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41104 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233912AbjGFIQ7 (ORCPT ); Thu, 6 Jul 2023 04:16:59 -0400 Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01on2075.outbound.protection.outlook.com [40.107.15.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9A6AF1BEB; Thu, 6 Jul 2023 01:16:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XdduhFNOoaYdYL4RnnsrUOR2zM2E8hPLvQxc248HsFSj0t5oWx236sGZDhjABALYQThGuU9ZjsXTNBLBqUNjhRlMj1UFO/04m6tElUHlxPQWQXAIfl5etKo+GWhjQDeFn+bTgu+DYL69e+4XDS4kaaplCccafy+YArKWfJCSrRK45Nf4XxtRrPhztWcIc6nAchJQJhqQQd+luGhAEDg5pReqG+xx34NkzP5DUjh1/gZ588DsxSOz/jg0gt9+R3Ckono5wZVKCn6iv48w7B9c9zP/kE1+WlH2GYHyZ4i3RJy7aG/mkkCJAxJZQpEHV9XkFgqyyatPqx7K6xum6vsVJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ALHnrNUVR0r6vkOh6Qg2LeUMUThQ/qrbFVs+hSXhMBQ=; b=A3k/GQETsIXK7WE806RarBYgCCAfx7EpXrwqudlUYvvYfMdI9JfBgatdVnoypoBzPIOlDy/9dSbIRDwX3QPF32NPEI8IRhwYeAuggwPkvCZaQWCDCD3CDOxCOhxbpG8mWegxozTLT7rnxiVcucmQfsLDgeYiQ1lbgRMl3akk0MTLRcPip7/lqplvXJUcMSo87pF+RrKPSC+pgKQoUCG8vG2E0nirqEL6WBjJq5LwPEzG4XQT5qjL5EJZltK8W0rtS1h/2OGx+wB6zsEDqPIpC9GyL8iBmtXOhhHhBl+L6C8zByp7kYG62sQn1vYIOO5A82XYxNWc/hjr+7MqA6yIJQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ALHnrNUVR0r6vkOh6Qg2LeUMUThQ/qrbFVs+hSXhMBQ=; b=jPKl58/j0haALTYorvkdBD+JfTs6WBrOW8vx18WeONvFLSHDxnv3wjqp7NJe1kAnxx+SKJE5FuhJAOOmEfBNA7wKparIVo/2ww0fLnkRHD5/asu6Ni9GkkpbIpDOCHMJiwuUF947cqwozX0+Y+KLgcvKYYyvaduNFod4ItEiwRI= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; Received: from AM5PR04MB3139.eurprd04.prod.outlook.com (2603:10a6:206:8::20) by DU2PR04MB9132.eurprd04.prod.outlook.com (2603:10a6:10:2f7::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6565.17; Thu, 6 Jul 2023 08:16:55 +0000 Received: from AM5PR04MB3139.eurprd04.prod.outlook.com ([fe80::1edd:68cb:85d0:29e0]) by AM5PR04MB3139.eurprd04.prod.outlook.com ([fe80::1edd:68cb:85d0:29e0%7]) with mapi id 15.20.6544.024; Thu, 6 Jul 2023 08:16:55 +0000 From: wei.fang@nxp.com To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, ast@kernel.org, daniel@iogearbox.net, hawk@kernel.org, john.fastabend@gmail.com, shenwei.wang@nxp.com, xiaoning.wang@nxp.com, netdev@vger.kernel.org Cc: linux-imx@nxp.com, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH V2 net 2/4] net: fec: recycle pages for transmitted XDP frames Date: Thu, 6 Jul 2023 16:10:10 +0800 Message-Id: <20230706081012.2278063-3-wei.fang@nxp.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230706081012.2278063-1-wei.fang@nxp.com> References: <20230706081012.2278063-1-wei.fang@nxp.com> X-ClientProxiedBy: SG2P153CA0018.APCP153.PROD.OUTLOOK.COM (2603:1096::28) To AM5PR04MB3139.eurprd04.prod.outlook.com (2603:10a6:206:8::20) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM5PR04MB3139:EE_|DU2PR04MB9132:EE_ X-MS-Office365-Filtering-Correlation-Id: e8fb4d4f-cf03-4017-5f70-08db7df95c14 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4OlyeJtU/PHVKS0waF9+3lF29WXkUH25vH22nhSDc47v5q7A3OX7Fa3IwaCNvZhukxyRdk0ku0hddbLv8boIzYACmxp9E40Zi44XA7v8c+m69nr+0bMW4Q3rJZmOVg/pv3yNet5DmVn8PTfbcW0ld4cadcgTPdssDDWnFT5AjZf1Ig5n9PRmsoRu/TCaxWHBIN+nBxwF4p6PV/UoLCd/CZWNm/2+wgcHz71vrcc0I/KUkJ3ENuK+CsXRc6Eu7LlmD+bVaTZGeR6VfF5jHgeWZmTdiWgE3uMc5V19dzv1W6KAleTz+xQTUk0obZe36cipz83/kZvnT1exWAPP3hPP1jQAEiNzG4oRfYqGGWh51yfz9uUDWmmnv08TGJz+99ESL5G5a/hCgWTxIdjnuezReqrPVuOcxmcm11FHgkCXb/ND91R52ngGtRb6Ai3Klm1+Fme4r9YfyAK1zSXCszaGd18IYZZWDYpqXrYaViZxMIwKtwpmNhMLBQZGBfBTmbUs6SThuHEey+BB1eI+54Vp80ASH1bh+RStHWIRkoT4Err5hnLacKzgajDibicftqPnOTtxxybQs3xR78RL8hs6Gyg9lLhv0XBzMZT5yIodhnPJvCjpBHopjAnH4y2C2oHLWXbFegdRDqy2bxAI3fr/ZQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM5PR04MB3139.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(366004)(136003)(396003)(376002)(39860400002)(346002)(451199021)(186003)(8676002)(8936002)(6506007)(26005)(2616005)(2906002)(66946007)(1076003)(5660300002)(7416002)(41300700001)(36756003)(921005)(6486002)(4326008)(316002)(66476007)(83380400001)(6666004)(66556008)(6512007)(478600001)(9686003)(52116002)(38350700002)(86362001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 0AC6w4k/JnzsKCYyEG4m16U5620k0zXSha2LaQi2CUTzxwH74ABWTyfhQTK7qiL4/AAhuDQmxiWmJuuxlIZHGjAhtOw9kWl1CF5jgA4+AK4i6MLkbw6prwCH/P6o/VG+5xwBg0ltH/HbbzC/Fz2bMbqD714G+NzuczqjSIVj1R3FeqFLZYwpcFEMbl5nGXsm5cMy9Ky2aVaTO7QmtQvK86Nq+0J0jiJxd3aIl+95Txw+hGf1YdLQzRZ8gFEoMg1cXwIlYVmxj7YvFwZIM/GHbLc2hD5XrXNPeYDsa63qW+CF5rjDOE5VPDrjnDfqrAnIfiUpmKidAQdN7LZXVYkTVB+5YnjCpk4aHIAsPSurwJBXH4ndkkKC8YExDV1XHreNXXfNHKkltZRklZfvNSSWm3gk9j0q+DrlDQAITdZQda/XB3x47rMkKEMmhQ4RVrZg+7lX0nKWHfxzSUw/81KFc9x6JExSFoS2eIScVIgIMF9JUJhYnusIcwtGN1ECuFq9RXIhfqnQ6gFmBqM1we8m4OJ5r5HDchMwWwgPCllu9GcmHo62rIM+G8DIZo45i0E3dPYsQbajE/VWmPwcbbC+2B4ZzitA/aI172+4AqDkJ6TrZiYnaPaPHsMeigmHao+NQS5QC/gKMBo/S1onE/v+USuLVo6WQ7Qe8I5Ks8xCL3k6p45dL69q8kZ6Dk56Ds8PQtbf4XTut+JPaPsTX1iGXBS0l1Y2cvj60qeswKK9PC48iyD8eNvR/t/hJCZzkyHDB3vC2GtoOYcZotj8d1XnKoHXBmnSHBaiSrg9z7Q9TvBxn85VPmZhOb/EvE9RJRa3ud4oKssOKm8u/0J6Sh9G8OIE09Xj563Ny5s2U+VUjg26AP3rYV+Ptb7V4Tkzs+Mza3OM4Xq7BhDMvZYfwSFByH4uXHhVBadYaHDBlUK56urPF1wKNYJn5jtu3cZh6uE04/6wf/b7NkeqQnu59Ms5AaATB9NwnpZNhxUJq5UM1AKyVPi0WmK0kw8yNwG0rr4b2KjWgOnDz18yqnBM5pOEpHGXhPt15iOajv6Pbxn1xmki8B2KYl6xDC69tqvoI1ss+rvsWJAL9XTmeyuayvSAy9tSbNeVILuhCVdyPf5E/6FdFP9/7LUjp/SZYcZMWN1CQ+ZzLFEDJ7mpzm9Y0QztNrXvlU/3oZmKRg3wuwb4mVx3BYcmwcIXogm+ngvEnUg/4vc1fxaLscAbB2cAR5jP4J2Yi385llEbWq6PBV1XiuCQuhv+x+7sefyireRC6KT9BUC7UDn5oe0hVMmPCVw1H+u/YXQVR/QmlkiT2XdVXVllD5WR9KVWHjlzPYB+nDbYBO027SzV9KXfGLGVLEF8W6e2dM8KzmhEzuAsymfPXAuWkPF4LRYOrhPxfccP8m3YyE9kSKV2GOwV1LvNc5qJjxAYhJLIB8N8ucEsRt4XfJfTDr20riIxBsnq2ns2hAe51+xs/ldkZMd6Yh7zxcEk90jJ3AQjAB26RzrodO7acEaRwSYEz2sQJrBRTM2dyN0S7QW6fAfXXpwnF0rfwsQ3y/J+DGGJbj1NzlgE+ZCkwfhuLyuQbQzg00NcEwjGXB6O X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: e8fb4d4f-cf03-4017-5f70-08db7df95c14 X-MS-Exchange-CrossTenant-AuthSource: AM5PR04MB3139.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jul 2023 08:16:55.1773 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: fYiU2CuJ2f2Y7j8jxBoUzWxcCj7sESj2wXquN1uAFbiVNtQhqU9w2HZdM1sVuSLJaLk/CAx8xPfx2GwTG0AAKA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR04MB9132 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770662143707860418?= X-GMAIL-MSGID: =?utf-8?q?1770662143707860418?= From: Wei Fang Once the XDP frames have been successfully transmitted through the ndo_xdp_xmit() interface, it's the driver responsibility to free the frames so that the page_pool can recycle the pages and reuse them. However, this action is not implemented in the fec driver. This leads to a user-visible problem that the console will print the following warning log. [ 157.568851] page_pool_release_retry() stalled pool shutdown 1389 inflight 60 sec [ 217.983446] page_pool_release_retry() stalled pool shutdown 1389 inflight 120 sec [ 278.399006] page_pool_release_retry() stalled pool shutdown 1389 inflight 181 sec [ 338.812885] page_pool_release_retry() stalled pool shutdown 1389 inflight 241 sec [ 399.226946] page_pool_release_retry() stalled pool shutdown 1389 inflight 302 sec Therefore, to solve this issue, we free XDP frames via xdp_return_frame() while cleaning the tx BD ring. Fixes: 6d6b39f180b8 ("net: fec: add initial XDP support") Signed-off-by: Wei Fang --- V2 change: No change. --- drivers/net/ethernet/freescale/fec.h | 15 ++- drivers/net/ethernet/freescale/fec_main.c | 148 +++++++++++++++------- 2 files changed, 115 insertions(+), 48 deletions(-) diff --git a/drivers/net/ethernet/freescale/fec.h b/drivers/net/ethernet/freescale/fec.h index 9939ccafb556..8c0226d061fe 100644 --- a/drivers/net/ethernet/freescale/fec.h +++ b/drivers/net/ethernet/freescale/fec.h @@ -544,10 +544,23 @@ enum { XDP_STATS_TOTAL, }; +enum fec_txbuf_type { + FEC_TXBUF_T_SKB, + FEC_TXBUF_T_XDP_NDO, +}; + +struct fec_tx_buffer { + union { + struct sk_buff *skb; + struct xdp_frame *xdp; + }; + enum fec_txbuf_type type; +}; + struct fec_enet_priv_tx_q { struct bufdesc_prop bd; unsigned char *tx_bounce[TX_RING_SIZE]; - struct sk_buff *tx_skbuff[TX_RING_SIZE]; + struct fec_tx_buffer tx_buf[TX_RING_SIZE]; unsigned short tx_stop_threshold; unsigned short tx_wake_threshold; diff --git a/drivers/net/ethernet/freescale/fec_main.c b/drivers/net/ethernet/freescale/fec_main.c index 9ce0319b33c3..940d3afe1d24 100644 --- a/drivers/net/ethernet/freescale/fec_main.c +++ b/drivers/net/ethernet/freescale/fec_main.c @@ -397,7 +397,7 @@ static void fec_dump(struct net_device *ndev) fec16_to_cpu(bdp->cbd_sc), fec32_to_cpu(bdp->cbd_bufaddr), fec16_to_cpu(bdp->cbd_datlen), - txq->tx_skbuff[index]); + txq->tx_buf[index].skb); bdp = fec_enet_get_nextdesc(bdp, &txq->bd); index++; } while (bdp != txq->bd.base); @@ -654,7 +654,7 @@ static int fec_enet_txq_submit_skb(struct fec_enet_priv_tx_q *txq, index = fec_enet_get_bd_index(last_bdp, &txq->bd); /* Save skb pointer */ - txq->tx_skbuff[index] = skb; + txq->tx_buf[index].skb = skb; /* Make sure the updates to rest of the descriptor are performed before * transferring ownership. @@ -672,9 +672,7 @@ static int fec_enet_txq_submit_skb(struct fec_enet_priv_tx_q *txq, skb_tx_timestamp(skb); - /* Make sure the update to bdp and tx_skbuff are performed before - * txq->bd.cur. - */ + /* Make sure the update to bdp is performed before txq->bd.cur. */ wmb(); txq->bd.cur = bdp; @@ -862,7 +860,7 @@ static int fec_enet_txq_submit_tso(struct fec_enet_priv_tx_q *txq, } /* Save skb pointer */ - txq->tx_skbuff[index] = skb; + txq->tx_buf[index].skb = skb; skb_tx_timestamp(skb); txq->bd.cur = bdp; @@ -952,16 +950,33 @@ static void fec_enet_bd_init(struct net_device *dev) for (i = 0; i < txq->bd.ring_size; i++) { /* Initialize the BD for every fragment in the page. */ bdp->cbd_sc = cpu_to_fec16(0); - if (bdp->cbd_bufaddr && - !IS_TSO_HEADER(txq, fec32_to_cpu(bdp->cbd_bufaddr))) - dma_unmap_single(&fep->pdev->dev, - fec32_to_cpu(bdp->cbd_bufaddr), - fec16_to_cpu(bdp->cbd_datlen), - DMA_TO_DEVICE); - if (txq->tx_skbuff[i]) { - dev_kfree_skb_any(txq->tx_skbuff[i]); - txq->tx_skbuff[i] = NULL; + if (txq->tx_buf[i].type == FEC_TXBUF_T_SKB) { + if (bdp->cbd_bufaddr && + !IS_TSO_HEADER(txq, fec32_to_cpu(bdp->cbd_bufaddr))) + dma_unmap_single(&fep->pdev->dev, + fec32_to_cpu(bdp->cbd_bufaddr), + fec16_to_cpu(bdp->cbd_datlen), + DMA_TO_DEVICE); + if (txq->tx_buf[i].skb) { + dev_kfree_skb_any(txq->tx_buf[i].skb); + txq->tx_buf[i].skb = NULL; + } + } else { + if (bdp->cbd_bufaddr) + dma_unmap_single(&fep->pdev->dev, + fec32_to_cpu(bdp->cbd_bufaddr), + fec16_to_cpu(bdp->cbd_datlen), + DMA_TO_DEVICE); + + if (txq->tx_buf[i].xdp) { + xdp_return_frame(txq->tx_buf[i].xdp); + txq->tx_buf[i].xdp = NULL; + } + + /* restore default tx buffer type: FEC_TXBUF_T_SKB */ + txq->tx_buf[i].type = FEC_TXBUF_T_SKB; } + bdp->cbd_bufaddr = cpu_to_fec32(0); bdp = fec_enet_get_nextdesc(bdp, &txq->bd); } @@ -1360,6 +1375,7 @@ static void fec_enet_tx_queue(struct net_device *ndev, u16 queue_id) { struct fec_enet_private *fep; + struct xdp_frame *xdpf; struct bufdesc *bdp; unsigned short status; struct sk_buff *skb; @@ -1387,16 +1403,31 @@ fec_enet_tx_queue(struct net_device *ndev, u16 queue_id) index = fec_enet_get_bd_index(bdp, &txq->bd); - skb = txq->tx_skbuff[index]; - txq->tx_skbuff[index] = NULL; - if (!IS_TSO_HEADER(txq, fec32_to_cpu(bdp->cbd_bufaddr))) - dma_unmap_single(&fep->pdev->dev, - fec32_to_cpu(bdp->cbd_bufaddr), - fec16_to_cpu(bdp->cbd_datlen), - DMA_TO_DEVICE); - bdp->cbd_bufaddr = cpu_to_fec32(0); - if (!skb) - goto skb_done; + if (txq->tx_buf[index].type == FEC_TXBUF_T_SKB) { + skb = txq->tx_buf[index].skb; + txq->tx_buf[index].skb = NULL; + if (bdp->cbd_bufaddr && + !IS_TSO_HEADER(txq, fec32_to_cpu(bdp->cbd_bufaddr))) + dma_unmap_single(&fep->pdev->dev, + fec32_to_cpu(bdp->cbd_bufaddr), + fec16_to_cpu(bdp->cbd_datlen), + DMA_TO_DEVICE); + bdp->cbd_bufaddr = cpu_to_fec32(0); + if (!skb) + goto tx_buf_done; + } else { + xdpf = txq->tx_buf[index].xdp; + if (bdp->cbd_bufaddr) + dma_unmap_single(&fep->pdev->dev, + fec32_to_cpu(bdp->cbd_bufaddr), + fec16_to_cpu(bdp->cbd_datlen), + DMA_TO_DEVICE); + bdp->cbd_bufaddr = cpu_to_fec32(0); + if (!xdpf) { + txq->tx_buf[index].type = FEC_TXBUF_T_SKB; + goto tx_buf_done; + } + } /* Check for errors. */ if (status & (BD_ENET_TX_HB | BD_ENET_TX_LC | @@ -1415,21 +1446,11 @@ fec_enet_tx_queue(struct net_device *ndev, u16 queue_id) ndev->stats.tx_carrier_errors++; } else { ndev->stats.tx_packets++; - ndev->stats.tx_bytes += skb->len; - } - /* NOTE: SKBTX_IN_PROGRESS being set does not imply it's we who - * are to time stamp the packet, so we still need to check time - * stamping enabled flag. - */ - if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS && - fep->hwts_tx_en) && - fep->bufdesc_ex) { - struct skb_shared_hwtstamps shhwtstamps; - struct bufdesc_ex *ebdp = (struct bufdesc_ex *)bdp; - - fec_enet_hwtstamp(fep, fec32_to_cpu(ebdp->ts), &shhwtstamps); - skb_tstamp_tx(skb, &shhwtstamps); + if (txq->tx_buf[index].type == FEC_TXBUF_T_SKB) + ndev->stats.tx_bytes += skb->len; + else + ndev->stats.tx_bytes += xdpf->len; } /* Deferred means some collisions occurred during transmit, @@ -1438,10 +1459,32 @@ fec_enet_tx_queue(struct net_device *ndev, u16 queue_id) if (status & BD_ENET_TX_DEF) ndev->stats.collisions++; - /* Free the sk buffer associated with this last transmit */ - dev_kfree_skb_any(skb); -skb_done: - /* Make sure the update to bdp and tx_skbuff are performed + if (txq->tx_buf[index].type == FEC_TXBUF_T_SKB) { + /* NOTE: SKBTX_IN_PROGRESS being set does not imply it's we who + * are to time stamp the packet, so we still need to check time + * stamping enabled flag. + */ + if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS && + fep->hwts_tx_en) && fep->bufdesc_ex) { + struct skb_shared_hwtstamps shhwtstamps; + struct bufdesc_ex *ebdp = (struct bufdesc_ex *)bdp; + + fec_enet_hwtstamp(fep, fec32_to_cpu(ebdp->ts), &shhwtstamps); + skb_tstamp_tx(skb, &shhwtstamps); + } + + /* Free the sk buffer associated with this last transmit */ + dev_kfree_skb_any(skb); + } else { + xdp_return_frame(xdpf); + + txq->tx_buf[index].xdp = NULL; + /* restore default tx buffer type: FEC_TXBUF_T_SKB */ + txq->tx_buf[index].type = FEC_TXBUF_T_SKB; + } + +tx_buf_done: + /* Make sure the update to bdp and tx_buf are performed * before dirty_tx */ wmb(); @@ -3249,9 +3292,19 @@ static void fec_enet_free_buffers(struct net_device *ndev) for (i = 0; i < txq->bd.ring_size; i++) { kfree(txq->tx_bounce[i]); txq->tx_bounce[i] = NULL; - skb = txq->tx_skbuff[i]; - txq->tx_skbuff[i] = NULL; - dev_kfree_skb(skb); + + if (txq->tx_buf[i].type == FEC_TXBUF_T_SKB) { + skb = txq->tx_buf[i].skb; + txq->tx_buf[i].skb = NULL; + dev_kfree_skb(skb); + } else { + if (txq->tx_buf[i].xdp) { + xdp_return_frame(txq->tx_buf[i].xdp); + txq->tx_buf[i].xdp = NULL; + } + + txq->tx_buf[i].type = FEC_TXBUF_T_SKB; + } } } } @@ -3817,7 +3870,8 @@ static int fec_enet_txq_xmit_frame(struct fec_enet_private *fep, ebdp->cbd_esc = cpu_to_fec32(estatus); } - txq->tx_skbuff[index] = NULL; + txq->tx_buf[index].type = FEC_TXBUF_T_XDP_NDO; + txq->tx_buf[index].xdp = frame; /* Make sure the updates to rest of the descriptor are performed before * transferring ownership.