From patchwork Thu Jul 6 08:10:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Fang X-Patchwork-Id: 116574 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f45:0:b0:3ea:f831:8777 with SMTP id v5csp2419262vqx; Thu, 6 Jul 2023 02:13:47 -0700 (PDT) X-Google-Smtp-Source: APBJJlF3+p4fT4BuGTr3/BmToNsaf0In2uTI5d1MTO1odkWa53D3y1reJwX8lM0q2LymrmYK33Og X-Received: by 2002:a05:6a20:1441:b0:12f:6500:84fb with SMTP id a1-20020a056a20144100b0012f650084fbmr1630857pzi.6.1688634826809; Thu, 06 Jul 2023 02:13:46 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1688634826; cv=pass; d=google.com; s=arc-20160816; b=Yb1vwQJFKYGnUQe7HfwudDuNqVfSotBqXhXaWD8yEYZpofpaEqloNF988Vy38Ljzal pMrScFnSggJbaiAKSLWnOYBtBbeCujToe6tiyfv7TJnjMz3/9iecZ1DgG+ufJRB+7D8b FbpAmckILM8JrUfEu5M6D1TUjpr4z2sdvccAqMnUWuI+CDls588jOGzORl4iXteqT9Yn 0Gha+K+SpAH/2hxZYvkfQGs/TZo1Kzj2SZQ+kBWQfBwDj37Y8gTj0YQ4xDb11uSLMUcS QOW32/vzvnHXAYFPPgBoiNDSJMubJEBXcJAL494Tg+HZZqWgCWSx26COOvDbp1igBNpg O4yw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:content-transfer-encoding :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=fj4R7Y5dbUrxgslYFEgNMVMvIZXmB1XTJV0y1X7ZxOQ=; fh=AE1PERU6bhr9O8eJNdZsAjdBxzfHJxzlkxEKIMP4fEw=; b=bFmanzzQ7I7Vqdits0ohPKMSdaiLDU1CmlgGmrm0UDZ1+88R67/Ky9GMp3/IMV4Aw1 wc72mZBsPvENZNGtyos8Du4LmAbfNUNno9uIv386ZFuDr4oMq/gYRotAhRd8esDCZ5o1 XyghL2Pli/noLyZQ0aNwB0saOwfzT1WbWc/zeKU0ITEKeByqIG0lY95bh9TgXuGWl7H7 PDUTM60PMHuJtqJ9/WD5efP3z++iy0gZq7yC0PA8klLkjF7hGEf6KmZooQB2vgf6Jjoc E+rOvzdQre0x0s2Jx4t6Dfz8DWybAV2Lhvx35k/+YHKX1otl301KABqrxNRtwqwhQ69P 5Q1Q== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@nxp.com header.s=selector2 header.b=S18NEYlt; arc=pass (i=1 spf=pass spfdomain=nxp.com dkim=pass dkdomain=nxp.com dmarc=pass fromdomain=nxp.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nxp.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l7-20020a63f307000000b0053f23ffad1csi1056668pgh.544.2023.07.06.02.13.32; Thu, 06 Jul 2023 02:13:46 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@nxp.com header.s=selector2 header.b=S18NEYlt; arc=pass (i=1 spf=pass spfdomain=nxp.com dkim=pass dkdomain=nxp.com dmarc=pass fromdomain=nxp.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nxp.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233757AbjGFIQv (ORCPT + 99 others); Thu, 6 Jul 2023 04:16:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233574AbjGFIQu (ORCPT ); Thu, 6 Jul 2023 04:16:50 -0400 Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01on2071.outbound.protection.outlook.com [40.107.15.71]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 41C3E1988; Thu, 6 Jul 2023 01:16:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JkpUu3JIDEEtnLRr9DVpehXTY1q5yIfyQra+AP3ag27HZoX1bgm8rsOriTTv7Qrv9w0H9kVxsXUeOfB5yT7gvR6gTUf/nwsLx47tM1seTKrQ9eAP09QYC9axIvk2c2Mh+w7TpiOcxh6zlQxWuxHtKgzvbKXWKzucBsp79XLC4VfQHtfdG0ZEKL9klt1nT1KvB0hzqLAxZZTlgJiYA8Hyjl1ai1d52FpfBFGm47HqDzoiZxOchr4yX3zvOZQzZ5JDOIAjsARN2BIOVqHqtQuQU+hviH/6QjQpqRjP2bM9o2Df9n/ql6s4mGwZDYvdQFwaq57bEf83CJSPMMktNfG/xA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fj4R7Y5dbUrxgslYFEgNMVMvIZXmB1XTJV0y1X7ZxOQ=; b=c3NRdMTRzfly9AA2rw8A9lR9CSQkYUAN+2HTcTZ8qseMYsPwdL918yXL8/BegI3020YF/dXUXZyG7pa8l3CZGZHskEWGboy2NuPPGDzgG9WDnaiOluIxlcurkx1DsrMQY+c1Qle5zllEcyriHiWwJM5rg+dbeyMR8riqMMBfyUPiAdEjf5qA9JcXeV9jZ760I1vB4Zy64xS3IUacHRZY9fxGQZUZ39RKegbFSCvLXzrwHh15bs8KBDRFHsWLDQKUrMjYjk3DxaumSLbcnOLa2jdLrWLiRwBYMc1h8peG2Dzyjtq76aA/wYr5qLEO2z5ckuT6cCCart+5nLt/zFmcyQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fj4R7Y5dbUrxgslYFEgNMVMvIZXmB1XTJV0y1X7ZxOQ=; b=S18NEYlt3zD1JhPRAam6z+9CmrmV6qkpAejqMvvuaxoZSd+h4beVmFp7sCI6x6KoxTvKV6jQOyyTsMp0qgZNBgHTeEiUWd4faQzfzlTZGyyZbEUoPYDXr0M7kmZGPBoWVdhFTTKTibxTRIvOEUo7h5MxbpiOfjR9IH6Iw2G1iIE= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; Received: from AM5PR04MB3139.eurprd04.prod.outlook.com (2603:10a6:206:8::20) by DU2PR04MB9132.eurprd04.prod.outlook.com (2603:10a6:10:2f7::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6565.17; Thu, 6 Jul 2023 08:16:44 +0000 Received: from AM5PR04MB3139.eurprd04.prod.outlook.com ([fe80::1edd:68cb:85d0:29e0]) by AM5PR04MB3139.eurprd04.prod.outlook.com ([fe80::1edd:68cb:85d0:29e0%7]) with mapi id 15.20.6544.024; Thu, 6 Jul 2023 08:16:44 +0000 From: wei.fang@nxp.com To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, ast@kernel.org, daniel@iogearbox.net, hawk@kernel.org, john.fastabend@gmail.com, shenwei.wang@nxp.com, xiaoning.wang@nxp.com, netdev@vger.kernel.org Cc: linux-imx@nxp.com, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH V2 net 1/4] net: fec: dynamically set the NETDEV_XDP_ACT_NDO_XMIT feature of XDP Date: Thu, 6 Jul 2023 16:10:09 +0800 Message-Id: <20230706081012.2278063-2-wei.fang@nxp.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230706081012.2278063-1-wei.fang@nxp.com> References: <20230706081012.2278063-1-wei.fang@nxp.com> X-ClientProxiedBy: SG2P153CA0018.APCP153.PROD.OUTLOOK.COM (2603:1096::28) To AM5PR04MB3139.eurprd04.prod.outlook.com (2603:10a6:206:8::20) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM5PR04MB3139:EE_|DU2PR04MB9132:EE_ X-MS-Office365-Filtering-Correlation-Id: 0c350074-a6a2-4163-7e23-08db7df9559b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: X3LyRV4Yek7A8Ydjrpj1VowhYWvBuLbu96s3OyVydnhqIs+Z8BbrnsTh86Mxggr/RusrkA9PpOLzDg+scTd70OQX8Ezo1Mwt+Rm7oeqW5hdPd8H3b90W7bLmRz/Ux6/1tjWbsYrg/dgaQQMwdYIg73iGL1fUSu6ml2/KA3EMnmjbO9Xk5sWh4K7JjNxVBaUxgQ0oCueTPzAqxE/E2Bd7GfgGRZucJjvt2UOlSH8wIJEAZtwx7bP34Ij30kcG+P/Q76OisuHkDPlMnUl+7BOnHe/ztc5F1K7aAwNrlV1pq1ytCe8cgzr5CsIfJm0AZo71spvN2NSGN8xSXz/S0260kUx3M6n5tb7qmPkpRUBkznHruzhPacvJ0el8PJ3fu5v86594ZfQ2BvxBmPuk7Ns3Oucop3Jyx7XV6OonS8x4MxnwMjPfXGflPsGm2VJ/ku69UgPIpYWRmPmM8rC0jrspAnqTUMbOS/+qnxbOsfIltklSKO8dyfysatiKBepZRMxYdJDnFUNzJV4COfrA03pDAxSHC3FqacpWKyyoYbnJNaG8m0PismnKGrWyt+LqietIXMPPDexTMMB7fGNX/ii466lrFMQTChyX/3w/eHx2Zsrj4yYzHp+prvGuBiJHutFuDIF3zFEQXSAKmhyzTEj7fQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM5PR04MB3139.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(366004)(136003)(396003)(376002)(39860400002)(346002)(451199021)(186003)(8676002)(8936002)(6506007)(26005)(2616005)(2906002)(66946007)(1076003)(5660300002)(7416002)(41300700001)(36756003)(921005)(6486002)(4326008)(316002)(66476007)(83380400001)(6666004)(66556008)(6512007)(478600001)(9686003)(52116002)(38350700002)(86362001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Sa0/J4gBsm2P3ie+43HQ6vSJkXNx5rCXlJs/y68bt44OIz6V7DRAEBga2UDSnVOhAqeopYCo7OiVaTJwcpU5/cvXzXUWvPCwi6+nkB+no/Qxuepwy3SpHXxh+7tb62c9E8DvRnbpyOaMZ+eOBvcjXeXEEQ/Oc0Mg3fSZ21L3ogrg7/8bFaPa3/beh/uF2O1PXE/udbpLjOFw7MFkAoQun2UK+XFxV2jMSub0WlH1G46wyCrp0b4ZN8Ck+je1TR6jOJ/qfYTo+HzgX6p22xIGxfMJZjeq6ctjTGm9xVns580CUkJeIuEd+ZU0pkI/+5mlBXPe+wxefzR4V71ZPMPIqHS0889NQ+2X1/H1tJ/sK5T/jo8f28YmiLyyfTA9T8iO+edB8G4BhWCAP/118W2FMrwRqaBgWSItGuzvDIbJSjkdUwjxvm7HaLeN5C6EOqmkzt2bP4ZAYy8Zz+hq1BHIETiSd5Ux2LErQ2KsNNPMQMCvkHjT+sO29kYSa9uja9UrIYaJ6NVwe81VZLoytCGpE72od6tAxVTy/9IXWKmCSTBQWKAzW/h9YkvXL5N/MdQ4ADVo3tiONQYZRWnwSv5x0c32B720WFAmtZnc3CmVWdSddTJefqRWVZ6l000XorflluPjXRb7Okfdm7zoYOX8YslSTQy9Iv4ZTgsX2l0KqawuHlK0APwBuXfB+CzA00L07nuNBZKrSJEFP/YR21WwKvRhwkavGI0ruN4GTyG4XvxS3PeDVfpJRQYGCGvKpVJaj0cJ0369ZDCK7pu2cz2COCVGyIYFlJku6OIIvmtKUSK0zcl9FjxxvEGsHp3xSKCznEbDRuCGp341G75Yb3fDykFveWxCtsKaO2ROy9zaIHIu9cdyEuv8bPxEUZSyWSDIwuXOeHKfsjZ+Qv5aF/nUQn64MR57bfZj1k2spko21091kqKOaSIeU4Tm5klWDlVXpnu3rQJGB5Nt1ewOouPEtgBo6kLQbTrd3uxwJcSotll51R1vb5wuP5tmLQQg6d8RBmhfLBAyB2E5L3wKbGhrACM/UwGqavo0/2U4YmHFS/3OTxySXjAVDIYyH3cxs4pFtypvoYt9bez6cMPCparuSffkB+0M5Z4NovelX3GBCvYCJ+vJ5WQHBxKw5GApBo1XDAEuNEjwjoKhXprisfBpr+eTm0Mim9E/XGwXH4fvJdcp4iJqD7OQcYIOOwlnjsrSaQe42UnzNwsFRbXho+QbBxv7+PEd4y0/r+6+aH/oIz0iqeEvnfqcMkGDtgNG34OChECHNdicBMI0ZiS6kwn/SiTAzLLxilrZImjxNreJre8YsM7Rtzj/zlFU86Ncpo/idCIl9331T6QN/wh0I1Yp7JyGuj6ARtlMWV3eT1+T6EHmFfHmH/rTZ99Fk9fhP4REy7QvTN2bQJE66TsG+qKASnFtzG/ir0dyj9SUH/u+RkcNYoS2eIOYWIKvjVzLWdC/HvKeaQCBGTL2Qw0FfjzYXuK6vDJnlh0clbREjyZE2I68uX1VYm3tbimkaxj8YhLQ8Cwifw6EomkX8AF48mc9QlKqb5qXQxvkp4VNO43Utu+EmCd9wH7kJ1M0CC5iHV+w X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0c350074-a6a2-4163-7e23-08db7df9559b X-MS-Exchange-CrossTenant-AuthSource: AM5PR04MB3139.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jul 2023 08:16:44.1693 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: FHm5gvQexlFtYOjFqmbFTQn3epXxD7rBaIg38h4EvV/5pkSOAThQiljz+5UzjPYsXe1o/4MRXvna5kHTP2IC8g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR04MB9132 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770661952488228669?= X-GMAIL-MSGID: =?utf-8?q?1770661952488228669?= From: Wei Fang When a XDP program is installed or uninstalled, fec_restart() will be invoked to reset MAC and buffer descriptor rings. It's reasonable not to transmit any packet during the process of reset. However, the NETDEV_XDP_ACT_NDO_XMIT bit of xdp_features is enabled by default, that is to say, it's possible that the fec_enet_xdp_xmit() will be invoked even if the process of reset is not finished. In this case, the redirected XDP frames might be dropped and available transmit BDs may be incorrectly deemed insufficient. So this patch disable the NETDEV_XDP_ACT_NDO_XMIT feature by default and dynamically configure this feature when the bpf program is installed or uninstalled. Fixes: e4ac7cc6e5a4 ("net: fec: turn on XDP features") Signed-off-by: Wei Fang --- V2 change: No change. --- drivers/net/ethernet/freescale/fec_main.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/freescale/fec_main.c b/drivers/net/ethernet/freescale/fec_main.c index 8fbe47703d47..9ce0319b33c3 100644 --- a/drivers/net/ethernet/freescale/fec_main.c +++ b/drivers/net/ethernet/freescale/fec_main.c @@ -3732,12 +3732,18 @@ static int fec_enet_bpf(struct net_device *dev, struct netdev_bpf *bpf) if (fep->quirks & FEC_QUIRK_SWAP_FRAME) return -EOPNOTSUPP; + if (!bpf->prog) + xdp_features_clear_redirect_target(dev); + if (is_run) { napi_disable(&fep->napi); netif_tx_disable(dev); } old_prog = xchg(&fep->xdp_prog, bpf->prog); + if (old_prog) + bpf_prog_put(old_prog); + fec_restart(dev); if (is_run) { @@ -3745,8 +3751,8 @@ static int fec_enet_bpf(struct net_device *dev, struct netdev_bpf *bpf) netif_tx_start_all_queues(dev); } - if (old_prog) - bpf_prog_put(old_prog); + if (bpf->prog) + xdp_features_set_redirect_target(dev, false); return 0; @@ -4016,8 +4022,7 @@ static int fec_enet_init(struct net_device *ndev) if (!(fep->quirks & FEC_QUIRK_SWAP_FRAME)) ndev->xdp_features = NETDEV_XDP_ACT_BASIC | - NETDEV_XDP_ACT_REDIRECT | - NETDEV_XDP_ACT_NDO_XMIT; + NETDEV_XDP_ACT_REDIRECT; fec_restart(ndev); From patchwork Thu Jul 6 08:10:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Fang X-Patchwork-Id: 116575 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f45:0:b0:3ea:f831:8777 with SMTP id v5csp2420623vqx; Thu, 6 Jul 2023 02:16:49 -0700 (PDT) X-Google-Smtp-Source: APBJJlG6MPWlUOVP9Cc+MoJSfRaQ9aYtvhyfbCgYHweuWhF2QeNdKiyy6hJKM0oPFSeD0QM9s84u X-Received: by 2002:a17:902:daca:b0:1b8:902c:f8e8 with SMTP id q10-20020a170902daca00b001b8902cf8e8mr1030870plx.60.1688635009263; Thu, 06 Jul 2023 02:16:49 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1688635009; cv=pass; d=google.com; s=arc-20160816; b=qqLWmdeMNEpbvX3KQ4Nmlm2OJzkaXV9hNivobo02djNwjc07v8YCaNRXBPXVveLjY4 K8cA1iht+VCh/h0wmB6zPw2eUgg0j+7RYvMOkqODu7+/Xy7ONavM3wxv8Nfi92XV3RKB ItPb7GDeijGPw8C6c6o6K+Znpc0JZpqQ55UqxG3PV7tlR6wEZrwKUcdZm8xdVbgtUr8G gQ6zQcox4p2AR3R3v9Q1aWJScQ/hUZHfyazGKrPtQKdvz3WsuyTSlJRiOKvDrhwmpIv4 IYf31QRu0HdDFB7ilJ9S0uWd3Hw0NKUZbaonr4FGLEgCNkL21lN0SGm77PukBN+kyeYd T8tw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:content-transfer-encoding :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ALHnrNUVR0r6vkOh6Qg2LeUMUThQ/qrbFVs+hSXhMBQ=; fh=AE1PERU6bhr9O8eJNdZsAjdBxzfHJxzlkxEKIMP4fEw=; b=PaW9GaXh4p20jMYab8Oa2HDociaROKFwRmnF4fFPGLFblijPhl9eXmQW9G/gEQBBkU BMHSAglTMe+2ihQQoFnTKnJlPLpH8eBpjA+XgTOUtec1n40VejtVyRXfz34bYwpFaFXz Ox1GgXVz2lYtHeX6cZvq+p1buKHBnUApUDuuThJv8H4GX6n0xBQaMMEfvgYOFfz7a9x8 5uX3kUBXSHGhEXMQfO1poJShJCY+lbYxMncVsyR3tqbbo8iYkGCOjVLEX3vtLC/mzAz9 KLzwDX/czbEh8VGEU3xjOxPqenPdHuJZWs21qjYSunWWokVmY3uHQhLJF8tAMQn5sVKl owtg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@nxp.com header.s=selector2 header.b="jPKl58/j"; arc=pass (i=1 spf=pass spfdomain=nxp.com dkim=pass dkdomain=nxp.com dmarc=pass fromdomain=nxp.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nxp.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b14-20020a170902650e00b001b872e20955si949484plk.30.2023.07.06.02.16.33; Thu, 06 Jul 2023 02:16:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@nxp.com header.s=selector2 header.b="jPKl58/j"; arc=pass (i=1 spf=pass spfdomain=nxp.com dkim=pass dkdomain=nxp.com dmarc=pass fromdomain=nxp.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nxp.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233921AbjGFIRD (ORCPT + 99 others); Thu, 6 Jul 2023 04:17:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41104 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233912AbjGFIQ7 (ORCPT ); Thu, 6 Jul 2023 04:16:59 -0400 Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01on2075.outbound.protection.outlook.com [40.107.15.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9A6AF1BEB; Thu, 6 Jul 2023 01:16:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XdduhFNOoaYdYL4RnnsrUOR2zM2E8hPLvQxc248HsFSj0t5oWx236sGZDhjABALYQThGuU9ZjsXTNBLBqUNjhRlMj1UFO/04m6tElUHlxPQWQXAIfl5etKo+GWhjQDeFn+bTgu+DYL69e+4XDS4kaaplCccafy+YArKWfJCSrRK45Nf4XxtRrPhztWcIc6nAchJQJhqQQd+luGhAEDg5pReqG+xx34NkzP5DUjh1/gZ588DsxSOz/jg0gt9+R3Ckono5wZVKCn6iv48w7B9c9zP/kE1+WlH2GYHyZ4i3RJy7aG/mkkCJAxJZQpEHV9XkFgqyyatPqx7K6xum6vsVJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ALHnrNUVR0r6vkOh6Qg2LeUMUThQ/qrbFVs+hSXhMBQ=; b=A3k/GQETsIXK7WE806RarBYgCCAfx7EpXrwqudlUYvvYfMdI9JfBgatdVnoypoBzPIOlDy/9dSbIRDwX3QPF32NPEI8IRhwYeAuggwPkvCZaQWCDCD3CDOxCOhxbpG8mWegxozTLT7rnxiVcucmQfsLDgeYiQ1lbgRMl3akk0MTLRcPip7/lqplvXJUcMSo87pF+RrKPSC+pgKQoUCG8vG2E0nirqEL6WBjJq5LwPEzG4XQT5qjL5EJZltK8W0rtS1h/2OGx+wB6zsEDqPIpC9GyL8iBmtXOhhHhBl+L6C8zByp7kYG62sQn1vYIOO5A82XYxNWc/hjr+7MqA6yIJQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ALHnrNUVR0r6vkOh6Qg2LeUMUThQ/qrbFVs+hSXhMBQ=; b=jPKl58/j0haALTYorvkdBD+JfTs6WBrOW8vx18WeONvFLSHDxnv3wjqp7NJe1kAnxx+SKJE5FuhJAOOmEfBNA7wKparIVo/2ww0fLnkRHD5/asu6Ni9GkkpbIpDOCHMJiwuUF947cqwozX0+Y+KLgcvKYYyvaduNFod4ItEiwRI= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; Received: from AM5PR04MB3139.eurprd04.prod.outlook.com (2603:10a6:206:8::20) by DU2PR04MB9132.eurprd04.prod.outlook.com (2603:10a6:10:2f7::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6565.17; Thu, 6 Jul 2023 08:16:55 +0000 Received: from AM5PR04MB3139.eurprd04.prod.outlook.com ([fe80::1edd:68cb:85d0:29e0]) by AM5PR04MB3139.eurprd04.prod.outlook.com ([fe80::1edd:68cb:85d0:29e0%7]) with mapi id 15.20.6544.024; Thu, 6 Jul 2023 08:16:55 +0000 From: wei.fang@nxp.com To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, ast@kernel.org, daniel@iogearbox.net, hawk@kernel.org, john.fastabend@gmail.com, shenwei.wang@nxp.com, xiaoning.wang@nxp.com, netdev@vger.kernel.org Cc: linux-imx@nxp.com, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH V2 net 2/4] net: fec: recycle pages for transmitted XDP frames Date: Thu, 6 Jul 2023 16:10:10 +0800 Message-Id: <20230706081012.2278063-3-wei.fang@nxp.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230706081012.2278063-1-wei.fang@nxp.com> References: <20230706081012.2278063-1-wei.fang@nxp.com> X-ClientProxiedBy: SG2P153CA0018.APCP153.PROD.OUTLOOK.COM (2603:1096::28) To AM5PR04MB3139.eurprd04.prod.outlook.com (2603:10a6:206:8::20) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM5PR04MB3139:EE_|DU2PR04MB9132:EE_ X-MS-Office365-Filtering-Correlation-Id: e8fb4d4f-cf03-4017-5f70-08db7df95c14 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4OlyeJtU/PHVKS0waF9+3lF29WXkUH25vH22nhSDc47v5q7A3OX7Fa3IwaCNvZhukxyRdk0ku0hddbLv8boIzYACmxp9E40Zi44XA7v8c+m69nr+0bMW4Q3rJZmOVg/pv3yNet5DmVn8PTfbcW0ld4cadcgTPdssDDWnFT5AjZf1Ig5n9PRmsoRu/TCaxWHBIN+nBxwF4p6PV/UoLCd/CZWNm/2+wgcHz71vrcc0I/KUkJ3ENuK+CsXRc6Eu7LlmD+bVaTZGeR6VfF5jHgeWZmTdiWgE3uMc5V19dzv1W6KAleTz+xQTUk0obZe36cipz83/kZvnT1exWAPP3hPP1jQAEiNzG4oRfYqGGWh51yfz9uUDWmmnv08TGJz+99ESL5G5a/hCgWTxIdjnuezReqrPVuOcxmcm11FHgkCXb/ND91R52ngGtRb6Ai3Klm1+Fme4r9YfyAK1zSXCszaGd18IYZZWDYpqXrYaViZxMIwKtwpmNhMLBQZGBfBTmbUs6SThuHEey+BB1eI+54Vp80ASH1bh+RStHWIRkoT4Err5hnLacKzgajDibicftqPnOTtxxybQs3xR78RL8hs6Gyg9lLhv0XBzMZT5yIodhnPJvCjpBHopjAnH4y2C2oHLWXbFegdRDqy2bxAI3fr/ZQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM5PR04MB3139.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(366004)(136003)(396003)(376002)(39860400002)(346002)(451199021)(186003)(8676002)(8936002)(6506007)(26005)(2616005)(2906002)(66946007)(1076003)(5660300002)(7416002)(41300700001)(36756003)(921005)(6486002)(4326008)(316002)(66476007)(83380400001)(6666004)(66556008)(6512007)(478600001)(9686003)(52116002)(38350700002)(86362001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 0AC6w4k/JnzsKCYyEG4m16U5620k0zXSha2LaQi2CUTzxwH74ABWTyfhQTK7qiL4/AAhuDQmxiWmJuuxlIZHGjAhtOw9kWl1CF5jgA4+AK4i6MLkbw6prwCH/P6o/VG+5xwBg0ltH/HbbzC/Fz2bMbqD714G+NzuczqjSIVj1R3FeqFLZYwpcFEMbl5nGXsm5cMy9Ky2aVaTO7QmtQvK86Nq+0J0jiJxd3aIl+95Txw+hGf1YdLQzRZ8gFEoMg1cXwIlYVmxj7YvFwZIM/GHbLc2hD5XrXNPeYDsa63qW+CF5rjDOE5VPDrjnDfqrAnIfiUpmKidAQdN7LZXVYkTVB+5YnjCpk4aHIAsPSurwJBXH4ndkkKC8YExDV1XHreNXXfNHKkltZRklZfvNSSWm3gk9j0q+DrlDQAITdZQda/XB3x47rMkKEMmhQ4RVrZg+7lX0nKWHfxzSUw/81KFc9x6JExSFoS2eIScVIgIMF9JUJhYnusIcwtGN1ECuFq9RXIhfqnQ6gFmBqM1we8m4OJ5r5HDchMwWwgPCllu9GcmHo62rIM+G8DIZo45i0E3dPYsQbajE/VWmPwcbbC+2B4ZzitA/aI172+4AqDkJ6TrZiYnaPaPHsMeigmHao+NQS5QC/gKMBo/S1onE/v+USuLVo6WQ7Qe8I5Ks8xCL3k6p45dL69q8kZ6Dk56Ds8PQtbf4XTut+JPaPsTX1iGXBS0l1Y2cvj60qeswKK9PC48iyD8eNvR/t/hJCZzkyHDB3vC2GtoOYcZotj8d1XnKoHXBmnSHBaiSrg9z7Q9TvBxn85VPmZhOb/EvE9RJRa3ud4oKssOKm8u/0J6Sh9G8OIE09Xj563Ny5s2U+VUjg26AP3rYV+Ptb7V4Tkzs+Mza3OM4Xq7BhDMvZYfwSFByH4uXHhVBadYaHDBlUK56urPF1wKNYJn5jtu3cZh6uE04/6wf/b7NkeqQnu59Ms5AaATB9NwnpZNhxUJq5UM1AKyVPi0WmK0kw8yNwG0rr4b2KjWgOnDz18yqnBM5pOEpHGXhPt15iOajv6Pbxn1xmki8B2KYl6xDC69tqvoI1ss+rvsWJAL9XTmeyuayvSAy9tSbNeVILuhCVdyPf5E/6FdFP9/7LUjp/SZYcZMWN1CQ+ZzLFEDJ7mpzm9Y0QztNrXvlU/3oZmKRg3wuwb4mVx3BYcmwcIXogm+ngvEnUg/4vc1fxaLscAbB2cAR5jP4J2Yi385llEbWq6PBV1XiuCQuhv+x+7sefyireRC6KT9BUC7UDn5oe0hVMmPCVw1H+u/YXQVR/QmlkiT2XdVXVllD5WR9KVWHjlzPYB+nDbYBO027SzV9KXfGLGVLEF8W6e2dM8KzmhEzuAsymfPXAuWkPF4LRYOrhPxfccP8m3YyE9kSKV2GOwV1LvNc5qJjxAYhJLIB8N8ucEsRt4XfJfTDr20riIxBsnq2ns2hAe51+xs/ldkZMd6Yh7zxcEk90jJ3AQjAB26RzrodO7acEaRwSYEz2sQJrBRTM2dyN0S7QW6fAfXXpwnF0rfwsQ3y/J+DGGJbj1NzlgE+ZCkwfhuLyuQbQzg00NcEwjGXB6O X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: e8fb4d4f-cf03-4017-5f70-08db7df95c14 X-MS-Exchange-CrossTenant-AuthSource: AM5PR04MB3139.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jul 2023 08:16:55.1773 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: fYiU2CuJ2f2Y7j8jxBoUzWxcCj7sESj2wXquN1uAFbiVNtQhqU9w2HZdM1sVuSLJaLk/CAx8xPfx2GwTG0AAKA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR04MB9132 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770662143707860418?= X-GMAIL-MSGID: =?utf-8?q?1770662143707860418?= From: Wei Fang Once the XDP frames have been successfully transmitted through the ndo_xdp_xmit() interface, it's the driver responsibility to free the frames so that the page_pool can recycle the pages and reuse them. However, this action is not implemented in the fec driver. This leads to a user-visible problem that the console will print the following warning log. [ 157.568851] page_pool_release_retry() stalled pool shutdown 1389 inflight 60 sec [ 217.983446] page_pool_release_retry() stalled pool shutdown 1389 inflight 120 sec [ 278.399006] page_pool_release_retry() stalled pool shutdown 1389 inflight 181 sec [ 338.812885] page_pool_release_retry() stalled pool shutdown 1389 inflight 241 sec [ 399.226946] page_pool_release_retry() stalled pool shutdown 1389 inflight 302 sec Therefore, to solve this issue, we free XDP frames via xdp_return_frame() while cleaning the tx BD ring. Fixes: 6d6b39f180b8 ("net: fec: add initial XDP support") Signed-off-by: Wei Fang --- V2 change: No change. --- drivers/net/ethernet/freescale/fec.h | 15 ++- drivers/net/ethernet/freescale/fec_main.c | 148 +++++++++++++++------- 2 files changed, 115 insertions(+), 48 deletions(-) diff --git a/drivers/net/ethernet/freescale/fec.h b/drivers/net/ethernet/freescale/fec.h index 9939ccafb556..8c0226d061fe 100644 --- a/drivers/net/ethernet/freescale/fec.h +++ b/drivers/net/ethernet/freescale/fec.h @@ -544,10 +544,23 @@ enum { XDP_STATS_TOTAL, }; +enum fec_txbuf_type { + FEC_TXBUF_T_SKB, + FEC_TXBUF_T_XDP_NDO, +}; + +struct fec_tx_buffer { + union { + struct sk_buff *skb; + struct xdp_frame *xdp; + }; + enum fec_txbuf_type type; +}; + struct fec_enet_priv_tx_q { struct bufdesc_prop bd; unsigned char *tx_bounce[TX_RING_SIZE]; - struct sk_buff *tx_skbuff[TX_RING_SIZE]; + struct fec_tx_buffer tx_buf[TX_RING_SIZE]; unsigned short tx_stop_threshold; unsigned short tx_wake_threshold; diff --git a/drivers/net/ethernet/freescale/fec_main.c b/drivers/net/ethernet/freescale/fec_main.c index 9ce0319b33c3..940d3afe1d24 100644 --- a/drivers/net/ethernet/freescale/fec_main.c +++ b/drivers/net/ethernet/freescale/fec_main.c @@ -397,7 +397,7 @@ static void fec_dump(struct net_device *ndev) fec16_to_cpu(bdp->cbd_sc), fec32_to_cpu(bdp->cbd_bufaddr), fec16_to_cpu(bdp->cbd_datlen), - txq->tx_skbuff[index]); + txq->tx_buf[index].skb); bdp = fec_enet_get_nextdesc(bdp, &txq->bd); index++; } while (bdp != txq->bd.base); @@ -654,7 +654,7 @@ static int fec_enet_txq_submit_skb(struct fec_enet_priv_tx_q *txq, index = fec_enet_get_bd_index(last_bdp, &txq->bd); /* Save skb pointer */ - txq->tx_skbuff[index] = skb; + txq->tx_buf[index].skb = skb; /* Make sure the updates to rest of the descriptor are performed before * transferring ownership. @@ -672,9 +672,7 @@ static int fec_enet_txq_submit_skb(struct fec_enet_priv_tx_q *txq, skb_tx_timestamp(skb); - /* Make sure the update to bdp and tx_skbuff are performed before - * txq->bd.cur. - */ + /* Make sure the update to bdp is performed before txq->bd.cur. */ wmb(); txq->bd.cur = bdp; @@ -862,7 +860,7 @@ static int fec_enet_txq_submit_tso(struct fec_enet_priv_tx_q *txq, } /* Save skb pointer */ - txq->tx_skbuff[index] = skb; + txq->tx_buf[index].skb = skb; skb_tx_timestamp(skb); txq->bd.cur = bdp; @@ -952,16 +950,33 @@ static void fec_enet_bd_init(struct net_device *dev) for (i = 0; i < txq->bd.ring_size; i++) { /* Initialize the BD for every fragment in the page. */ bdp->cbd_sc = cpu_to_fec16(0); - if (bdp->cbd_bufaddr && - !IS_TSO_HEADER(txq, fec32_to_cpu(bdp->cbd_bufaddr))) - dma_unmap_single(&fep->pdev->dev, - fec32_to_cpu(bdp->cbd_bufaddr), - fec16_to_cpu(bdp->cbd_datlen), - DMA_TO_DEVICE); - if (txq->tx_skbuff[i]) { - dev_kfree_skb_any(txq->tx_skbuff[i]); - txq->tx_skbuff[i] = NULL; + if (txq->tx_buf[i].type == FEC_TXBUF_T_SKB) { + if (bdp->cbd_bufaddr && + !IS_TSO_HEADER(txq, fec32_to_cpu(bdp->cbd_bufaddr))) + dma_unmap_single(&fep->pdev->dev, + fec32_to_cpu(bdp->cbd_bufaddr), + fec16_to_cpu(bdp->cbd_datlen), + DMA_TO_DEVICE); + if (txq->tx_buf[i].skb) { + dev_kfree_skb_any(txq->tx_buf[i].skb); + txq->tx_buf[i].skb = NULL; + } + } else { + if (bdp->cbd_bufaddr) + dma_unmap_single(&fep->pdev->dev, + fec32_to_cpu(bdp->cbd_bufaddr), + fec16_to_cpu(bdp->cbd_datlen), + DMA_TO_DEVICE); + + if (txq->tx_buf[i].xdp) { + xdp_return_frame(txq->tx_buf[i].xdp); + txq->tx_buf[i].xdp = NULL; + } + + /* restore default tx buffer type: FEC_TXBUF_T_SKB */ + txq->tx_buf[i].type = FEC_TXBUF_T_SKB; } + bdp->cbd_bufaddr = cpu_to_fec32(0); bdp = fec_enet_get_nextdesc(bdp, &txq->bd); } @@ -1360,6 +1375,7 @@ static void fec_enet_tx_queue(struct net_device *ndev, u16 queue_id) { struct fec_enet_private *fep; + struct xdp_frame *xdpf; struct bufdesc *bdp; unsigned short status; struct sk_buff *skb; @@ -1387,16 +1403,31 @@ fec_enet_tx_queue(struct net_device *ndev, u16 queue_id) index = fec_enet_get_bd_index(bdp, &txq->bd); - skb = txq->tx_skbuff[index]; - txq->tx_skbuff[index] = NULL; - if (!IS_TSO_HEADER(txq, fec32_to_cpu(bdp->cbd_bufaddr))) - dma_unmap_single(&fep->pdev->dev, - fec32_to_cpu(bdp->cbd_bufaddr), - fec16_to_cpu(bdp->cbd_datlen), - DMA_TO_DEVICE); - bdp->cbd_bufaddr = cpu_to_fec32(0); - if (!skb) - goto skb_done; + if (txq->tx_buf[index].type == FEC_TXBUF_T_SKB) { + skb = txq->tx_buf[index].skb; + txq->tx_buf[index].skb = NULL; + if (bdp->cbd_bufaddr && + !IS_TSO_HEADER(txq, fec32_to_cpu(bdp->cbd_bufaddr))) + dma_unmap_single(&fep->pdev->dev, + fec32_to_cpu(bdp->cbd_bufaddr), + fec16_to_cpu(bdp->cbd_datlen), + DMA_TO_DEVICE); + bdp->cbd_bufaddr = cpu_to_fec32(0); + if (!skb) + goto tx_buf_done; + } else { + xdpf = txq->tx_buf[index].xdp; + if (bdp->cbd_bufaddr) + dma_unmap_single(&fep->pdev->dev, + fec32_to_cpu(bdp->cbd_bufaddr), + fec16_to_cpu(bdp->cbd_datlen), + DMA_TO_DEVICE); + bdp->cbd_bufaddr = cpu_to_fec32(0); + if (!xdpf) { + txq->tx_buf[index].type = FEC_TXBUF_T_SKB; + goto tx_buf_done; + } + } /* Check for errors. */ if (status & (BD_ENET_TX_HB | BD_ENET_TX_LC | @@ -1415,21 +1446,11 @@ fec_enet_tx_queue(struct net_device *ndev, u16 queue_id) ndev->stats.tx_carrier_errors++; } else { ndev->stats.tx_packets++; - ndev->stats.tx_bytes += skb->len; - } - /* NOTE: SKBTX_IN_PROGRESS being set does not imply it's we who - * are to time stamp the packet, so we still need to check time - * stamping enabled flag. - */ - if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS && - fep->hwts_tx_en) && - fep->bufdesc_ex) { - struct skb_shared_hwtstamps shhwtstamps; - struct bufdesc_ex *ebdp = (struct bufdesc_ex *)bdp; - - fec_enet_hwtstamp(fep, fec32_to_cpu(ebdp->ts), &shhwtstamps); - skb_tstamp_tx(skb, &shhwtstamps); + if (txq->tx_buf[index].type == FEC_TXBUF_T_SKB) + ndev->stats.tx_bytes += skb->len; + else + ndev->stats.tx_bytes += xdpf->len; } /* Deferred means some collisions occurred during transmit, @@ -1438,10 +1459,32 @@ fec_enet_tx_queue(struct net_device *ndev, u16 queue_id) if (status & BD_ENET_TX_DEF) ndev->stats.collisions++; - /* Free the sk buffer associated with this last transmit */ - dev_kfree_skb_any(skb); -skb_done: - /* Make sure the update to bdp and tx_skbuff are performed + if (txq->tx_buf[index].type == FEC_TXBUF_T_SKB) { + /* NOTE: SKBTX_IN_PROGRESS being set does not imply it's we who + * are to time stamp the packet, so we still need to check time + * stamping enabled flag. + */ + if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS && + fep->hwts_tx_en) && fep->bufdesc_ex) { + struct skb_shared_hwtstamps shhwtstamps; + struct bufdesc_ex *ebdp = (struct bufdesc_ex *)bdp; + + fec_enet_hwtstamp(fep, fec32_to_cpu(ebdp->ts), &shhwtstamps); + skb_tstamp_tx(skb, &shhwtstamps); + } + + /* Free the sk buffer associated with this last transmit */ + dev_kfree_skb_any(skb); + } else { + xdp_return_frame(xdpf); + + txq->tx_buf[index].xdp = NULL; + /* restore default tx buffer type: FEC_TXBUF_T_SKB */ + txq->tx_buf[index].type = FEC_TXBUF_T_SKB; + } + +tx_buf_done: + /* Make sure the update to bdp and tx_buf are performed * before dirty_tx */ wmb(); @@ -3249,9 +3292,19 @@ static void fec_enet_free_buffers(struct net_device *ndev) for (i = 0; i < txq->bd.ring_size; i++) { kfree(txq->tx_bounce[i]); txq->tx_bounce[i] = NULL; - skb = txq->tx_skbuff[i]; - txq->tx_skbuff[i] = NULL; - dev_kfree_skb(skb); + + if (txq->tx_buf[i].type == FEC_TXBUF_T_SKB) { + skb = txq->tx_buf[i].skb; + txq->tx_buf[i].skb = NULL; + dev_kfree_skb(skb); + } else { + if (txq->tx_buf[i].xdp) { + xdp_return_frame(txq->tx_buf[i].xdp); + txq->tx_buf[i].xdp = NULL; + } + + txq->tx_buf[i].type = FEC_TXBUF_T_SKB; + } } } } @@ -3817,7 +3870,8 @@ static int fec_enet_txq_xmit_frame(struct fec_enet_private *fep, ebdp->cbd_esc = cpu_to_fec32(estatus); } - txq->tx_skbuff[index] = NULL; + txq->tx_buf[index].type = FEC_TXBUF_T_XDP_NDO; + txq->tx_buf[index].xdp = frame; /* Make sure the updates to rest of the descriptor are performed before * transferring ownership. From patchwork Thu Jul 6 08:10:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Fang X-Patchwork-Id: 116580 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f45:0:b0:3ea:f831:8777 with SMTP id v5csp2423144vqx; Thu, 6 Jul 2023 02:22:41 -0700 (PDT) X-Google-Smtp-Source: APBJJlG7RC9QLmsFY+3YG3don/ux1KKpN/9g2TavaZl+na7CpiAtYyH07rsiW57r7y2J0bK8b9GE X-Received: by 2002:a17:90a:4a0e:b0:263:41fc:95f4 with SMTP id e14-20020a17090a4a0e00b0026341fc95f4mr717428pjh.47.1688635361188; Thu, 06 Jul 2023 02:22:41 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1688635361; cv=pass; d=google.com; s=arc-20160816; b=IAjQAGQzLP5pEaXRkcLyKcN7Egae8ubkD3ts85T71sIZbzzoUU4/xgBLygD+HLZuL2 ryyczJJSJcaEGE4ybVASN0x7RIA5cDSCKSOwn4rGj1GguXpv79CxgtfuNbGLfQXUIa2M GUFhBUQzjQNHiwYnszCn5rxxYKxbLnqLeODw/yfR3jI+uNDAEmEZFBqgicb5fWqGQwZV xaS8mg/tTOo8W8yuCpMacXn7eAONufabDobJAFkfSIrbrIpLgi9iP99xKMyqBjjul2fp 2DwO/K/6kdPdIjnX+BnZz5M5tb62vi6AvULGEU+Ps0eyP9tuICrCMlNL9dc7zAUMjGt4 J0Qw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:content-transfer-encoding :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Zs4qsLvA4I1EDVqaUbosKk+vfTqETBtDOnTvr6bjeeg=; fh=AE1PERU6bhr9O8eJNdZsAjdBxzfHJxzlkxEKIMP4fEw=; b=h+IgM5Pr3U8kgyHepo6kC8vUDJct2fLp/C0VxLf3VIPqJa2eOHf6KG2EYSLq0ML1cP TGZKR0KcjyEbeaFFIe9pusGUnAzzk8yf0P6vN+UdumOWyhy8Oi+4NFEfB01ALXppW80B QIG1zdlfKWi0RSQy6TuET9fLQmSWeLm03tXIXQoJb9o2ftMHTONyZ0J+tuEYLDhTWI9X cW+UEc/ExLDCUUFPM7A9wNNFST0sizGnI+xFM30n2VJrIqRfsekEgJdCEwWCdoKTdpmU wRGjJNy/R3sBii3LdqnyzGu9PkWo3NTz6HhVfsRC1cXLxA5n+0Odarpcyq/3WQWkpCsL 7acw== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@nxp.com header.s=selector2 header.b=BMcp4QvM; arc=pass (i=1 spf=pass spfdomain=nxp.com dkim=pass dkdomain=nxp.com dmarc=pass fromdomain=nxp.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nxp.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i15-20020a17090acf8f00b002563fbb8a85si1201848pju.111.2023.07.06.02.22.25; Thu, 06 Jul 2023 02:22:41 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@nxp.com header.s=selector2 header.b=BMcp4QvM; arc=pass (i=1 spf=pass spfdomain=nxp.com dkim=pass dkdomain=nxp.com dmarc=pass fromdomain=nxp.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nxp.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233962AbjGFIRL (ORCPT + 99 others); Thu, 6 Jul 2023 04:17:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41236 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233912AbjGFIRH (ORCPT ); Thu, 6 Jul 2023 04:17:07 -0400 Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01on2048.outbound.protection.outlook.com [40.107.15.48]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 309561BD3; Thu, 6 Jul 2023 01:17:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LiVWv1qRUk+57Ru4j7BGi/KS4RGhFMmH19z1y2zRHNA+FP5emv9eIlJSRqOQcm4zpYQA7n//wI+MZk+r9i8qD+6tGm487hk/8Udt1+C7DmYnVd7w+BqrBxrZiEHRIjByeBXISTSPiaYw9mlKrlEpKa3QUy/ucohEvehc4Im1kc4HohuHQ1bW1EsJ2uj8xxxFGQ/kh3vC/V6Q7vY63do97DaHoihxPdtdyoPF0FaUmI+1AZrTLV5iPnPUnS6vKiKiujIVSZJt3dgT/hRukC6GjujdNzmxHRh/QYIz6z5C5Lw0v8y05+os88YS8ce4sFbUxG1xks393VBq2DXCi1cm7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Zs4qsLvA4I1EDVqaUbosKk+vfTqETBtDOnTvr6bjeeg=; b=bWxOAIHh0uGS8Uidlw0NeLPrpTXKwq523BRRf2s+yTcX1PivnYHtGEWqMYUNUK5EA4rlawjTXjrnEkaj8YSNlS7kvketSZHe8Y066a7V0wmJSfaF61ty1sSaa51xZkrrPZRGFAJSfAUewNZzlbdKwESxGw/TVyBNgLspIhIMpRC5ckDZiQwB7FCZNFQvckezBf2ofP4P9PMkJj1yzN4PmFgRj6CzBvCyn0rubn3/FozqifS5MVd3DhfL4U0dcMurUCwt59in9LpcltyMurndQygZoddqxtBbF0f0phQx7e4pkcZN0bxnjW1mk28lEKsPbUXwkRSJiUgMm7H9ELpd0w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Zs4qsLvA4I1EDVqaUbosKk+vfTqETBtDOnTvr6bjeeg=; b=BMcp4QvMZGHcsorLjp49qL42jDoojm8k/FRpfXd6q2JO1c+MM2GZfpZPuw0BSlfl8AIKKi0PxkN5rFR/WcqYab+qvPQ2Tjwz2tPf8Fjwbe8ZYdfKG+HuPQXpdKRjnyHNLG+7uDyWdtmVwyGEEBZOtSzVYIG64Hj/JkFGFMBzYmY= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; Received: from AM5PR04MB3139.eurprd04.prod.outlook.com (2603:10a6:206:8::20) by DU2PR04MB9132.eurprd04.prod.outlook.com (2603:10a6:10:2f7::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6565.17; Thu, 6 Jul 2023 08:17:00 +0000 Received: from AM5PR04MB3139.eurprd04.prod.outlook.com ([fe80::1edd:68cb:85d0:29e0]) by AM5PR04MB3139.eurprd04.prod.outlook.com ([fe80::1edd:68cb:85d0:29e0%7]) with mapi id 15.20.6544.024; Thu, 6 Jul 2023 08:17:00 +0000 From: wei.fang@nxp.com To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, ast@kernel.org, daniel@iogearbox.net, hawk@kernel.org, john.fastabend@gmail.com, shenwei.wang@nxp.com, xiaoning.wang@nxp.com, netdev@vger.kernel.org Cc: linux-imx@nxp.com, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH V2 net 3/4] net: fec: increase the size of tx ring and update tx_wake_threshold Date: Thu, 6 Jul 2023 16:10:11 +0800 Message-Id: <20230706081012.2278063-4-wei.fang@nxp.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230706081012.2278063-1-wei.fang@nxp.com> References: <20230706081012.2278063-1-wei.fang@nxp.com> X-ClientProxiedBy: SG2P153CA0018.APCP153.PROD.OUTLOOK.COM (2603:1096::28) To AM5PR04MB3139.eurprd04.prod.outlook.com (2603:10a6:206:8::20) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM5PR04MB3139:EE_|DU2PR04MB9132:EE_ X-MS-Office365-Filtering-Correlation-Id: 57d93597-865b-4b38-73ed-08db7df95f86 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JB8J4T4OnHgNGN9UBp2wOykthXpOI4zfhZg/OpTCVhKY6VaFSPU4o/ZV3EB4cDGJOf/IYup6OavXDUUq1zWBiGcJrcXcpwvkYycDnmU+2PezLaNY8503ao/sAN6T+UtyvtAHbzP82Bw0RoPPwyOK/uMXchbXIyROw18GuXcJqib8Mldxd3sFFTWQ1l4zn6f5N3KJUReRvQDNuh9sQRxOq7u0cZav0TCjgSw0po51M3Iujk7BPI2Hai41XpHUbsjtdqhU4wa4ipGQmct1VNMzHjIUK3Ew3pkKr9/4vOTcpfhjSnkKAYfUSaXDCm8kaCxTND32sGvE6xDtx47N0xUZVZo5AKbJLxvd2Wr/C6WppmWmD8ORdFh2ykhOTD/QPkDel3MFt8eXKlt/xybn0fFcghDWiQGxED9ns0c91FCLixAu/S1r0Rz2AjExcUxtqDx9j2e0p8ORFZ37d5qYELOjrwYgBejvJ3jRKtun5uzRVttGiIefKzaYqmVliBddOELKUttpAx+iF8cpMIFIPgYGmhjswOohZ/971oMnBtZUm+AZON8jZza6jUFMVGfBWHX73f9pTWAdJIPzHLCKoF8dC8x9ot2/KmfwrzbO+t6B8j4h4SwBtysMHvSzVUULtMSf9gCO45E5ijsNlcSj2e9NEA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM5PR04MB3139.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(366004)(136003)(396003)(376002)(39860400002)(346002)(451199021)(186003)(8676002)(8936002)(6506007)(26005)(2616005)(2906002)(66946007)(1076003)(5660300002)(15650500001)(7416002)(41300700001)(36756003)(921005)(6486002)(4326008)(316002)(66476007)(83380400001)(6666004)(66556008)(6512007)(478600001)(9686003)(52116002)(38350700002)(86362001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: NsLzbDFU2q7ZspDxpja/sZ2RoQ034+O+wtoNO/YypxA3S6ioi5B5pau829gkBDK65Gw5unVX1SIjAbst0xDNpTgxkTXCMz4n9ccOl3AesfLQVdOwStISUC6r70s4joynDDzcXKRxm7Rc1d/6Fix5VSehx0Y9UbfpIHWEuuroiKyXWVKRbC96Y09PwobBBeWOwHSFmmSACR8M320/OKf8MayRlQOOBrouNUOUXamG6j98h9dBdcPFlViBkb3Qn2070LMmDMOeflIiM/5peag8ICPHsO5DJr6I3F3qVug0AkNAvwXcshe2OYkcyLweaW1kNN8vr9fbuZcw+LCawPSfZQAEjSgaG2tn8tSWscSsa7vWVJaEIL3j++nzm4kCbAEK1QRxxUxgeXyxd9ZeylusVq3r7IhjgVvH9DJzPiwRS1PRXktv5tPc4ln+KLbDkolDtMITV7SvLEFP675i0CZ3z/SPQL67UJTLnKqxtVGsqpAp1EXqwRBrh91P5WzxzV3LyR8u+zTideyo7ejmUsWwYYkAjUFL8pw31CZHuOhcNbA6uuAPKypxAmzuSPtR83G4/rUc9WZ/TsNsHb323cTw5xJszmclvY2N/3MTW1Km0+6O/aLFVMeTr/TGUGhp0VXrjqulrcB/TZ6N6QSezycN7rucQzaybDzAD6o9myr2nA/gn0Ld5q/8vDfHH8z+sgBLmAL1Q4sj4ZpmNZy6CrwZQzCPcu0kza4pVZDwOHnvBfaPswo+/+DPWPA0mtNSm8P7YEZEGF4bzBtcIPSHTpwMydRipuS321QsdIInfOqH5gbFJjWJbGnS1IVoFTbWo+Z3QCI3rn8R+HlQKSN/JkbyNWvEdn4fhIZLrdeVIG11v/xdigAGOdn07gVRR1NWuL6j3ORqZZbcHOz10meavRc6Q27qm38fWTdyHJMXfEVq03UbhExssC15wZBUUxCLu4Om0mKwQhP7BrSCkf0yWeIPVM240DJ8FHYL0XJiKVFx+0BJmMaa76WIq3nNQY9RqPG4AaZvyt8xgYYf2RRdgnmuCfrGmq2z62zkzdvBboTjxhfKwzIKRjFbCt8lBnjj6dcxmK58cJTwDv1jidgAsnRvoPFBikFXG28vPAaM0Yrlvvy1kfT0o8miD+UTWaG+eQKL6ShrV0VUM9uup9gxFvsRHVDO5sUoVMfJs08cj3xnD8MYU9aUXy7rMruOU1u4SIp6dufFN/CTiuHdzLjV7YJyqGOLKJ9GeJ9tVl3glAYWmmRzWMQHBTfakC4+gkCw1lcU/Nk2gKmcZkN36CTPJmqGhzFPLZ6fgg3NktvtqnF77aO+Wu0RisfDj02iOwVnRvsajj2YjwRLUjvFd9cJFR4xbhwoyXLHfmjJ/9A+NAyhbQ6P/WncKCK+WqfZbKJOBzP8lKFjfssC9n/Wyj9L5Xi4DoV0g2x+HbdRY1ZthIF86ht/x8QH3l/jPNAJDi7F3xC/HBdjltTJnFPh3k3arjn93vtXRsLs9a1DgLjJBP3zEUOv3wxggFVSm7CxLu9lsJwsW5MNi7OOXswIslhkn4Rgcw/nAl+EhdXfSSbrEPeQcRFZ/lLQunCxjUED6P1v4Yih X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 57d93597-865b-4b38-73ed-08db7df95f86 X-MS-Exchange-CrossTenant-AuthSource: AM5PR04MB3139.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jul 2023 08:17:00.8383 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hpMMIuojwzE0BzkjX+gYbJTV+qTZ9MqA0Qrj9UifvHwMAEjIKcgFLbIwpLPSgvunkMQFWA9a5dET6DvtDCHD0w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR04MB9132 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770662512219706487?= X-GMAIL-MSGID: =?utf-8?q?1770662512219706487?= From: Wei Fang When the XDP feature is enabled and with heavy XDP frames to be transmitted, there is a considerable probability that available tx BDs are insufficient. This will lead to some XDP frames to be discarded and the "NOT enough BD for SG!" error log will appear in the console (as shown below). [ 160.013112] fec 30be0000.ethernet eth0: NOT enough BD for SG! [ 160.023116] fec 30be0000.ethernet eth0: NOT enough BD for SG! [ 160.028926] fec 30be0000.ethernet eth0: NOT enough BD for SG! [ 160.038946] fec 30be0000.ethernet eth0: NOT enough BD for SG! [ 160.044758] fec 30be0000.ethernet eth0: NOT enough BD for SG! In the case of heavy XDP traffic, sometimes the speed of recycling tx BDs may be slower than the speed of sending XDP frames. There may be several specific reasons, such as the interrupt is not responsed in time, the efficiency of the NAPI callback function is too low due to all the queues (tx queues and rx queues) share the same NAPI, and so on. After trying various methods, I think that increase the size of tx BD ring is simple and effective. Maybe the best resolution is that allocate NAPI for each queue to improve the efficiency of the NAPI callback, but this change is a bit big and I didn't try this method. Perheps this method will be implemented in a future patch. This patch also updates the tx_wake_threshold of tx ring which is related to the size of tx ring in the previous logic. Otherwise, the tx_wake_threshold will be too high (403 BDs), which is more likely to impact the slow path in the case of heavy XDP traffic, because XDP path and slow path share the tx BD rings. According to Jakub's suggestion, the tx_wake_threshold is at least equal to tx_stop_threshold + 2 * MAX_SKB_FRAGS, if a queue of hundreds of entries is overflowing, we should be able to apply a hysteresis of a few tens of entries. Fixes: 6d6b39f180b8 ("net: fec: add initial XDP support") Signed-off-by: Wei Fang --- V2 change: 1. Keep the tx_stop_threshold unchanged and change the tx_wake_threshold to tx_stop_threshold + 2 * MAX_SKB_FRAGS. 2. Split the change, the netdev_err_once() is moved to a separate patch. 3. Modify the commit message and subject ppropriately. --- drivers/net/ethernet/freescale/fec.h | 2 +- drivers/net/ethernet/freescale/fec_main.c | 3 +-- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/freescale/fec.h b/drivers/net/ethernet/freescale/fec.h index 8c0226d061fe..63a053dea819 100644 --- a/drivers/net/ethernet/freescale/fec.h +++ b/drivers/net/ethernet/freescale/fec.h @@ -355,7 +355,7 @@ struct bufdesc_ex { #define RX_RING_SIZE (FEC_ENET_RX_FRPPG * FEC_ENET_RX_PAGES) #define FEC_ENET_TX_FRSIZE 2048 #define FEC_ENET_TX_FRPPG (PAGE_SIZE / FEC_ENET_TX_FRSIZE) -#define TX_RING_SIZE 512 /* Must be power of two */ +#define TX_RING_SIZE 1024 /* Must be power of two */ #define TX_RING_MOD_MASK 511 /* for this to work */ #define BD_ENET_RX_INT 0x00800000 diff --git a/drivers/net/ethernet/freescale/fec_main.c b/drivers/net/ethernet/freescale/fec_main.c index 940d3afe1d24..c59576ab8c7a 100644 --- a/drivers/net/ethernet/freescale/fec_main.c +++ b/drivers/net/ethernet/freescale/fec_main.c @@ -3349,8 +3349,7 @@ static int fec_enet_alloc_queue(struct net_device *ndev) fep->total_tx_ring_size += fep->tx_queue[i]->bd.ring_size; txq->tx_stop_threshold = FEC_MAX_SKB_DESCS; - txq->tx_wake_threshold = - (txq->bd.ring_size - txq->tx_stop_threshold) / 2; + txq->tx_wake_threshold = FEC_MAX_SKB_DESCS + 2 * MAX_SKB_FRAGS; txq->tso_hdrs = dma_alloc_coherent(&fep->pdev->dev, txq->bd.ring_size * TSO_HEADER_SIZE, From patchwork Thu Jul 6 08:10:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Fang X-Patchwork-Id: 116562 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:9f45:0:b0:3ea:f831:8777 with SMTP id v5csp2405080vqx; Thu, 6 Jul 2023 01:40:47 -0700 (PDT) X-Google-Smtp-Source: APBJJlFW60iBWlBlWLFKQg4S3/a2WyG5HHboFoF1uIvT0cDJRGKaCJYw+oWNmvZsq1+CqQMWwo7R X-Received: by 2002:a05:6808:150:b0:3a0:4faf:accd with SMTP id h16-20020a056808015000b003a04fafaccdmr938436oie.32.1688632847282; Thu, 06 Jul 2023 01:40:47 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1688632847; cv=pass; d=google.com; s=arc-20160816; b=Cj8rCXBTn8cxicUTfR9gdVp3aDI0wKeMQmAniqmvgbvR/WQ17CtqumhMAmlepAQED3 NFrV/q+utUWved2ub7pNO2RWRLO93YgnkVA8dLMaksw9z9dhN8Lr8Dxs6PLumTKhizO+ o1+TyAA55SoOuUy/fGYxNhVlFvup7/o90+VyJn4uDMqADl0c40iU3L2SuKA8PuqPucFr JpWrIEqgpmPfpqxJecx/NxuWqlDab3JnxxlNo89rJxQSzCuH5ZwhGzPq7VUBSuE7F2dj +kuZ6TOVYSfXeDtTgKnZj8LWjxp2Wu9bsSDJ+LEvvx5ab3WNwIV5TF2DqyxjN9JGFlEd O7Iw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:content-transfer-encoding :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=tLRwXCv59vuptQfXAOAJE6Y1TBzkjUHbkt8CpsUbpuY=; fh=AE1PERU6bhr9O8eJNdZsAjdBxzfHJxzlkxEKIMP4fEw=; b=eIXcg47VWsN2wBfU9ZX66yulnwTUJtfMgZx1hgKiGy/OJV6FZfOuLa051W6GaTMb3/ z/wUxMFJ1blQ/wfgf8xYOhd9y9Sta3RYNrlf1XVJXZSa8B68UzPRdsJR7/C1Glfypwpi O2/TK4Z7TxaudznrQZfWn3EhXouV69Vc9PHBVWeSynbms6cG66DeTTLWBu9Vqc9De67W sKsaGtTaeCsbCCS2ObegOu557Zq18SMg7T+/QqDXKfnhstOXZYlzmKNh5VCkKrSQBjvM QeRYybH1jtFVFjDaDDfoOXqJn3+YyTG3t+uAgZqPmo1tHXQ9LwNZ1UU49dhm8QV4UPny tW5Q== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@nxp.com header.s=selector2 header.b=pZ9fCUzO; arc=pass (i=1 spf=pass spfdomain=nxp.com dkim=pass dkdomain=nxp.com dmarc=pass fromdomain=nxp.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nxp.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i4-20020a63e444000000b00528d0d8b7bbsi996278pgk.595.2023.07.06.01.40.32; Thu, 06 Jul 2023 01:40:47 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@nxp.com header.s=selector2 header.b=pZ9fCUzO; arc=pass (i=1 spf=pass spfdomain=nxp.com dkim=pass dkdomain=nxp.com dmarc=pass fromdomain=nxp.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nxp.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233889AbjGFIR0 (ORCPT + 99 others); Thu, 6 Jul 2023 04:17:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41522 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233750AbjGFIRU (ORCPT ); Thu, 6 Jul 2023 04:17:20 -0400 Received: from EUR03-AM7-obe.outbound.protection.outlook.com (mail-am7eur03on2077.outbound.protection.outlook.com [40.107.105.77]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1967C1BD3; Thu, 6 Jul 2023 01:17:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hQGD6lzOeA/IAg8gn6w5yzNpVDx332n45gZdjh5vCGPq6AzplvNCbGsbIeoDhW4nKjGpEUDWxU1S2Zeg5x8dovv+CZM2BXBUR/LDuArAxqFW6DGCIzs7VdHZSZJnSiRaDwu6b88Kv6G+iIKzitFlCWcvX4KOzD+c/4WYVjmA5Jw4xKDBFxFvTVJD1UJcmiZ7LeP0pDZxJ2WrI5pFpDReZor8GLK1UWi8yw8N3kFLO6jSV2SvvjuNEMLwIcgKRW6TxEeFGpfrXjBjVx8dBc2MuFRoYH6BZlBPTXyrUT7JD4kcLvAGn2m4ixianxdtvo7c1Azo4bZ4gTAtRo5Iw8tqqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tLRwXCv59vuptQfXAOAJE6Y1TBzkjUHbkt8CpsUbpuY=; b=LHDS2uwaNQlQD278GERsdnCtYgA+y/3lnxY5FNe3WfAWAd54y8xmXkDeB0rxEQD7LlDr5JfjFv1KRAXqvVzHxI6ZZ/8cWMaU74LVoJZH2owfwZEhXp+Ka+kkw+Bt9ujX5eULxWbEtPJI8AdhkhiaNl5SkakIUxRg2drlEYZz8fAy1N9w4Z/+V6vhNTBEYgyFIWPyAtUzhsgCmSEdgPBD3Rj+WErhTT6kQIeaQfeYCo5ZM2V8a81jI5qrF3/7E3VUxmYYXAO0YM9UJT2dw3wssqs3jbA5AdNWuCROwJgHU16uKb/CnI/8O2EKQhPvz7p0g8z/d/ixmepaqrUcr+9l9g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tLRwXCv59vuptQfXAOAJE6Y1TBzkjUHbkt8CpsUbpuY=; b=pZ9fCUzObE1L1/7Aa1i3Jt/Eotn2BatwUoFFqpy0nbge1nIuBaV/OZMkvpWX/QKDea6Yy1o9ySC+o1OnYBZVp0sQbUS5T6vpidzutrISutsFqOQPOWfmYmtMavUSdj6F1MZTE4A/0BZ3TClflThRp5j9P9RH2qLmNSTxLgODrsg= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; Received: from AM5PR04MB3139.eurprd04.prod.outlook.com (2603:10a6:206:8::20) by DU2PR04MB9132.eurprd04.prod.outlook.com (2603:10a6:10:2f7::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6565.17; Thu, 6 Jul 2023 08:17:09 +0000 Received: from AM5PR04MB3139.eurprd04.prod.outlook.com ([fe80::1edd:68cb:85d0:29e0]) by AM5PR04MB3139.eurprd04.prod.outlook.com ([fe80::1edd:68cb:85d0:29e0%7]) with mapi id 15.20.6544.024; Thu, 6 Jul 2023 08:17:09 +0000 From: wei.fang@nxp.com To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, ast@kernel.org, daniel@iogearbox.net, hawk@kernel.org, john.fastabend@gmail.com, shenwei.wang@nxp.com, xiaoning.wang@nxp.com, netdev@vger.kernel.org Cc: linux-imx@nxp.com, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH V2 net 4/4] net: fec: use netdev_err_once() instead of netdev_err() Date: Thu, 6 Jul 2023 16:10:12 +0800 Message-Id: <20230706081012.2278063-5-wei.fang@nxp.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230706081012.2278063-1-wei.fang@nxp.com> References: <20230706081012.2278063-1-wei.fang@nxp.com> X-ClientProxiedBy: SG2P153CA0018.APCP153.PROD.OUTLOOK.COM (2603:1096::28) To AM5PR04MB3139.eurprd04.prod.outlook.com (2603:10a6:206:8::20) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM5PR04MB3139:EE_|DU2PR04MB9132:EE_ X-MS-Office365-Filtering-Correlation-Id: 47a163be-09a6-466b-cc15-08db7df964cb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: oPiufv+bBynqxI6pdXhIdn89YwMoVWuRoyh4ZaKMWfMr4MHElC9Y64HVAg8OqKYuiE78gTQXKUX89RzgLK2m87HZy+a1gAeSVDPyRSIC8DdCK8SZeKxDocYELdkeLwvtFhulVhtPRVSE49cqX5Lma6ANzFGb4u1ueTHaENq02ACQIo6AY99ptTl0vBx9/iot6QmudM7KL9b+oblyyOB7eZ0FtiKHzJ9qFoX6ZzYRuCck1D6INlL5Tar2qOWz5GleFRo+MHA6DFXz1/a0+8KqgVYxcFSdurZDVe/fHWSjDwdGRBGHw5T4smgEV+HEWV/HaOZ2c0aXLLzpjC2z6CZ5AsF6+VkjGmOAX8qLNosL1hRJsN7PxvKOr3B6FzEGgy/JNNMU1MNMPg52+FIkj2t65Evt9IiX2r2uhd/k4b5PiU/tWUNyzNOq0kk8qj2twfIT75tp/j8SWJUuW4xSdNOP1Bp06e0JsKAyahUBUZDAKX8DwW7wxsAs7SpnQJsUk7q6sNtwYUT4yNFxdeYzymZ47T2a2jNcYWFW/ml0ggwI7M5YcrbwTz3iqIWJchT8nAgt5jRnHKes+i8KcAWGsDorq6Q8Sdws9yTt6dgSLozIiQ1yTvVr/0lj9ytqJyQ9BRfo X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AM5PR04MB3139.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(366004)(136003)(396003)(376002)(39860400002)(346002)(451199021)(186003)(8676002)(8936002)(6506007)(26005)(2616005)(2906002)(66946007)(1076003)(5660300002)(7416002)(41300700001)(36756003)(921005)(6486002)(4326008)(316002)(66476007)(83380400001)(6666004)(66556008)(6512007)(478600001)(9686003)(52116002)(38350700002)(86362001)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: IcuYFogPkWaeYi2+UU0sUQ6PJ0nCoidFEIPd31XQeiG22w0grRhLaDlZM/lApXSt73siLxWaag8paO6H+j8GU5L/RsPbgsyLsq59oR8tx1aVp0oar09QHrANMjiiDTPyr03aHq1t8Laoa54E+FZBHwdpoWPOPVjtEZJggnv+2h7pfAjnXdpHMNQWhHQj4meV7Owm7AFbFMOdUlnlK/JIV7CMlPw7yeL+1+ZMyySH6+JB9E3xati7WsJ/UE+aUSZ88QvLB94+ga7KKm/Ph7YEO54fu+0gTe5eypjS+yYRo9bfQl/VlkcLxJDmUj6hakw/ONiYoiIs73tALt92VD2A5Q/G6S0CB9qh2ftHguXDsp4/HM7X4HA54R7Be8yHUaaW41FH5csLGG7LIFI3j5Vn3bT7AXTfWqhsjVINRTxLXWA8RsufCG8iRKW35Vg0L7oBph4zKQMJNU6b//d5ruRHQ3RS0ovtZWkl/gGiSbbPpjwTVc8WYLsPIt7IzdhWZ+vXkEDTZgSRjnbEWR2Bgy4yYOmrRIZXD2KnRItUTiGBaiHT4VABBIQ8mkm6RPeeCsxHatG4jeIuuY401uopCIEjWjWpv6q5HKBs9rhUxe8WLmgZMCNJ6PlkwHUsjkZMa75339iNETHXfsUIs1Uz/Yugs/7ohtr+wZZdv07S7s8rsnJf3iHZSZEKD/NC5l3sPZ0lIBfBrVrjjuYRF49uj5xmP3pJ1uBPmVHzLXH9caRaJLmgKq8grUL4l7eox2XfAl45dueoPeA5lkJ7sKeS9Jmd8GY6ILSu2xV+zudTLAmvQ9PEgSvMp252zVjQpP4YdfoeKhcwvFzuSad8zBaA/HujdD7Kf+yOUA3g9L9S6OzprybdrqwtjwSmmxOCj57aPG8QcaxKaL0eOB3dw9VTBkPQcJg0o+B1L5sdmXGuaCakiqiF/edCWUykk6SvbMdTR6jGW42PrixSjZPPeQ6+dlA9jRS052Oml1bpJ3QaEaP5FEyMV14bGoinWHfVK1MVY+xh8cpmNdlHwASyGSuiVnm5de1aSHBEkd786LCsW7iK6BayhhLCwH3Rr4jbxkqnl/qFLvUayCSPfMmrMZzXWYYkdqa1/Iy7jg7ZQiy4sCd5RzTPmG+U2XD0SP5bOn6KQvNObvN+dPaN0Nv3m0yUR36XsWr82W52Pf6fyDcM6NTY/ttb3lkPmnnudrFWJzxYv4ICQD4pEN6tcRQFvKoT4oS1Z9Rr4Q/LqZYJJ78LHvVWx8JC5OzMeXGmOWfxNOmKkEunpUskJQBkofEWpHaKP8oCUoNUK6s3qq7247cfmfEuiCJvVxou4xMu06m8Md/hIfMahOE+4hQzsigC0bpdkszoZ4+TNBaR1KKeBpvHJbtGRlohxYScIg2uZ2iIPzFHQe3Zd1rwuBg5y+dBQEQ8XXWZ54LJxW+dmrSRA6X1jsHN0+SjLVPatX06z44CP5cSUxev6T7PRjNVNMk9B7ZgMG403+T2Q+e7+CSK5uEPl9INyIO8sg1ids3bc7oghaxgi2OcygZtpAhW+wm/aAXDZOpV2bxq7NzzA/zYgUlBZdxDomEmgyY8wYgO59IlwX1ICzRo X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 47a163be-09a6-466b-cc15-08db7df964cb X-MS-Exchange-CrossTenant-AuthSource: AM5PR04MB3139.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jul 2023 08:17:09.6154 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: onmyxbH86IqXyG3cPqAsP7Kb+tqY/lRz0/oTvQbtEEQdHgO94H/jOgqS6QD/qr6DY03KVzKpZdKwGTHZk4TMBg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR04MB9132 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1770659876070925375?= X-GMAIL-MSGID: =?utf-8?q?1770659876070925375?= From: Wei Fang In the case of heavy XDP traffic to be transmitted, the console will print the error log continuously if there are lack of enough BDs to accommodate the frames. The log looks like below. [ 160.013112] fec 30be0000.ethernet eth0: NOT enough BD for SG! [ 160.023116] fec 30be0000.ethernet eth0: NOT enough BD for SG! [ 160.028926] fec 30be0000.ethernet eth0: NOT enough BD for SG! [ 160.038946] fec 30be0000.ethernet eth0: NOT enough BD for SG! [ 160.044758] fec 30be0000.ethernet eth0: NOT enough BD for SG! Not only will this log be replicated and redundant, it will also degrade XDP performance. So we use netdev_err_once() instead of netdev_err() now. Fixes: 6d6b39f180b8 ("net: fec: add initial XDP support") Signed-off-by: Wei Fang --- V2 change: This is a new patch which is separated from the patch 3. --- drivers/net/ethernet/freescale/fec_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/net/ethernet/freescale/fec_main.c b/drivers/net/ethernet/freescale/fec_main.c index c59576ab8c7a..ec9e4bdb0c06 100644 --- a/drivers/net/ethernet/freescale/fec_main.c +++ b/drivers/net/ethernet/freescale/fec_main.c @@ -3836,7 +3836,7 @@ static int fec_enet_txq_xmit_frame(struct fec_enet_private *fep, entries_free = fec_enet_get_free_txdesc_num(txq); if (entries_free < MAX_SKB_FRAGS + 1) { - netdev_err(fep->netdev, "NOT enough BD for SG!\n"); + netdev_err_once(fep->netdev, "NOT enough BD for SG!\n"); return -EBUSY; }