From patchwork Wed Dec 20 23:47:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Thompson X-Patchwork-Id: 181901 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2483:b0:fb:cd0c:d3e with SMTP id q3csp80845dyi; Wed, 20 Dec 2023 15:48:17 -0800 (PST) X-Google-Smtp-Source: AGHT+IEwkviDG0uTzG7eK04p5WJ0za2vrA5yIUUTX3bJYBADb1QI0+cuupsjbRJmV+LiWAJh+Tci X-Received: by 2002:ac8:5747:0:b0:412:14a8:24c8 with SMTP id 7-20020ac85747000000b0041214a824c8mr30895952qtx.8.1703116097418; Wed, 20 Dec 2023 15:48:17 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1703116097; cv=pass; d=google.com; s=arc-20160816; b=GttR5BxVU09od+jHuW8x6K+lliuxXkvc4c2meV8oZ9ewMjRe7CG/EeCR6FWZL8wLdv mHCVFQcFpa931/qv5mI17yXFbTh57Ck+WiiIYhCqYW+XurTEvhbvZA09s3bNbhqW8/8g QOHXfK75kp4y/+o2qgQwaSwHh+WRfFXEH8exzoMhxWamPZga/zjdTUQknLvKq40vyk5r 1z3SjtWX1i9KqNVhoOpBHeWHP45v9GuucqpHxfQsmjzdwZM2giVqWIest35UwyJgz5F6 9pBJbd5gt96ZPzDVrtZ7aK3oQc/R9Y4RAb0zQ2lIkwnnhmYetxsclOnnCXbRjTjnUYph V9fA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:message-id:date:subject:cc:to :from:dkim-signature; bh=OaW2dTWc2+3XAGXo26TktKIaKbuqfT0hs/IF6zDNT4M=; fh=r2NLFV0L3vwSsifO+wLOZLE8CwrO84VuslyOZSlv//U=; b=NXYboRlA2AycmumNHOem9MagnQ0WZ/vOVqMxeeXyGGOfzQhwgBP23IaWylySk6c8lm lLvV71BaiQSl7xYpdmK8Dah+rwKcpdM5OqBdERsHy7DgZIHsSjYq00FnB0eoBYYxfgj4 wsdWVynRFfV6MnpM6QSdUaWQKBpHvMdy/lzZ4IUrBpaJ+eQM3cts2Fl2zDCE/mUSu86C Mm++pd8JS7DGtaCV46y8cJ8UfpgXpA3Uds76VZAQ1BCmzMQmTrMlh9W7DFC7bTBztlz5 x4kd6f+8eIC1t0AROvLY+I3F99iivq5F8/VYQSN9TFC7k5QZtWY7d/6cisVhxW0yMwq1 lqPA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=ty5gjEs+; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel+bounces-7648-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-7648-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id p18-20020a05622a049200b00427916fd347si907174qtx.92.2023.12.20.15.48.17 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Dec 2023 15:48:17 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-7648-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@Nvidia.com header.s=selector2 header.b=ty5gjEs+; arc=pass (i=1 spf=pass spfdomain=nvidia.com dmarc=pass fromdomain=nvidia.com); spf=pass (google.com: domain of linux-kernel+bounces-7648-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-7648-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=nvidia.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 2B2CC1C2148F for ; Wed, 20 Dec 2023 23:48:17 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4529C4B5BB; Wed, 20 Dec 2023 23:47:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="ty5gjEs+" X-Original-To: linux-kernel@vger.kernel.org Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2055.outbound.protection.outlook.com [40.107.244.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 774E64AF64; Wed, 20 Dec 2023 23:47:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VcUJX7ByYBEkedut5ASmDInXnTlfWJa6biBWgmB4+da/xQ44USthG6Xp2D3oWBVYgGGs7Q8y0xMQzIQoYYR3nt5iIV1EbU7frTma0WCJVNJLwN2xriNm40yQbVIO4XAMfA5m20uVMmlwRmLhW+AI1YbSdjIXF3cgx45EcGFiavCTfXLJqbj+wXhvBjrBFQIbLQ2GY39GyHzL7CIUGqbY2CXIN49797ecAUOO1ItFpwJFdctaUEVqAO5TEQC3t5D4wissLYhr0or8+vcqUaK4NMxwqZ7sYKvQqpjV4YlMB5673n+m7W3hMBYxFt35jHNeTjd8IsyAz2oHkxKKJekFEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OaW2dTWc2+3XAGXo26TktKIaKbuqfT0hs/IF6zDNT4M=; b=E9H8VS5je3JG3FeXXQUeTBABQ2qXVvDJwk6zeIT6kGSVJtuhgCMOJWlueoqHoy4KVZJ1g8PXRixnxGV6/5WCkieuPDujHub845cREVrAZtzLi9GM7S+xfyW/DR5HHkbFk2xMARKwQjVkLxp0nYz3Tka0nWmPpn3MrJ560Xgg0DxldbumYxEgeE+IO7IEW5vUwTwIAYnY5PR89uhm8LJjss9NZ4Pik3bw+UmCE7dUIZukwPR1GcMfwc4NZ01Cwws65rrThGW2LF1dxuEbkWyLa6v886d2n4GpknvJbvdExwwgNR0ySWCvPTHDkqBQd+GWjhrLIc0glv1vkqFCgik0NA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OaW2dTWc2+3XAGXo26TktKIaKbuqfT0hs/IF6zDNT4M=; b=ty5gjEs+cR22travmniPyaPPf//c8dk6ekqrynLbWEecb4MI2CVCN7qk+XBjItioQTlN6oAoNa1Xd7IzZ9nRVg8VfGaweCqei45WYz/otiOtPLpmqcTXXJH7z7oHsevPeBceY8XmlF9pJu6ngiNZ82StK/PNCJFoH7QQT2QHjtW6odmNHY+OQHhvpatAkIg2f2QlLoSaNZLUx39xiTkPjTia9vMAAxgLmSl4cY+1cas39Cw3T4gXwkw/k0eyqOxUXtWAYG/fzNwIpqa8WwkiaG5+apy9vBHIoLzeoBnlUXhvRroHgjp2RRgK99dV42YakDlP7BcdRH7wmIYR1dgkFw== Received: from CYXPR02CA0077.namprd02.prod.outlook.com (2603:10b6:930:ce::25) by PH0PR12MB7009.namprd12.prod.outlook.com (2603:10b6:510:21c::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7113.18; Wed, 20 Dec 2023 23:47:51 +0000 Received: from CY4PEPF0000FCC3.namprd03.prod.outlook.com (2603:10b6:930:ce:cafe::f9) by CYXPR02CA0077.outlook.office365.com (2603:10b6:930:ce::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7113.19 via Frontend Transport; Wed, 20 Dec 2023 23:47:50 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by CY4PEPF0000FCC3.mail.protection.outlook.com (10.167.242.105) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7113.20 via Frontend Transport; Wed, 20 Dec 2023 23:47:50 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Wed, 20 Dec 2023 15:47:46 -0800 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Wed, 20 Dec 2023 15:47:46 -0800 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server id 15.2.986.41 via Frontend Transport; Wed, 20 Dec 2023 15:47:45 -0800 From: David Thompson To: , , , , , CC: , , , David Thompson Subject: [PATCH net v2] mlxbf_gige: fix receive packet race condition Date: Wed, 20 Dec 2023 18:47:39 -0500 Message-ID: <20231220234739.13753-1-davthompson@nvidia.com> X-Mailer: git-send-email 2.30.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000FCC3:EE_|PH0PR12MB7009:EE_ X-MS-Office365-Filtering-Correlation-Id: 496fdd6b-7b27-4fdc-36bd-08dc01b613c0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: nPmOUO+v5bNxiICRw0lKa/tEPdiHz60bR8hbI3eFjFvsN1xtcYnbOA/zvg06pWaDiGnA4oJTuouGNLfQGjunFmq3vYcHcVI9AmBT4cjFysKUDkJNKwEOmxXPs7sk7JBh4Rera7geYVcAfMNoHHf5+LaAsn3PWaixo6o7cPuUsNkF32itdfOWbxvRiSIjLfM7JKi2ADRkC6kBSjUJxDhQsPWAqJv/YAxAHWEKzClrgEnI8TekxExYvbuAUt3HPgDfaGIUCFDdJClg2QcHh/u8ztK3kjkVnzTj+xCq5j3hyyMyf53fWr1KeWfHSIuhriVqRCVckbReoFYDTekAgeGEsArjyn0LvAflxm/DpstnVhvQdvYsT1EGzlmM9DOWxDWnPyYccICkq50D0FZFDwc7muTYi3Nogvzb0ove3RuBvaDsl+gxxqjAd65Ydijhkdq66ERxepOD63Akjil11AlfThrTz3oFXnIRUFUgUdU+pYZfqMxwhoJQLJ61Vg8wU9tPVxWhGhwuHiiEshG0cmmzay65pyX44UiR8QqgNSDLb86k9iEWgCHA1i7yyiCS2VACVfj3MDPqNpSIlBmkH/PZ+eYVPank0DYfGrHujVQuF9r0mPH4SbzJ92xzovw+FJo3Y/i+ZcvVXo3fD6BxSfF2Omwv3KycYgvw4avwrWTUhdlLRFY4jy8Kex+8X5c5K9nzVO61itI1sW6V2Zj7Nu7gJ+sEf4A7+mDX1cjeWNcNqVZSsQ4+jxEonRvhg3ZRdHwv X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(396003)(346002)(376002)(136003)(39860400002)(230922051799003)(1800799012)(451199024)(82310400011)(186009)(64100799003)(46966006)(36840700001)(40470700004)(40460700003)(40480700001)(1076003)(4326008)(426003)(83380400001)(7696005)(5660300002)(336012)(54906003)(110136005)(26005)(8676002)(8936002)(70206006)(70586007)(2616005)(107886003)(316002)(478600001)(36860700001)(7636003)(36756003)(356005)(6666004)(82740400003)(47076005)(2906002)(41300700001)(86362001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Dec 2023 23:47:50.5869 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 496fdd6b-7b27-4fdc-36bd-08dc01b613c0 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000FCC3.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB7009 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1785846664887825669 X-GMAIL-MSGID: 1785846664887825669 Under heavy traffic, the BlueField Gigabit interface can become unresponsive. This is due to a possible race condition in the mlxbf_gige_rx_packet function, where the function exits with producer and consumer indices equal but there are remaining packet(s) to be processed. In order to prevent this situation, read receive consumer index *before* the HW replenish so that the mlxbf_gige_rx_packet function returns an accurate return value even if a packet is received into just-replenished buffer prior to exiting this routine. If the just-replenished buffer is received and occupies the last RX ring entry, the interface would not recover and instead would encounter RX packet drops related to internal buffer shortages since the driver RX logic is not being triggered to drain the RX ring. This patch will address and prevent this "ring full" condition. Fixes: f92e1869d74e ("Add Mellanox BlueField Gigabit Ethernet driver") Reviewed-by: Asmaa Mnebhi Signed-off-by: David Thompson --- v2 * removed logic that affected RX_DMA enable/disable --- drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_rx.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_rx.c b/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_rx.c index 0d5a41a2ae01..227d01cace3f 100644 --- a/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_rx.c +++ b/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_rx.c @@ -267,6 +267,13 @@ static bool mlxbf_gige_rx_packet(struct mlxbf_gige *priv, int *rx_pkts) priv->stats.rx_truncate_errors++; } + /* Read receive consumer index before replenish so that this routine + * returns accurate return value even if packet is received into + * just-replenished buffer prior to exiting this routine. + */ + rx_ci = readq(priv->base + MLXBF_GIGE_RX_CQE_PACKET_CI); + rx_ci_rem = rx_ci % priv->rx_q_entries; + /* Let hardware know we've replenished one buffer */ rx_pi++; @@ -279,8 +286,6 @@ static bool mlxbf_gige_rx_packet(struct mlxbf_gige *priv, int *rx_pkts) rx_pi_rem = rx_pi % priv->rx_q_entries; if (rx_pi_rem == 0) priv->valid_polarity ^= 1; - rx_ci = readq(priv->base + MLXBF_GIGE_RX_CQE_PACKET_CI); - rx_ci_rem = rx_ci % priv->rx_q_entries; if (skb) netif_receive_skb(skb);