From patchwork Wed Aug 9 10:22:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Souradeep Chakrabarti X-Patchwork-Id: 133218 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:c44e:0:b0:3f2:4152:657d with SMTP id w14csp2780507vqr; Wed, 9 Aug 2023 06:00:05 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHUumHDY8400fXUty3Ds9gDb5+lky5t6GFx2SpN6Y3+EcII3FP+4Rs7U3W/gFPbf3jRwM0y X-Received: by 2002:a17:902:bb8c:b0:1bc:7001:6e58 with SMTP id m12-20020a170902bb8c00b001bc70016e58mr2291848pls.33.1691586005157; Wed, 09 Aug 2023 06:00:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691586005; cv=none; d=google.com; s=arc-20160816; b=lA3qm6HFgGsubmGRAVIzstVtyb6Lyzd1fBrg8GNU9RrQprS9rytZGC7OstAKk6TPo1 yX4SaWfUnGFIzlGcJccKJ+7FFC623RUumguCwWJoaWcpEzZqMS418qtxmrE772pEBOYa bExfdXvn5SNKxGRa1cqpFWp7Q+086lx/gbnR4CF2MJ8Fql6/oSHbE+Y9YKfZVL9OTmKN lGVxOeo7rlKqoaNWZ3QeVR/FvR62VM48amvEzI0bttEwaCGt/NfDM2mpP7tfEQNPwHiN 5IWqfxXoCCIHdW47unRXSDWl0c4YyY2n762AO5BolaEAhQZzE8YGe77ctvVxRG8QL2gX Fxpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:message-id:date:subject:cc:to:from :dkim-signature:dkim-filter; bh=s/Hd32b4XFC5qXe0hMfXzG8vSHV2xgLG+0kVH3hmJUk=; fh=CRXlYauY4/Fm9OvL6NavBYnlK8EESZebSVy0W94b4Sg=; b=ve+oBlXz4U9Vm15MaoABsJj6i3rsI1hAQ07SMajbYDdJNQ2G4WjYCDYZr952im1mST 8sPwsPhT7xcTh7ePdcYO0q5lPbLZgZ35Tq87ysgVbibqYUte6QSQLgAxc6y/UC/MCv+8 RTaVAfXz6XRiEeQPxj3OR0BTHteWqoF8hOQbIYPojDdD3c4jfsDEbcZQSKuJLtEam/Jd Csks0e16Zz6gGPq9mGfetsP7fO4xo8xqKvV9uYnfqF62Pun8HsJOo3rcE2wHcd/JobXz rE1SL9IUK5Ioau1kLHifp1x1I4WwOnL4U6KwlHrpPtrpT4E8rgBr2eXJn0yLDqeD9RN5 F1Ww== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=kyOStzjy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q17-20020a170902dad100b001b89613687bsi9812446plx.439.2023.08.09.05.59.49; Wed, 09 Aug 2023 06:00:05 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=kyOStzjy; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231857AbjHIKW1 (ORCPT + 99 others); Wed, 9 Aug 2023 06:22:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52434 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230146AbjHIKW0 (ORCPT ); Wed, 9 Aug 2023 06:22:26 -0400 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id E1DC698; Wed, 9 Aug 2023 03:22:25 -0700 (PDT) Received: by linux.microsoft.com (Postfix, from userid 1099) id 68B4B20FC3E4; Wed, 9 Aug 2023 03:22:25 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 68B4B20FC3E4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1691576545; bh=s/Hd32b4XFC5qXe0hMfXzG8vSHV2xgLG+0kVH3hmJUk=; h=From:To:Cc:Subject:Date:From; b=kyOStzjyWsOPbM5+GPS6YfLyRyueAoJDuK/3FnslrQcYWOMx67IxnAqm9InYveCN0 N31+cX0/E8BFYKY20z9QN+dSOWKPYAmjZVsw0D3NXp9IQYlgDaHQQ7o2DDboLeAm++ reoj2X8SQKDn/sS5U0H0nmPTzAXXG+J+A7/S6GSI= From: Souradeep Chakrabarti To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, longli@microsoft.com, sharmaajay@microsoft.com, leon@kernel.org, cai.huoqing@linux.dev, ssengar@linux.microsoft.com, vkuznets@redhat.com, tglx@linutronix.de, linux-hyperv@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org Cc: schakrabarti@microsoft.com, Souradeep Chakrabarti , stable@vger.kernel.org Subject: [PATCH V8 net] net: mana: Fix MANA VF unload when hardware is unresponsive Date: Wed, 9 Aug 2023 03:22:05 -0700 Message-Id: <1691576525-24271-1-git-send-email-schakrabarti@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 X-Spam-Status: No, score=-17.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_BLOCKED, SPF_HELO_PASS,SPF_PASS,USER_IN_DEF_DKIM_WL,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1773756486997361790 X-GMAIL-MSGID: 1773756486997361790 When unloading the MANA driver, mana_dealloc_queues() waits for the MANA hardware to complete any inflight packets and set the pending send count to zero. But if the hardware has failed, mana_dealloc_queues() could wait forever. Fix this by adding a timeout to the wait. Set the timeout to 120 seconds, which is a somewhat arbitrary value that is more than long enough for functional hardware to complete any sends. Cc: stable@vger.kernel.org Fixes: ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)") Signed-off-by: Souradeep Chakrabarti --- V7 -> V8: * Replaced dev_consume_skb_any with dev_kfree_skb_any. * Added extra braces to use assignment as truth value. V6 -> V7: * Optimized the while loop for freeing skb. V5 -> V6: * Added pcie_flr to reset the pci after timeout. * Fixed the position of changelog. * Removed unused variable like cq. V4 -> V5: * Added fixes tag * Changed the usleep_range from static to incremental value. * Initialized timeout in the begining. V3 -> V4: * Removed the unnecessary braces from mana_dealloc_queues(). V2 -> V3: * Removed the unnecessary braces from mana_dealloc_queues(). V1 -> V2: * Added net branch * Removed the typecasting to (struct mana_context*) of void pointer * Repositioned timeout variable in mana_dealloc_queues() * Repositioned vf_unload_timeout in mana_context struct, to utilise the 6 bytes hole --- drivers/net/ethernet/microsoft/mana/mana_en.c | 37 +++++++++++++++++-- 1 file changed, 33 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c index a499e460594b..c2ad0921e893 100644 --- a/drivers/net/ethernet/microsoft/mana/mana_en.c +++ b/drivers/net/ethernet/microsoft/mana/mana_en.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include @@ -2345,9 +2346,12 @@ int mana_attach(struct net_device *ndev) static int mana_dealloc_queues(struct net_device *ndev) { struct mana_port_context *apc = netdev_priv(ndev); + unsigned long timeout = jiffies + 120 * HZ; struct gdma_dev *gd = apc->ac->gdma_dev; struct mana_txq *txq; + struct sk_buff *skb; int i, err; + u32 tsleep; if (apc->port_is_up) return -EINVAL; @@ -2363,15 +2367,40 @@ static int mana_dealloc_queues(struct net_device *ndev) * to false, but it doesn't matter since mana_start_xmit() drops any * new packets due to apc->port_is_up being false. * - * Drain all the in-flight TX packets + * Drain all the in-flight TX packets. + * A timeout of 120 seconds for all the queues is used. + * This will break the while loop when h/w is not responding. + * This value of 120 has been decided here considering max + * number of queues. */ + for (i = 0; i < apc->num_queues; i++) { txq = &apc->tx_qp[i].txq; - - while (atomic_read(&txq->pending_sends) > 0) - usleep_range(1000, 2000); + tsleep = 1000; + while (atomic_read(&txq->pending_sends) > 0 && + time_before(jiffies, timeout)) { + usleep_range(tsleep, tsleep + 1000); + tsleep <<= 1; + } + if (atomic_read(&txq->pending_sends)) { + err = pcie_flr(to_pci_dev(gd->gdma_context->dev)); + if (err) { + netdev_err(ndev, "flr failed %d with %d pkts pending in txq %u\n", + err, atomic_read(&txq->pending_sends), + txq->gdma_txq_id); + } + break; + } } + for (i = 0; i < apc->num_queues; i++) { + txq = &apc->tx_qp[i].txq; + while ((skb = skb_dequeue(&txq->pending_skbs))) { + mana_unmap_skb(skb, apc); + dev_kfree_skb_any(skb); + } + atomic_set(&txq->pending_sends, 0); + } /* We're 100% sure the queues can no longer be woken up, because * we're sure now mana_poll_tx_cq() can't be running. */