From patchwork Wed Dec 7 17:53:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ajit Khaparde X-Patchwork-Id: 30968 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp328804wrr; Wed, 7 Dec 2022 10:02:13 -0800 (PST) X-Google-Smtp-Source: AA0mqf6bPyt3ecG2Brd3tHTqP8/t3w2s5idlC4wwzMC3i1mUfVmN3/yrr6hhqQJcJSslRNI4SbFr X-Received: by 2002:a17:906:5e42:b0:7c1:22a6:818d with SMTP id b2-20020a1709065e4200b007c122a6818dmr2190550eju.43.1670436118196; Wed, 07 Dec 2022 10:01:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670436116; cv=none; d=google.com; s=arc-20160816; b=zj/wH3AYtNNiyR6uj9QIzYu0mxgbs2SGb3Y83ue7Jiy+1VKpoXUNZIEN2+yWhBMaXF MZ3FS5KkULBqSfMn1wOiLVSl9Mwdk9LulqkDKWbHDQpH27qKKvJqa1z+MWUeAwu7VX4Y 20M/vmLZpvbF2y2Vyqgw3O2q9RuHfAgKEWqCmlYDcH7rg6BCzbPc1xnTQTxxVnkmUauZ 0zDHsSbRDqC09FqyV/9jIFXLuklkYr9bD1oFtWEomoZI0QFvk/hVyl7/Lw9Bkx2wAbP5 DAAP+jaT4PdAZwZ/BGrs2thve2YrKZSy2LRK6Zk6vkvKwJ6mmGeuUM5t784Kt6xLV5yF EMsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=MrT1Oag1H6EvRsi7C0jpMgCgoL3CjGIUUPRj3ChA8Tk=; b=LfKIVTIRKOsA2XtxB9Y3+FbmF8KOU4oIx/wfuN5FIzsl8Vfh56Qbef/PdqAawaSLxk hMTLO4bVFmQtshf0bcgfs0Dm9dldATsqQjyUEMRwRcUfkBrdLuPg/fRFm606FDjaqvlR Hfq9lmFwaCrybnLUDKx44TInt+Tu89XnBnPmNWe9mlXi05lMC103eWeACGnfaUAAy7lg beN2iAJROepyr/3k53k7OoOzeznpea6e8Jcclqcg7LvYZgnE9KR2HABKLYwYVSjkLdmr THHLOEaUFDKoiAjjTnrZprbqeEoWhsSJ6VNmHdEWzMODVecIjWiBOtvhWb6a9C6dE8+p Iqjg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@broadcom.com header.s=google header.b="NwO/BWNk"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=broadcom.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id m23-20020aa7d357000000b004614db9789asi256785edr.127.2022.12.07.10.01.31; Wed, 07 Dec 2022 10:01:55 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@broadcom.com header.s=google header.b="NwO/BWNk"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=broadcom.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230155AbiLGRxY (ORCPT + 99 others); Wed, 7 Dec 2022 12:53:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56048 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230144AbiLGRxU (ORCPT ); Wed, 7 Dec 2022 12:53:20 -0500 Received: from mail-pl1-x630.google.com (mail-pl1-x630.google.com [IPv6:2607:f8b0:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 38CD02D1DF for ; Wed, 7 Dec 2022 09:53:18 -0800 (PST) Received: by mail-pl1-x630.google.com with SMTP id y4so17737387plb.2 for ; Wed, 07 Dec 2022 09:53:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=MrT1Oag1H6EvRsi7C0jpMgCgoL3CjGIUUPRj3ChA8Tk=; b=NwO/BWNkRJttTp9n3Qj55/G76MnoJEwyBOwLEwr8MHUrVsbjcHy1QtHW3t76EV4Rb2 oDH7tbquVFYJp20OP5rBcmB/JwouQiV6YKECNKGUdwJfXkToALP1VX3NHRGAfd320Ync g+/lzZZUsHtYmJ6IC2ak3X+vJltlrM6ZGnRp4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=MrT1Oag1H6EvRsi7C0jpMgCgoL3CjGIUUPRj3ChA8Tk=; b=jOM7t/3IHrxo//3vSNtlQVYxFNudDXEW9+6gpJogTSbUM2m6H+sPMk1Z9l+hcHND9k zeC9hiYgZ+9VxExBDTn+UEKJq4CR/cAzQWRsuG6tdWOTbFOyq5NvyZRZktiEDUgKrVF2 Jx5S+yzFJNvQzrz2X567Vme3At8/MybnC22aQsNsWt9p1SheMy0FekyhESHt1XEy7kZF NXrKQZ3CvkGI+PBs4bzZEV+GrlS7SrFiFISw7qHZEf3i5i5YqxU7wy1txSqzpJLbH/Cl OA/8m8QQtULFfGcCNn/lB3HB5APNfi73HOyomnhY2KGCjNDWP/rAS2Q9hI5AtiX/b9PL FwHQ== X-Gm-Message-State: ANoB5pl1FTFXXlfbwFXCAQWHZpdPe0sFg3Ch+wT4W1uL3/DGB7P2RtcO +AieJ5xGcad/z80hHfqdf2+lDQ== X-Received: by 2002:a17:902:d389:b0:189:cea0:bc98 with SMTP id e9-20020a170902d38900b00189cea0bc98mr18470152pld.70.1670435597639; Wed, 07 Dec 2022 09:53:17 -0800 (PST) Received: from C02GC2QQMD6T.wifi.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id l6-20020a622506000000b005748aca80fesm13862242pfl.32.2022.12.07.09.53.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Dec 2022 09:53:17 -0800 (PST) From: Ajit Khaparde To: ajit.khaparde@broadcom.com Cc: andrew.gospodarek@broadcom.com, davem@davemloft.net, edumazet@google.com, jgg@ziepe.ca, kuba@kernel.org, leon@kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, michael.chan@broadcom.com, netdev@vger.kernel.org, pabeni@redhat.com, selvin.xavier@broadcom.com Subject: [PATCH v5 1/7] bnxt_en: Add auxiliary driver support Date: Wed, 7 Dec 2022 09:53:04 -0800 Message-Id: <20221207175310.23656-2-ajit.khaparde@broadcom.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20221207175310.23656-1-ajit.khaparde@broadcom.com> References: <20221207175310.23656-1-ajit.khaparde@broadcom.com> MIME-Version: 1.0 X-Spam-Status: No, score=0.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SORTED_RECIPS,SPF_HELO_NONE,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751579221271465994?= X-GMAIL-MSGID: =?utf-8?q?1751579221271465994?= Add auxiliary driver support. An auxiliary device will be created if the hardware indicates support for RDMA. The bnxt_ulp_probe() function has been removed and a new bnxt_rdma_aux_device_add() function has been added. The bnxt_free_msix_vecs() and bnxt_req_msix_vecs() will now hold the RTNL lock when they call the bnxt_close_nic()and bnxt_open_nic() since the device close and open need to be protected under RTNL lock. The operations between the bnxt_en and bnxt_re will be protected using the en_ops_lock. This will be used by the bnxt_re driver in a follow-on patch to create ROCE interfaces. Signed-off-by: Ajit Khaparde Reviewed-by: Andy Gospodarek Reviewed-by: Selvin Xavier --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 8 +- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 8 + drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 177 +++++++++++++++--- drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h | 7 +- 4 files changed, 169 insertions(+), 31 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 0fe164b42c5d..2769d36b7d39 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -13154,6 +13154,9 @@ static void bnxt_remove_one(struct pci_dev *pdev) struct net_device *dev = pci_get_drvdata(pdev); struct bnxt *bp = netdev_priv(dev); + bnxt_rdma_aux_device_uninit(bp->aux_dev); + bnxt_aux_dev_free(bp); + if (BNXT_PF(bp)) bnxt_sriov_disable(bp); @@ -13747,11 +13750,13 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) bnxt_dl_fw_reporters_create(bp); + bnxt_rdma_aux_device_init(bp); + bnxt_print_device_info(bp); pci_save_state(pdev); - return 0; + return 0; init_err_cleanup: bnxt_dl_unregister(bp); init_err_dl: @@ -13795,7 +13800,6 @@ static void bnxt_shutdown(struct pci_dev *pdev) if (netif_running(dev)) dev_close(dev); - bnxt_ulp_shutdown(bp); bnxt_clear_int_mode(bp); pci_disable_device(pdev); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index 41c6dd0ae447..04181f6e8a17 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -1623,6 +1624,12 @@ struct bnxt_fw_health { #define BNXT_FW_IF_RETRY 10 #define BNXT_FW_SLOT_RESET_RETRY 4 +struct bnxt_aux_dev { + struct auxiliary_device aux_dev; + struct bnxt_en_dev *edev; + int id; +}; + enum board_idx { BCM57301, BCM57302, @@ -1844,6 +1851,7 @@ struct bnxt { #define BNXT_CHIP_P4_PLUS(bp) \ (BNXT_CHIP_P4(bp) || BNXT_CHIP_P5(bp)) + struct bnxt_aux_dev *aux_dev; struct bnxt_en_dev *edev; struct bnxt_napi **bnapi; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c index 2e54bf4fc7a7..7ffd61ec2683 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c @@ -25,32 +25,37 @@ #include "bnxt_hwrm.h" #include "bnxt_ulp.h" +static DEFINE_IDA(bnxt_aux_dev_ids); + static int bnxt_register_dev(struct bnxt_en_dev *edev, unsigned int ulp_id, struct bnxt_ulp_ops *ulp_ops, void *handle) { struct net_device *dev = edev->net; struct bnxt *bp = netdev_priv(dev); struct bnxt_ulp *ulp; + int rc = 0; - ASSERT_RTNL(); if (ulp_id >= BNXT_MAX_ULP) return -EINVAL; ulp = &edev->ulp_tbl[ulp_id]; if (rcu_access_pointer(ulp->ulp_ops)) { netdev_err(bp->dev, "ulp id %d already registered\n", ulp_id); - return -EBUSY; + rc = -EBUSY; + goto exit; } if (ulp_id == BNXT_ROCE_ULP) { unsigned int max_stat_ctxs; max_stat_ctxs = bnxt_get_max_func_stat_ctxs(bp); if (max_stat_ctxs <= BNXT_MIN_ROCE_STAT_CTXS || - bp->cp_nr_rings == max_stat_ctxs) - return -ENOMEM; + bp->cp_nr_rings == max_stat_ctxs) { + rc = -ENOMEM; + goto exit; + } } - atomic_set(&ulp->ref_count, 0); + atomic_set(&ulp->ref_count, 1); ulp->handle = handle; rcu_assign_pointer(ulp->ulp_ops, ulp_ops); @@ -59,7 +64,8 @@ static int bnxt_register_dev(struct bnxt_en_dev *edev, unsigned int ulp_id, bnxt_hwrm_vnic_cfg(bp, 0); } - return 0; +exit: + return rc; } static int bnxt_unregister_dev(struct bnxt_en_dev *edev, unsigned int ulp_id) @@ -69,10 +75,11 @@ static int bnxt_unregister_dev(struct bnxt_en_dev *edev, unsigned int ulp_id) struct bnxt_ulp *ulp; int i = 0; - ASSERT_RTNL(); if (ulp_id >= BNXT_MAX_ULP) return -EINVAL; + edev->flags |= BNXT_EN_FLAG_ULP_STOPPED; + ulp = &edev->ulp_tbl[ulp_id]; if (!rcu_access_pointer(ulp->ulp_ops)) { netdev_err(bp->dev, "ulp id %d not registered\n", ulp_id); @@ -126,7 +133,6 @@ static int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, unsigned int ulp_id, int total_vecs; int rc = 0; - ASSERT_RTNL(); if (ulp_id != BNXT_ROCE_ULP) return -EINVAL; @@ -149,6 +155,7 @@ static int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, unsigned int ulp_id, max_idx = min_t(int, bp->total_irqs, max_cp_rings); idx = max_idx - avail_msix; } + edev->ulp_tbl[ulp_id].msix_base = idx; edev->ulp_tbl[ulp_id].msix_requested = avail_msix; hw_resc = &bp->hw_resc; @@ -156,8 +163,10 @@ static int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, unsigned int ulp_id, if (bp->total_irqs < total_vecs || (BNXT_NEW_RM(bp) && hw_resc->resv_irqs < total_vecs)) { if (netif_running(dev)) { + rtnl_lock(); bnxt_close_nic(bp, true, false); rc = bnxt_open_nic(bp, true, false); + rtnl_unlock(); } else { rc = bnxt_reserve_rings(bp, true); } @@ -184,7 +193,6 @@ static int bnxt_free_msix_vecs(struct bnxt_en_dev *edev, unsigned int ulp_id) struct net_device *dev = edev->net; struct bnxt *bp = netdev_priv(dev); - ASSERT_RTNL(); if (ulp_id != BNXT_ROCE_ULP) return -EINVAL; @@ -194,9 +202,12 @@ static int bnxt_free_msix_vecs(struct bnxt_en_dev *edev, unsigned int ulp_id) edev->ulp_tbl[ulp_id].msix_requested = 0; edev->flags &= ~BNXT_EN_FLAG_MSIX_REQUESTED; if (netif_running(dev) && !(edev->flags & BNXT_EN_FLAG_ULP_STOPPED)) { + rtnl_lock(); bnxt_close_nic(bp, true, false); bnxt_open_nic(bp, true, false); + rtnl_unlock(); } + return 0; } @@ -347,25 +358,6 @@ void bnxt_ulp_sriov_cfg(struct bnxt *bp, int num_vfs) } } -void bnxt_ulp_shutdown(struct bnxt *bp) -{ - struct bnxt_en_dev *edev = bp->edev; - struct bnxt_ulp_ops *ops; - int i; - - if (!edev) - return; - - for (i = 0; i < BNXT_MAX_ULP; i++) { - struct bnxt_ulp *ulp = &edev->ulp_tbl[i]; - - ops = rtnl_dereference(ulp->ulp_ops); - if (!ops || !ops->ulp_shutdown) - continue; - ops->ulp_shutdown(ulp->handle); - } -} - void bnxt_ulp_irq_stop(struct bnxt *bp) { struct bnxt_en_dev *edev = bp->edev; @@ -475,6 +467,135 @@ static const struct bnxt_en_ops bnxt_en_ops_tbl = { .bnxt_register_fw_async_events = bnxt_register_async_events, }; +void bnxt_aux_dev_free(struct bnxt *bp) +{ + kfree(bp->aux_dev); + bp->aux_dev = NULL; +} + +static struct bnxt_aux_dev *bnxt_aux_dev_alloc(struct bnxt *bp) +{ + struct bnxt_aux_dev *bnxt_adev; + + bnxt_adev = kzalloc(sizeof(*bnxt_adev), GFP_KERNEL); + if (!bnxt_adev) + return ERR_PTR(-ENOMEM); + + return bnxt_adev; +} + +void bnxt_rdma_aux_device_uninit(struct bnxt_aux_dev *bnxt_adev) +{ + struct auxiliary_device *adev; + + if (IS_ERR_OR_NULL(bnxt_adev)) + return; + + adev = &bnxt_adev->aux_dev; + auxiliary_device_delete(adev); + auxiliary_device_uninit(adev); + if (bnxt_adev->id >= 0) + ida_free(&bnxt_aux_dev_ids, bnxt_adev->id); +} + +void bnxt_rdma_aux_device_init(struct bnxt *bp) +{ + struct net_device *dev = bp->dev; + int rc; + + if (bp->flags & BNXT_FLAG_ROCE_CAP) { + bp->aux_dev = bnxt_aux_dev_alloc(bp); + if (IS_ERR_OR_NULL(bp->aux_dev)) { + netdev_warn(dev, "Failed to init auxiliary device for ROCE\n"); + goto skip_aux_init; + } + + bp->aux_dev->id = ida_alloc(&bnxt_aux_dev_ids, GFP_KERNEL); + if (bp->aux_dev->id < 0) { + netdev_warn(dev, "ida alloc failed for ROCE auxiliary device\n"); + bnxt_aux_dev_free(bp); + goto skip_aux_init; + } + + /* If aux bus init fails, continue with netdev init. */ + rc = bnxt_rdma_aux_device_add(bp); + if (rc) { + netdev_warn(dev, "Failed to add auxiliary device for ROCE\n"); + ida_free(&bnxt_aux_dev_ids, bp->aux_dev->id); + bnxt_aux_dev_free(bp); + } + } +skip_aux_init: + return; +} + +void bnxt_aux_dev_release(struct device *dev) +{ + struct bnxt_aux_dev *bnxt_adev = + container_of(dev, struct bnxt_aux_dev, aux_dev.dev); + struct bnxt *bp = netdev_priv(bnxt_adev->edev->net); + + bnxt_adev->edev->en_ops = NULL; + kfree(bnxt_adev->edev); + bnxt_adev->edev = NULL; + bp->edev = NULL; +} + +static inline void bnxt_set_edev_info(struct bnxt_en_dev *edev, struct bnxt *bp) +{ + edev->en_ops = &bnxt_en_ops_tbl; + edev->net = bp->dev; + edev->pdev = bp->pdev; + edev->l2_db_size = bp->db_size; + edev->l2_db_size_nc = bp->db_size; + + if (bp->flags & BNXT_FLAG_ROCEV1_CAP) + edev->flags |= BNXT_EN_FLAG_ROCEV1_CAP; + if (bp->flags & BNXT_FLAG_ROCEV2_CAP) + edev->flags |= BNXT_EN_FLAG_ROCEV2_CAP; +} + +int bnxt_rdma_aux_device_add(struct bnxt *bp) +{ + struct bnxt_aux_dev *bnxt_adev = bp->aux_dev; + struct bnxt_en_dev *edev = bnxt_adev->edev; + struct auxiliary_device *aux_dev; + int ret; + + aux_dev = &bnxt_adev->aux_dev; + aux_dev->id = bnxt_adev->id; + aux_dev->name = "rdma"; + aux_dev->dev.parent = &bp->pdev->dev; + aux_dev->dev.release = bnxt_aux_dev_release; + + if (!edev) { + edev = kzalloc(sizeof(*edev), GFP_KERNEL); + if (!edev) + return -ENOMEM; + } + + bnxt_set_edev_info(edev, bp); + bnxt_adev->edev = edev; + bp->edev = edev; + + ret = auxiliary_device_init(aux_dev); + if (ret) { + kfree(edev); + kfree(bnxt_adev); + return ret; + } + + ret = auxiliary_device_add(aux_dev); + if (ret) { + kfree(edev); + kfree(bnxt_adev); + auxiliary_device_uninit(aux_dev); + return ret; + } + + return 0; +} + struct bnxt_en_dev *bnxt_ulp_probe(struct net_device *dev) { struct bnxt *bp = netdev_priv(dev); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h index 42b50abc3e91..8b85c663d881 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h @@ -17,6 +17,7 @@ #define BNXT_MIN_ROCE_STAT_CTXS 1 struct hwrm_async_event_cmpl; +struct bnxt_aux_dev; struct bnxt; struct bnxt_msix_entry { @@ -102,10 +103,14 @@ int bnxt_get_ulp_stat_ctxs(struct bnxt *bp); void bnxt_ulp_stop(struct bnxt *bp); void bnxt_ulp_start(struct bnxt *bp, int err); void bnxt_ulp_sriov_cfg(struct bnxt *bp, int num_vfs); -void bnxt_ulp_shutdown(struct bnxt *bp); void bnxt_ulp_irq_stop(struct bnxt *bp); void bnxt_ulp_irq_restart(struct bnxt *bp, int err); void bnxt_ulp_async_events(struct bnxt *bp, struct hwrm_async_event_cmpl *cmpl); +void bnxt_aux_dev_release(struct device *dev); +int bnxt_rdma_aux_device_add(struct bnxt *bp); +void bnxt_rdma_aux_device_uninit(struct bnxt_aux_dev *bnxt_adev); +void bnxt_rdma_aux_device_init(struct bnxt *bp); +void bnxt_aux_dev_free(struct bnxt *bp); struct bnxt_en_dev *bnxt_ulp_probe(struct net_device *dev); #endif From patchwork Wed Dec 7 17:53:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ajit Khaparde X-Patchwork-Id: 30969 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp329647wrr; Wed, 7 Dec 2022 10:03:28 -0800 (PST) X-Google-Smtp-Source: AA0mqf6GZLPC/FE1z0eQWo+xQvOkTpwl81tv97fE7aFgACl8/mZknH9eYlfOwQfcW3Gz9riAFDSO X-Received: by 2002:a17:90a:4886:b0:211:42a9:d132 with SMTP id b6-20020a17090a488600b0021142a9d132mr101509547pjh.8.1670436207706; Wed, 07 Dec 2022 10:03:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670436207; cv=none; d=google.com; s=arc-20160816; b=xeRu36fm3469QB/a625mRjl+rICTbX2bX/vnHwiVsYDltdV7u6JMK9+6pwK5RioPyU nbZXmV2Vj6rBOuK1y+SdFrbBwQudGySjpt7vO5EPzlm+YmtTPP5GaMNljV6Wvu3MZyiB 9G358l3R3Gn9GNIQvzVaPDy2Iye06FiGFVnJH+WKcrNSn8Hbmgl83Io/fbiL6NCVkPcF FnCJu3zZr/9UukZUedGHkmn0pzpl8covglAnDuFERqiiMGsxlyWS1UaDgBIHFMwJWU+r r4drqktmf4mQvuZq1ZopswUHkVmnaAo0cxgWtifoAcnCBmsF86N5nXwhJmanFPsjuPkc yPCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=O/fqABfoRtHoQPF4AamhrcAP2+ZdzJ4cAGjUaysAVYw=; b=ukPVuxrJHBtbhoFVRFOU6y2piwyHJJ9FaBBLy+pmLM+kz44BwmaMoi/X9bxmX/fMSa J8gxZaIG8IqBhq3tr7EVyOU8bXFYAe0B5xNkAWY0vTDz7EOrCY+sX0dT0OGCsyOIKHo1 voPyDbeHU4fmDt5as6KjwpkiJlXm8+RWgnDT2nhKhyLbsCYYdCXnkOOnWy//DWKIivMW ue9ho8hB7g46t+2XMIg0+X3Ue06yAJBueTMlBFD4JpOp0DMr0jwCMLzVflUsikd73rfh Of2VKcUz845qVZ5zWKUstmcB1lfrv3fXpjIdzL6oDi0nT9y7hWzPesRvTOpk3tKrSH8F d0sA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@broadcom.com header.s=google header.b=I5KvliVo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=broadcom.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l31-20020a63571f000000b00477b0e028e2si21581583pgb.810.2022.12.07.10.03.10; Wed, 07 Dec 2022 10:03:26 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@broadcom.com header.s=google header.b=I5KvliVo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=broadcom.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230172AbiLGRxb (ORCPT + 99 others); Wed, 7 Dec 2022 12:53:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229449AbiLGRxV (ORCPT ); Wed, 7 Dec 2022 12:53:21 -0500 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 141A62F67A for ; Wed, 7 Dec 2022 09:53:20 -0800 (PST) Received: by mail-pj1-x102f.google.com with SMTP id cm20so18366256pjb.1 for ; Wed, 07 Dec 2022 09:53:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=O/fqABfoRtHoQPF4AamhrcAP2+ZdzJ4cAGjUaysAVYw=; b=I5KvliVoftoINTg7kDAHcOKsQocxfgbGSxrUsbKuYKmRIwq8t5MoUkwb3EYfDtcncN htF62Tpc0HryUFiL3YAoVe1zIUYSb/+aBhanxNehtdBPRp/AQL1jqwD4/F5pIH19Uwto T8gQN20A+t1dRVqx7jaxXsXnwYPnYtNyCmdpA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=O/fqABfoRtHoQPF4AamhrcAP2+ZdzJ4cAGjUaysAVYw=; b=gr1ZdGABz85U5p4tHZvWds5vE3RepsGcqbgAVx2GsbIT8eS4TGAbH0juHIny7gFBJQ AERjB2nfumviWVxc+LfwrjT2WJJP6/Z0zTDteCVXhC0Vrq42pIrS4CNMDxwPMYIqSk4/ OEsH70aWNeYDkVW082nbPxtmb3r8w80dSqOdiMwQQFtpYVRwi1Zx9Ohd0bqTuEJFNCKW IEroQFjGDobsvOBQlWXUer5fErnsK2W5fqTpN3k7zXMVLyrpI0dKILbNt25iVoB1YZRp S92/WiE6n8/LZ2MzX9cjcKoVhVwyeqlpsl2Re/JOEGjMBQDjfPt6YiLU5bRAOOELSWLo sMcw== X-Gm-Message-State: ANoB5pkQRG7AaIaLbzgwBtka5aw1UwSGohb//ZIuOTFe9J3ItYQfFais /qlMQtn26h0lZ5V8+F+JXAdFHQ== X-Received: by 2002:a17:903:555:b0:189:959a:84cc with SMTP id jo21-20020a170903055500b00189959a84ccmr43918708plb.41.1670435599376; Wed, 07 Dec 2022 09:53:19 -0800 (PST) Received: from C02GC2QQMD6T.wifi.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id l6-20020a622506000000b005748aca80fesm13862242pfl.32.2022.12.07.09.53.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Dec 2022 09:53:18 -0800 (PST) From: Ajit Khaparde To: ajit.khaparde@broadcom.com Cc: andrew.gospodarek@broadcom.com, davem@davemloft.net, edumazet@google.com, jgg@ziepe.ca, kuba@kernel.org, leon@kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, michael.chan@broadcom.com, netdev@vger.kernel.org, pabeni@redhat.com, selvin.xavier@broadcom.com Subject: [PATCH v5 2/7] RDMA/bnxt_re: Use auxiliary driver interface Date: Wed, 7 Dec 2022 09:53:05 -0800 Message-Id: <20221207175310.23656-3-ajit.khaparde@broadcom.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20221207175310.23656-1-ajit.khaparde@broadcom.com> References: <20221207175310.23656-1-ajit.khaparde@broadcom.com> MIME-Version: 1.0 X-Spam-Status: No, score=0.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SORTED_RECIPS,SPF_HELO_NONE,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751579317273773462?= X-GMAIL-MSGID: =?utf-8?q?1751579317273773462?= Use auxiliary driver interface for driver load, unload ROCE driver. The driver does not need to register the interface using the netdev notifier anymore. Removed the bnxt_re_dev_list which is not needed. Currently probe, remove and shutdown ops have been implemented for the auxiliary device. Also remove exccessve validation checks for rdev. Signed-off-by: Ajit Khaparde Reviewed-by: Andy Gospodarek Reviewed-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/bnxt_re.h | 9 +- drivers/infiniband/hw/bnxt_re/main.c | 395 ++++++------------ drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 29 +- drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h | 3 - 4 files changed, 132 insertions(+), 304 deletions(-) diff --git a/drivers/infiniband/hw/bnxt_re/bnxt_re.h b/drivers/infiniband/hw/bnxt_re/bnxt_re.h index 785c37cae3c0..b0465c8d229a 100644 --- a/drivers/infiniband/hw/bnxt_re/bnxt_re.h +++ b/drivers/infiniband/hw/bnxt_re/bnxt_re.h @@ -89,13 +89,6 @@ struct bnxt_re_ring_attr { u8 mode; }; -struct bnxt_re_work { - struct work_struct work; - unsigned long event; - struct bnxt_re_dev *rdev; - struct net_device *vlan_dev; -}; - struct bnxt_re_sqp_entries { struct bnxt_qplib_sge sge; u64 wrid; @@ -132,6 +125,7 @@ struct bnxt_re_dev { #define BNXT_RE_FLAG_ERR_DEVICE_DETACHED 17 #define BNXT_RE_FLAG_ISSUE_ROCE_STATS 29 struct net_device *netdev; + struct notifier_block nb; unsigned int version, major, minor; struct bnxt_qplib_chip_ctx *chip_ctx; struct bnxt_en_dev *en_dev; @@ -194,5 +188,4 @@ static inline struct device *rdev_to_dev(struct bnxt_re_dev *rdev) return &rdev->ibdev.dev; return NULL; } - #endif diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index 8c0c80a8d338..165ae11b927d 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -48,6 +48,7 @@ #include #include #include +#include #include #include @@ -74,14 +75,14 @@ MODULE_DESCRIPTION(BNXT_RE_DESC " Driver"); MODULE_LICENSE("Dual BSD/GPL"); /* globals */ -static struct list_head bnxt_re_dev_list = LIST_HEAD_INIT(bnxt_re_dev_list); -/* Mutex to protect the list of bnxt_re devices added */ -static DEFINE_MUTEX(bnxt_re_dev_lock); -static struct workqueue_struct *bnxt_re_wq; -static void bnxt_re_remove_device(struct bnxt_re_dev *rdev); -static void bnxt_re_dealloc_driver(struct ib_device *ib_dev); +static DEFINE_MUTEX(bnxt_re_mutex); + static void bnxt_re_stop_irq(void *handle); static void bnxt_re_dev_stop(struct bnxt_re_dev *rdev); +static int bnxt_re_netdev_event(struct notifier_block *notifier, + unsigned long event, void *ptr); +static struct bnxt_re_dev *bnxt_re_from_netdev(struct net_device *netdev); +static void bnxt_re_dev_uninit(struct bnxt_re_dev *rdev); static void bnxt_re_set_drv_mode(struct bnxt_re_dev *rdev, u8 mode) { @@ -233,7 +234,6 @@ static void bnxt_re_stop(void *p) if (!rdev) return; - ASSERT_RTNL(); /* L2 driver invokes this callback during device error/crash or device * reset. Current RoCE driver doesn't recover the device in case of @@ -269,9 +269,6 @@ static void bnxt_re_sriov_config(void *p, int num_vfs) { struct bnxt_re_dev *rdev = p; - if (!rdev) - return; - if (test_bit(BNXT_RE_FLAG_ERR_DEVICE_DETACHED, &rdev->flags)) return; rdev->num_vfs = num_vfs; @@ -282,16 +279,14 @@ static void bnxt_re_sriov_config(void *p, int num_vfs) } } -static void bnxt_re_shutdown(void *p) +static void bnxt_re_shutdown(struct auxiliary_device *adev) { - struct bnxt_re_dev *rdev = p; + struct bnxt_re_dev *rdev = auxiliary_get_drvdata(adev); if (!rdev) return; - ASSERT_RTNL(); - /* Release the MSIx vectors before queuing unregister */ - bnxt_re_stop_irq(rdev); - ib_unregister_device_queued(&rdev->ibdev); + ib_unregister_device(&rdev->ibdev); + bnxt_re_dev_uninit(rdev); } static void bnxt_re_stop_irq(void *handle) @@ -346,11 +341,9 @@ static void bnxt_re_start_irq(void *handle, struct bnxt_msix_entry *ent) } static struct bnxt_ulp_ops bnxt_re_ulp_ops = { - .ulp_async_notifier = NULL, .ulp_stop = bnxt_re_stop, .ulp_start = bnxt_re_start, .ulp_sriov_config = bnxt_re_sriov_config, - .ulp_shutdown = bnxt_re_shutdown, .ulp_irq_stop = bnxt_re_stop_irq, .ulp_irq_restart = bnxt_re_start_irq }; @@ -380,9 +373,6 @@ static int bnxt_re_register_netdev(struct bnxt_re_dev *rdev) struct bnxt_en_dev *en_dev; int rc = 0; - if (!rdev) - return -EINVAL; - en_dev = rdev->en_dev; rc = en_dev->en_ops->bnxt_register_device(en_dev, BNXT_ROCE_ULP, @@ -401,7 +391,6 @@ static int bnxt_re_free_msix(struct bnxt_re_dev *rdev) en_dev = rdev->en_dev; - rc = en_dev->en_ops->bnxt_free_msix(rdev->en_dev, BNXT_ROCE_ULP); return rc; @@ -412,9 +401,6 @@ static int bnxt_re_request_msix(struct bnxt_re_dev *rdev) int rc = 0, num_msix_want = BNXT_RE_MAX_MSIX, num_msix_got; struct bnxt_en_dev *en_dev; - if (!rdev) - return -EINVAL; - en_dev = rdev->en_dev; num_msix_want = min_t(u32, BNXT_RE_MAX_MSIX, num_online_cpus()); @@ -458,12 +444,17 @@ static void bnxt_re_fill_fw_msg(struct bnxt_fw_msg *fw_msg, void *msg, static int bnxt_re_net_ring_free(struct bnxt_re_dev *rdev, u16 fw_ring_id, int type) { - struct bnxt_en_dev *en_dev = rdev->en_dev; + struct bnxt_en_dev *en_dev; struct hwrm_ring_free_input req = {0}; struct hwrm_ring_free_output resp; struct bnxt_fw_msg fw_msg; int rc = -EINVAL; + if (!rdev) + return rc; + + en_dev = rdev->en_dev; + if (!en_dev) return rc; @@ -584,21 +575,6 @@ static int bnxt_re_net_stats_ctx_alloc(struct bnxt_re_dev *rdev, /* Device */ -static bool is_bnxt_re_dev(struct net_device *netdev) -{ - struct ethtool_drvinfo drvinfo; - - if (netdev->ethtool_ops && netdev->ethtool_ops->get_drvinfo) { - memset(&drvinfo, 0, sizeof(drvinfo)); - netdev->ethtool_ops->get_drvinfo(netdev, &drvinfo); - - if (strcmp(drvinfo.driver, "bnxt_en")) - return false; - return true; - } - return false; -} - static struct bnxt_re_dev *bnxt_re_from_netdev(struct net_device *netdev) { struct ib_device *ibdev = @@ -609,31 +585,6 @@ static struct bnxt_re_dev *bnxt_re_from_netdev(struct net_device *netdev) return container_of(ibdev, struct bnxt_re_dev, ibdev); } -static struct bnxt_en_dev *bnxt_re_dev_probe(struct net_device *netdev) -{ - struct bnxt_en_dev *en_dev; - struct pci_dev *pdev; - - en_dev = bnxt_ulp_probe(netdev); - if (IS_ERR(en_dev)) - return en_dev; - - pdev = en_dev->pdev; - if (!pdev) - return ERR_PTR(-EINVAL); - - if (!(en_dev->flags & BNXT_EN_FLAG_ROCE_CAP)) { - dev_info(&pdev->dev, - "%s: probe error: RoCE is not supported on this device", - ROCE_DRV_MODULE_NAME); - return ERR_PTR(-ENODEV); - } - - dev_hold(netdev); - - return en_dev; -} - static ssize_t hw_rev_show(struct device *device, struct device_attribute *attr, char *buf) { @@ -679,7 +630,6 @@ static const struct ib_device_ops bnxt_re_dev_ops = { .create_qp = bnxt_re_create_qp, .create_srq = bnxt_re_create_srq, .create_user_ah = bnxt_re_create_ah, - .dealloc_driver = bnxt_re_dealloc_driver, .dealloc_pd = bnxt_re_dealloc_pd, .dealloc_ucontext = bnxt_re_dealloc_ucontext, .del_gid = bnxt_re_del_gid, @@ -744,18 +694,7 @@ static int bnxt_re_register_ib(struct bnxt_re_dev *rdev) return ib_register_device(ibdev, "bnxt_re%d", &rdev->en_dev->pdev->dev); } -static void bnxt_re_dev_remove(struct bnxt_re_dev *rdev) -{ - dev_put(rdev->netdev); - rdev->netdev = NULL; - mutex_lock(&bnxt_re_dev_lock); - list_del_rcu(&rdev->list); - mutex_unlock(&bnxt_re_dev_lock); - - synchronize_rcu(); -} - -static struct bnxt_re_dev *bnxt_re_dev_add(struct net_device *netdev, +static struct bnxt_re_dev *bnxt_re_dev_add(struct bnxt_aux_dev *aux_dev, struct bnxt_en_dev *en_dev) { struct bnxt_re_dev *rdev; @@ -768,8 +707,8 @@ static struct bnxt_re_dev *bnxt_re_dev_add(struct net_device *netdev, return NULL; } /* Default values */ - rdev->netdev = netdev; - dev_hold(rdev->netdev); + rdev->nb.notifier_call = NULL; + rdev->netdev = en_dev->net; rdev->en_dev = en_dev; rdev->id = rdev->en_dev->pdev->devfn; INIT_LIST_HEAD(&rdev->qp_list); @@ -784,9 +723,6 @@ static struct bnxt_re_dev *bnxt_re_dev_add(struct net_device *netdev, rdev->cosq[0] = 0xFFFF; rdev->cosq[1] = 0xFFFF; - mutex_lock(&bnxt_re_dev_lock); - list_add_tail_rcu(&rdev->list, &bnxt_re_dev_list); - mutex_unlock(&bnxt_re_dev_lock); return rdev; } @@ -1323,7 +1259,7 @@ static int bnxt_re_ib_init(struct bnxt_re_dev *rdev) pr_err("Failed to register with IB: %#x\n", rc); return rc; } - dev_info(rdev_to_dev(rdev), "Device registered successfully"); + dev_info(rdev_to_dev(rdev), "Device registered with IB successfully"); ib_get_eth_speed(&rdev->ibdev, 1, &rdev->active_speed, &rdev->active_width); set_bit(BNXT_RE_FLAG_ISSUE_ROCE_STATS, &rdev->flags); @@ -1541,135 +1477,43 @@ static int bnxt_re_dev_init(struct bnxt_re_dev *rdev, u8 wqe_mode) return rc; } -static void bnxt_re_dev_unreg(struct bnxt_re_dev *rdev) -{ - struct net_device *netdev = rdev->netdev; - - bnxt_re_dev_remove(rdev); - - if (netdev) - dev_put(netdev); -} - -static int bnxt_re_dev_reg(struct bnxt_re_dev **rdev, struct net_device *netdev) +static int bnxt_re_add_device(struct auxiliary_device *adev, u8 wqe_mode) { + struct bnxt_aux_dev *aux_dev = + container_of(adev, struct bnxt_aux_dev, aux_dev); struct bnxt_en_dev *en_dev; + struct bnxt_re_dev *rdev; int rc = 0; - if (!is_bnxt_re_dev(netdev)) - return -ENODEV; + /* en_dev should never be NULL as long as adev and aux_dev are valid. */ + en_dev = aux_dev->edev; - en_dev = bnxt_re_dev_probe(netdev); - if (IS_ERR(en_dev)) { - if (en_dev != ERR_PTR(-ENODEV)) - ibdev_err(&(*rdev)->ibdev, "%s: Failed to probe\n", - ROCE_DRV_MODULE_NAME); - rc = PTR_ERR(en_dev); - goto exit; - } - *rdev = bnxt_re_dev_add(netdev, en_dev); - if (!*rdev) { + rdev = bnxt_re_dev_add(aux_dev, en_dev); + if (!rdev || !rdev_to_dev(rdev)) { rc = -ENOMEM; - dev_put(netdev); goto exit; } -exit: - return rc; -} - -static void bnxt_re_remove_device(struct bnxt_re_dev *rdev) -{ - bnxt_re_dev_uninit(rdev); - pci_dev_put(rdev->en_dev->pdev); - bnxt_re_dev_unreg(rdev); -} - -static int bnxt_re_add_device(struct bnxt_re_dev **rdev, - struct net_device *netdev, u8 wqe_mode) -{ - int rc; - rc = bnxt_re_dev_reg(rdev, netdev); - if (rc == -ENODEV) - return rc; - if (rc) { - pr_err("Failed to register with the device %s: %#x\n", - netdev->name, rc); - return rc; - } + rc = bnxt_re_dev_init(rdev, wqe_mode); + if (rc) + goto re_dev_dealloc; - pci_dev_get((*rdev)->en_dev->pdev); - rc = bnxt_re_dev_init(*rdev, wqe_mode); + rc = bnxt_re_ib_init(rdev); if (rc) { - pci_dev_put((*rdev)->en_dev->pdev); - bnxt_re_dev_unreg(*rdev); + pr_err("Failed to register with IB: %s", + aux_dev->aux_dev.name); + goto re_dev_uninit; } + auxiliary_set_drvdata(adev, rdev); - return rc; -} - -static void bnxt_re_dealloc_driver(struct ib_device *ib_dev) -{ - struct bnxt_re_dev *rdev = - container_of(ib_dev, struct bnxt_re_dev, ibdev); - - dev_info(rdev_to_dev(rdev), "Unregistering Device"); - - rtnl_lock(); - bnxt_re_remove_device(rdev); - rtnl_unlock(); -} - -/* Handle all deferred netevents tasks */ -static void bnxt_re_task(struct work_struct *work) -{ - struct bnxt_re_work *re_work; - struct bnxt_re_dev *rdev; - int rc = 0; - - re_work = container_of(work, struct bnxt_re_work, work); - rdev = re_work->rdev; - - if (re_work->event == NETDEV_REGISTER) { - rc = bnxt_re_ib_init(rdev); - if (rc) { - ibdev_err(&rdev->ibdev, - "Failed to register with IB: %#x", rc); - rtnl_lock(); - bnxt_re_remove_device(rdev); - rtnl_unlock(); - goto exit; - } - goto exit; - } - - if (!ib_device_try_get(&rdev->ibdev)) - goto exit; + return 0; - switch (re_work->event) { - case NETDEV_UP: - bnxt_re_dispatch_event(&rdev->ibdev, NULL, 1, - IB_EVENT_PORT_ACTIVE); - break; - case NETDEV_DOWN: - bnxt_re_dev_stop(rdev); - break; - case NETDEV_CHANGE: - if (!netif_carrier_ok(rdev->netdev)) - bnxt_re_dev_stop(rdev); - else if (netif_carrier_ok(rdev->netdev)) - bnxt_re_dispatch_event(&rdev->ibdev, NULL, 1, - IB_EVENT_PORT_ACTIVE); - ib_get_eth_speed(&rdev->ibdev, 1, &rdev->active_speed, - &rdev->active_width); - break; - default: - break; - } - ib_device_put(&rdev->ibdev); +re_dev_uninit: + bnxt_re_dev_uninit(rdev); +re_dev_dealloc: + ib_dealloc_device(&rdev->ibdev); exit: - put_device(&rdev->ibdev.dev); - kfree(re_work); + return rc; } /* @@ -1690,109 +1534,130 @@ static int bnxt_re_netdev_event(struct notifier_block *notifier, unsigned long event, void *ptr) { struct net_device *real_dev, *netdev = netdev_notifier_info_to_dev(ptr); - struct bnxt_re_work *re_work; struct bnxt_re_dev *rdev; - int rc = 0; - bool sch_work = false; - bool release = true; real_dev = rdma_vlan_dev_real_dev(netdev); if (!real_dev) real_dev = netdev; - rdev = bnxt_re_from_netdev(real_dev); - if (!rdev && event != NETDEV_REGISTER) - return NOTIFY_OK; - if (real_dev != netdev) goto exit; - switch (event) { - case NETDEV_REGISTER: - if (rdev) - break; - rc = bnxt_re_add_device(&rdev, real_dev, - BNXT_QPLIB_WQE_MODE_STATIC); - if (!rc) - sch_work = true; - release = false; - break; + rdev = bnxt_re_from_netdev(real_dev); + if (!rdev) + return NOTIFY_DONE; - case NETDEV_UNREGISTER: - ib_unregister_device_queued(&rdev->ibdev); - break; + switch (event) { + case NETDEV_UP: + case NETDEV_DOWN: + case NETDEV_CHANGE: + bnxt_re_dispatch_event(&rdev->ibdev, NULL, 1, + netif_carrier_ok(real_dev) ? + IB_EVENT_PORT_ACTIVE : + IB_EVENT_PORT_ERR); + break; default: - sch_work = true; break; } - if (sch_work) { - /* Allocate for the deferred task */ - re_work = kzalloc(sizeof(*re_work), GFP_KERNEL); - if (re_work) { - get_device(&rdev->ibdev.dev); - re_work->rdev = rdev; - re_work->event = event; - re_work->vlan_dev = (real_dev == netdev ? - NULL : netdev); - INIT_WORK(&re_work->work, bnxt_re_task); - queue_work(bnxt_re_wq, &re_work->work); - } - } - + ib_device_put(&rdev->ibdev); exit: - if (rdev && release) - ib_device_put(&rdev->ibdev); return NOTIFY_DONE; } -static struct notifier_block bnxt_re_netdev_notifier = { - .notifier_call = bnxt_re_netdev_event -}; +#define BNXT_ADEV_NAME "bnxt_en" -static int __init bnxt_re_mod_init(void) +static void bnxt_re_remove(struct auxiliary_device *adev) { - int rc = 0; + struct bnxt_re_dev *rdev = auxiliary_get_drvdata(adev); - pr_info("%s: %s", ROCE_DRV_MODULE_NAME, version); + if (!rdev) + return; - bnxt_re_wq = create_singlethread_workqueue("bnxt_re"); - if (!bnxt_re_wq) - return -ENOMEM; + mutex_lock(&bnxt_re_mutex); + if (rdev->nb.notifier_call) { + unregister_netdevice_notifier(&rdev->nb); + rdev->nb.notifier_call = NULL; + } else { + /* If notifier is null, we should have already done a + * clean up before coming here. + */ + goto skip_remove; + } + + ib_unregister_device(&rdev->ibdev); + ib_dealloc_device(&rdev->ibdev); + bnxt_re_dev_uninit(rdev); +skip_remove: + mutex_unlock(&bnxt_re_mutex); +} + +static int bnxt_re_probe(struct auxiliary_device *adev, + const struct auxiliary_device_id *id) +{ + struct bnxt_re_dev *rdev; + int rc; + + mutex_lock(&bnxt_re_mutex); + rc = bnxt_re_add_device(adev, BNXT_QPLIB_WQE_MODE_STATIC); + if (rc) { + mutex_unlock(&bnxt_re_mutex); + return rc; + } - INIT_LIST_HEAD(&bnxt_re_dev_list); + rdev = auxiliary_get_drvdata(adev); - rc = register_netdevice_notifier(&bnxt_re_netdev_notifier); + rdev->nb.notifier_call = bnxt_re_netdev_event; + rc = register_netdevice_notifier(&rdev->nb); if (rc) { + rdev->nb.notifier_call = NULL; pr_err("%s: Cannot register to netdevice_notifier", ROCE_DRV_MODULE_NAME); - goto err_netdev; + goto err; } + + mutex_unlock(&bnxt_re_mutex); return 0; -err_netdev: - destroy_workqueue(bnxt_re_wq); +err: + mutex_unlock(&bnxt_re_mutex); + bnxt_re_remove(adev); return rc; } -static void __exit bnxt_re_mod_exit(void) +static const struct auxiliary_device_id bnxt_re_id_table[] = { + { .name = BNXT_ADEV_NAME ".rdma", }, + {}, +}; + +MODULE_DEVICE_TABLE(auxiliary, bnxt_re_id_table); + +static struct auxiliary_driver bnxt_re_driver = { + .name = "rdma", + .probe = bnxt_re_probe, + .remove = bnxt_re_remove, + .shutdown = bnxt_re_shutdown, + .id_table = bnxt_re_id_table, +}; + +static int __init bnxt_re_mod_init(void) { - struct bnxt_re_dev *rdev; + int rc = 0; - unregister_netdevice_notifier(&bnxt_re_netdev_notifier); - if (bnxt_re_wq) - destroy_workqueue(bnxt_re_wq); - list_for_each_entry(rdev, &bnxt_re_dev_list, list) { - /* VF device removal should be called before the removal - * of PF device. Queue VFs unregister first, so that VFs - * shall be removed before the PF during the call of - * ib_unregister_driver. - */ - if (rdev->is_virtfn) - ib_unregister_device(&rdev->ibdev); + pr_info("%s: %s", ROCE_DRV_MODULE_NAME, version); + rc = auxiliary_driver_register(&bnxt_re_driver); + if (rc) { + pr_err("%s: Failed to register auxiliary driver\n", + ROCE_DRV_MODULE_NAME); + return rc; } - ib_unregister_driver(RDMA_DRIVER_BNXT_RE); + return 0; +} + +static void __exit bnxt_re_mod_exit(void) +{ + auxiliary_driver_unregister(&bnxt_re_driver); } module_init(bnxt_re_mod_init); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c index 7ffd61ec2683..1d2d30f97ed9 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c @@ -19,6 +19,7 @@ #include #include #include +#include #include "bnxt_hsi.h" #include "bnxt.h" @@ -78,8 +79,6 @@ static int bnxt_unregister_dev(struct bnxt_en_dev *edev, unsigned int ulp_id) if (ulp_id >= BNXT_MAX_ULP) return -EINVAL; - edev->flags |= BNXT_EN_FLAG_ULP_STOPPED; - ulp = &edev->ulp_tbl[ulp_id]; if (!rcu_access_pointer(ulp->ulp_ops)) { netdev_err(bp->dev, "ulp id %d not registered\n", ulp_id); @@ -595,29 +594,3 @@ int bnxt_rdma_aux_device_add(struct bnxt *bp) return 0; } - -struct bnxt_en_dev *bnxt_ulp_probe(struct net_device *dev) -{ - struct bnxt *bp = netdev_priv(dev); - struct bnxt_en_dev *edev; - - edev = bp->edev; - if (!edev) { - edev = kzalloc(sizeof(*edev), GFP_KERNEL); - if (!edev) - return ERR_PTR(-ENOMEM); - edev->en_ops = &bnxt_en_ops_tbl; - edev->net = dev; - edev->pdev = bp->pdev; - edev->l2_db_size = bp->db_size; - edev->l2_db_size_nc = bp->db_size; - bp->edev = edev; - } - edev->flags &= ~BNXT_EN_FLAG_ROCE_CAP; - if (bp->flags & BNXT_FLAG_ROCEV1_CAP) - edev->flags |= BNXT_EN_FLAG_ROCEV1_CAP; - if (bp->flags & BNXT_FLAG_ROCEV2_CAP) - edev->flags |= BNXT_EN_FLAG_ROCEV2_CAP; - return bp->edev; -} -EXPORT_SYMBOL(bnxt_ulp_probe); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h index 8b85c663d881..aaf55d847505 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h @@ -32,7 +32,6 @@ struct bnxt_ulp_ops { void (*ulp_stop)(void *); void (*ulp_start)(void *); void (*ulp_sriov_config)(void *, int); - void (*ulp_shutdown)(void *); void (*ulp_irq_stop)(void *); void (*ulp_irq_restart)(void *, struct bnxt_msix_entry *); }; @@ -111,6 +110,4 @@ int bnxt_rdma_aux_device_add(struct bnxt *bp); void bnxt_rdma_aux_device_uninit(struct bnxt_aux_dev *bnxt_adev); void bnxt_rdma_aux_device_init(struct bnxt *bp); void bnxt_aux_dev_free(struct bnxt *bp); -struct bnxt_en_dev *bnxt_ulp_probe(struct net_device *dev); - #endif From patchwork Wed Dec 7 17:53:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ajit Khaparde X-Patchwork-Id: 30970 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp330557wrr; Wed, 7 Dec 2022 10:04:56 -0800 (PST) X-Google-Smtp-Source: AA0mqf4gaGZYFR8LiyxoMLr2j4pRn7kdyUK/PzWKBew16WsjKyoQ3Em/p9H1VeRM9YkN7slnGGeW X-Received: by 2002:a63:7d1:0:b0:478:e7f8:faab with SMTP id 200-20020a6307d1000000b00478e7f8faabmr6246761pgh.390.1670436295718; Wed, 07 Dec 2022 10:04:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670436295; cv=none; d=google.com; s=arc-20160816; b=1H5XvNi2KtpKTzzfv5riR+RX9Pvj7cXmYU5r5w1W74gs37wvUkWnCbnvS/TB5Fj+2k iK4a6u4+nAkulK/7Qq0xPhUTSkfnwhtlJOrzHrtE2rk6fxerxsP8WvetluadOXRA++XB vHRTY4lkozhFpSVaTTUJR1ErwMFUYjTW3GA+lV7tNAqN3+FRhHBv9m2m4vXQK3xDW1ik 435+HvVTJGj5Puvjb4FNoBZ/1wOxl7XFN52ZCgTKhzxCerhJlB9YbIQOnZghDGAf7gfY vlpPg6JoaVBaAl2aWk+JJffuF5ZhCZj5f/OE9b8ANrL9odlT2qhotd+h/taP1ho/ROsN qOUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=9VL3Qhs20DgSf6LIRHTif4r/kPTgwWPj3NUbn0V3VnI=; b=GlolyDZ4rcPBX5OOOaofPxd5W168xmibgUScMRPyoz4ZkHCEYunDXVvx47sUt8Tw7N HH9U9zKbzFK8CSsuKTcf2YBtojfRjdktFi3o5KR4kRmE+yr4Spvcp8jFMl+QJIkJQ36y yVWVHVwetlUUGCSRul6GjOK84LaHTKkHPqJG1tKAjHLvGKFzy61V1778PFP034wHJnOe DIM5SE9J5ffBL9ce6fIFtk46oSSEwjYuIeMJHzaQjGpqDaW1UWuirG2poP6r0yvnuUNW XnM9o2Q1eRS3GwyfkRGaikuLmQS7ZIBfEKKgy5Q6iuz4SOXFC7PXUEqTQ49H8jaHa8uB tj/g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@broadcom.com header.s=google header.b="CX/bBxnS"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=broadcom.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l31-20020a63571f000000b00477b0e028e2si21581583pgb.810.2022.12.07.10.04.40; Wed, 07 Dec 2022 10:04:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@broadcom.com header.s=google header.b="CX/bBxnS"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=broadcom.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230169AbiLGRxu (ORCPT + 99 others); Wed, 7 Dec 2022 12:53:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56234 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230163AbiLGRx1 (ORCPT ); Wed, 7 Dec 2022 12:53:27 -0500 Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C77C14045F for ; Wed, 7 Dec 2022 09:53:21 -0800 (PST) Received: by mail-pj1-x102e.google.com with SMTP id b11so18368276pjp.2 for ; Wed, 07 Dec 2022 09:53:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=9VL3Qhs20DgSf6LIRHTif4r/kPTgwWPj3NUbn0V3VnI=; b=CX/bBxnSygQOw2KL/9zJi3Isar260GrUkGDuMtt2c9zSArDAbJL2tvBNKIp3hSX0NR Wfu4RTl10zHiEjEIiRQtn4PZe8AF4KAi8SuHSLRDDxtVxrkN7h4/8DSCq5tstNymdH/p HuDdbT9FShmt99dN+j9SmpWf5MJ+63UHFNKt0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9VL3Qhs20DgSf6LIRHTif4r/kPTgwWPj3NUbn0V3VnI=; b=lwRZWSPRUUKw5Cj1B68JKoi3xK0pRN8Ex6WUBzsDImBkMgUctbDFWwmS3XwVZ5tKaO QtPdtiaFW0Gx7GmeBk6cLHmpEyHQu8H7lM2UZNK1NyCZpYIVPmdGORZMJ8gUy5w+OZzp RhjGTe8qw3+MQsQO/4/scH9JlJB5Pxq/Mo4hoxAX8STc/H7pTVSeSN0KQs86DmGUw/Cr TFiEZ6xy/JdXnPCozwNzmw4S+BybBz89aBR2w7IIYAH7oCDG/ieB7bux4XL+kIMrbRUm dQCROW4VJSBVxJJf7CfdybguXvDOPrc1/IvKJaiQNsTxhrNlaZ8gENPPvpgG/xT3RY+4 9o2Q== X-Gm-Message-State: ANoB5pli1t7e4MULZJ7NQAhv6NX3EzMsRnbFvorTI+29BBsi5Vknh6pj rW+esYCAOVwxKSl/JUvGPqHY+A== X-Received: by 2002:a17:902:7fc6:b0:189:680d:f063 with SMTP id t6-20020a1709027fc600b00189680df063mr59422348plb.55.1670435601097; Wed, 07 Dec 2022 09:53:21 -0800 (PST) Received: from C02GC2QQMD6T.wifi.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id l6-20020a622506000000b005748aca80fesm13862242pfl.32.2022.12.07.09.53.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Dec 2022 09:53:20 -0800 (PST) From: Ajit Khaparde To: ajit.khaparde@broadcom.com Cc: andrew.gospodarek@broadcom.com, davem@davemloft.net, edumazet@google.com, jgg@ziepe.ca, kuba@kernel.org, leon@kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, michael.chan@broadcom.com, netdev@vger.kernel.org, pabeni@redhat.com, selvin.xavier@broadcom.com, Leon Romanovsky Subject: [PATCH v5 3/7] bnxt_en: Remove usage of ulp_id Date: Wed, 7 Dec 2022 09:53:06 -0800 Message-Id: <20221207175310.23656-4-ajit.khaparde@broadcom.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20221207175310.23656-1-ajit.khaparde@broadcom.com> References: <20221207175310.23656-1-ajit.khaparde@broadcom.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751579409644884019?= X-GMAIL-MSGID: =?utf-8?q?1751579409644884019?= Since the driver continues to use the single ULP model, the extra complexity and indirection is unnecessary. Remove the usage of ulp_id from the code. Suggested-by: Leon Romanovsky Signed-off-by: Ajit Khaparde Reviewed-by: Andy Gospodarek Reviewed-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/main.c | 24 +- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 2 +- drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 215 ++++++++---------- drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h | 26 +-- 4 files changed, 115 insertions(+), 152 deletions(-) diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index 165ae11b927d..814f1d2e31b5 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -363,8 +363,7 @@ static int bnxt_re_unregister_netdev(struct bnxt_re_dev *rdev) en_dev = rdev->en_dev; - rc = en_dev->en_ops->bnxt_unregister_device(rdev->en_dev, - BNXT_ROCE_ULP); + rc = en_dev->en_ops->bnxt_unregister_device(rdev->en_dev); return rc; } @@ -375,7 +374,7 @@ static int bnxt_re_register_netdev(struct bnxt_re_dev *rdev) en_dev = rdev->en_dev; - rc = en_dev->en_ops->bnxt_register_device(en_dev, BNXT_ROCE_ULP, + rc = en_dev->en_ops->bnxt_register_device(en_dev, &bnxt_re_ulp_ops, rdev); rdev->qplib_res.pdev = rdev->en_dev->pdev; return rc; @@ -384,16 +383,15 @@ static int bnxt_re_register_netdev(struct bnxt_re_dev *rdev) static int bnxt_re_free_msix(struct bnxt_re_dev *rdev) { struct bnxt_en_dev *en_dev; - int rc; if (!rdev) return -EINVAL; en_dev = rdev->en_dev; - rc = en_dev->en_ops->bnxt_free_msix(rdev->en_dev, BNXT_ROCE_ULP); + en_dev->en_ops->bnxt_free_msix(rdev->en_dev); - return rc; + return 0; } static int bnxt_re_request_msix(struct bnxt_re_dev *rdev) @@ -405,7 +403,7 @@ static int bnxt_re_request_msix(struct bnxt_re_dev *rdev) num_msix_want = min_t(u32, BNXT_RE_MAX_MSIX, num_online_cpus()); - num_msix_got = en_dev->en_ops->bnxt_request_msix(en_dev, BNXT_ROCE_ULP, + num_msix_got = en_dev->en_ops->bnxt_request_msix(en_dev, rdev->msix_entries, num_msix_want); if (num_msix_got < BNXT_RE_MIN_MSIX) { @@ -468,7 +466,7 @@ static int bnxt_re_net_ring_free(struct bnxt_re_dev *rdev, req.ring_id = cpu_to_le16(fw_ring_id); bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); - rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, BNXT_ROCE_ULP, &fw_msg); + rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, &fw_msg); if (rc) ibdev_err(&rdev->ibdev, "Failed to free HW ring:%d :%#x", req.ring_id, rc); @@ -505,7 +503,7 @@ static int bnxt_re_net_ring_alloc(struct bnxt_re_dev *rdev, req.int_mode = ring_attr->mode; bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); - rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, BNXT_ROCE_ULP, &fw_msg); + rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, &fw_msg); if (!rc) *fw_ring_id = le16_to_cpu(resp.ring_id); @@ -533,7 +531,7 @@ static int bnxt_re_net_stats_ctx_free(struct bnxt_re_dev *rdev, req.stat_ctx_id = cpu_to_le32(fw_stats_ctx_id); bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); - rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, BNXT_ROCE_ULP, &fw_msg); + rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, &fw_msg); if (rc) ibdev_err(&rdev->ibdev, "Failed to free HW stats context %#x", rc); @@ -566,7 +564,7 @@ static int bnxt_re_net_stats_ctx_alloc(struct bnxt_re_dev *rdev, req.stat_ctx_flags = STAT_CTX_ALLOC_REQ_STAT_CTX_FLAGS_ROCE; bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); - rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, BNXT_ROCE_ULP, &fw_msg); + rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, &fw_msg); if (!rc) *fw_stats_ctx_id = le32_to_cpu(resp.stat_ctx_id); @@ -1052,7 +1050,7 @@ static int bnxt_re_query_hwrm_pri2cos(struct bnxt_re_dev *rdev, u8 dir, bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); - rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, BNXT_ROCE_ULP, &fw_msg); + rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, &fw_msg); if (rc) return rc; @@ -1235,7 +1233,7 @@ static void bnxt_re_query_hwrm_intf_version(struct bnxt_re_dev *rdev) req.hwrm_intf_upd = HWRM_VERSION_UPDATE; bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); - rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, BNXT_ROCE_ULP, &fw_msg); + rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, &fw_msg); if (rc) { ibdev_err(&rdev->ibdev, "Failed to query HW version, rc = 0x%x", rc); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 2769d36b7d39..e8e32039d3a1 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -5533,7 +5533,7 @@ int bnxt_hwrm_vnic_cfg(struct bnxt *bp, u16 vnic_id) #endif if ((bp->flags & BNXT_FLAG_STRIP_VLAN) || def_vlan) req->flags |= cpu_to_le32(VNIC_CFG_REQ_FLAGS_VLAN_STRIP_MODE); - if (!vnic_id && bnxt_ulp_registered(bp->edev, BNXT_ROCE_ULP)) + if (!vnic_id && bnxt_ulp_registered(bp->edev)) req->flags |= cpu_to_le32(bnxt_get_roce_vnic_mode(bp)); return hwrm_req_send(bp, req); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c index 1d2d30f97ed9..69fb7d594f1d 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c @@ -28,64 +28,44 @@ static DEFINE_IDA(bnxt_aux_dev_ids); -static int bnxt_register_dev(struct bnxt_en_dev *edev, unsigned int ulp_id, - struct bnxt_ulp_ops *ulp_ops, void *handle) +static int bnxt_register_dev(struct bnxt_en_dev *edev, + struct bnxt_ulp_ops *ulp_ops, + void *handle) { struct net_device *dev = edev->net; struct bnxt *bp = netdev_priv(dev); + unsigned int max_stat_ctxs; struct bnxt_ulp *ulp; - int rc = 0; - if (ulp_id >= BNXT_MAX_ULP) - return -EINVAL; + max_stat_ctxs = bnxt_get_max_func_stat_ctxs(bp); + if (max_stat_ctxs <= BNXT_MIN_ROCE_STAT_CTXS || + bp->cp_nr_rings == max_stat_ctxs) + return -ENOMEM; - ulp = &edev->ulp_tbl[ulp_id]; - if (rcu_access_pointer(ulp->ulp_ops)) { - netdev_err(bp->dev, "ulp id %d already registered\n", ulp_id); - rc = -EBUSY; - goto exit; - } - if (ulp_id == BNXT_ROCE_ULP) { - unsigned int max_stat_ctxs; - - max_stat_ctxs = bnxt_get_max_func_stat_ctxs(bp); - if (max_stat_ctxs <= BNXT_MIN_ROCE_STAT_CTXS || - bp->cp_nr_rings == max_stat_ctxs) { - rc = -ENOMEM; - goto exit; - } - } + ulp = kzalloc(sizeof(*ulp), GFP_KERNEL); + if (!ulp) + return -ENOMEM; - atomic_set(&ulp->ref_count, 1); + edev->ulp_tbl = ulp; ulp->handle = handle; rcu_assign_pointer(ulp->ulp_ops, ulp_ops); - if (ulp_id == BNXT_ROCE_ULP) { - if (test_bit(BNXT_STATE_OPEN, &bp->state)) - bnxt_hwrm_vnic_cfg(bp, 0); - } + if (test_bit(BNXT_STATE_OPEN, &bp->state)) + bnxt_hwrm_vnic_cfg(bp, 0); -exit: - return rc; + return 0; } -static int bnxt_unregister_dev(struct bnxt_en_dev *edev, unsigned int ulp_id) +static int bnxt_unregister_dev(struct bnxt_en_dev *edev) { struct net_device *dev = edev->net; struct bnxt *bp = netdev_priv(dev); struct bnxt_ulp *ulp; int i = 0; - if (ulp_id >= BNXT_MAX_ULP) - return -EINVAL; - - ulp = &edev->ulp_tbl[ulp_id]; - if (!rcu_access_pointer(ulp->ulp_ops)) { - netdev_err(bp->dev, "ulp id %d not registered\n", ulp_id); - return -EINVAL; - } - if (ulp_id == BNXT_ROCE_ULP && ulp->msix_requested) - edev->en_ops->bnxt_free_msix(edev, ulp_id); + ulp = edev->ulp_tbl; + if (ulp->msix_requested) + edev->en_ops->bnxt_free_msix(edev); if (ulp->max_async_event_id) bnxt_hwrm_func_drv_rgtr(bp, NULL, 0, true); @@ -98,6 +78,8 @@ static int bnxt_unregister_dev(struct bnxt_en_dev *edev, unsigned int ulp_id) msleep(100); i++; } + kfree(ulp); + edev->ulp_tbl = NULL; return 0; } @@ -106,8 +88,8 @@ static void bnxt_fill_msix_vecs(struct bnxt *bp, struct bnxt_msix_entry *ent) struct bnxt_en_dev *edev = bp->edev; int num_msix, idx, i; - num_msix = edev->ulp_tbl[BNXT_ROCE_ULP].msix_requested; - idx = edev->ulp_tbl[BNXT_ROCE_ULP].msix_base; + num_msix = edev->ulp_tbl->msix_requested; + idx = edev->ulp_tbl->msix_base; for (i = 0; i < num_msix; i++) { ent[i].vector = bp->irq_tbl[idx + i].vector; ent[i].ring_idx = idx + i; @@ -121,8 +103,9 @@ static void bnxt_fill_msix_vecs(struct bnxt *bp, struct bnxt_msix_entry *ent) } } -static int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, unsigned int ulp_id, - struct bnxt_msix_entry *ent, int num_msix) +static int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, + struct bnxt_msix_entry *ent, + int num_msix) { struct net_device *dev = edev->net; struct bnxt *bp = netdev_priv(dev); @@ -132,13 +115,10 @@ static int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, unsigned int ulp_id, int total_vecs; int rc = 0; - if (ulp_id != BNXT_ROCE_ULP) - return -EINVAL; - if (!(bp->flags & BNXT_FLAG_USING_MSIX)) return -ENODEV; - if (edev->ulp_tbl[ulp_id].msix_requested) + if (edev->ulp_tbl->msix_requested) return -EAGAIN; max_cp_rings = bnxt_get_max_func_cp_rings(bp); @@ -155,8 +135,8 @@ static int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, unsigned int ulp_id, idx = max_idx - avail_msix; } - edev->ulp_tbl[ulp_id].msix_base = idx; - edev->ulp_tbl[ulp_id].msix_requested = avail_msix; + edev->ulp_tbl->msix_base = idx; + edev->ulp_tbl->msix_requested = avail_msix; hw_resc = &bp->hw_resc; total_vecs = idx + avail_msix; if (bp->total_irqs < total_vecs || @@ -171,7 +151,7 @@ static int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, unsigned int ulp_id, } } if (rc) { - edev->ulp_tbl[ulp_id].msix_requested = 0; + edev->ulp_tbl->msix_requested = 0; return -EAGAIN; } @@ -180,25 +160,22 @@ static int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, unsigned int ulp_id, resv_msix = hw_resc->resv_irqs - bp->cp_nr_rings; avail_msix = min_t(int, resv_msix, avail_msix); - edev->ulp_tbl[ulp_id].msix_requested = avail_msix; + edev->ulp_tbl->msix_requested = avail_msix; } bnxt_fill_msix_vecs(bp, ent); edev->flags |= BNXT_EN_FLAG_MSIX_REQUESTED; return avail_msix; } -static int bnxt_free_msix_vecs(struct bnxt_en_dev *edev, unsigned int ulp_id) +static void bnxt_free_msix_vecs(struct bnxt_en_dev *edev) { struct net_device *dev = edev->net; struct bnxt *bp = netdev_priv(dev); - if (ulp_id != BNXT_ROCE_ULP) - return -EINVAL; - if (!(edev->flags & BNXT_EN_FLAG_MSIX_REQUESTED)) - return 0; + return; - edev->ulp_tbl[ulp_id].msix_requested = 0; + edev->ulp_tbl->msix_requested = 0; edev->flags &= ~BNXT_EN_FLAG_MSIX_REQUESTED; if (netif_running(dev) && !(edev->flags & BNXT_EN_FLAG_ULP_STOPPED)) { rtnl_lock(); @@ -207,43 +184,43 @@ static int bnxt_free_msix_vecs(struct bnxt_en_dev *edev, unsigned int ulp_id) rtnl_unlock(); } - return 0; + return; } int bnxt_get_ulp_msix_num(struct bnxt *bp) { - if (bnxt_ulp_registered(bp->edev, BNXT_ROCE_ULP)) { + if (bnxt_ulp_registered(bp->edev)) { struct bnxt_en_dev *edev = bp->edev; - return edev->ulp_tbl[BNXT_ROCE_ULP].msix_requested; + return edev->ulp_tbl->msix_requested; } return 0; } int bnxt_get_ulp_msix_base(struct bnxt *bp) { - if (bnxt_ulp_registered(bp->edev, BNXT_ROCE_ULP)) { + if (bnxt_ulp_registered(bp->edev)) { struct bnxt_en_dev *edev = bp->edev; - if (edev->ulp_tbl[BNXT_ROCE_ULP].msix_requested) - return edev->ulp_tbl[BNXT_ROCE_ULP].msix_base; + if (edev->ulp_tbl->msix_requested) + return edev->ulp_tbl->msix_base; } return 0; } int bnxt_get_ulp_stat_ctxs(struct bnxt *bp) { - if (bnxt_ulp_registered(bp->edev, BNXT_ROCE_ULP)) { + if (bnxt_ulp_registered(bp->edev)) { struct bnxt_en_dev *edev = bp->edev; - if (edev->ulp_tbl[BNXT_ROCE_ULP].msix_requested) + if (edev->ulp_tbl->msix_requested) return BNXT_MIN_ROCE_STAT_CTXS; } return 0; } -static int bnxt_send_msg(struct bnxt_en_dev *edev, unsigned int ulp_id, +static int bnxt_send_msg(struct bnxt_en_dev *edev, struct bnxt_fw_msg *fw_msg) { struct net_device *dev = edev->net; @@ -253,7 +230,7 @@ static int bnxt_send_msg(struct bnxt_en_dev *edev, unsigned int ulp_id, u32 resp_len; int rc; - if (ulp_id != BNXT_ROCE_ULP && bp->fw_reset_state) + if (bp->fw_reset_state) return -EBUSY; rc = hwrm_req_init(bp, req, 0 /* don't care */); @@ -292,27 +269,24 @@ void bnxt_ulp_stop(struct bnxt *bp) { struct bnxt_en_dev *edev = bp->edev; struct bnxt_ulp_ops *ops; - int i; + struct bnxt_ulp *ulp; if (!edev) return; edev->flags |= BNXT_EN_FLAG_ULP_STOPPED; - for (i = 0; i < BNXT_MAX_ULP; i++) { - struct bnxt_ulp *ulp = &edev->ulp_tbl[i]; - - ops = rtnl_dereference(ulp->ulp_ops); - if (!ops || !ops->ulp_stop) - continue; - ops->ulp_stop(ulp->handle); - } + ulp = edev->ulp_tbl; + ops = rtnl_dereference(ulp->ulp_ops); + if (!ops || !ops->ulp_stop) + return; + ops->ulp_stop(ulp->handle); } void bnxt_ulp_start(struct bnxt *bp, int err) { struct bnxt_en_dev *edev = bp->edev; struct bnxt_ulp_ops *ops; - int i; + struct bnxt_ulp *ulp; if (!edev) return; @@ -322,39 +296,33 @@ void bnxt_ulp_start(struct bnxt *bp, int err) if (err) return; - for (i = 0; i < BNXT_MAX_ULP; i++) { - struct bnxt_ulp *ulp = &edev->ulp_tbl[i]; - - ops = rtnl_dereference(ulp->ulp_ops); - if (!ops || !ops->ulp_start) - continue; - ops->ulp_start(ulp->handle); - } + ulp = edev->ulp_tbl; + ops = rtnl_dereference(ulp->ulp_ops); + if (!ops || !ops->ulp_start) + return; + ops->ulp_start(ulp->handle); } void bnxt_ulp_sriov_cfg(struct bnxt *bp, int num_vfs) { struct bnxt_en_dev *edev = bp->edev; struct bnxt_ulp_ops *ops; - int i; + struct bnxt_ulp *ulp; if (!edev) return; + ulp = edev->ulp_tbl; - for (i = 0; i < BNXT_MAX_ULP; i++) { - struct bnxt_ulp *ulp = &edev->ulp_tbl[i]; - - rcu_read_lock(); - ops = rcu_dereference(ulp->ulp_ops); - if (!ops || !ops->ulp_sriov_config) { - rcu_read_unlock(); - continue; - } - bnxt_ulp_get(ulp); + rcu_read_lock(); + ops = rcu_dereference(ulp->ulp_ops); + if (!ops || !ops->ulp_sriov_config) { rcu_read_unlock(); - ops->ulp_sriov_config(ulp->handle, num_vfs); - bnxt_ulp_put(ulp); + return; } + bnxt_ulp_get(ulp); + rcu_read_unlock(); + ops->ulp_sriov_config(ulp->handle, num_vfs); + bnxt_ulp_put(ulp); } void bnxt_ulp_irq_stop(struct bnxt *bp) @@ -365,8 +333,8 @@ void bnxt_ulp_irq_stop(struct bnxt *bp) if (!edev || !(edev->flags & BNXT_EN_FLAG_MSIX_REQUESTED)) return; - if (bnxt_ulp_registered(bp->edev, BNXT_ROCE_ULP)) { - struct bnxt_ulp *ulp = &edev->ulp_tbl[BNXT_ROCE_ULP]; + if (bnxt_ulp_registered(bp->edev)) { + struct bnxt_ulp *ulp = edev->ulp_tbl; if (!ulp->msix_requested) return; @@ -386,8 +354,8 @@ void bnxt_ulp_irq_restart(struct bnxt *bp, int err) if (!edev || !(edev->flags & BNXT_EN_FLAG_MSIX_REQUESTED)) return; - if (bnxt_ulp_registered(bp->edev, BNXT_ROCE_ULP)) { - struct bnxt_ulp *ulp = &edev->ulp_tbl[BNXT_ROCE_ULP]; + if (bnxt_ulp_registered(bp->edev)) { + struct bnxt_ulp *ulp = edev->ulp_tbl; struct bnxt_msix_entry *ent = NULL; if (!ulp->msix_requested) @@ -414,41 +382,38 @@ void bnxt_ulp_async_events(struct bnxt *bp, struct hwrm_async_event_cmpl *cmpl) u16 event_id = le16_to_cpu(cmpl->event_id); struct bnxt_en_dev *edev = bp->edev; struct bnxt_ulp_ops *ops; - int i; + struct bnxt_ulp *ulp; - if (!edev) + if (!bnxt_ulp_registered(edev)) return; + ulp = edev->ulp_tbl; + rcu_read_lock(); - for (i = 0; i < BNXT_MAX_ULP; i++) { - struct bnxt_ulp *ulp = &edev->ulp_tbl[i]; - - ops = rcu_dereference(ulp->ulp_ops); - if (!ops || !ops->ulp_async_notifier) - continue; - if (!ulp->async_events_bmap || - event_id > ulp->max_async_event_id) - continue; - - /* Read max_async_event_id first before testing the bitmap. */ - smp_rmb(); - if (test_bit(event_id, ulp->async_events_bmap)) - ops->ulp_async_notifier(ulp->handle, cmpl); - } + + ops = rcu_dereference(ulp->ulp_ops); + if (!ops || !ops->ulp_async_notifier) + goto exit; + if (!ulp->async_events_bmap || event_id > ulp->max_async_event_id) + goto exit; + + /* Read max_async_event_id first before testing the bitmap. */ + smp_rmb(); + if (test_bit(event_id, ulp->async_events_bmap)) + ops->ulp_async_notifier(ulp->handle, cmpl); +exit: rcu_read_unlock(); } -static int bnxt_register_async_events(struct bnxt_en_dev *edev, unsigned int ulp_id, - unsigned long *events_bmap, u16 max_id) +static int bnxt_register_async_events(struct bnxt_en_dev *edev, + unsigned long *events_bmap, + u16 max_id) { struct net_device *dev = edev->net; struct bnxt *bp = netdev_priv(dev); struct bnxt_ulp *ulp; - if (ulp_id >= BNXT_MAX_ULP) - return -EINVAL; - - ulp = &edev->ulp_tbl[ulp_id]; + ulp = edev->ulp_tbl; ulp->async_events_bmap = events_bmap; /* Make sure bnxt_ulp_async_events() sees this order */ smp_wmb(); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h index aaf55d847505..7b2b829300b4 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h @@ -65,7 +65,7 @@ struct bnxt_en_dev { #define BNXT_EN_FLAG_MSIX_REQUESTED 0x4 #define BNXT_EN_FLAG_ULP_STOPPED 0x8 const struct bnxt_en_ops *en_ops; - struct bnxt_ulp ulp_tbl[BNXT_MAX_ULP]; + struct bnxt_ulp *ulp_tbl; int l2_db_size; /* Doorbell BAR size in * bytes mapped by L2 * driver. @@ -77,21 +77,21 @@ struct bnxt_en_dev { }; struct bnxt_en_ops { - int (*bnxt_register_device)(struct bnxt_en_dev *, unsigned int, - struct bnxt_ulp_ops *, void *); - int (*bnxt_unregister_device)(struct bnxt_en_dev *, unsigned int); - int (*bnxt_request_msix)(struct bnxt_en_dev *, unsigned int, - struct bnxt_msix_entry *, int); - int (*bnxt_free_msix)(struct bnxt_en_dev *, unsigned int); - int (*bnxt_send_fw_msg)(struct bnxt_en_dev *, unsigned int, - struct bnxt_fw_msg *); - int (*bnxt_register_fw_async_events)(struct bnxt_en_dev *, unsigned int, - unsigned long *, u16); + int (*bnxt_register_device)(struct bnxt_en_dev *edev, + struct bnxt_ulp_ops *ulp_ops, void *handle); + int (*bnxt_unregister_device)(struct bnxt_en_dev *edev); + int (*bnxt_request_msix)(struct bnxt_en_dev *edev, + struct bnxt_msix_entry *ent, int num_msix); + void (*bnxt_free_msix)(struct bnxt_en_dev *edev); + int (*bnxt_send_fw_msg)(struct bnxt_en_dev *edev, + struct bnxt_fw_msg *fw_msg); + int (*bnxt_register_fw_async_events)(struct bnxt_en_dev *edev, + unsigned long *events_bmap, u16 max_id); }; -static inline bool bnxt_ulp_registered(struct bnxt_en_dev *edev, int ulp_id) +static inline bool bnxt_ulp_registered(struct bnxt_en_dev *edev) { - if (edev && rcu_access_pointer(edev->ulp_tbl[ulp_id].ulp_ops)) + if (edev && edev->ulp_tbl) return true; return false; } From patchwork Wed Dec 7 17:53:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ajit Khaparde X-Patchwork-Id: 30971 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp331704wrr; Wed, 7 Dec 2022 10:06:55 -0800 (PST) X-Google-Smtp-Source: AA0mqf6hfrjMfbOwMc7inhvD57CTGwebSGkEFT11SaCMoVZzK9iFAgRU7b5uM6xdTP+AhcTJk9zn X-Received: by 2002:a17:90a:4d48:b0:219:4487:f763 with SMTP id l8-20020a17090a4d4800b002194487f763mr46339523pjh.201.1670436415427; Wed, 07 Dec 2022 10:06:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670436415; cv=none; d=google.com; s=arc-20160816; b=PliswQ8HHEccBLF+PrAKSo03suIrDMVmCUe1YctZxTpAhZLZcf2UDLVDAp3hG/1akL RfbVyHMLPtMCZlR7mWzILcPVcHIgzSI63Wz1otcLi34hq61mZQQ52dgnS2kCFij4mhW2 CgGMsRtY9S5MK/9Z94bOmtLkV+/5pnkG0/I9xVc//r34y5msOhfRU7tVulvn/wXXLLbY YzWiTqjTxIryXgJaLkYvp+2579hYFmQp8LqUhTX0O7reRFAD6q9ImNbj/Ev1KLM43Dyp l+A1ogjXtXmn7iy9u+udICCZKik2JY252uR5ul/Rhjms/DDdHJ2CKDOEh/JAfa9hhZjM v5mw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=iij+9pIDYF3FpiFGUOe7X9O1UKl6KRzMwNUDrUBY1lI=; b=W6GgAWzc4GVqHrjCbN1Fi1iAg3apMoRifetuidFa3ZEDSOjBi5lkxZT+N52wGn9Xr6 RvEzMLTYvMyMSxqyOHOi9AC2lcBaEmgkVX4fV+kKrgLqmzFyw4jMP2KT9DII02Fr5lMM js00BHKiO7WwKVlyyCec9Co9VEunq3zb7Gapw4fa9bB9aVR3nHVgvqHPy9PDU/aFzPaz a84iAbZwpyR4z/pe/GBE2kaPixJJvfRs63RtbifAlbc50YN2cPpQt5WlbHDVfyFn1Y1A jxfVooLFOoT4b3qdnJdSgN2nsWeiZWGFq4drHsK6eVb2Uoi+7s531bcVSwAH1tAR/Qd4 Rd7w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@broadcom.com header.s=google header.b=N8Pesd0A; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=broadcom.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id l31-20020a63571f000000b00477b0e028e2si21581583pgb.810.2022.12.07.10.06.41; Wed, 07 Dec 2022 10:06:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@broadcom.com header.s=google header.b=N8Pesd0A; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=broadcom.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230090AbiLGRx4 (ORCPT + 99 others); Wed, 7 Dec 2022 12:53:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229475AbiLGRxd (ORCPT ); Wed, 7 Dec 2022 12:53:33 -0500 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 783CE5132E for ; Wed, 7 Dec 2022 09:53:23 -0800 (PST) Received: by mail-pj1-x102f.google.com with SMTP id t17so18348330pjo.3 for ; Wed, 07 Dec 2022 09:53:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=iij+9pIDYF3FpiFGUOe7X9O1UKl6KRzMwNUDrUBY1lI=; b=N8Pesd0A+hMuNPg+wO1cnRr4fxCDmSwDRO9UbQIcNez5pnRzc7cWqSJ5aXveKyRhTg Vh0ezEcQawwEOeJ48Gpfp9EaH8PxReodHiuUyuuYD7PQ6ZOQFBYm+gYWld4neEUuqcLH m8xyerNIs9L6uK+GoOGyrUot1d0/8PHVPbhbc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=iij+9pIDYF3FpiFGUOe7X9O1UKl6KRzMwNUDrUBY1lI=; b=pLT7Fwy2A4KidujsgD1CutCLosI4H6z+CpeTl2kfAUHpKS25rjSFLcIZmREOT7h8kV PQ1l6YIxzsxRwdy7s3RMg1YS0XRV93KAKcoIOObn7E3uq73FwrmHhQLQFtW4DWsldza8 GBPjqyKh6Vt9Sbi5ZWSBr77tRcqxusqPTVsXznvVFQRbcY6GpJjOWquYXoQ99ztoM5SJ pyiLxE+BKLYRMhClL18q8MTRRhBOp2jzO5Ro18G+ghAQcgelKQPmAoKd4HIU2llTjnoA 1D1gm4mSQd/wZNt6a+E4UyF+vTGcXskZeKE0ceOhOgZzi05lFtdvFUOvSZym/kVnnHsP e9Zg== X-Gm-Message-State: ANoB5plfuiIcIyk2TsTvKINKj5jHfG2OnVqAWnERXSc0sJcbsfsGnDlE uZsvAm73TYbx49A+K4tWLomDdg== X-Received: by 2002:a17:902:db09:b0:186:def0:4886 with SMTP id m9-20020a170902db0900b00186def04886mr79521043plx.152.1670435602822; Wed, 07 Dec 2022 09:53:22 -0800 (PST) Received: from C02GC2QQMD6T.wifi.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id l6-20020a622506000000b005748aca80fesm13862242pfl.32.2022.12.07.09.53.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Dec 2022 09:53:22 -0800 (PST) From: Ajit Khaparde To: ajit.khaparde@broadcom.com Cc: andrew.gospodarek@broadcom.com, davem@davemloft.net, edumazet@google.com, jgg@ziepe.ca, kuba@kernel.org, leon@kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, michael.chan@broadcom.com, netdev@vger.kernel.org, pabeni@redhat.com, selvin.xavier@broadcom.com, Leon Romanovsky Subject: [PATCH v5 4/7] bnxt_en: Use direct API instead of indirection Date: Wed, 7 Dec 2022 09:53:07 -0800 Message-Id: <20221207175310.23656-5-ajit.khaparde@broadcom.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20221207175310.23656-1-ajit.khaparde@broadcom.com> References: <20221207175310.23656-1-ajit.khaparde@broadcom.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751579534711393752?= X-GMAIL-MSGID: =?utf-8?q?1751579534711393752?= For a single ULP user there is no need for complicating function indirection calls. Remove all this complexity in favour of direct function calls exported by the bnxt_en driver. This allows to simplify the code greatly. Suggested-by: Leon Romanovsky Signed-off-by: Ajit Khaparde Reviewed-by: Andy Gospodarek Reviewed-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/main.c | 71 +++++-------------- drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 48 ++++++------- drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h | 23 +++--- 3 files changed, 45 insertions(+), 97 deletions(-) diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index 814f1d2e31b5..e1777c8e6d5e 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -350,23 +350,6 @@ static struct bnxt_ulp_ops bnxt_re_ulp_ops = { /* RoCE -> Net driver */ -/* Driver registration routines used to let the networking driver (bnxt_en) - * to know that the RoCE driver is now installed - */ -static int bnxt_re_unregister_netdev(struct bnxt_re_dev *rdev) -{ - struct bnxt_en_dev *en_dev; - int rc; - - if (!rdev) - return -EINVAL; - - en_dev = rdev->en_dev; - - rc = en_dev->en_ops->bnxt_unregister_device(rdev->en_dev); - return rc; -} - static int bnxt_re_register_netdev(struct bnxt_re_dev *rdev) { struct bnxt_en_dev *en_dev; @@ -374,26 +357,12 @@ static int bnxt_re_register_netdev(struct bnxt_re_dev *rdev) en_dev = rdev->en_dev; - rc = en_dev->en_ops->bnxt_register_device(en_dev, - &bnxt_re_ulp_ops, rdev); - rdev->qplib_res.pdev = rdev->en_dev->pdev; + rc = bnxt_register_dev(en_dev, &bnxt_re_ulp_ops, rdev); + if (!rc) + rdev->qplib_res.pdev = rdev->en_dev->pdev; return rc; } -static int bnxt_re_free_msix(struct bnxt_re_dev *rdev) -{ - struct bnxt_en_dev *en_dev; - - if (!rdev) - return -EINVAL; - - en_dev = rdev->en_dev; - - en_dev->en_ops->bnxt_free_msix(rdev->en_dev); - - return 0; -} - static int bnxt_re_request_msix(struct bnxt_re_dev *rdev) { int rc = 0, num_msix_want = BNXT_RE_MAX_MSIX, num_msix_got; @@ -403,9 +372,9 @@ static int bnxt_re_request_msix(struct bnxt_re_dev *rdev) num_msix_want = min_t(u32, BNXT_RE_MAX_MSIX, num_online_cpus()); - num_msix_got = en_dev->en_ops->bnxt_request_msix(en_dev, - rdev->msix_entries, - num_msix_want); + num_msix_got = bnxt_req_msix_vecs(en_dev, + rdev->msix_entries, + num_msix_want); if (num_msix_got < BNXT_RE_MIN_MSIX) { rc = -EINVAL; goto done; @@ -466,7 +435,7 @@ static int bnxt_re_net_ring_free(struct bnxt_re_dev *rdev, req.ring_id = cpu_to_le16(fw_ring_id); bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); - rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, &fw_msg); + rc = bnxt_send_msg(en_dev, &fw_msg); if (rc) ibdev_err(&rdev->ibdev, "Failed to free HW ring:%d :%#x", req.ring_id, rc); @@ -503,7 +472,7 @@ static int bnxt_re_net_ring_alloc(struct bnxt_re_dev *rdev, req.int_mode = ring_attr->mode; bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); - rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, &fw_msg); + rc = bnxt_send_msg(en_dev, &fw_msg); if (!rc) *fw_ring_id = le16_to_cpu(resp.ring_id); @@ -531,7 +500,7 @@ static int bnxt_re_net_stats_ctx_free(struct bnxt_re_dev *rdev, req.stat_ctx_id = cpu_to_le32(fw_stats_ctx_id); bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); - rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, &fw_msg); + rc = bnxt_send_msg(en_dev, &fw_msg); if (rc) ibdev_err(&rdev->ibdev, "Failed to free HW stats context %#x", rc); @@ -564,7 +533,7 @@ static int bnxt_re_net_stats_ctx_alloc(struct bnxt_re_dev *rdev, req.stat_ctx_flags = STAT_CTX_ALLOC_REQ_STAT_CTX_FLAGS_ROCE; bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); - rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, &fw_msg); + rc = bnxt_send_msg(en_dev, &fw_msg); if (!rc) *fw_stats_ctx_id = le32_to_cpu(resp.stat_ctx_id); @@ -1050,7 +1019,7 @@ static int bnxt_re_query_hwrm_pri2cos(struct bnxt_re_dev *rdev, u8 dir, bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); - rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, &fw_msg); + rc = bnxt_send_msg(en_dev, &fw_msg); if (rc) return rc; @@ -1233,7 +1202,7 @@ static void bnxt_re_query_hwrm_intf_version(struct bnxt_re_dev *rdev) req.hwrm_intf_upd = HWRM_VERSION_UPDATE; bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); - rc = en_dev->en_ops->bnxt_send_fw_msg(en_dev, &fw_msg); + rc = bnxt_send_msg(en_dev, &fw_msg); if (rc) { ibdev_err(&rdev->ibdev, "Failed to query HW version, rc = 0x%x", rc); @@ -1296,20 +1265,12 @@ static void bnxt_re_dev_uninit(struct bnxt_re_dev *rdev) bnxt_re_net_ring_free(rdev, rdev->rcfw.creq.ring_id, type); bnxt_qplib_free_rcfw_channel(&rdev->rcfw); } - if (test_and_clear_bit(BNXT_RE_FLAG_GOT_MSIX, &rdev->flags)) { - rc = bnxt_re_free_msix(rdev); - if (rc) - ibdev_warn(&rdev->ibdev, - "Failed to free MSI-X vectors: %#x", rc); - } + if (test_and_clear_bit(BNXT_RE_FLAG_GOT_MSIX, &rdev->flags)) + bnxt_free_msix_vecs(rdev->en_dev); bnxt_re_destroy_chip_ctx(rdev); - if (test_and_clear_bit(BNXT_RE_FLAG_NETDEV_REGISTERED, &rdev->flags)) { - rc = bnxt_re_unregister_netdev(rdev); - if (rc) - ibdev_warn(&rdev->ibdev, - "Failed to unregister with netdev: %#x", rc); - } + if (test_and_clear_bit(BNXT_RE_FLAG_NETDEV_REGISTERED, &rdev->flags)) + bnxt_unregister_dev(rdev->en_dev); } /* worker thread for polling periodic events. Now used for QoS programming*/ diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c index 69fb7d594f1d..edc4d49502ab 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c @@ -28,9 +28,9 @@ static DEFINE_IDA(bnxt_aux_dev_ids); -static int bnxt_register_dev(struct bnxt_en_dev *edev, - struct bnxt_ulp_ops *ulp_ops, - void *handle) +int bnxt_register_dev(struct bnxt_en_dev *edev, + struct bnxt_ulp_ops *ulp_ops, + void *handle) { struct net_device *dev = edev->net; struct bnxt *bp = netdev_priv(dev); @@ -55,8 +55,9 @@ static int bnxt_register_dev(struct bnxt_en_dev *edev, return 0; } +EXPORT_SYMBOL(bnxt_register_dev); -static int bnxt_unregister_dev(struct bnxt_en_dev *edev) +void bnxt_unregister_dev(struct bnxt_en_dev *edev) { struct net_device *dev = edev->net; struct bnxt *bp = netdev_priv(dev); @@ -65,7 +66,7 @@ static int bnxt_unregister_dev(struct bnxt_en_dev *edev) ulp = edev->ulp_tbl; if (ulp->msix_requested) - edev->en_ops->bnxt_free_msix(edev); + bnxt_free_msix_vecs(edev); if (ulp->max_async_event_id) bnxt_hwrm_func_drv_rgtr(bp, NULL, 0, true); @@ -80,8 +81,9 @@ static int bnxt_unregister_dev(struct bnxt_en_dev *edev) } kfree(ulp); edev->ulp_tbl = NULL; - return 0; + return; } +EXPORT_SYMBOL(bnxt_unregister_dev); static void bnxt_fill_msix_vecs(struct bnxt *bp, struct bnxt_msix_entry *ent) { @@ -103,7 +105,7 @@ static void bnxt_fill_msix_vecs(struct bnxt *bp, struct bnxt_msix_entry *ent) } } -static int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, +int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, struct bnxt_msix_entry *ent, int num_msix) { @@ -166,8 +168,9 @@ static int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, edev->flags |= BNXT_EN_FLAG_MSIX_REQUESTED; return avail_msix; } +EXPORT_SYMBOL(bnxt_req_msix_vecs); -static void bnxt_free_msix_vecs(struct bnxt_en_dev *edev) +void bnxt_free_msix_vecs(struct bnxt_en_dev *edev) { struct net_device *dev = edev->net; struct bnxt *bp = netdev_priv(dev); @@ -186,6 +189,7 @@ static void bnxt_free_msix_vecs(struct bnxt_en_dev *edev) return; } +EXPORT_SYMBOL(bnxt_free_msix_vecs); int bnxt_get_ulp_msix_num(struct bnxt *bp) { @@ -220,7 +224,7 @@ int bnxt_get_ulp_stat_ctxs(struct bnxt *bp) return 0; } -static int bnxt_send_msg(struct bnxt_en_dev *edev, +int bnxt_send_msg(struct bnxt_en_dev *edev, struct bnxt_fw_msg *fw_msg) { struct net_device *dev = edev->net; @@ -254,6 +258,7 @@ static int bnxt_send_msg(struct bnxt_en_dev *edev, hwrm_req_drop(bp, req); return rc; } +EXPORT_SYMBOL(bnxt_send_msg); static void bnxt_ulp_get(struct bnxt_ulp *ulp) { @@ -313,14 +318,11 @@ void bnxt_ulp_sriov_cfg(struct bnxt *bp, int num_vfs) return; ulp = edev->ulp_tbl; - rcu_read_lock(); ops = rcu_dereference(ulp->ulp_ops); - if (!ops || !ops->ulp_sriov_config) { - rcu_read_unlock(); + if (!ops || !ops->ulp_sriov_config) return; - } + bnxt_ulp_get(ulp); - rcu_read_unlock(); ops->ulp_sriov_config(ulp->handle, num_vfs); bnxt_ulp_put(ulp); } @@ -405,9 +407,9 @@ void bnxt_ulp_async_events(struct bnxt *bp, struct hwrm_async_event_cmpl *cmpl) rcu_read_unlock(); } -static int bnxt_register_async_events(struct bnxt_en_dev *edev, - unsigned long *events_bmap, - u16 max_id) +int bnxt_register_async_events(struct bnxt_en_dev *edev, + unsigned long *events_bmap, + u16 max_id) { struct net_device *dev = edev->net; struct bnxt *bp = netdev_priv(dev); @@ -421,15 +423,7 @@ static int bnxt_register_async_events(struct bnxt_en_dev *edev, bnxt_hwrm_func_drv_rgtr(bp, events_bmap, max_id + 1, true); return 0; } - -static const struct bnxt_en_ops bnxt_en_ops_tbl = { - .bnxt_register_device = bnxt_register_dev, - .bnxt_unregister_device = bnxt_unregister_dev, - .bnxt_request_msix = bnxt_req_msix_vecs, - .bnxt_free_msix = bnxt_free_msix_vecs, - .bnxt_send_fw_msg = bnxt_send_msg, - .bnxt_register_fw_async_events = bnxt_register_async_events, -}; +EXPORT_SYMBOL(bnxt_register_async_events); void bnxt_aux_dev_free(struct bnxt *bp) { @@ -499,7 +493,6 @@ void bnxt_aux_dev_release(struct device *dev) container_of(dev, struct bnxt_aux_dev, aux_dev.dev); struct bnxt *bp = netdev_priv(bnxt_adev->edev->net); - bnxt_adev->edev->en_ops = NULL; kfree(bnxt_adev->edev); bnxt_adev->edev = NULL; bp->edev = NULL; @@ -507,7 +500,6 @@ void bnxt_aux_dev_release(struct device *dev) static inline void bnxt_set_edev_info(struct bnxt_en_dev *edev, struct bnxt *bp) { - edev->en_ops = &bnxt_en_ops_tbl; edev->net = bp->dev; edev->pdev = bp->pdev; edev->l2_db_size = bp->db_size; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h index 7b2b829300b4..b16dbf217dcd 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h @@ -64,7 +64,6 @@ struct bnxt_en_dev { BNXT_EN_FLAG_ROCEV2_CAP) #define BNXT_EN_FLAG_MSIX_REQUESTED 0x4 #define BNXT_EN_FLAG_ULP_STOPPED 0x8 - const struct bnxt_en_ops *en_ops; struct bnxt_ulp *ulp_tbl; int l2_db_size; /* Doorbell BAR size in * bytes mapped by L2 @@ -76,19 +75,6 @@ struct bnxt_en_dev { */ }; -struct bnxt_en_ops { - int (*bnxt_register_device)(struct bnxt_en_dev *edev, - struct bnxt_ulp_ops *ulp_ops, void *handle); - int (*bnxt_unregister_device)(struct bnxt_en_dev *edev); - int (*bnxt_request_msix)(struct bnxt_en_dev *edev, - struct bnxt_msix_entry *ent, int num_msix); - void (*bnxt_free_msix)(struct bnxt_en_dev *edev); - int (*bnxt_send_fw_msg)(struct bnxt_en_dev *edev, - struct bnxt_fw_msg *fw_msg); - int (*bnxt_register_fw_async_events)(struct bnxt_en_dev *edev, - unsigned long *events_bmap, u16 max_id); -}; - static inline bool bnxt_ulp_registered(struct bnxt_en_dev *edev) { if (edev && edev->ulp_tbl) @@ -110,4 +96,13 @@ int bnxt_rdma_aux_device_add(struct bnxt *bp); void bnxt_rdma_aux_device_uninit(struct bnxt_aux_dev *bnxt_adev); void bnxt_rdma_aux_device_init(struct bnxt *bp); void bnxt_aux_dev_free(struct bnxt *bp); +int bnxt_register_dev(struct bnxt_en_dev *edev, struct bnxt_ulp_ops *ulp_ops, + void *handle); +void bnxt_unregister_dev(struct bnxt_en_dev *edev); +int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, struct bnxt_msix_entry *ent, + int num_msix); +void bnxt_free_msix_vecs(struct bnxt_en_dev *edev); +int bnxt_send_msg(struct bnxt_en_dev *edev, struct bnxt_fw_msg *fw_msg); +int bnxt_register_async_events(struct bnxt_en_dev *edev, + unsigned long *events_bmap, u16 max_id); #endif From patchwork Wed Dec 7 17:53:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ajit Khaparde X-Patchwork-Id: 30972 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp332117wrr; Wed, 7 Dec 2022 10:07:37 -0800 (PST) X-Google-Smtp-Source: AA0mqf50HGhdHrBIldk0XJLwuWVdm9E9tcSLEyelTLyO7caEYFgdpaxtO4GXiZ1CREb+Le0ZWD6C X-Received: by 2002:a17:906:58d4:b0:7c1:2981:405 with SMTP id e20-20020a17090658d400b007c129810405mr434562ejs.180.1670436457713; Wed, 07 Dec 2022 10:07:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670436457; cv=none; d=google.com; s=arc-20160816; b=wCztZaEyW7FpmbukDM80n4Xg2AXM1JEirKiLqN/8w2p7dcQbDWOVTEgHw7jPJlxVSJ Xzx4TIUup7xsRNTaaMgmnDoUXe9ihzjpGn/BIRE+9GeTBM8tYgA9an0ADFF/OfncP2w+ ed1t9FfYhq1UCx8GJXiJ6iNie5aPw/0t/jSQnO6YJp7bwwbP+2qp9UFkACrjcV0L0BJL J2zqA/nXwtNa9SZqsJKyoZVKsDY1hgbeGVs6B+5erZm02fkzQ6JFPwDsAt9NDHyznEg8 IVkd79i+TmMQq0FyBfEIuRd3IFtEVpQyLwGtKy6Je7JG2+Be0f6dIYSTmSu00AEf5qvy 19Og== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=zVcPUCW6LwhNTSW75TuHtuQAx/LyretTAkaau2izCww=; b=E5zn9LWg92o4ORYLEqWdc9T/iGoGu70z2aU+iGp7J4wMxvdMUE+qEirrxtmqcVq7Bn fx/FKGlCFDrgcrpCOH9ZTfN0n1GVPRi5dP/a/w62ByHJN2aUxsKeXofc2izXbdeyNSQZ nooK/XaP3bm4HmkWr9iTeq/tadphY1LUywrDg6TgnB02zYA9X4lBlwLRI/WPrxgRy9FD Sr86tLtUi/+n7D+vX8NT902adYn8GtjbtAisTCVn9d/WrqTOZg5sCkACgHlok9golXeh VyP6Xk+7QEjMXYhN0gJ6ZXhZkpdt/pGhs42O9kGO8SVy8tgsE2NrjW2Q9j5wWl/8n8b/ VRSg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@broadcom.com header.s=google header.b=CjbYs8K0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=broadcom.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id gt13-20020a1709072d8d00b0078e1206d934si19320517ejc.162.2022.12.07.10.07.13; Wed, 07 Dec 2022 10:07:37 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@broadcom.com header.s=google header.b=CjbYs8K0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=broadcom.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229947AbiLGRyO (ORCPT + 99 others); Wed, 7 Dec 2022 12:54:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56796 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230186AbiLGRxs (ORCPT ); Wed, 7 Dec 2022 12:53:48 -0500 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DBD89554FC for ; Wed, 7 Dec 2022 09:53:26 -0800 (PST) Received: by mail-pl1-x633.google.com with SMTP id g10so17707612plo.11 for ; Wed, 07 Dec 2022 09:53:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=zVcPUCW6LwhNTSW75TuHtuQAx/LyretTAkaau2izCww=; b=CjbYs8K0vlIe679eX2ACCViVOfWd7nvVuPqHkfJ/jkXWua9PXDSdetbZ7tATiIhhhL QALkx+nQna0xgxdYBB5R6t0GxGNvm913BuIWYtLCNH+DTA7dQ44UXU2dHWqrkriqZeG5 iBJJWrn559S8g5G9cVCCr9twOi/ycTcJWirMU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=zVcPUCW6LwhNTSW75TuHtuQAx/LyretTAkaau2izCww=; b=JXP1oG8t038WIjYHZ5TisovpvKBTJTq4zzS9+oBTvJK+Nt26JUpPgrz2JH7Hk8AIAw oi86lQJ7mo+FNnP9b/QRo/ILeHGOeyeLAfaXoyWPCpgW5XQMov47143SZKfI3xJy6ugn DyaBMG3yGqtsRzfjeJb9JMBJSJP7Oz8psDqtSYlaS3JmP7WILVlSa0SwgRwbr5uemTXk Mq/DfqzP8bskrgipoL9oh32955+ZGgJpxys0WChg5eyxv1ViiTZkeEfz6ZWpAniKv1fS WMrN0OuOSOTYgQJdnD02+JVB3hG+p7oZPfr9yGnAaVNYo93h2nXFNYfCRjBR96ouHSPG nAKA== X-Gm-Message-State: ANoB5pmHytCM/PDrbkcNVMUwMi/b4n162BKJOhOPgut48x8IEiHVesrX 9Tnh5GVd+I42FC12lyx+1h+sQA== X-Received: by 2002:a17:902:7e08:b0:189:fae5:e955 with SMTP id b8-20020a1709027e0800b00189fae5e955mr2475519plm.107.1670435605919; Wed, 07 Dec 2022 09:53:25 -0800 (PST) Received: from C02GC2QQMD6T.wifi.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id l6-20020a622506000000b005748aca80fesm13862242pfl.32.2022.12.07.09.53.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Dec 2022 09:53:25 -0800 (PST) From: Ajit Khaparde To: ajit.khaparde@broadcom.com Cc: andrew.gospodarek@broadcom.com, davem@davemloft.net, edumazet@google.com, jgg@ziepe.ca, kuba@kernel.org, leon@kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, michael.chan@broadcom.com, netdev@vger.kernel.org, pabeni@redhat.com, selvin.xavier@broadcom.com Subject: [PATCH v5 5/7] bnxt_en: Use auxiliary bus calls over proprietary calls Date: Wed, 7 Dec 2022 09:53:08 -0800 Message-Id: <20221207175310.23656-6-ajit.khaparde@broadcom.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20221207175310.23656-1-ajit.khaparde@broadcom.com> References: <20221207175310.23656-1-ajit.khaparde@broadcom.com> MIME-Version: 1.0 X-Spam-Status: No, score=0.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SORTED_RECIPS,SPF_HELO_NONE,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751579579300864996?= X-GMAIL-MSGID: =?utf-8?q?1751579579300864996?= Wherever possible use the function ops provided by auxiliary bus instead of using proprietary ops. Defined bnxt_re_suspend and bnxt_re_resume calls which can be invoked by the bnxt_en driver instead of the ULP stop/start calls. Signed-off-by: Ajit Khaparde Reviewed-by: Andy Gospodarek Reviewed-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/main.c | 102 +++++++++++------- drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 40 ++++--- drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h | 2 - 3 files changed, 87 insertions(+), 57 deletions(-) diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index e1777c8e6d5e..ae04198c33b7 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -226,45 +226,6 @@ static void bnxt_re_set_resource_limits(struct bnxt_re_dev *rdev) bnxt_re_limit_vf_res(&rdev->qplib_ctx, num_vfs); } -/* for handling bnxt_en callbacks later */ -static void bnxt_re_stop(void *p) -{ - struct bnxt_re_dev *rdev = p; - struct bnxt *bp; - - if (!rdev) - return; - - /* L2 driver invokes this callback during device error/crash or device - * reset. Current RoCE driver doesn't recover the device in case of - * error. Handle the error by dispatching fatal events to all qps - * ie. by calling bnxt_re_dev_stop and release the MSIx vectors as - * L2 driver want to modify the MSIx table. - */ - bp = netdev_priv(rdev->netdev); - - ibdev_info(&rdev->ibdev, "Handle device stop call from L2 driver"); - /* Check the current device state from L2 structure and move the - * device to detached state if FW_FATAL_COND is set. - * This prevents more commands to HW during clean-up, - * in case the device is already in error. - */ - if (test_bit(BNXT_STATE_FW_FATAL_COND, &bp->state)) - set_bit(ERR_DEVICE_DETACHED, &rdev->rcfw.cmdq.flags); - - bnxt_re_dev_stop(rdev); - bnxt_re_stop_irq(rdev); - /* Move the device states to detached and avoid sending any more - * commands to HW - */ - set_bit(BNXT_RE_FLAG_ERR_DEVICE_DETACHED, &rdev->flags); - set_bit(ERR_DEVICE_DETACHED, &rdev->rcfw.cmdq.flags); -} - -static void bnxt_re_start(void *p) -{ -} - static void bnxt_re_sriov_config(void *p, int num_vfs) { struct bnxt_re_dev *rdev = p; @@ -341,8 +302,6 @@ static void bnxt_re_start_irq(void *handle, struct bnxt_msix_entry *ent) } static struct bnxt_ulp_ops bnxt_re_ulp_ops = { - .ulp_stop = bnxt_re_stop, - .ulp_start = bnxt_re_start, .ulp_sriov_config = bnxt_re_sriov_config, .ulp_irq_stop = bnxt_re_stop_irq, .ulp_irq_restart = bnxt_re_start_irq @@ -1585,6 +1544,65 @@ static int bnxt_re_probe(struct auxiliary_device *adev, return rc; } +static int bnxt_re_suspend(struct auxiliary_device *adev, pm_message_t state) +{ + struct bnxt_re_dev *rdev = auxiliary_get_drvdata(adev); + struct bnxt *bp; + + if (!rdev) + return 0; + + mutex_lock(&bnxt_re_mutex); + /* L2 driver may invoke this callback during device error/crash or device + * reset. Current RoCE driver doesn't recover the device in case of + * error. Handle the error by dispatching fatal events to all qps + * ie. by calling bnxt_re_dev_stop and release the MSIx vectors as + * L2 driver want to modify the MSIx table. + */ + bp = netdev_priv(rdev->netdev); + + ibdev_info(&rdev->ibdev, "Handle device suspend call"); + /* Check the current device state from L2 structure and move the + * device to detached state if FW_FATAL_COND is set. + * This prevents more commands to HW during clean-up, + * in case the device is already in error. + */ + if (test_bit(BNXT_STATE_FW_FATAL_COND, &bp->state)) + set_bit(ERR_DEVICE_DETACHED, &rdev->rcfw.cmdq.flags); + + bnxt_re_dev_stop(rdev); + bnxt_re_stop_irq(rdev); + /* Move the device states to detached and avoid sending any more + * commands to HW + */ + set_bit(BNXT_RE_FLAG_ERR_DEVICE_DETACHED, &rdev->flags); + set_bit(ERR_DEVICE_DETACHED, &rdev->rcfw.cmdq.flags); + mutex_unlock(&bnxt_re_mutex); + + return 0; +} + +static int bnxt_re_resume(struct auxiliary_device *adev) +{ + struct bnxt_re_dev *rdev = auxiliary_get_drvdata(adev); + + if (!rdev) + return 0; + + mutex_lock(&bnxt_re_mutex); + /* L2 driver may invoke this callback during device recovery, resume. + * reset. Current RoCE driver doesn't recover the device in case of + * error. Handle the error by dispatching fatal events to all qps + * ie. by calling bnxt_re_dev_stop and release the MSIx vectors as + * L2 driver want to modify the MSIx table. + */ + + ibdev_info(&rdev->ibdev, "Handle device resume call"); + mutex_unlock(&bnxt_re_mutex); + + return 0; +} + static const struct auxiliary_device_id bnxt_re_id_table[] = { { .name = BNXT_ADEV_NAME ".rdma", }, {}, @@ -1597,6 +1615,8 @@ static struct auxiliary_driver bnxt_re_driver = { .probe = bnxt_re_probe, .remove = bnxt_re_remove, .shutdown = bnxt_re_shutdown, + .suspend = bnxt_re_suspend, + .resume = bnxt_re_resume, .id_table = bnxt_re_id_table, }; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c index edc4d49502ab..8e11603387dd 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c @@ -272,26 +272,31 @@ static void bnxt_ulp_put(struct bnxt_ulp *ulp) void bnxt_ulp_stop(struct bnxt *bp) { + struct bnxt_aux_dev *bnxt_aux = bp->aux_dev; struct bnxt_en_dev *edev = bp->edev; - struct bnxt_ulp_ops *ops; - struct bnxt_ulp *ulp; if (!edev) return; edev->flags |= BNXT_EN_FLAG_ULP_STOPPED; - ulp = edev->ulp_tbl; - ops = rtnl_dereference(ulp->ulp_ops); - if (!ops || !ops->ulp_stop) - return; - ops->ulp_stop(ulp->handle); + if (bnxt_aux) { + struct auxiliary_device *adev; + + adev = &bnxt_aux->aux_dev; + if (adev->dev.driver) { + struct auxiliary_driver *adrv; + pm_message_t pm = {}; + + adrv = to_auxiliary_drv(adev->dev.driver); + adrv->suspend(adev, pm); + } + } } void bnxt_ulp_start(struct bnxt *bp, int err) { + struct bnxt_aux_dev *bnxt_aux = bp->aux_dev; struct bnxt_en_dev *edev = bp->edev; - struct bnxt_ulp_ops *ops; - struct bnxt_ulp *ulp; if (!edev) return; @@ -301,11 +306,18 @@ void bnxt_ulp_start(struct bnxt *bp, int err) if (err) return; - ulp = edev->ulp_tbl; - ops = rtnl_dereference(ulp->ulp_ops); - if (!ops || !ops->ulp_start) - return; - ops->ulp_start(ulp->handle); + if (bnxt_aux) { + struct auxiliary_device *adev; + + adev = &bnxt_aux->aux_dev; + if (adev->dev.driver) { + struct auxiliary_driver *adrv; + + adrv = to_auxiliary_drv(adev->dev.driver); + adrv->resume(adev); + } + } + } void bnxt_ulp_sriov_cfg(struct bnxt *bp, int num_vfs) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h index b16dbf217dcd..526afd1e06b2 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h @@ -29,8 +29,6 @@ struct bnxt_msix_entry { struct bnxt_ulp_ops { /* async_notifier() cannot sleep (in BH context) */ void (*ulp_async_notifier)(void *, struct hwrm_async_event_cmpl *); - void (*ulp_stop)(void *); - void (*ulp_start)(void *); void (*ulp_sriov_config)(void *, int); void (*ulp_irq_stop)(void *); void (*ulp_irq_restart)(void *, struct bnxt_msix_entry *); From patchwork Wed Dec 7 17:53:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ajit Khaparde X-Patchwork-Id: 30974 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp332895wrr; Wed, 7 Dec 2022 10:09:06 -0800 (PST) X-Google-Smtp-Source: AA0mqf4mj7xy3NhBgBsXeCsCYF4v1UKOMle++XW3TRjC2lXq3g72vHVI9xeo/62VXKXzeV/2Y9EN X-Received: by 2002:a17:906:d782:b0:7bc:9d14:a2db with SMTP id pj2-20020a170906d78200b007bc9d14a2dbmr45704877ejb.770.1670436546703; Wed, 07 Dec 2022 10:09:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670436546; cv=none; d=google.com; s=arc-20160816; b=XCQW0YcIX6t8nGao1ih6XncwR6y/PLDfMm2qJfTMmGGM0WWyqBm+P/xu9IApm11WgZ sPcn7XxoAZu7W9t9nPnrHwg+vZig59qJ5izzrAcwB8L9w9PaW037rQs/3ogugvyG7KUZ tLeV+FWIY/tcs5Sqbo53rw1rUQ6TL18sG+b/pc2244qZ70cxi6ah+emZrVLpfqYifZoX UJgrmeA0ZMQ20KIok+ZB0TvN6FGNFH1RTiY4p0S4pGHsAqvtNwUUXdOncneqxT7qbqrB XQ7TAQT4OfjcWkFfpVjM11NMhCo4N3mFCgFtSDvaXYRXYYDaLzcHZgIbtwl5u61FMUdf pwDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=771fb57CUKzXRlNeEnrFYSiwSB4SOkuaX8SqIgYEjDI=; b=plr5P1C4bhZlpOUy78CKw+MxALHLN4oOvbrtCr83uDVegOgkDbJCH15e1a2b2mvdbj AskxPuzxM8aPuvgeHqvRqFzwnPkvgAvKiJhV2ejhBPeQlNN2Ts0W2SWb/lx1Dp1/IG1B Oe+1JN3enjFkMjnv5Qut3kJFbnn7lYB5FGDOJTTR4kIFVSdXfhsPn+udjuHXa+mR1IvK MiAlIhze0FL2P2Bvi5vpvpNkdcpcaAgUWqVpKVKlmlv8VKyIszaSe8CEPzHtAGVx2Qam 3ID2ch0FiUbEPcYVtRFD+5bX3SuK/CIacX+gsYZOKIh18bSOG1OYfetMj5xmI2W6sUAN qW7Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@broadcom.com header.s=google header.b=A8pJmMp6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=broadcom.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id sd39-20020a1709076e2700b0078db6f488cesi17766416ejc.112.2022.12.07.10.08.42; Wed, 07 Dec 2022 10:09:06 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@broadcom.com header.s=google header.b=A8pJmMp6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=broadcom.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230201AbiLGRyR (ORCPT + 99 others); Wed, 7 Dec 2022 12:54:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56826 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230200AbiLGRxt (ORCPT ); Wed, 7 Dec 2022 12:53:49 -0500 Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 750405657D for ; Wed, 7 Dec 2022 09:53:29 -0800 (PST) Received: by mail-pf1-x42f.google.com with SMTP id g1so10086305pfk.2 for ; Wed, 07 Dec 2022 09:53:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=771fb57CUKzXRlNeEnrFYSiwSB4SOkuaX8SqIgYEjDI=; b=A8pJmMp6ZE1AMz3/lSUOLnustv1Tc952J5HrWaBrbPDuJxilYVC9CMB7RDJ5moHK9H mn9UXRZ1m71YS0LuMpy2XgwNRbENFIpzvInpjTS/Iqbpm9ykc/v9PEOhnrLrB6U+jkvW uR4JuS1iMLwRcYIc8Vh1sEIz8qreSq+JRFOkw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=771fb57CUKzXRlNeEnrFYSiwSB4SOkuaX8SqIgYEjDI=; b=bOnZSNYiINFqMvKiuLvbbGzOQD17QW77M2BOGT/kGMQXRjOVoI6B/dSbcH1ft6ABv7 DaGeHdcR1PGb55UY3lVXtEZ86XAihZWjQCzho8thAMo7oxT34xVDdA422nYeVCaIUWGs mhsrIYs2Ww8T/BAlGp4udeBcPCm+QG/S3O4osCRVNNbuX3IvLZPsn+hgJSAaHnd+fTzo 9BFHh/PCMvy/VSHoTICfUL/IoCirmykEu2VhOWbMkzmBDDfR3c5ZuQLySyjtYD0n4zEL 83bBasLDCnIw/QPMbSrS7YjO6msjwPkRbD4UbJFv7Ekcipctq/svGfJjO81IDjytzQLj 1yQw== X-Gm-Message-State: ANoB5pm0xik6+YeknQEML7FnyjNdgcmlByiYf+nbBs2uyzRfI+ynMNnZ exJgrXO2Mk2YLuMwp8an469tEQ== X-Received: by 2002:a63:c046:0:b0:477:b0d0:bbff with SMTP id z6-20020a63c046000000b00477b0d0bbffmr61935497pgi.364.1670435608568; Wed, 07 Dec 2022 09:53:28 -0800 (PST) Received: from C02GC2QQMD6T.wifi.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id l6-20020a622506000000b005748aca80fesm13862242pfl.32.2022.12.07.09.53.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Dec 2022 09:53:27 -0800 (PST) From: Ajit Khaparde To: ajit.khaparde@broadcom.com Cc: andrew.gospodarek@broadcom.com, davem@davemloft.net, edumazet@google.com, jgg@ziepe.ca, kuba@kernel.org, leon@kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, michael.chan@broadcom.com, netdev@vger.kernel.org, pabeni@redhat.com, selvin.xavier@broadcom.com, Hongguang Gao Subject: [PATCH v5 6/7] bnxt_en: Remove struct bnxt access from RoCE driver Date: Wed, 7 Dec 2022 09:53:09 -0800 Message-Id: <20221207175310.23656-7-ajit.khaparde@broadcom.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20221207175310.23656-1-ajit.khaparde@broadcom.com> References: <20221207175310.23656-1-ajit.khaparde@broadcom.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751579672584869326?= X-GMAIL-MSGID: =?utf-8?q?1751579672584869326?= From: Hongguang Gao Decouple RoCE driver from directly accessing L2's private bnxt structure. Move the fields needed by RoCE driver into bnxt_en_dev. They'll be passed to RoCE driver by bnxt_rdma_aux_device_add() function. Signed-off-by: Hongguang Gao Signed-off-by: Ajit Khaparde Reviewed-by: Andy Gospodarek Reviewed-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/main.c | 22 ++++++------------- drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 9 ++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h | 11 ++++++++++ 3 files changed, 27 insertions(+), 15 deletions(-) diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index ae04198c33b7..3251078e9fe3 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -112,16 +112,14 @@ static int bnxt_re_setup_chip_ctx(struct bnxt_re_dev *rdev, u8 wqe_mode) { struct bnxt_qplib_chip_ctx *chip_ctx; struct bnxt_en_dev *en_dev; - struct bnxt *bp; en_dev = rdev->en_dev; - bp = netdev_priv(en_dev->net); chip_ctx = kzalloc(sizeof(*chip_ctx), GFP_KERNEL); if (!chip_ctx) return -ENOMEM; - chip_ctx->chip_num = bp->chip_num; - chip_ctx->hw_stats_size = bp->hw_ring_stats_size; + chip_ctx->chip_num = en_dev->chip_num; + chip_ctx->hw_stats_size = en_dev->hw_ring_stats_size; rdev->chip_ctx = chip_ctx; /* rest members to follow eventually */ @@ -129,7 +127,7 @@ static int bnxt_re_setup_chip_ctx(struct bnxt_re_dev *rdev, u8 wqe_mode) rdev->qplib_res.cctx = rdev->chip_ctx; rdev->rcfw.res = &rdev->qplib_res; rdev->qplib_res.dattr = &rdev->dev_attr; - rdev->qplib_res.is_vf = BNXT_VF(bp); + rdev->qplib_res.is_vf = BNXT_EN_VF(en_dev); bnxt_re_set_drv_mode(rdev, wqe_mode); if (bnxt_qplib_determine_atomics(en_dev->pdev)) @@ -142,10 +140,7 @@ static int bnxt_re_setup_chip_ctx(struct bnxt_re_dev *rdev, u8 wqe_mode) static void bnxt_re_get_sriov_func_type(struct bnxt_re_dev *rdev) { - struct bnxt *bp; - - bp = netdev_priv(rdev->en_dev->net); - if (BNXT_VF(bp)) + if (BNXT_EN_VF(rdev->en_dev)) rdev->is_virtfn = 1; } @@ -957,7 +952,6 @@ static int bnxt_re_query_hwrm_pri2cos(struct bnxt_re_dev *rdev, u8 dir, u64 *cid_map) { struct hwrm_queue_pri2cos_qcfg_input req = {0}; - struct bnxt *bp = netdev_priv(rdev->netdev); struct hwrm_queue_pri2cos_qcfg_output resp; struct bnxt_en_dev *en_dev = rdev->en_dev; struct bnxt_fw_msg fw_msg; @@ -974,7 +968,7 @@ static int bnxt_re_query_hwrm_pri2cos(struct bnxt_re_dev *rdev, u8 dir, flags |= (dir & 0x01); flags |= HWRM_QUEUE_PRI2COS_QCFG_INPUT_FLAGS_IVLAN; req.flags = cpu_to_le32(flags); - req.port_id = bp->pf.port_id; + req.port_id = en_dev->pf_port_id; bnxt_re_fill_fw_msg(&fw_msg, (void *)&req, sizeof(req), (void *)&resp, sizeof(resp), DFLT_HWRM_CMD_TIMEOUT); @@ -1547,7 +1541,6 @@ static int bnxt_re_probe(struct auxiliary_device *adev, static int bnxt_re_suspend(struct auxiliary_device *adev, pm_message_t state) { struct bnxt_re_dev *rdev = auxiliary_get_drvdata(adev); - struct bnxt *bp; if (!rdev) return 0; @@ -1559,15 +1552,14 @@ static int bnxt_re_suspend(struct auxiliary_device *adev, pm_message_t state) * ie. by calling bnxt_re_dev_stop and release the MSIx vectors as * L2 driver want to modify the MSIx table. */ - bp = netdev_priv(rdev->netdev); ibdev_info(&rdev->ibdev, "Handle device suspend call"); - /* Check the current device state from L2 structure and move the + /* Check the current device state from bnxt_en_dev and move the * device to detached state if FW_FATAL_COND is set. * This prevents more commands to HW during clean-up, * in case the device is already in error. */ - if (test_bit(BNXT_STATE_FW_FATAL_COND, &bp->state)) + if (test_bit(BNXT_STATE_FW_FATAL_COND, &rdev->en_dev->en_state)) set_bit(ERR_DEVICE_DETACHED, &rdev->rcfw.cmdq.flags); bnxt_re_dev_stop(rdev); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c index 8e11603387dd..ca612a8f38c2 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c @@ -288,6 +288,7 @@ void bnxt_ulp_stop(struct bnxt *bp) pm_message_t pm = {}; adrv = to_auxiliary_drv(adev->dev.driver); + edev->en_state = bp->state; adrv->suspend(adev, pm); } } @@ -314,6 +315,7 @@ void bnxt_ulp_start(struct bnxt *bp, int err) struct auxiliary_driver *adrv; adrv = to_auxiliary_drv(adev->dev.driver); + edev->en_state = bp->state; adrv->resume(adev); } } @@ -521,6 +523,13 @@ static inline void bnxt_set_edev_info(struct bnxt_en_dev *edev, struct bnxt *bp) edev->flags |= BNXT_EN_FLAG_ROCEV1_CAP; if (bp->flags & BNXT_FLAG_ROCEV2_CAP) edev->flags |= BNXT_EN_FLAG_ROCEV2_CAP; + if (bp->flags & BNXT_FLAG_VF) + edev->flags |= BNXT_EN_FLAG_VF; + + edev->chip_num = bp->chip_num; + edev->hw_ring_stats_size = bp->hw_ring_stats_size; + edev->pf_port_id = bp->pf.port_id; + edev->en_state = bp->state; } int bnxt_rdma_aux_device_add(struct bnxt *bp) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h index 526afd1e06b2..ad0c63da11d0 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h @@ -62,6 +62,9 @@ struct bnxt_en_dev { BNXT_EN_FLAG_ROCEV2_CAP) #define BNXT_EN_FLAG_MSIX_REQUESTED 0x4 #define BNXT_EN_FLAG_ULP_STOPPED 0x8 + #define BNXT_EN_FLAG_VF 0x10 +#define BNXT_EN_VF(edev) ((edev)->flags & BNXT_EN_FLAG_VF) + struct bnxt_ulp *ulp_tbl; int l2_db_size; /* Doorbell BAR size in * bytes mapped by L2 @@ -71,6 +74,14 @@ struct bnxt_en_dev { * bytes mapped as non- * cacheable. */ + u16 chip_num; + u16 hw_ring_stats_size; + u16 pf_port_id; + unsigned long en_state; /* Could be checked in + * RoCE driver suspend + * mode only. Will be + * updated in resume. + */ }; static inline bool bnxt_ulp_registered(struct bnxt_en_dev *edev) From patchwork Wed Dec 7 17:53:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ajit Khaparde X-Patchwork-Id: 30975 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:f944:0:0:0:0:0 with SMTP id q4csp332925wrr; Wed, 7 Dec 2022 10:09:09 -0800 (PST) X-Google-Smtp-Source: AA0mqf6DL1fF04MaNkuAvF1K82R2YuH0VaF9kExIC3c+DN+daKnqZhE75WwtLcGhQOAlpR70zB5o X-Received: by 2002:a17:906:3096:b0:7ae:eae9:25a5 with SMTP id 22-20020a170906309600b007aeeae925a5mr77454737ejv.394.1670436549366; Wed, 07 Dec 2022 10:09:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670436549; cv=none; d=google.com; s=arc-20160816; b=yxr08LCF09apedTCI6/SbwpSQzROBfEqmIjZWAvZ497GKwrg1FZT6vqXXp7jY7G2ZP 488n7b7GbAzKehqK/DRikmjtz2X0fsXl0TLCJkXfxWEc8/HPk7mOYtUHVj0BtD5YWLL/ 2PRa2SDzdh8vfiK3UyvBCMXED4WzHlFdx7TX9ELOfxlsYjTnjv9tTNTiyK7x2okU66uf 9s3jTwYkY/cBrLFlIE39BjtlqTP1ojGcVL6vGoNXCAwdhL75hhGOD8pV5QLsEtyB6b2N M6hzIzImHfQMi6bJxQefiwNhzlEn/zXT5MDcItZg5TbutFe+cdpkb4HAbHR0TPrJja4S EC5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=3hiqdIA1I9YlcIQPcBlqppYeAtfPaDFb7PLfRxvaBzU=; b=G/9V9a9vguOuiOB0kYdgoLROFfnMv+VbBHDRueE3p6O1xUzaPsnVgWfHw5aCaUTy/B E0WZ8AVXVszzToqHqhxYFv2bU1T9IkypHMt+SIJVQs/RXDoehGsXHSwPLbqfz2jb1+ld flntZ/74OdXXYEPg6rzXp24Q841guqis0FP7M9spD7khfahW5/2i/jmo0WK717H4TTuY MEhsxWzl+gyIrynN1BchtX3+sSMkgIxkmGbCYZaEZYarI962f/RZa8zBhrr0cRA2LH9/ p/QpEUv4wl+HAm9ymiFCjtGVvr4sduHUnPvMrV757PvbhS0e2nEIKVs4Easu1bh60BmX rPvQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@broadcom.com header.s=google header.b=VeutLNgO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=broadcom.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id eb10-20020a0564020d0a00b0046c77f0b6bcsi5129348edb.562.2022.12.07.10.08.45; Wed, 07 Dec 2022 10:09:08 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@broadcom.com header.s=google header.b=VeutLNgO; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=broadcom.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230032AbiLGRyi (ORCPT + 99 others); Wed, 7 Dec 2022 12:54:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56226 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230218AbiLGRxv (ORCPT ); Wed, 7 Dec 2022 12:53:51 -0500 Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 238D05CD0F for ; Wed, 7 Dec 2022 09:53:30 -0800 (PST) Received: by mail-pf1-x42b.google.com with SMTP id a14so14081070pfa.1 for ; Wed, 07 Dec 2022 09:53:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=3hiqdIA1I9YlcIQPcBlqppYeAtfPaDFb7PLfRxvaBzU=; b=VeutLNgObgpq4BvlX7uXlUBHnH8AHLVKwwyvIILwPbhULvoKh6atOV2HTKAd8SfOt+ eEZMxQLTHLebv7SGMHyUJO56PA7HnR19xieLAbyt5cBlbztEKX6oxmW4a2P0ulVyCdZB ywkhO1uyOWAqx3+PKrYsLp9nasJcqtHDROxMQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3hiqdIA1I9YlcIQPcBlqppYeAtfPaDFb7PLfRxvaBzU=; b=Aqn7HnM2tYWBfE1fuI6T58W2GGzl6zkhUr6GUQQMlG2ZqLrRlM6xIoYNaqtNhe0Mwe coOEQ/L3PK7uQsOB3RLgP13RcHUXyGOOju7zb4Kb0A8yvjS3Ozfp6pqK9GcUVOO4A8SW p180VjPQAldGiEMcobcGN2QXMSiS6MYCaRMep1TsbPNw8b34QQQwfT/9jyIN2m8F7oHU OdZkBheYF+S90IYS+R2TGgG+TneAwPWnCmHauAQlVge+x2Z+P/7JF/vSIudygpmEfSoW MIktlGwu8oVZSqFSpOFuQMyN+MLPVltF9K74dCG9jNtfsE0r8lifRb55ouPgrUoo1lga Vo2w== X-Gm-Message-State: ANoB5pk+m/ZYMAnQKTBSKm3Qr/oBgRcwp+0OHmkXyRiz9hiwahzvS9y8 PPmfBOK+7E/4Q67zkRLthjPCxw== X-Received: by 2002:a63:4944:0:b0:46f:ec9f:dcb0 with SMTP id y4-20020a634944000000b0046fec9fdcb0mr67861828pgk.202.1670435610188; Wed, 07 Dec 2022 09:53:30 -0800 (PST) Received: from C02GC2QQMD6T.wifi.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id l6-20020a622506000000b005748aca80fesm13862242pfl.32.2022.12.07.09.53.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Dec 2022 09:53:29 -0800 (PST) From: Ajit Khaparde To: ajit.khaparde@broadcom.com Cc: andrew.gospodarek@broadcom.com, davem@davemloft.net, edumazet@google.com, jgg@ziepe.ca, kuba@kernel.org, leon@kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, michael.chan@broadcom.com, netdev@vger.kernel.org, pabeni@redhat.com, selvin.xavier@broadcom.com, Leon Romanovsky Subject: [PATCH v5 7/7] RDMA/bnxt_re: Remove the sriov config callback Date: Wed, 7 Dec 2022 09:53:10 -0800 Message-Id: <20221207175310.23656-8-ajit.khaparde@broadcom.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20221207175310.23656-1-ajit.khaparde@broadcom.com> References: <20221207175310.23656-1-ajit.khaparde@broadcom.com> MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1751579675046844247?= X-GMAIL-MSGID: =?utf-8?q?1751579675046844247?= Remove the SRIOV config callback which the bnxt_en was calling to reconfigure the chip resources for a PF device when VFs are created. The code is now modified to provision the VF resources based on the total VF count instead of the actual VF count. This allows the SRIOV config callback to be removed from the list of ulp_ops. Suggested-by: Leon Romanovsky Signed-off-by: Ajit Khaparde --- drivers/infiniband/hw/bnxt_re/main.c | 7 ++++- .../net/ethernet/broadcom/bnxt/bnxt_sriov.c | 7 +---- drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 29 ------------------- drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h | 1 - 4 files changed, 7 insertions(+), 37 deletions(-) diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index 3251078e9fe3..5811385e0439 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -297,7 +297,6 @@ static void bnxt_re_start_irq(void *handle, struct bnxt_msix_entry *ent) } static struct bnxt_ulp_ops bnxt_re_ulp_ops = { - .ulp_sriov_config = bnxt_re_sriov_config, .ulp_irq_stop = bnxt_re_stop_irq, .ulp_irq_restart = bnxt_re_start_irq }; @@ -1361,6 +1360,12 @@ static int bnxt_re_dev_init(struct bnxt_re_dev *rdev, u8 wqe_mode) set_bit(BNXT_RE_FLAG_RESOURCES_INITIALIZED, &rdev->flags); if (!rdev->is_virtfn) { + /* + * Use the total VF count since the actual VF count may not be + * available at this point. + */ + bnxt_re_sriov_config(rdev, + pci_sriov_get_totalvfs(rdev->en_dev->pdev)); rc = bnxt_re_setup_qos(rdev); if (rc) ibdev_info(&rdev->ibdev, diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c index a4cba7cb2783..3ed3a2b3b3a9 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c @@ -749,7 +749,6 @@ int bnxt_cfg_hw_sriov(struct bnxt *bp, int *num_vfs, bool reset) *num_vfs = rc; } - bnxt_ulp_sriov_cfg(bp, *num_vfs); return 0; } @@ -823,10 +822,8 @@ static int bnxt_sriov_enable(struct bnxt *bp, int *num_vfs) goto err_out2; rc = pci_enable_sriov(bp->pdev, *num_vfs); - if (rc) { - bnxt_ulp_sriov_cfg(bp, 0); + if (rc) goto err_out2; - } return 0; @@ -872,8 +869,6 @@ void bnxt_sriov_disable(struct bnxt *bp) rtnl_lock(); bnxt_restore_pf_fw_resources(bp); rtnl_unlock(); - - bnxt_ulp_sriov_cfg(bp, 0); } int bnxt_sriov_configure(struct pci_dev *pdev, int num_vfs) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c index ca612a8f38c2..a7c81cb950dc 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c @@ -260,16 +260,6 @@ int bnxt_send_msg(struct bnxt_en_dev *edev, } EXPORT_SYMBOL(bnxt_send_msg); -static void bnxt_ulp_get(struct bnxt_ulp *ulp) -{ - atomic_inc(&ulp->ref_count); -} - -static void bnxt_ulp_put(struct bnxt_ulp *ulp) -{ - atomic_dec(&ulp->ref_count); -} - void bnxt_ulp_stop(struct bnxt *bp) { struct bnxt_aux_dev *bnxt_aux = bp->aux_dev; @@ -322,25 +312,6 @@ void bnxt_ulp_start(struct bnxt *bp, int err) } -void bnxt_ulp_sriov_cfg(struct bnxt *bp, int num_vfs) -{ - struct bnxt_en_dev *edev = bp->edev; - struct bnxt_ulp_ops *ops; - struct bnxt_ulp *ulp; - - if (!edev) - return; - ulp = edev->ulp_tbl; - - ops = rcu_dereference(ulp->ulp_ops); - if (!ops || !ops->ulp_sriov_config) - return; - - bnxt_ulp_get(ulp); - ops->ulp_sriov_config(ulp->handle, num_vfs); - bnxt_ulp_put(ulp); -} - void bnxt_ulp_irq_stop(struct bnxt *bp) { struct bnxt_en_dev *edev = bp->edev; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h index ad0c63da11d0..fd8fd419602c 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h @@ -29,7 +29,6 @@ struct bnxt_msix_entry { struct bnxt_ulp_ops { /* async_notifier() cannot sleep (in BH context) */ void (*ulp_async_notifier)(void *, struct hwrm_async_event_cmpl *); - void (*ulp_sriov_config)(void *, int); void (*ulp_irq_stop)(void *); void (*ulp_irq_restart)(void *, struct bnxt_msix_entry *); };