From patchwork Fri Nov 4 16:27:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ajit Khaparde X-Patchwork-Id: 15673 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a5d:6687:0:0:0:0:0 with SMTP id l7csp508314wru; Fri, 4 Nov 2022 09:29:00 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5Pa/qdOFrhWf3r2rplFtPAatMuldCTBR3MS381pDN7VQCXyYP3/QTMzTX+ZsXdLNwMUSMH X-Received: by 2002:aa7:d710:0:b0:463:bd7b:2b44 with SMTP id t16-20020aa7d710000000b00463bd7b2b44mr20188765edq.385.1667579339795; Fri, 04 Nov 2022 09:28:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667579339; cv=none; d=google.com; s=arc-20160816; b=pLySbDWqtTu/ZUp8a8qaxQwbcliToW/nkUunbM7wirACtNLmLqjnwDoAQeDCtt7Nw8 c7VVD6Wp8CgM/YPinNomPTksDTmXjF9h+HotkDdxBA/Vu2ByaDNxwxuE0x+OZenqF4sK JXII3BypkBFHPjz2+LCmAAwgSCpBxLcFl9480lgZ/wScn7TfjGXENCRLJUdXu6Bi6YG5 Ze8NerMQ/l47ofMd/Dn/Sa/meckPOeaV1gmeHve/WIKO4VGIn7AOyjuVfPypkIaWIJOH cjl3YPpDwksfPJEwp0zBoikArnOsqhYq5kqCrhX0rFw1LiX7KRKnWH8wlidVGDDQXt0k s04A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=Y1Z1P1pDBb2+LZCzW/Bi2EtBA3AuPzBXMxd4YPHoUEk=; b=yMg5jQneRlRWRayHUNaNuvVPvet4AEjE+0666xhUJw5PrZq5Gy+ONjmmSd4S0kGPRE fCsUVcVg6Ng3nsNV3QAZhWtNsRQvdBJ33CPM51ZHZEXQxVcmmWsuo89Ql5AkbvS0DAuJ h8RG/Vum+JhU+0lZO5dJfR6ceIOXl2uIoa8HUgvKSLPPfL2CNtBtBGXrVA4ksR2u6sVK VJ5jON5uR8RPzlc8v3JfxWwT9ppmUSoe2lv2McOqbAszUCNsnQsBNLwfDYMfFjVbHenH cT8UKnhGPPfRq1jJNhBVqnD7WkpsLUv4zr91APTAX5fxJialhlSWtcGNkgcm7FyaVZll PEdg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@broadcom.com header.s=google header.b=GcjwwZHA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=broadcom.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id k24-20020a1709062a5800b0078dcdbb3e87si4630668eje.530.2022.11.04.09.28.34; Fri, 04 Nov 2022 09:28:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@broadcom.com header.s=google header.b=GcjwwZHA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=broadcom.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231583AbiKDQ16 (ORCPT + 99 others); Fri, 4 Nov 2022 12:27:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53836 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230508AbiKDQ1r (ORCPT ); Fri, 4 Nov 2022 12:27:47 -0400 Received: from mail-qv1-xf35.google.com (mail-qv1-xf35.google.com [IPv6:2607:f8b0:4864:20::f35]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E0EA04731F for ; Fri, 4 Nov 2022 09:27:45 -0700 (PDT) Received: by mail-qv1-xf35.google.com with SMTP id j6so3509050qvn.12 for ; Fri, 04 Nov 2022 09:27:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=Y1Z1P1pDBb2+LZCzW/Bi2EtBA3AuPzBXMxd4YPHoUEk=; b=GcjwwZHAw5Vhknbove+EUnn3OEVgVZnn9BTlsC1lhoQQtD3RDCKRL3hH45q66sphHZ EI5GU0QffZ2CsOnQb+c3hvf2DADb+8F4+TQSFqVVfW3SdwzL4LMb8/l8aF7sjHBT2cwI jl49ctYBe1zt4PdFCF8RnSlRCZ3+rOAUxTT84= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Y1Z1P1pDBb2+LZCzW/Bi2EtBA3AuPzBXMxd4YPHoUEk=; b=QPMV4XNoSr/OKoknLtUT+w4ruk7gPhuyW9wxjTu+mnjP3PyBasqYT1lFweydClam8x ADXyGlAQlNfcUlo6jBWZKorYk4s+E2+eyL/nexgOcDzcZjsMQ2+MJniSHZfpNBUWHItS slKSGswv3wZN6EpZcwu8gQW+waxeeAK5Deoq3zoN8lFLC0fFH78P1RKVXDVBAJMxRdwh UWj44OBuE9biFr3YqLjqCtdkFsMQq+TvpFGgDw7Plmb1q+YjfblBxLtx7UTGq414OEZK kwe+KMfVfHe7WUI9MEjlxCp1UKJDZoONkpNF5amyLcTBu0RWWFur8/ap+rqy5awNmBvX fe2A== X-Gm-Message-State: ACrzQf1tTq2Am7gwbZ0he3+QEEDD3q/7/U7u5OFaXiR8BvFAErD4hTTv K0GGsiUxb4SjTQ6UAsPAdWF7Pg== X-Received: by 2002:a05:6214:dab:b0:4bb:b0d2:da37 with SMTP id h11-20020a0562140dab00b004bbb0d2da37mr32149770qvh.24.1667579264882; Fri, 04 Nov 2022 09:27:44 -0700 (PDT) Received: from C02GC2QQMD6T.wifi.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id x17-20020a05620a449100b006fa31bf2f3dsm3290395qkp.47.2022.11.04.09.27.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Nov 2022 09:27:43 -0700 (PDT) From: Ajit Khaparde To: ajit.khaparde@broadcom.com Cc: andrew.gospodarek@broadcom.com, davem@davemloft.net, edumazet@google.com, jgg@ziepe.ca, kuba@kernel.org, leon@kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, michael.chan@broadcom.com, netdev@vger.kernel.org, pabeni@redhat.com, selvin.xavier@broadcom.com Subject: [PATCH v3 1/6] bnxt_en: Add auxiliary driver support Date: Fri, 4 Nov 2022 09:27:28 -0700 Message-Id: <20221104162733.73345-2-ajit.khaparde@broadcom.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20221104162733.73345-1-ajit.khaparde@broadcom.com> References: <20221104162733.73345-1-ajit.khaparde@broadcom.com> MIME-Version: 1.0 X-Spam-Status: No, score=-0.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SORTED_RECIPS,SPF_HELO_NONE,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1748583674099440669?= X-GMAIL-MSGID: =?utf-8?q?1748583674099440669?= Add auxiliary driver support. An auxiliary device will be created if the hardware indicates support for RDMA. The bnxt_ulp_probe() function has been removed and a new bnxt_rdma_aux_device_add() function has been added. The bnxt_free_msix_vecs() and bnxt_req_msix_vecs() will now hold the RTNL lock when they call the bnxt_close_nic()and bnxt_open_nic() since the device close and open need to be protected under RTNL lock. The operations between the bnxt_en and bnxt_re will be protected using the en_ops_lock. This will be used by the bnxt_re driver in a follow-on patch to create ROCE interfaces. Signed-off-by: Ajit Khaparde Reviewed-by: Andy Gospodarek --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 8 +- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 8 + drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c | 177 +++++++++++++++--- drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h | 7 +- 4 files changed, 169 insertions(+), 31 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 04cf7684f1b0..3cbedbecf3b3 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -13119,6 +13119,9 @@ static void bnxt_remove_one(struct pci_dev *pdev) struct net_device *dev = pci_get_drvdata(pdev); struct bnxt *bp = netdev_priv(dev); + bnxt_rdma_aux_device_uninit(bp->aux_dev); + bnxt_aux_dev_free(bp); + if (BNXT_PF(bp)) bnxt_sriov_disable(bp); @@ -13716,11 +13719,13 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) devlink_port_type_eth_set(&bp->dl_port, bp->dev); bnxt_dl_fw_reporters_create(bp); + bnxt_rdma_aux_device_init(bp); + bnxt_print_device_info(bp); pci_save_state(pdev); - return 0; + return 0; init_err_cleanup: bnxt_dl_unregister(bp); init_err_dl: @@ -13764,7 +13769,6 @@ static void bnxt_shutdown(struct pci_dev *pdev) if (netif_running(dev)) dev_close(dev); - bnxt_ulp_shutdown(bp); bnxt_clear_int_mode(bp); pci_disable_device(pdev); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index 91a1ba0a914d..284e39d5ae49 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -1622,6 +1623,12 @@ struct bnxt_fw_health { #define BNXT_FW_RETRY 5 #define BNXT_FW_IF_RETRY 10 +struct bnxt_aux_dev { + struct auxiliary_device aux_dev; + struct bnxt_en_dev *edev; + int id; +}; + enum board_idx { BCM57301, BCM57302, @@ -1843,6 +1850,7 @@ struct bnxt { #define BNXT_CHIP_P4_PLUS(bp) \ (BNXT_CHIP_P4(bp) || BNXT_CHIP_P5(bp)) + struct bnxt_aux_dev *aux_dev; struct bnxt_en_dev *edev; struct bnxt_napi **bnapi; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c index 2e54bf4fc7a7..7ffd61ec2683 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c @@ -25,32 +25,37 @@ #include "bnxt_hwrm.h" #include "bnxt_ulp.h" +static DEFINE_IDA(bnxt_aux_dev_ids); + static int bnxt_register_dev(struct bnxt_en_dev *edev, unsigned int ulp_id, struct bnxt_ulp_ops *ulp_ops, void *handle) { struct net_device *dev = edev->net; struct bnxt *bp = netdev_priv(dev); struct bnxt_ulp *ulp; + int rc = 0; - ASSERT_RTNL(); if (ulp_id >= BNXT_MAX_ULP) return -EINVAL; ulp = &edev->ulp_tbl[ulp_id]; if (rcu_access_pointer(ulp->ulp_ops)) { netdev_err(bp->dev, "ulp id %d already registered\n", ulp_id); - return -EBUSY; + rc = -EBUSY; + goto exit; } if (ulp_id == BNXT_ROCE_ULP) { unsigned int max_stat_ctxs; max_stat_ctxs = bnxt_get_max_func_stat_ctxs(bp); if (max_stat_ctxs <= BNXT_MIN_ROCE_STAT_CTXS || - bp->cp_nr_rings == max_stat_ctxs) - return -ENOMEM; + bp->cp_nr_rings == max_stat_ctxs) { + rc = -ENOMEM; + goto exit; + } } - atomic_set(&ulp->ref_count, 0); + atomic_set(&ulp->ref_count, 1); ulp->handle = handle; rcu_assign_pointer(ulp->ulp_ops, ulp_ops); @@ -59,7 +64,8 @@ static int bnxt_register_dev(struct bnxt_en_dev *edev, unsigned int ulp_id, bnxt_hwrm_vnic_cfg(bp, 0); } - return 0; +exit: + return rc; } static int bnxt_unregister_dev(struct bnxt_en_dev *edev, unsigned int ulp_id) @@ -69,10 +75,11 @@ static int bnxt_unregister_dev(struct bnxt_en_dev *edev, unsigned int ulp_id) struct bnxt_ulp *ulp; int i = 0; - ASSERT_RTNL(); if (ulp_id >= BNXT_MAX_ULP) return -EINVAL; + edev->flags |= BNXT_EN_FLAG_ULP_STOPPED; + ulp = &edev->ulp_tbl[ulp_id]; if (!rcu_access_pointer(ulp->ulp_ops)) { netdev_err(bp->dev, "ulp id %d not registered\n", ulp_id); @@ -126,7 +133,6 @@ static int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, unsigned int ulp_id, int total_vecs; int rc = 0; - ASSERT_RTNL(); if (ulp_id != BNXT_ROCE_ULP) return -EINVAL; @@ -149,6 +155,7 @@ static int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, unsigned int ulp_id, max_idx = min_t(int, bp->total_irqs, max_cp_rings); idx = max_idx - avail_msix; } + edev->ulp_tbl[ulp_id].msix_base = idx; edev->ulp_tbl[ulp_id].msix_requested = avail_msix; hw_resc = &bp->hw_resc; @@ -156,8 +163,10 @@ static int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, unsigned int ulp_id, if (bp->total_irqs < total_vecs || (BNXT_NEW_RM(bp) && hw_resc->resv_irqs < total_vecs)) { if (netif_running(dev)) { + rtnl_lock(); bnxt_close_nic(bp, true, false); rc = bnxt_open_nic(bp, true, false); + rtnl_unlock(); } else { rc = bnxt_reserve_rings(bp, true); } @@ -184,7 +193,6 @@ static int bnxt_free_msix_vecs(struct bnxt_en_dev *edev, unsigned int ulp_id) struct net_device *dev = edev->net; struct bnxt *bp = netdev_priv(dev); - ASSERT_RTNL(); if (ulp_id != BNXT_ROCE_ULP) return -EINVAL; @@ -194,9 +202,12 @@ static int bnxt_free_msix_vecs(struct bnxt_en_dev *edev, unsigned int ulp_id) edev->ulp_tbl[ulp_id].msix_requested = 0; edev->flags &= ~BNXT_EN_FLAG_MSIX_REQUESTED; if (netif_running(dev) && !(edev->flags & BNXT_EN_FLAG_ULP_STOPPED)) { + rtnl_lock(); bnxt_close_nic(bp, true, false); bnxt_open_nic(bp, true, false); + rtnl_unlock(); } + return 0; } @@ -347,25 +358,6 @@ void bnxt_ulp_sriov_cfg(struct bnxt *bp, int num_vfs) } } -void bnxt_ulp_shutdown(struct bnxt *bp) -{ - struct bnxt_en_dev *edev = bp->edev; - struct bnxt_ulp_ops *ops; - int i; - - if (!edev) - return; - - for (i = 0; i < BNXT_MAX_ULP; i++) { - struct bnxt_ulp *ulp = &edev->ulp_tbl[i]; - - ops = rtnl_dereference(ulp->ulp_ops); - if (!ops || !ops->ulp_shutdown) - continue; - ops->ulp_shutdown(ulp->handle); - } -} - void bnxt_ulp_irq_stop(struct bnxt *bp) { struct bnxt_en_dev *edev = bp->edev; @@ -475,6 +467,135 @@ static const struct bnxt_en_ops bnxt_en_ops_tbl = { .bnxt_register_fw_async_events = bnxt_register_async_events, }; +void bnxt_aux_dev_free(struct bnxt *bp) +{ + kfree(bp->aux_dev); + bp->aux_dev = NULL; +} + +static struct bnxt_aux_dev *bnxt_aux_dev_alloc(struct bnxt *bp) +{ + struct bnxt_aux_dev *bnxt_adev; + + bnxt_adev = kzalloc(sizeof(*bnxt_adev), GFP_KERNEL); + if (!bnxt_adev) + return ERR_PTR(-ENOMEM); + + return bnxt_adev; +} + +void bnxt_rdma_aux_device_uninit(struct bnxt_aux_dev *bnxt_adev) +{ + struct auxiliary_device *adev; + + if (IS_ERR_OR_NULL(bnxt_adev)) + return; + + adev = &bnxt_adev->aux_dev; + auxiliary_device_delete(adev); + auxiliary_device_uninit(adev); + if (bnxt_adev->id >= 0) + ida_free(&bnxt_aux_dev_ids, bnxt_adev->id); +} + +void bnxt_rdma_aux_device_init(struct bnxt *bp) +{ + struct net_device *dev = bp->dev; + int rc; + + if (bp->flags & BNXT_FLAG_ROCE_CAP) { + bp->aux_dev = bnxt_aux_dev_alloc(bp); + if (IS_ERR_OR_NULL(bp->aux_dev)) { + netdev_warn(dev, "Failed to init auxiliary device for ROCE\n"); + goto skip_aux_init; + } + + bp->aux_dev->id = ida_alloc(&bnxt_aux_dev_ids, GFP_KERNEL); + if (bp->aux_dev->id < 0) { + netdev_warn(dev, "ida alloc failed for ROCE auxiliary device\n"); + bnxt_aux_dev_free(bp); + goto skip_aux_init; + } + + /* If aux bus init fails, continue with netdev init. */ + rc = bnxt_rdma_aux_device_add(bp); + if (rc) { + netdev_warn(dev, "Failed to add auxiliary device for ROCE\n"); + ida_free(&bnxt_aux_dev_ids, bp->aux_dev->id); + bnxt_aux_dev_free(bp); + } + } +skip_aux_init: + return; +} + +void bnxt_aux_dev_release(struct device *dev) +{ + struct bnxt_aux_dev *bnxt_adev = + container_of(dev, struct bnxt_aux_dev, aux_dev.dev); + struct bnxt *bp = netdev_priv(bnxt_adev->edev->net); + + bnxt_adev->edev->en_ops = NULL; + kfree(bnxt_adev->edev); + bnxt_adev->edev = NULL; + bp->edev = NULL; +} + +static inline void bnxt_set_edev_info(struct bnxt_en_dev *edev, struct bnxt *bp) +{ + edev->en_ops = &bnxt_en_ops_tbl; + edev->net = bp->dev; + edev->pdev = bp->pdev; + edev->l2_db_size = bp->db_size; + edev->l2_db_size_nc = bp->db_size; + + if (bp->flags & BNXT_FLAG_ROCEV1_CAP) + edev->flags |= BNXT_EN_FLAG_ROCEV1_CAP; + if (bp->flags & BNXT_FLAG_ROCEV2_CAP) + edev->flags |= BNXT_EN_FLAG_ROCEV2_CAP; +} + +int bnxt_rdma_aux_device_add(struct bnxt *bp) +{ + struct bnxt_aux_dev *bnxt_adev = bp->aux_dev; + struct bnxt_en_dev *edev = bnxt_adev->edev; + struct auxiliary_device *aux_dev; + int ret; + + aux_dev = &bnxt_adev->aux_dev; + aux_dev->id = bnxt_adev->id; + aux_dev->name = "rdma"; + aux_dev->dev.parent = &bp->pdev->dev; + aux_dev->dev.release = bnxt_aux_dev_release; + + if (!edev) { + edev = kzalloc(sizeof(*edev), GFP_KERNEL); + if (!edev) + return -ENOMEM; + } + + bnxt_set_edev_info(edev, bp); + bnxt_adev->edev = edev; + bp->edev = edev; + + ret = auxiliary_device_init(aux_dev); + if (ret) { + kfree(edev); + kfree(bnxt_adev); + return ret; + } + + ret = auxiliary_device_add(aux_dev); + if (ret) { + kfree(edev); + kfree(bnxt_adev); + auxiliary_device_uninit(aux_dev); + return ret; + } + + return 0; +} + struct bnxt_en_dev *bnxt_ulp_probe(struct net_device *dev) { struct bnxt *bp = netdev_priv(dev); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h index 42b50abc3e91..8b85c663d881 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h @@ -17,6 +17,7 @@ #define BNXT_MIN_ROCE_STAT_CTXS 1 struct hwrm_async_event_cmpl; +struct bnxt_aux_dev; struct bnxt; struct bnxt_msix_entry { @@ -102,10 +103,14 @@ int bnxt_get_ulp_stat_ctxs(struct bnxt *bp); void bnxt_ulp_stop(struct bnxt *bp); void bnxt_ulp_start(struct bnxt *bp, int err); void bnxt_ulp_sriov_cfg(struct bnxt *bp, int num_vfs); -void bnxt_ulp_shutdown(struct bnxt *bp); void bnxt_ulp_irq_stop(struct bnxt *bp); void bnxt_ulp_irq_restart(struct bnxt *bp, int err); void bnxt_ulp_async_events(struct bnxt *bp, struct hwrm_async_event_cmpl *cmpl); +void bnxt_aux_dev_release(struct device *dev); +int bnxt_rdma_aux_device_add(struct bnxt *bp); +void bnxt_rdma_aux_device_uninit(struct bnxt_aux_dev *bnxt_adev); +void bnxt_rdma_aux_device_init(struct bnxt *bp); +void bnxt_aux_dev_free(struct bnxt *bp); struct bnxt_en_dev *bnxt_ulp_probe(struct net_device *dev); #endif