From patchwork Wed Jan 31 10:55:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Konstantin Taranov X-Patchwork-Id: 194672 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp1808186dyb; Wed, 31 Jan 2024 03:03:51 -0800 (PST) X-Google-Smtp-Source: AGHT+IFY+qh+bxifi4M+et4QqEhWx4cZkaPR47hDACeGBvNavv2UzPdmKkMTcMwVyIHfdzCCMkV+ X-Received: by 2002:aa7:9821:0:b0:6de:25b9:573b with SMTP id q1-20020aa79821000000b006de25b9573bmr1335658pfl.5.1706699030940; Wed, 31 Jan 2024 03:03:50 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706699030; cv=pass; d=google.com; s=arc-20160816; b=Gk9HzECtR+iG2HtWvHsCSXUNgfsaED2JUXCjHGfCQYxulC5sA/6xN0cPIzXstLY8Rb VmihosoER4tccaty/TQGrbl7LkNEFPpi9fVUMv9bHoSN8ih7/RCsDoXufp5KE8aS4IAK gFAeSGno5OekeLMYQUHOk4p9K4cbVvQTLBVmvCU+azHzCg4Wjoxd7FYprLRl0rkI3E/0 h74nLz32OYomNDTVZKRG2rDvAtMpU7qDQuOjgpVMoByRcDmnmfiKIcI9Mgbp81t4X4/7 CsbGa5QyaiI48wb/JRFswbm9FHhDRbDBBIFw+uz9tRLZTWhehIhNeSI7E0VG1cVAq6yS rpaA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-id:precedence:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-filter; bh=saDdL5sYNRP0RdrEfccKfGdzxJcVkUQ2VWnSyr2q/F4=; fh=dCJIycT8vWf+/757e31dDLkEJpLykzgP25n053FPmZc=; b=rNnuyfgDPCCPsPs4xaWnkOiyrdftE3VdbPG6xLh1Qx4eOJYREII2aqK8lUkHmK/E66 hyb0I0edrDNc/xgUg39eTqV+zPYZNYxRj1K5vEMNUrpfCMp7CHnnTV/sxEBgrUXsI8q9 2RShlzQ4HidWfM1XAS30VnBqUWpU/APZxRVKmptUItjpoVzDhVFM5nh13LvmAoU6Q8DR tVjprodw0M+/1nTiYhLVy/ENybamTGFPl4H9jw230RsMwVoGxG+iDo7Kzci3AgpueKc7 XsvEL44pCUR64dUgYCo4Aivha5MPCgEUGgR7vm0Nu2fKzEOnbAH702eRip2csOEukEG1 n8/Q==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=gNepmmLp; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-46275-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-46275-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com X-Forwarded-Encrypted: i=1; AJvYcCUFI+lc9FM2ClSUjkY6z8BIW9YElWXHv8wajasOWyxjmqcjQE0KA4CRyRjso1jQ6UqnYhmSMnw8mnACYJ5CdEc8dURgsA== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id 15-20020a63000f000000b005d8fdbbd5d7si2955836pga.606.2024.01.31.03.03.50 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 03:03:50 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-46275-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=gNepmmLp; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-46275-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-46275-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 9B331284865 for ; Wed, 31 Jan 2024 10:57:34 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7FC3378B77; Wed, 31 Jan 2024 10:56:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b="gNepmmLp" Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 063D669E07; Wed, 31 Jan 2024 10:55:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=13.77.154.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706698559; cv=none; b=GxojwrV6PbypJhb3zmdRK4UuI1H9YqYrNe0eSx/Wx0s8JhnJ9pnRlPP2CqEI8Jg1q4f8vh3sh6IgZoRbXgZJN9Wq3tEDT75e3ORJ/par+Hq+Xjg63ouExhay1QTaQ9s2WLQVwx+GcsulkPLTarSoJx9Syuxb6bUejK8cASqpOxc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706698559; c=relaxed/simple; bh=4ZmPsc+rDgX7rX3N3RoK8DZaj+dzW3V5EQi2cFZw9c4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=bVrNBZfi1uQCvvq3ggLbGPkwx1IRUfzzmqvR/N9RV+7yp9stIyUdEFlQVD/jnvXi7nfyAgD+aZsGmBPPyCwl/IBvZroTYihxIMJbjS6vm956y3DO0d0rA7ZXBlRUkealJIwwWFCtvotfYhkP+vrSGlpeD1Eb/jPOmUWL1dlpvXQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com; spf=pass smtp.mailfrom=linux.microsoft.com; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b=gNepmmLp; arc=none smtp.client-ip=13.77.154.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Received: from linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net (linux.microsoft.com [13.77.154.182]) by linux.microsoft.com (Postfix) with ESMTPSA id AAE0720B2001; Wed, 31 Jan 2024 02:55:57 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com AAE0720B2001 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1706698557; bh=saDdL5sYNRP0RdrEfccKfGdzxJcVkUQ2VWnSyr2q/F4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gNepmmLp+bdtSeHkAgPlZ4++Kj2eo3GdZt371DiIVOMy8sQ1+PqN8T2ayv+z3lwCZ wJnazxG3Z5n5Ciy28iJO2vQV406v9lcRpGeKmWBtopB1rYZ0P5mxtNZzYz58Dj/SA2 uNVMXBcu1KV4fa2RVkJ5852mtYi9wuwMcJbWwTWA= From: Konstantin Taranov To: kotaranov@microsoft.com, sharmaajay@microsoft.com, longli@microsoft.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH rdma-next v1 1/5] RDMA/mana_ib: Add EQ creation for rnic adapter Date: Wed, 31 Jan 2024 02:55:48 -0800 Message-Id: <1706698552-25383-2-git-send-email-kotaranov@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1706698552-25383-1-git-send-email-kotaranov@linux.microsoft.com> References: <1706698552-25383-1-git-send-email-kotaranov@linux.microsoft.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789603642852727919 X-GMAIL-MSGID: 1789603642852727919 This patch introduces functions for RNIC creation and creates one EQ for RNIC creation. Signed-off-by: Konstantin Taranov --- drivers/infiniband/hw/mana/device.c | 9 ++++-- drivers/infiniband/hw/mana/main.c | 53 ++++++++++++++++++++++++++++++++++++ drivers/infiniband/hw/mana/mana_ib.h | 5 ++++ 3 files changed, 64 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/hw/mana/device.c b/drivers/infiniband/hw/mana/device.c index 6fa902e..d8e8b10 100644 --- a/drivers/infiniband/hw/mana/device.c +++ b/drivers/infiniband/hw/mana/device.c @@ -92,15 +92,19 @@ static int mana_ib_probe(struct auxiliary_device *adev, goto deregister_device; } + mana_ib_gd_create_rnic_adapter(dev); + ret = ib_register_device(&dev->ib_dev, "mana_%d", mdev->gdma_context->dev); if (ret) - goto deregister_device; + goto destroy_rnic_adapter; dev_set_drvdata(&adev->dev, dev); return 0; +destroy_rnic_adapter: + mana_ib_gd_destroy_rnic_adapter(dev); deregister_device: mana_gd_deregister_device(dev->gdma_dev); free_ib_device: @@ -113,9 +117,8 @@ static void mana_ib_remove(struct auxiliary_device *adev) struct mana_ib_dev *dev = dev_get_drvdata(&adev->dev); ib_unregister_device(&dev->ib_dev); - + mana_ib_gd_destroy_rnic_adapter(dev); mana_gd_deregister_device(dev->gdma_dev); - ib_dealloc_device(&dev->ib_dev); } diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c index 29dd243..c64d569 100644 --- a/drivers/infiniband/hw/mana/main.c +++ b/drivers/infiniband/hw/mana/main.c @@ -548,3 +548,56 @@ int mana_ib_gd_query_adapter_caps(struct mana_ib_dev *dev) return 0; } + +static int mana_ib_create_eqs(struct mana_ib_dev *mdev) +{ + struct gdma_context *gc = mdev_to_gc(mdev); + struct gdma_queue_spec spec = {}; + int err; + + spec.type = GDMA_EQ; + spec.monitor_avl_buf = false; + spec.queue_size = EQ_SIZE; + spec.eq.callback = NULL; + spec.eq.context = mdev; + spec.eq.log2_throttle_limit = LOG2_EQ_THROTTLE; + spec.eq.msix_index = 0; + + err = mana_gd_create_mana_eq(&gc->mana_ib, &spec, &mdev->fatal_err_eq); + if (err) + return err; + + return 0; +} + +static void mana_ib_destroy_eqs(struct mana_ib_dev *mdev) +{ + if (!mdev->fatal_err_eq) + return; + + mana_gd_destroy_queue(mdev_to_gc(mdev), mdev->fatal_err_eq); + mdev->fatal_err_eq = NULL; +} + +void mana_ib_gd_create_rnic_adapter(struct mana_ib_dev *mdev) +{ + int err; + + err = mana_ib_create_eqs(mdev); + if (err) { + ibdev_err(&mdev->ib_dev, "Failed to create EQs for RNIC err %d", err); + goto cleanup; + } + + return; + +cleanup: + ibdev_warn(&mdev->ib_dev, + "RNIC is not available. Only RAW QPs are supported"); + mana_ib_destroy_eqs(mdev); +} + +void mana_ib_gd_destroy_rnic_adapter(struct mana_ib_dev *mdev) +{ + mana_ib_destroy_eqs(mdev); +} diff --git a/drivers/infiniband/hw/mana/mana_ib.h b/drivers/infiniband/hw/mana/mana_ib.h index 6a03ae6..a4b94ee 100644 --- a/drivers/infiniband/hw/mana/mana_ib.h +++ b/drivers/infiniband/hw/mana/mana_ib.h @@ -48,6 +48,7 @@ struct mana_ib_adapter_caps { struct mana_ib_dev { struct ib_device ib_dev; struct gdma_dev *gdma_dev; + struct gdma_queue *fatal_err_eq; struct mana_ib_adapter_caps adapter_caps; }; @@ -228,4 +229,8 @@ int mana_ib_query_gid(struct ib_device *ibdev, u32 port, int index, void mana_ib_disassociate_ucontext(struct ib_ucontext *ibcontext); int mana_ib_gd_query_adapter_caps(struct mana_ib_dev *mdev); + +void mana_ib_gd_create_rnic_adapter(struct mana_ib_dev *mdev); + +void mana_ib_gd_destroy_rnic_adapter(struct mana_ib_dev *mdev); #endif From patchwork Wed Jan 31 10:55:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Konstantin Taranov X-Patchwork-Id: 194665 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp1804714dyb; Wed, 31 Jan 2024 02:57:35 -0800 (PST) X-Google-Smtp-Source: AGHT+IGVRiePjh4cf6QtxZFvgedJuBUr9LRwSu+lKkfJJ05G7XAe+FZfBrGa9YEwedpYZaJaSh0D X-Received: by 2002:a05:6214:2021:b0:686:1c06:4436 with SMTP id 1-20020a056214202100b006861c064436mr1172024qvf.48.1706698655673; Wed, 31 Jan 2024 02:57:35 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706698655; cv=pass; d=google.com; s=arc-20160816; b=Ylgr/KhrDyyQGIkV06pXafSuaZxa/VRv3QU/hWJBptZHxR2GpGKuwWa/Zv/pW/iBFv TeHr1bTbQTVZlvQIFp35EmiTSC8ir1My9aHvaJ3rtVgjLhA6NLwe7ZiaXB8DCvEw4cU/ o9K6qhSCxsBm7ycWb19Vtpy5CsIOZGd96Zf6MIsjlqklMgF6uYOEyBSoIXipsuTaaXwl 2m83OZ8s1ydQsZ2A+QBwxi8ZQJehXv7+t6jf5ZTyijW9ypAo0RHaTZO3VM4q0khsPbqk 9a+0AROqSPgrygDV4gLaf3DZHsQgx0w9k8Now7zaemiO8WGy4s6fyhyXj2dyMQopQ4l+ +P3g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-id:precedence:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-filter; bh=FQMJvC08Xx+rruS+xJDdv53mAl3GJPxfohVLYNulxEA=; fh=4C7a762SwFeOC49REwRXzFkD35HrnmN6getpI9p9UT4=; b=X4nt6GsrwrwzTL5ErzbbmGU3HvvPwDlIMgQozCRPww1G/nrd9zsh5IE0jh7/VDEMyn xxvWIaojwt0ODfvQUNcaUGlhM5XLKlNDHBrm3MIh2dT19Uj6BG2jMBKzyOtDfYgh9XW7 0JOTOdLHMVnauO14PPwj1vVR4HVrFeS6YygY2wAdS9Itlls0XWRpa0xXUBKzOJ4oPiK9 bFL5wlDGoHGu9KPDyObiJLXSSUQiyx5GMGCSaVYbAUbOxmiBLTLTTibK4p0o3CLFUuy5 yqL3ofRbuk0hb9INlwKr4qd/gdHK6XdCrDO5FQyQEQJA3gGAWT3CUsg0n53Cdoo0oW47 Xkug==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=S3JrxK1p; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-46276-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-46276-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com X-Forwarded-Encrypted: i=1; AJvYcCWQhKczVLQuGcke9jdAmH1Vperwjcw65qZ8Y8vOLg2cCQ9rORrg4rwxK6CL92LBVhYQXi1usDQdRv+4yKtL2oKlJ12ovw== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id o26-20020a05620a0d5a00b00783e0fac2adsi10126730qkl.205.2024.01.31.02.57.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 02:57:35 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-46276-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=S3JrxK1p; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-46276-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-46276-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 766661C21B05 for ; Wed, 31 Jan 2024 10:57:35 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9593278685; Wed, 31 Jan 2024 10:56:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b="S3JrxK1p" Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 397536A030; Wed, 31 Jan 2024 10:55:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=13.77.154.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706698559; cv=none; b=rFpROYn1MjJezdru3mpaR95J/kb7Ip+y7uzeso18wBcUQTNCOq4n7ESQAvcq39xm4DrrS6Lkmb6UUZ9uK6f/91sFzC7kXysyH8YZoKBMa547cnml+xz2x/TrrpDDZspIWnJ+7dM6UdWxPnlbphHg/yylWHwszSBq4HThlwuHnBE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706698559; c=relaxed/simple; bh=GI6QY98SEMz0zqOuQidUbYonmgSgitJXQUyspqe4rNo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=fpYF+IoV17o2REa2Tn1PCvaJUZqmrS8QTvwh2/N9p8QvRlZyKNYW6xi3EvvFoqiixxImEm/d1qPi440k3k+arU+1pVXKLjGA0D16uj5XGpbskUivQ23ZhrJNzomYV8OYAo7KZ8GjWxjqi0YGN9VVrlIfL3ev2WvQu3M5ebdTSA0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com; spf=pass smtp.mailfrom=linux.microsoft.com; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b=S3JrxK1p; arc=none smtp.client-ip=13.77.154.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Received: from linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net (linux.microsoft.com [13.77.154.182]) by linux.microsoft.com (Postfix) with ESMTPSA id C2D7F20B2002; Wed, 31 Jan 2024 02:55:57 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com C2D7F20B2002 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1706698557; bh=FQMJvC08Xx+rruS+xJDdv53mAl3GJPxfohVLYNulxEA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=S3JrxK1pO/X+3876uWpPrkKDWepe9Vt4MtQXVlmCX64d3WHz/zd7d+5oyQgIYhwdO umuzfIPe5kdz3Z+Pt0J0sWk06449UeLvCdOdjXosVxc53ZsLQ1ojYj283Y0kgxEJJ1 zCjsENNIHlMzHWM35uivZp6tyUSr22W+J5DixdAE= From: Konstantin Taranov To: kotaranov@microsoft.com, sharmaajay@microsoft.com, longli@microsoft.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH rdma-next v1 2/5] RDMA/mana_ib: Create and destroy rnic adapter Date: Wed, 31 Jan 2024 02:55:49 -0800 Message-Id: <1706698552-25383-3-git-send-email-kotaranov@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1706698552-25383-1-git-send-email-kotaranov@linux.microsoft.com> References: <1706698552-25383-1-git-send-email-kotaranov@linux.microsoft.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789603249535286549 X-GMAIL-MSGID: 1789603249535286549 This patch adds RNIC creation and destruction. If creation of RNIC fails, we support only RAW QPs as they are served by ethernet driver. Signed-off-by: Konstantin Taranov --- drivers/infiniband/hw/mana/main.c | 31 +++++++++++++++++++++++++++++++ drivers/infiniband/hw/mana/mana_ib.h | 29 +++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+) diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c index c64d569..33cd69e 100644 --- a/drivers/infiniband/hw/mana/main.c +++ b/drivers/infiniband/hw/mana/main.c @@ -581,14 +581,31 @@ static void mana_ib_destroy_eqs(struct mana_ib_dev *mdev) void mana_ib_gd_create_rnic_adapter(struct mana_ib_dev *mdev) { + struct mana_rnic_create_adapter_resp resp = {}; + struct mana_rnic_create_adapter_req req = {}; + struct gdma_context *gc = mdev_to_gc(mdev); int err; + mdev->adapter_handle = INVALID_MANA_HANDLE; + err = mana_ib_create_eqs(mdev); if (err) { ibdev_err(&mdev->ib_dev, "Failed to create EQs for RNIC err %d", err); goto cleanup; } + mana_gd_init_req_hdr(&req.hdr, MANA_IB_CREATE_ADAPTER, sizeof(req), sizeof(resp)); + req.hdr.req.msg_version = GDMA_MESSAGE_V2; + req.hdr.dev_id = gc->mana_ib.dev_id; + req.notify_eq_id = mdev->fatal_err_eq->id; + + err = mana_gd_send_request(gc, sizeof(req), &req, sizeof(resp), &resp); + if (err) { + ibdev_err(&mdev->ib_dev, "Failed to create RNIC adapter err %d", err); + goto cleanup; + } + mdev->adapter_handle = resp.adapter; + return; cleanup: @@ -599,5 +616,19 @@ void mana_ib_gd_create_rnic_adapter(struct mana_ib_dev *mdev) void mana_ib_gd_destroy_rnic_adapter(struct mana_ib_dev *mdev) { + struct mana_rnic_destroy_adapter_resp resp = {}; + struct mana_rnic_destroy_adapter_req req = {}; + struct gdma_context *gc; + + if (!rnic_is_enabled(mdev)) + return; + + gc = mdev_to_gc(mdev); + mana_gd_init_req_hdr(&req.hdr, MANA_IB_DESTROY_ADAPTER, sizeof(req), sizeof(resp)); + req.hdr.dev_id = gc->mana_ib.dev_id; + req.adapter = mdev->adapter_handle; + + mana_gd_send_request(gc, sizeof(req), &req, sizeof(resp), &resp); + mdev->adapter_handle = INVALID_MANA_HANDLE; mana_ib_destroy_eqs(mdev); } diff --git a/drivers/infiniband/hw/mana/mana_ib.h b/drivers/infiniband/hw/mana/mana_ib.h index a4b94ee..96454cf 100644 --- a/drivers/infiniband/hw/mana/mana_ib.h +++ b/drivers/infiniband/hw/mana/mana_ib.h @@ -48,6 +48,7 @@ struct mana_ib_adapter_caps { struct mana_ib_dev { struct ib_device ib_dev; struct gdma_dev *gdma_dev; + mana_handle_t adapter_handle; struct gdma_queue *fatal_err_eq; struct mana_ib_adapter_caps adapter_caps; }; @@ -115,6 +116,8 @@ struct mana_ib_rwq_ind_table { enum mana_ib_command_code { MANA_IB_GET_ADAPTER_CAP = 0x30001, + MANA_IB_CREATE_ADAPTER = 0x30002, + MANA_IB_DESTROY_ADAPTER = 0x30003, }; struct mana_ib_query_adapter_caps_req { @@ -143,6 +146,32 @@ struct mana_ib_query_adapter_caps_resp { u32 max_inline_data_size; }; /* HW Data */ +struct mana_rnic_create_adapter_req { + struct gdma_req_hdr hdr; + u32 notify_eq_id; + u32 reserved; + u64 feature_flags; +}; /*HW Data */ + +struct mana_rnic_create_adapter_resp { + struct gdma_resp_hdr hdr; + mana_handle_t adapter; +}; /* HW Data */ + +struct mana_rnic_destroy_adapter_req { + struct gdma_req_hdr hdr; + mana_handle_t adapter; +}; /*HW Data */ + +struct mana_rnic_destroy_adapter_resp { + struct gdma_resp_hdr hdr; +}; /* HW Data */ + +static inline bool rnic_is_enabled(struct mana_ib_dev *mdev) +{ + return mdev->adapter_handle != INVALID_MANA_HANDLE; +} + static inline struct gdma_context *mdev_to_gc(struct mana_ib_dev *mdev) { return mdev->gdma_dev->gdma_context; From patchwork Wed Jan 31 10:55:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Konstantin Taranov X-Patchwork-Id: 194671 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp1808179dyb; Wed, 31 Jan 2024 03:03:50 -0800 (PST) X-Google-Smtp-Source: AGHT+IHUb5bXidfhLzvQfLrgL0ERo5QoA+5GqYby0RkNKyw+UN2ZGU/+ScsIEncBJm4xGz3MsZQf X-Received: by 2002:a17:902:9a0a:b0:1d7:3403:ba7 with SMTP id v10-20020a1709029a0a00b001d734030ba7mr894052plp.135.1706699030593; Wed, 31 Jan 2024 03:03:50 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706699030; cv=pass; d=google.com; s=arc-20160816; b=tizCCVOcJvsZ7KSjazxjO3QEEJGGJFRp3CrD1BCUhql+5pCGm1LOFJNnqnoYyb3Psx yR7tlQfTy0qS/7g2wTHax5LttThq8Gx/dNo0re9eeR5DWl4qdUK5BVmP8P9Sou4Vomqa 6YZyBBjIsvfEm3Vtr/9vz0OjAz9Ad1n/pnPW2fAmfRV4e6quVmZ2JwqksC0j+/7bWC37 PFg9jK3KxMeEOuMgT42jTADMTHsCXEWaiS39uAwTgykjpwp1jYmeNffi1UoQ/H9Co7C9 85wQP+KdSS2xPzm7wOtGe7HrOk46w0ef0QptnxK+p9eRLcXQixQ0DnMOiTxRv6CQZGeg fiNg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-id:precedence:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-filter; bh=5yXFaEp9vKBshRa+NgGc9Zh0uI+t9u7kQk3cCqXTndQ=; fh=6QRzlnZrEnr5xlwn0lfhaV5zi/g/vCx0qQRdKbAepek=; b=jf0kz22tHAv933YNNvdcPP5DGQ0JEec+pGLLZHkeYYAcY9pZv9ppcfytxc/FaGdyTY dEDdkpt3Li5mOD0fTNw6fm2LA+iG7wR5xsXwHbde/eu5gKsCFp2qnxqSJipbPI42XTVs FMrDWZLtKQCsKCv10nb/N/r/bWqKN/qMy3iLFVqfAQFjlnK7UMzr1Pa0wLuEShRNZMyO IlTP4BZADeLRsHHhiucM31sysioSg/yInjHzCoLO40TG3i67jJTEhOWrCwLkueCxGk0w U7GrO6NAIJ4C4VK+E34LcUwkXJX32y6gzfyGLswpbWbJqByAMQAGzskM4RtcKhF7lpW4 953g==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=GB+vxnbL; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-46277-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-46277-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com X-Forwarded-Encrypted: i=1; AJvYcCUHfJOYjCJu/q2FbNMxXwH2RkaWErhtF1ksFTi+6C9t8sul1/MDq66gfcMsGMcBL8F3a9FczxlzHVRylx2PociJjvnXoA== Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id b5-20020a170902ed0500b001d92e40f697si986462pld.428.2024.01.31.03.03.50 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 03:03:50 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-46277-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=GB+vxnbL; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-46277-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-46277-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id EDF1E284918 for ; Wed, 31 Jan 2024 10:57:33 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7B71578B76; Wed, 31 Jan 2024 10:56:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b="GB+vxnbL" Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 417F86A353; Wed, 31 Jan 2024 10:55:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=13.77.154.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706698559; cv=none; b=bweNXF22iXTxdROeX9blPwcPVgoiA6lc8l8kiBNclhrlYTbQqYYpWaXeF2bavneiLHIaeR8JHxSRKarHABEYBlbpH4J2DpkFqBFxG+Hd0jI9BxsRobrYoVqe2bVoZuVfqVeyc2zbBP9qtORlozNqg2w/OepLN/XuyTTYXMG8vFs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706698559; c=relaxed/simple; bh=lwkOSAnGEnMRxgE2eYpnXuty91szYnqPYwpQeHF0pik=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=uQJlTUtoLWCf0BKHjR5cdNs7+s64dqAscWIP7paIraP22Fru7WUdWHrO9+yTvNfOfm017LDyiYtIXzX3XvkNgxtRq5KgrClCBgQck7I7j/mYvM/3PGTE0/ZGKkM2njuO1IhX2rPTk6vqB2x6LDcq9z54451lFaTfYxSVcA29Cmw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com; spf=pass smtp.mailfrom=linux.microsoft.com; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b=GB+vxnbL; arc=none smtp.client-ip=13.77.154.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Received: from linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net (linux.microsoft.com [13.77.154.182]) by linux.microsoft.com (Postfix) with ESMTPSA id DB96C20B2003; Wed, 31 Jan 2024 02:55:57 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com DB96C20B2003 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1706698557; bh=5yXFaEp9vKBshRa+NgGc9Zh0uI+t9u7kQk3cCqXTndQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GB+vxnbLTjT5Cnt7unO+5vs7vkPwsXIQa+KctyE3Mac1SWpuauMpA/7NTrY3DtkI1 sUNbJdGmFonKb7p7PPqw+sfZxDcRQ6RPXJb3hzFqDbDhm8Kutug1KLwv4S48AZS0BU 1rCrPaeAerPUAqgKgZvX1bzGT5bltj7dNZ5gMp04= From: Konstantin Taranov To: kotaranov@microsoft.com, sharmaajay@microsoft.com, longli@microsoft.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH rdma-next v1 3/5] RDMA/mana_ib: Implement port parameters Date: Wed, 31 Jan 2024 02:55:50 -0800 Message-Id: <1706698552-25383-4-git-send-email-kotaranov@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1706698552-25383-1-git-send-email-kotaranov@linux.microsoft.com> References: <1706698552-25383-1-git-send-email-kotaranov@linux.microsoft.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789603642977497783 X-GMAIL-MSGID: 1789603642977497783 Implement port parameters for RNIC. Only port 1 is RoCEv2 capable. Signed-off-by: Konstantin Taranov --- drivers/infiniband/hw/mana/device.c | 2 ++ drivers/infiniband/hw/mana/main.c | 37 +++++++++++++++++++++++++++++++++++- drivers/infiniband/hw/mana/mana_ib.h | 4 ++++ 3 files changed, 42 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/mana/device.c b/drivers/infiniband/hw/mana/device.c index d8e8b10..11b0410 100644 --- a/drivers/infiniband/hw/mana/device.c +++ b/drivers/infiniband/hw/mana/device.c @@ -29,12 +29,14 @@ .destroy_rwq_ind_table = mana_ib_destroy_rwq_ind_table, .destroy_wq = mana_ib_destroy_wq, .disassociate_ucontext = mana_ib_disassociate_ucontext, + .get_link_layer = mana_ib_get_link_layer, .get_port_immutable = mana_ib_get_port_immutable, .mmap = mana_ib_mmap, .modify_qp = mana_ib_modify_qp, .modify_wq = mana_ib_modify_wq, .query_device = mana_ib_query_device, .query_gid = mana_ib_query_gid, + .query_pkey = mana_ib_query_pkey, .query_port = mana_ib_query_port, .reg_user_mr = mana_ib_reg_user_mr, diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c index 33cd69e..3e05a62 100644 --- a/drivers/infiniband/hw/mana/main.c +++ b/drivers/infiniband/hw/mana/main.c @@ -492,7 +492,42 @@ int mana_ib_query_device(struct ib_device *ibdev, struct ib_device_attr *props, int mana_ib_query_port(struct ib_device *ibdev, u32 port, struct ib_port_attr *props) { - /* This version doesn't return port properties */ + struct net_device *ndev = mana_ib_get_netdev(ibdev, port); + + if (!ndev) + return -EINVAL; + + memset(props, 0, sizeof(*props)); + props->max_mtu = IB_MTU_4096; + props->active_mtu = ib_mtu_int_to_enum(ndev->mtu); + + if (netif_carrier_ok(ndev) && netif_running(ndev)) { + props->state = IB_PORT_ACTIVE; + props->phys_state = IB_PORT_PHYS_STATE_LINK_UP; + } else { + props->state = IB_PORT_DOWN; + props->phys_state = IB_PORT_PHYS_STATE_DISABLED; + } + + props->active_width = IB_WIDTH_4X; + props->active_speed = IB_SPEED_EDR; + props->pkey_tbl_len = 1; + if (port == 1) + props->gid_tbl_len = 16; + + return 0; +} + +enum rdma_link_layer mana_ib_get_link_layer(struct ib_device *device, u32 port_num) +{ + return IB_LINK_LAYER_ETHERNET; +} + +int mana_ib_query_pkey(struct ib_device *ibdev, u32 port, u16 index, u16 *pkey) +{ + if (index != 0) + return -EINVAL; + *pkey = IB_DEFAULT_PKEY_FULL; return 0; } diff --git a/drivers/infiniband/hw/mana/mana_ib.h b/drivers/infiniband/hw/mana/mana_ib.h index 96454cf..196f3c8 100644 --- a/drivers/infiniband/hw/mana/mana_ib.h +++ b/drivers/infiniband/hw/mana/mana_ib.h @@ -262,4 +262,8 @@ int mana_ib_query_gid(struct ib_device *ibdev, u32 port, int index, void mana_ib_gd_create_rnic_adapter(struct mana_ib_dev *mdev); void mana_ib_gd_destroy_rnic_adapter(struct mana_ib_dev *mdev); + +int mana_ib_query_pkey(struct ib_device *ibdev, u32 port, u16 index, u16 *pkey); + +enum rdma_link_layer mana_ib_get_link_layer(struct ib_device *device, u32 port_num); #endif From patchwork Wed Jan 31 10:55:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Konstantin Taranov X-Patchwork-Id: 194698 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp1822334dyb; Wed, 31 Jan 2024 03:33:57 -0800 (PST) X-Google-Smtp-Source: AGHT+IHyjWEgEeUe+w4huaqTs4Mwz40ajBQFN8urYaHDb1MpxFL9mMziNBP+htKyAq62EFCO0SXr X-Received: by 2002:a05:6870:cb4b:b0:210:ad69:e985 with SMTP id oz11-20020a056870cb4b00b00210ad69e985mr1317535oab.51.1706700837712; Wed, 31 Jan 2024 03:33:57 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706700837; cv=pass; d=google.com; s=arc-20160816; b=qxCzxA75PnL46gCOMszrPC15HfpzSDdnhYR5czA7fF5DgV2RzqHxKlMch3iEFmfWoZ WQVfXW/yfrKo6noSFSTpsGujJ9LEodvX4JgA+hAQeNTNJaj34JzxYpZWluLS0AHY+xdb CE5kS8CaADzhWRQqkAlmHDK5eJBUBZlWbyVOdz3YM7mY51k4PnhWM1k/hHNEUJkte9cG mAd5YXBa8YprTSG+scA5HvuAZaTJ10maxWSNljfkK/1O8GzjHJg3RSexkepLSWsIqV7G 37Da/T7QuPB6xMiCxc7FnEhTMSeOyCf8170ClkHEmMTMHQUr7ZoApJOC2yAHmT4O3qAk 98Gg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-id:precedence:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-filter; bh=snXzVvd1Es1P5EFCe7tEDBGYBTXVzvrZs/jkgBz6PiU=; fh=5LxVqZgWMSBKJcLnZmCgFbWiTfZzK+Rg0XhdMsjnqjQ=; b=rajI3D5qYqh5y+9+a4s5b0sL6K5/Yrzz1tAJZ7Zlm2m85Myim9RZCNXNOxXUawO4dq 7nAQ+w3YGgqCl4uaLga7gTHI199Lot0T3qKM2GAvmyZIedh1140KJCsoMuYXEcJASgpG aCGDAw/SImxWoUIlOfSFSn3w7YmTK+veY+MwFVMBMyIN69CXfkyC6vykI15flHwbm/ba 3rvKMh3qSTG5+TdJgIhDsqAvWfoTPRxnwXEFmP7l6PLjh4pQ+Z8byhHwiNU5gg/DF5aL V8bmf6Wbg+9KlisFibnfw1KqFFzgQ5KrJA/o86zOgmZ0Bg3+uu+yKQPLnCc3wPeFh04V m5Dw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=iII5BczB; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-46278-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-46278-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com X-Forwarded-Encrypted: i=1; AJvYcCVLN3hnb14V9uubjBvolFxBE/B9acxdiZiH4/FRGNtfvgpo/NHHK8/JRBO+zZKP55Ia5f2iimJAGkbBs4+lbndnuctrDA== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [147.75.48.161]) by mx.google.com with ESMTPS id l17-20020a635b51000000b005c1b58a321asi9417378pgm.411.2024.01.31.03.33.57 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 03:33:57 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-46278-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=iII5BczB; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-46278-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) smtp.mailfrom="linux-kernel+bounces-46278-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 838FBB30A1A for ; Wed, 31 Jan 2024 10:57:54 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E0D9B79DB2; Wed, 31 Jan 2024 10:56:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b="iII5BczB" Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 418506D1D4; Wed, 31 Jan 2024 10:55:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=13.77.154.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706698561; cv=none; b=s+zox8VCtKW2//vuYXAnlpmOFNol80pQqZ0TSs/i0YugamRxNp3kpDtxFP+UPh0keYPK7X2JeFPmDLrIrNl8rsNHztNTxMGorbLsBw4knRBg8JafKQaEoXUnSYI4rLNvim/L/r10uEIBowlZ73e7ixLu7YTfEozPkK5jYn0N+ZQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706698561; c=relaxed/simple; bh=ykFVDqFPB4lY6V66losZpGAUIa9JbgeucCmBSnLGxnU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=SgRGU8J/2k2pEe71ZXyf9sZLmwJEiPrD00GpytQ2zb8X1q2mLOkCHD/kyJTRPurVy6DyfZT2ki6lYVRZpoP4StSht/j57nnUswd2PqMg5+lBaeZ88bgyaErJnoUTUgtItuZ9w5iWapJgw5VhW7Xge8A4LO/TfawpHHFhwFAwnvY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com; spf=pass smtp.mailfrom=linux.microsoft.com; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b=iII5BczB; arc=none smtp.client-ip=13.77.154.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Received: from linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net (linux.microsoft.com [13.77.154.182]) by linux.microsoft.com (Postfix) with ESMTPSA id F3EF920B2004; Wed, 31 Jan 2024 02:55:57 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com F3EF920B2004 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1706698558; bh=snXzVvd1Es1P5EFCe7tEDBGYBTXVzvrZs/jkgBz6PiU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iII5BczBqDghQYdz16lH6P0WkkFIkrwADtobdOVYg48E8A/y6nDssi7bcU7GilcPt JxTK6JBmLHB3cG7sRC1ZYLfEW7JOvSESXczdslow/HlU+mI7SQSgHbeqgwMDPdzcF9 wvrF9w8cUCFoffgzm1B/GFZSkxabAneZfzWj8XnQ= From: Konstantin Taranov To: kotaranov@microsoft.com, sharmaajay@microsoft.com, longli@microsoft.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH rdma-next v1 4/5] RDMA/mana_ib: Enable RoCE on port 1 Date: Wed, 31 Jan 2024 02:55:51 -0800 Message-Id: <1706698552-25383-5-git-send-email-kotaranov@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1706698552-25383-1-git-send-email-kotaranov@linux.microsoft.com> References: <1706698552-25383-1-git-send-email-kotaranov@linux.microsoft.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789605537482080886 X-GMAIL-MSGID: 1789605537482080886 Set netdev and RoCEv2 flag to be used in GID population. mana_ib is auxiliary device, thus we need GIDs of the master netdev. Signed-off-by: Konstantin Taranov --- drivers/infiniband/hw/mana/device.c | 14 ++++++++++++++ drivers/infiniband/hw/mana/main.c | 16 ++++++++++++---- 2 files changed, 26 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/hw/mana/device.c b/drivers/infiniband/hw/mana/device.c index 11b0410..b9ff3fd 100644 --- a/drivers/infiniband/hw/mana/device.c +++ b/drivers/infiniband/hw/mana/device.c @@ -53,6 +53,7 @@ static int mana_ib_probe(struct auxiliary_device *adev, { struct mana_adev *madev = container_of(adev, struct mana_adev, adev); struct gdma_dev *mdev = madev->mdev; + struct net_device *upper_ndev; struct mana_context *mc; struct mana_ib_dev *dev; int ret; @@ -79,6 +80,19 @@ static int mana_ib_probe(struct auxiliary_device *adev, dev->ib_dev.num_comp_vectors = 1; dev->ib_dev.dev.parent = mdev->gdma_context->dev; + rcu_read_lock(); /* required to get upper dev */ + upper_ndev = netdev_master_upper_dev_get_rcu(mc->ports[0]); + rcu_read_unlock(); + if (!upper_ndev) { + ibdev_err(&dev->ib_dev, "Failed to get master netdev"); + goto free_ib_device; + } + ret = ib_device_set_netdev(&dev->ib_dev, upper_ndev, 1); + if (ret) { + ibdev_err(&dev->ib_dev, "Failed to set ib netdev, ret %d", ret); + goto free_ib_device; + } + ret = mana_gd_register_device(&mdev->gdma_context->mana_ib); if (ret) { ibdev_err(&dev->ib_dev, "Failed to register device, ret %d", diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c index 3e05a62..645abf3 100644 --- a/drivers/infiniband/hw/mana/main.c +++ b/drivers/infiniband/hw/mana/main.c @@ -462,11 +462,19 @@ int mana_ib_mmap(struct ib_ucontext *ibcontext, struct vm_area_struct *vma) int mana_ib_get_port_immutable(struct ib_device *ibdev, u32 port_num, struct ib_port_immutable *immutable) { - /* - * This version only support RAW_PACKET - * other values need to be filled for other types - */ + struct mana_ib_dev *mdev = container_of(ibdev, struct mana_ib_dev, ib_dev); + struct ib_port_attr attr; + int err; + + err = ib_query_port(ibdev, port_num, &attr); + if (err) + return err; + + immutable->pkey_tbl_len = attr.pkey_tbl_len; + immutable->gid_tbl_len = attr.gid_tbl_len; immutable->core_cap_flags = RDMA_CORE_PORT_RAW_PACKET; + if (port_num == 1 && rnic_is_enabled(mdev)) + immutable->core_cap_flags |= RDMA_CORE_PORT_IBA_ROCE_UDP_ENCAP; return 0; } From patchwork Wed Jan 31 10:55:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Konstantin Taranov X-Patchwork-Id: 194666 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:2087:b0:106:209c:c626 with SMTP id gs7csp1804919dyb; Wed, 31 Jan 2024 02:58:12 -0800 (PST) X-Google-Smtp-Source: AGHT+IEJmvQa/Hldh6YsF/wesZValsVtuNa3+mIPOTeDNAoC1SqI/O4fDZRFaKFYQ12EgflfTKdv X-Received: by 2002:a17:906:4c54:b0:a31:7c5a:bf5b with SMTP id d20-20020a1709064c5400b00a317c5abf5bmr890428ejw.5.1706698692432; Wed, 31 Jan 2024 02:58:12 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706698692; cv=pass; d=google.com; s=arc-20160816; b=QdIZCTwD+ZRimtTBtCnPS5vIGsWZ0iKu8AAAubXc3rqEns5y+nm1bBcK3eMyfd+yg7 F/VVCUrzW7DFL/e/WKR4HxbFR364dcyFS1lDlKFfswJ+rypc0ZOO0qMcMWFKS376f1TC BaIRIO30XVf0UDARu88HYX50IC/U/eUiItt48E9SDqEbggcV4Ez018EaVjRK4Hd4fZP9 oXAPNhU7i5jkDHQjCBoodCWUK1Lf9TuJh9+XJdWQecK3Q5IglNxfWxEA5gNy5ZMoTDAC VmRnT93N9EBVQG4zyO05mjzZLpf5rhHxu23QoBYhOlsp0s5by4RTjtlk7i/45DBj5QOb X1Zw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-id:precedence:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-filter; bh=MsUo2v+//cFlf6nJMuZ1DMVlU3JLOjmKGXGXkSEimVc=; fh=8cSdD2SqmPcUE0CahARU+BdvX89R4Uu2s29jd8JN50I=; b=avV+XRpcRxaQPnwaZCZYB8YavUU+GmHCzbsEc5hREszJZOVpsoX3saF9NVh39vehOD JLAEx3lidg2GIjctobQ7LBWMbTTI4dQB51WBl7HNwuLx0DQVHOi9Nd0/xFjc9mjgbHPj BVC/Kp2YG68fvfBODFbcwk1qOplmax1ScjOVPg8czBaB5gQcIkrKXsbTL3db9ivBikSA ZHNZ4JLefyiutF6qUvkgEnWnkRPP1LJ/bQoauTAXJQZjWu+CkMxrIqXI1t42AehWOx/Q 6LvlfKA2Rj8mO//QDywWWlyBfcG7FVYNZh3VjonhyXYLsOSWflR6gWLmgGxl0edyxEc9 QcGQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=L7xBFSUO; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-46279-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-46279-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com X-Forwarded-Encrypted: i=1; AJvYcCVzWnWUtyJNmfdqprXvsvpmOsCoDVNt/i1LMkK2HTL0FkPrRTffKl0jYwEJRWpf2J1hufBM7ajrivK2P68LkXNviExxtA== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id kg27-20020a17090776fb00b00a332e92241csi5355205ejc.307.2024.01.31.02.58.12 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 02:58:12 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-46279-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=L7xBFSUO; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-46279-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-46279-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 81D761F2190A for ; Wed, 31 Jan 2024 10:58:05 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0046C79DAE; Wed, 31 Jan 2024 10:56:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b="L7xBFSUO" Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id AADA26EB6E; Wed, 31 Jan 2024 10:55:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=13.77.154.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706698562; cv=none; b=rqiMAmIWyl1rbYzFsiC5qw69D0kNVdLFegsSsr6hKblcGL3Ome+Slfoe25qR9QKQpkedXFsjUto45t195K8C/A3uXqJPxB5MKP7Of3lMbKd2LfeTGojgP9HKijSPh1cmUTodm9zDP/b+8HSI7XObST4HdiwEi9vs9r8b3OExf3g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706698562; c=relaxed/simple; bh=LHBIj/8uoDBRrS1dlZPeCz2qf0GZ0mc5vqMb8Dcyy6U=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=sSfJqTRaFhIlREW0cMpPSoS3WWTve1HRpfs/X/LuJhQgILvlJcGvq5E+W20HYvaLNJ4+iO2SktQrlmAnxWQozpv+tEGgHCqG9pmncRgiYPB9j12SsC+fa6uDcN8jAnrqt1fmXUQcVFFQbkAwHplkZwhvQtoW6wsNLOlVgRhKM9s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com; spf=pass smtp.mailfrom=linux.microsoft.com; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b=L7xBFSUO; arc=none smtp.client-ip=13.77.154.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Received: from linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net (linux.microsoft.com [13.77.154.182]) by linux.microsoft.com (Postfix) with ESMTPSA id 1859020B2005; Wed, 31 Jan 2024 02:55:58 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 1859020B2005 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1706698558; bh=MsUo2v+//cFlf6nJMuZ1DMVlU3JLOjmKGXGXkSEimVc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=L7xBFSUON0y+jHp0RHicnx4GOcP3xCMFb6wyib2qcVkrCV4YXENV/EBgdnPq/AW1T t6fjax+zreYGHyrT/FTObwgcTI2XXzPsHWPiSMa5jFGT72oV3Ng9dnZiazoqVyWkX1 Rd8KPyE++IipAvl46yMwOVL88ODVAnYR0pVsveXY= From: Konstantin Taranov To: kotaranov@microsoft.com, sharmaajay@microsoft.com, longli@microsoft.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH rdma-next v1 5/5] RDMA/mana_ib: Adding and deleting GIDs Date: Wed, 31 Jan 2024 02:55:52 -0800 Message-Id: <1706698552-25383-6-git-send-email-kotaranov@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1706698552-25383-1-git-send-email-kotaranov@linux.microsoft.com> References: <1706698552-25383-1-git-send-email-kotaranov@linux.microsoft.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789603288201511785 X-GMAIL-MSGID: 1789603288201511785 Implement add_gid and del_gid for RNIC. We support ipv4 and ipv6 addresses. Signed-off-by: Konstantin Taranov --- drivers/infiniband/hw/mana/device.c | 2 ++ drivers/infiniband/hw/mana/main.c | 66 ++++++++++++++++++++++++++++++++++++ drivers/infiniband/hw/mana/mana_ib.h | 37 ++++++++++++++++++++ 3 files changed, 105 insertions(+) diff --git a/drivers/infiniband/hw/mana/device.c b/drivers/infiniband/hw/mana/device.c index b9ff3fd..9655307 100644 --- a/drivers/infiniband/hw/mana/device.c +++ b/drivers/infiniband/hw/mana/device.c @@ -15,6 +15,7 @@ .driver_id = RDMA_DRIVER_MANA, .uverbs_abi_ver = MANA_IB_UVERBS_ABI_VERSION, + .add_gid = mana_ib_gd_add_gid, .alloc_pd = mana_ib_alloc_pd, .alloc_ucontext = mana_ib_alloc_ucontext, .create_cq = mana_ib_create_cq, @@ -23,6 +24,7 @@ .create_wq = mana_ib_create_wq, .dealloc_pd = mana_ib_dealloc_pd, .dealloc_ucontext = mana_ib_dealloc_ucontext, + .del_gid = mana_ib_gd_del_gid, .dereg_mr = mana_ib_dereg_mr, .destroy_cq = mana_ib_destroy_cq, .destroy_qp = mana_ib_destroy_qp, diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c index 645abf3..282c024 100644 --- a/drivers/infiniband/hw/mana/main.c +++ b/drivers/infiniband/hw/mana/main.c @@ -675,3 +675,69 @@ void mana_ib_gd_destroy_rnic_adapter(struct mana_ib_dev *mdev) mdev->adapter_handle = INVALID_MANA_HANDLE; mana_ib_destroy_eqs(mdev); } + +int mana_ib_gd_add_gid(const struct ib_gid_attr *attr, void **context) +{ + struct mana_ib_dev *mdev = container_of(attr->device, struct mana_ib_dev, ib_dev); + enum rdma_network_type ntype = rdma_gid_attr_network_type(attr); + struct mana_rnic_config_addr_resp resp = {}; + struct gdma_context *gc = mdev_to_gc(mdev); + struct mana_rnic_config_addr_req req = {}; + int err; + + if (!rnic_is_enabled(mdev)) + return -EINVAL; + + if (ntype != RDMA_NETWORK_IPV4 && ntype != RDMA_NETWORK_IPV6) { + ibdev_dbg(&mdev->ib_dev, "Unsupported rdma network type %d", ntype); + return -EINVAL; + } + + mana_gd_init_req_hdr(&req.hdr, MANA_IB_CONFIG_IP_ADDR, sizeof(req), sizeof(resp)); + req.hdr.dev_id = gc->mana_ib.dev_id; + req.adapter = mdev->adapter_handle; + req.op = ADDR_OP_ADD; + req.sgid_type = (ntype == RDMA_NETWORK_IPV6) ? SGID_TYPE_IPV6 : SGID_TYPE_IPV4; + copy_in_reverse(req.ip_addr, attr->gid.raw, sizeof(union ib_gid)); + + err = mana_gd_send_request(gc, sizeof(req), &req, sizeof(resp), &resp); + if (err) { + ibdev_err(&mdev->ib_dev, "Failed to config IP addr err %d\n", err); + return err; + } + + return 0; +} + +int mana_ib_gd_del_gid(const struct ib_gid_attr *attr, void **context) +{ + struct mana_ib_dev *mdev = container_of(attr->device, struct mana_ib_dev, ib_dev); + enum rdma_network_type ntype = rdma_gid_attr_network_type(attr); + struct mana_rnic_config_addr_resp resp = {}; + struct gdma_context *gc = mdev_to_gc(mdev); + struct mana_rnic_config_addr_req req = {}; + int err; + + if (!rnic_is_enabled(mdev)) + return -EINVAL; + + if (ntype != RDMA_NETWORK_IPV4 && ntype != RDMA_NETWORK_IPV6) { + ibdev_dbg(&mdev->ib_dev, "Unsupported rdma network type %d", ntype); + return -EINVAL; + } + + mana_gd_init_req_hdr(&req.hdr, MANA_IB_CONFIG_IP_ADDR, sizeof(req), sizeof(resp)); + req.hdr.dev_id = gc->mana_ib.dev_id; + req.adapter = mdev->adapter_handle; + req.op = ADDR_OP_REMOVE; + req.sgid_type = (ntype == RDMA_NETWORK_IPV6) ? SGID_TYPE_IPV6 : SGID_TYPE_IPV4; + copy_in_reverse(req.ip_addr, attr->gid.raw, sizeof(union ib_gid)); + + err = mana_gd_send_request(gc, sizeof(req), &req, sizeof(resp), &resp); + if (err) { + ibdev_err(&mdev->ib_dev, "Failed to config IP addr err %d\n", err); + return err; + } + + return 0; +} diff --git a/drivers/infiniband/hw/mana/mana_ib.h b/drivers/infiniband/hw/mana/mana_ib.h index 196f3c8..2a3e3b0 100644 --- a/drivers/infiniband/hw/mana/mana_ib.h +++ b/drivers/infiniband/hw/mana/mana_ib.h @@ -118,6 +118,7 @@ enum mana_ib_command_code { MANA_IB_GET_ADAPTER_CAP = 0x30001, MANA_IB_CREATE_ADAPTER = 0x30002, MANA_IB_DESTROY_ADAPTER = 0x30003, + MANA_IB_CONFIG_IP_ADDR = 0x30004, }; struct mana_ib_query_adapter_caps_req { @@ -167,6 +168,30 @@ struct mana_rnic_destroy_adapter_resp { struct gdma_resp_hdr hdr; }; /* HW Data */ +enum mana_ib_addr_op { + ADDR_OP_ADD = 1, + ADDR_OP_REMOVE, +}; + +enum sgid_entry_type { + SGID_TYPE_INVALID = 0, + SGID_TYPE_IPV4 = 1, + SGID_TYPE_IPV6 = 2, + SGID_TYPE_HYBRID = 3 +}; + +struct mana_rnic_config_addr_req { + struct gdma_req_hdr hdr; + mana_handle_t adapter; + enum mana_ib_addr_op op; + enum sgid_entry_type sgid_type; + u8 ip_addr[16]; +}; /* HW Data */ + +struct mana_rnic_config_addr_resp { + struct gdma_resp_hdr hdr; +}; /* HW Data */ + static inline bool rnic_is_enabled(struct mana_ib_dev *mdev) { return mdev->adapter_handle != INVALID_MANA_HANDLE; @@ -188,6 +213,14 @@ static inline struct net_device *mana_ib_get_netdev(struct ib_device *ibdev, u32 return mc->ports[port - 1]; } +static inline void copy_in_reverse(u8 *dst, const u8 *src, u32 size) +{ + u32 i; + + for (i = 0; i < size; i++) + dst[size - 1 - i] = src[i]; +} + int mana_ib_install_cq_cb(struct mana_ib_dev *mdev, struct mana_ib_cq *cq); int mana_ib_gd_create_dma_region(struct mana_ib_dev *dev, struct ib_umem *umem, @@ -266,4 +299,8 @@ int mana_ib_query_gid(struct ib_device *ibdev, u32 port, int index, int mana_ib_query_pkey(struct ib_device *ibdev, u32 port, u16 index, u16 *pkey); enum rdma_link_layer mana_ib_get_link_layer(struct ib_device *device, u32 port_num); + +int mana_ib_gd_add_gid(const struct ib_gid_attr *attr, void **context); + +int mana_ib_gd_del_gid(const struct ib_gid_attr *attr, void **context); #endif