From patchwork Fri Feb 2 15:06:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Konstantin Taranov X-Patchwork-Id: 195882 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:9bc1:b0:106:209c:c626 with SMTP id op1csp491862dyc; Fri, 2 Feb 2024 07:09:28 -0800 (PST) X-Google-Smtp-Source: AGHT+IH82cAQfyXDht81KjZiwQ5NqO0uJh8Ty1YU/z81mkN3ozmGXk/NB9j5tMRGI0/XPEJKBpmx X-Received: by 2002:a05:6808:3985:b0:3bf:b81a:473 with SMTP id gq5-20020a056808398500b003bfb81a0473mr3133036oib.47.1706886568471; Fri, 02 Feb 2024 07:09:28 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706886568; cv=pass; d=google.com; s=arc-20160816; b=rKyKsVhuPYXXdRDrKa2sfjiTuNNSi5fXk9zq9AlMYT2JxXRDnNVvItE73LHtZwVomt J1Nexu6Y/Hl5GqfroME2R29Wpy7DIgXD0X8QcUSY4EY6qc3i7QW+QibQjEGLd3OqnPhW i8//W7SuEORi/BZVqIhdY6p70V+yQL+VhkT39gwhOJvpPjwA1651R1rZiDoYd0r/qkQ1 7jDCt39d7V9j2FAhHAKYgTb018bG87DzkR5oq6cmM0fTJ/pegLRMXn3n/b8lXe66vynh 16PmEg7RAvMYgFnfruhy5593OusPSgHTufQ3lmGnyc7kIuY9z/Eud9mOGPCAE+x6cwC+ 6nGA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-id:precedence:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-filter; bh=saDdL5sYNRP0RdrEfccKfGdzxJcVkUQ2VWnSyr2q/F4=; fh=It2lASTis4E47x1/Igg0EufTexmuBXhVqgKK5P2QS0E=; b=o93bKNcbtLa6nX5InphN8zafLeuCoFQVxbhbm/LEBEEXRqw2bMmKwGJwHPrb4+LdJo UU5CoCBT0iGDAQugX3DVG2H7tWXh42/IeZXYUZbm0d4o7eNcfg18v7AU4VRxLFyZysWx 0d7Tnw4srW9Swn+wEnFEjLTvVs0/ucvAaSKoWLhdeRjhXEqjMtvPsJGxoJjRminCMWKE AYP112JMdCr5Eh49NwCCuFR0gAcqP/BBRDB6fKcRBU8NL8pc0aQUe/o/6ugod5QZJTpu ySmyT84Uardhi/MYP1LIS/H/5QrKaLgEICKL/VQDE05MOwQZrL3YRWVCV1PHFOZ770Nr OUTQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=GwseLGBm; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-49989-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-49989-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com X-Forwarded-Encrypted: i=1; AJvYcCV5PeAZFd4p65YMwcKtJ+O5ha+gR63iqHNl+TiyVWjFZCQ7pA7Qqj8b5qQoU0yf/9ATbRBUigdezp9UlvcR1DmHSeoOVQ== Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id c15-20020ab0078f000000b007cd4ad1a08fsi328987uaf.8.2024.02.02.07.09.28 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Feb 2024 07:09:28 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-49989-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=GwseLGBm; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-49989-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-49989-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id BE4F81C2513A for ; Fri, 2 Feb 2024 15:08:28 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 50D071482EA; Fri, 2 Feb 2024 15:06:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b="GwseLGBm" Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8BE86145346; Fri, 2 Feb 2024 15:06:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=13.77.154.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706886414; cv=none; b=BOXEbKintWuRtHq1dHmIng1ytwCru0q9lbax9brGTXEpMox9H5CzqBgrOteUPhPgZe8sFKgUORM9VRD1Jt3ghRxCa/8Yv0P6vn2dzPoKp2naTp+SWcN76nJoWmcQDZ0/BV7yxetB2pcds16Fj4Yi6p++qHwdA7JmtnZzFoNeEeg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706886414; c=relaxed/simple; bh=4ZmPsc+rDgX7rX3N3RoK8DZaj+dzW3V5EQi2cFZw9c4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Qm3vqxmGtksFdHr+VGMfwGTpb4JBsJDRmMfn9X2CYp4OUmm1dqH4XIhjV70a4qYcGF3XXri6MQ+bk/m61WsBHVFuJ0YdesbgZ3QGNvRKmnPiBiIF99Q/a4OIQsEvgecfkX9g3hxGdcI3J9TXLfcPOVTGj5EfFdBKQ/xSWhJeoGM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com; spf=pass smtp.mailfrom=linux.microsoft.com; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b=GwseLGBm; arc=none smtp.client-ip=13.77.154.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Received: from linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net (linux.microsoft.com [13.77.154.182]) by linux.microsoft.com (Postfix) with ESMTPSA id 541DB20B2001; Fri, 2 Feb 2024 07:06:45 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 541DB20B2001 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1706886405; bh=saDdL5sYNRP0RdrEfccKfGdzxJcVkUQ2VWnSyr2q/F4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GwseLGBmF4IF6c7qv7lLjifT7x3kKd0qA/BBxUa4uzHH7xKaY/iH9Augap+q1B3DF zLP00weP8S2MoiWqXKwbOUyboPwIqX6VhONzKLhlWw5uENIA3IfqzO3+OBY4r9bqWB TugSxtJGaYXQrbSNXjemOk1hdrlXSD5uFhD9aCtY= From: Konstantin Taranov To: kotaranov@microsoft.com, sharmaajay@microsoft.com, longli@microsoft.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH rdma-next v2 1/5] RDMA/mana_ib: Add EQ creation for rnic adapter Date: Fri, 2 Feb 2024 07:06:33 -0800 Message-Id: <1706886397-16600-2-git-send-email-kotaranov@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1706886397-16600-1-git-send-email-kotaranov@linux.microsoft.com> References: <1706886397-16600-1-git-send-email-kotaranov@linux.microsoft.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789800290332924693 X-GMAIL-MSGID: 1789800290332924693 This patch introduces functions for RNIC creation and creates one EQ for RNIC creation. Signed-off-by: Konstantin Taranov --- drivers/infiniband/hw/mana/device.c | 9 ++++-- drivers/infiniband/hw/mana/main.c | 53 ++++++++++++++++++++++++++++++++++++ drivers/infiniband/hw/mana/mana_ib.h | 5 ++++ 3 files changed, 64 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/hw/mana/device.c b/drivers/infiniband/hw/mana/device.c index 6fa902e..d8e8b10 100644 --- a/drivers/infiniband/hw/mana/device.c +++ b/drivers/infiniband/hw/mana/device.c @@ -92,15 +92,19 @@ static int mana_ib_probe(struct auxiliary_device *adev, goto deregister_device; } + mana_ib_gd_create_rnic_adapter(dev); + ret = ib_register_device(&dev->ib_dev, "mana_%d", mdev->gdma_context->dev); if (ret) - goto deregister_device; + goto destroy_rnic_adapter; dev_set_drvdata(&adev->dev, dev); return 0; +destroy_rnic_adapter: + mana_ib_gd_destroy_rnic_adapter(dev); deregister_device: mana_gd_deregister_device(dev->gdma_dev); free_ib_device: @@ -113,9 +117,8 @@ static void mana_ib_remove(struct auxiliary_device *adev) struct mana_ib_dev *dev = dev_get_drvdata(&adev->dev); ib_unregister_device(&dev->ib_dev); - + mana_ib_gd_destroy_rnic_adapter(dev); mana_gd_deregister_device(dev->gdma_dev); - ib_dealloc_device(&dev->ib_dev); } diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c index 29dd243..c64d569 100644 --- a/drivers/infiniband/hw/mana/main.c +++ b/drivers/infiniband/hw/mana/main.c @@ -548,3 +548,56 @@ int mana_ib_gd_query_adapter_caps(struct mana_ib_dev *dev) return 0; } + +static int mana_ib_create_eqs(struct mana_ib_dev *mdev) +{ + struct gdma_context *gc = mdev_to_gc(mdev); + struct gdma_queue_spec spec = {}; + int err; + + spec.type = GDMA_EQ; + spec.monitor_avl_buf = false; + spec.queue_size = EQ_SIZE; + spec.eq.callback = NULL; + spec.eq.context = mdev; + spec.eq.log2_throttle_limit = LOG2_EQ_THROTTLE; + spec.eq.msix_index = 0; + + err = mana_gd_create_mana_eq(&gc->mana_ib, &spec, &mdev->fatal_err_eq); + if (err) + return err; + + return 0; +} + +static void mana_ib_destroy_eqs(struct mana_ib_dev *mdev) +{ + if (!mdev->fatal_err_eq) + return; + + mana_gd_destroy_queue(mdev_to_gc(mdev), mdev->fatal_err_eq); + mdev->fatal_err_eq = NULL; +} + +void mana_ib_gd_create_rnic_adapter(struct mana_ib_dev *mdev) +{ + int err; + + err = mana_ib_create_eqs(mdev); + if (err) { + ibdev_err(&mdev->ib_dev, "Failed to create EQs for RNIC err %d", err); + goto cleanup; + } + + return; + +cleanup: + ibdev_warn(&mdev->ib_dev, + "RNIC is not available. Only RAW QPs are supported"); + mana_ib_destroy_eqs(mdev); +} + +void mana_ib_gd_destroy_rnic_adapter(struct mana_ib_dev *mdev) +{ + mana_ib_destroy_eqs(mdev); +} diff --git a/drivers/infiniband/hw/mana/mana_ib.h b/drivers/infiniband/hw/mana/mana_ib.h index 6a03ae6..a4b94ee 100644 --- a/drivers/infiniband/hw/mana/mana_ib.h +++ b/drivers/infiniband/hw/mana/mana_ib.h @@ -48,6 +48,7 @@ struct mana_ib_adapter_caps { struct mana_ib_dev { struct ib_device ib_dev; struct gdma_dev *gdma_dev; + struct gdma_queue *fatal_err_eq; struct mana_ib_adapter_caps adapter_caps; }; @@ -228,4 +229,8 @@ int mana_ib_query_gid(struct ib_device *ibdev, u32 port, int index, void mana_ib_disassociate_ucontext(struct ib_ucontext *ibcontext); int mana_ib_gd_query_adapter_caps(struct mana_ib_dev *mdev); + +void mana_ib_gd_create_rnic_adapter(struct mana_ib_dev *mdev); + +void mana_ib_gd_destroy_rnic_adapter(struct mana_ib_dev *mdev); #endif From patchwork Fri Feb 2 15:06:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Konstantin Taranov X-Patchwork-Id: 195902 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:9bc1:b0:106:209c:c626 with SMTP id op1csp503678dyc; Fri, 2 Feb 2024 07:27:43 -0800 (PST) X-Google-Smtp-Source: AGHT+IHj7OGnx9kyBGjZzK6vsQ4nrm1O9XbuQiq8cKbyvGB3jPQn2eNRr1DzBwtEVOlQkwkZCDrV X-Received: by 2002:a05:6a20:4306:b0:19e:4f18:b95a with SMTP id h6-20020a056a20430600b0019e4f18b95amr2943720pzk.4.1706887662909; Fri, 02 Feb 2024 07:27:42 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706887662; cv=pass; d=google.com; s=arc-20160816; b=DukB2PlW/huE6gtfU9hvRLIAmvlovoOwisHw8OqJiFb3IJymqQ3xMCufr3y07CNYeW y7pn0AmNhLd1M1tKrelAQ7S+TiXq4QF5DtFFgSHE7X4bcf8s/Ovjf35rditjjpTHwQhY d48kkgMWP3HQFGGS+w4UFTDIy57SL1GHqMnVhDK6sTGhgMLydddDSTVLdmo1rIazYs+n fl5ad42DzDXEY48wJfg6crk8idJac7QeemThDs57B947bi6D4I6he2mUEguZNHesYbk4 9iR5wV/OKm1rUfWIjxUAAN5FR8BXEsocRS9vzMCj1q1476RlaI58E0C9ChOS+6QfhxeW 9dSg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-id:precedence:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-filter; bh=FQMJvC08Xx+rruS+xJDdv53mAl3GJPxfohVLYNulxEA=; fh=QY2GPBS1UDoCU+wnd2caFy63qZ4iYBEMZHfg9JEfk9I=; b=HlM5EMOVx8Sp3hl+31oHmTAdl1QgSRmxYMR2L8+nEBQSLVCkV7EVEObrxZRYtJ2NwR ++va6RLhuqfxnOyMq/aZwbCF03Hu7Y+9Fq8OibGSvOvmaly9JoZWY956NVpP4loSH5oC yKfYp1RzC4Qw5B6fmhkFXtITZveArQC0mdPIoAHvNYyh4GSHhZgeV6gQMUSAtOTIJLmU X6jE+w/eWivdfslZnS5JMlTlnAFfMg3m8HN+t51dgNM1BMwoNeM45W+WZw8X/guAYxty 8t5srtPILUDOrzKmY9sFhsJMn4qCld2k8HNe4YBE8i9Akp3pySQvYDUxdOPhr7Y2ezE/ ZAgw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b="J/s/unMB"; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-49987-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-49987-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com X-Forwarded-Encrypted: i=1; AJvYcCXt20Ua1qtx889SJUqU/UX/ZVzzN/ONP4c11t6VaR+QvPgxozyoRD2ARwU6RWrRbV4cE/V15q+zk6+2YWNM8PFPF/PHOg== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id f1-20020a63dc41000000b005d8b69f8844si1796312pgj.498.2024.02.02.07.27.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Feb 2024 07:27:42 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-49987-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b="J/s/unMB"; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-49987-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-49987-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id A2397B2B7FE for ; Fri, 2 Feb 2024 15:08:06 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 069B2145B3B; Fri, 2 Feb 2024 15:06:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b="J/s/unMB" Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8BD0114463E; Fri, 2 Feb 2024 15:06:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=13.77.154.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706886413; cv=none; b=iYYAY7i1HQMYKuHD5cIZsZDTZmpbOsblObWb/oo2XMDqW8QOdImXMz+cGrSeaRMTIpdhtLYA7Gi/8z1TGS3rT5TSrSiNtm3sY83bi8x3UqRCynyWhtJtE7nVdgz8jADQQQv8sjFYXNA+ZU/SE7wGK5X9YCk4BuloKugUfSrNOQY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706886413; c=relaxed/simple; bh=GI6QY98SEMz0zqOuQidUbYonmgSgitJXQUyspqe4rNo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=PWVvSadJq5ghWeiQJauKfmQP3AYkMhnaMEDlQpsKztHrcXMNp0jblNE3isR5gYtUKXg6+U6uF2O5R4uvze16zGKIqark1qyzeyQs5TzLGCJsV1TvhpiMmcK0w3jg1eMktse3CUwm5fTsOmDy7df1boIlpct6v+E7PpnyNH0iF6o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com; spf=pass smtp.mailfrom=linux.microsoft.com; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b=J/s/unMB; arc=none smtp.client-ip=13.77.154.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Received: from linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net (linux.microsoft.com [13.77.154.182]) by linux.microsoft.com (Postfix) with ESMTPSA id 6C4A720B2002; Fri, 2 Feb 2024 07:06:45 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 6C4A720B2002 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1706886405; bh=FQMJvC08Xx+rruS+xJDdv53mAl3GJPxfohVLYNulxEA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=J/s/unMBx0HErlSr3q27qi3Jy0v2LjutTru7nKUQkJ9T77pL2tqEGDkOMfiGlNRj8 gIM2iy/jCEsVFWuNugwTIUrtTc/gFpmjWJkT6kLhawfrSP7Q2tpXr/BTMkrDmXNEAU nozq7QsV7+RUKp87FRko2QuYQA0dBH129DfsThbw= From: Konstantin Taranov To: kotaranov@microsoft.com, sharmaajay@microsoft.com, longli@microsoft.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH rdma-next v2 2/5] RDMA/mana_ib: Create and destroy rnic adapter Date: Fri, 2 Feb 2024 07:06:34 -0800 Message-Id: <1706886397-16600-3-git-send-email-kotaranov@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1706886397-16600-1-git-send-email-kotaranov@linux.microsoft.com> References: <1706886397-16600-1-git-send-email-kotaranov@linux.microsoft.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789801437916026001 X-GMAIL-MSGID: 1789801437916026001 This patch adds RNIC creation and destruction. If creation of RNIC fails, we support only RAW QPs as they are served by ethernet driver. Signed-off-by: Konstantin Taranov --- drivers/infiniband/hw/mana/main.c | 31 +++++++++++++++++++++++++++++++ drivers/infiniband/hw/mana/mana_ib.h | 29 +++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+) diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c index c64d569..33cd69e 100644 --- a/drivers/infiniband/hw/mana/main.c +++ b/drivers/infiniband/hw/mana/main.c @@ -581,14 +581,31 @@ static void mana_ib_destroy_eqs(struct mana_ib_dev *mdev) void mana_ib_gd_create_rnic_adapter(struct mana_ib_dev *mdev) { + struct mana_rnic_create_adapter_resp resp = {}; + struct mana_rnic_create_adapter_req req = {}; + struct gdma_context *gc = mdev_to_gc(mdev); int err; + mdev->adapter_handle = INVALID_MANA_HANDLE; + err = mana_ib_create_eqs(mdev); if (err) { ibdev_err(&mdev->ib_dev, "Failed to create EQs for RNIC err %d", err); goto cleanup; } + mana_gd_init_req_hdr(&req.hdr, MANA_IB_CREATE_ADAPTER, sizeof(req), sizeof(resp)); + req.hdr.req.msg_version = GDMA_MESSAGE_V2; + req.hdr.dev_id = gc->mana_ib.dev_id; + req.notify_eq_id = mdev->fatal_err_eq->id; + + err = mana_gd_send_request(gc, sizeof(req), &req, sizeof(resp), &resp); + if (err) { + ibdev_err(&mdev->ib_dev, "Failed to create RNIC adapter err %d", err); + goto cleanup; + } + mdev->adapter_handle = resp.adapter; + return; cleanup: @@ -599,5 +616,19 @@ void mana_ib_gd_create_rnic_adapter(struct mana_ib_dev *mdev) void mana_ib_gd_destroy_rnic_adapter(struct mana_ib_dev *mdev) { + struct mana_rnic_destroy_adapter_resp resp = {}; + struct mana_rnic_destroy_adapter_req req = {}; + struct gdma_context *gc; + + if (!rnic_is_enabled(mdev)) + return; + + gc = mdev_to_gc(mdev); + mana_gd_init_req_hdr(&req.hdr, MANA_IB_DESTROY_ADAPTER, sizeof(req), sizeof(resp)); + req.hdr.dev_id = gc->mana_ib.dev_id; + req.adapter = mdev->adapter_handle; + + mana_gd_send_request(gc, sizeof(req), &req, sizeof(resp), &resp); + mdev->adapter_handle = INVALID_MANA_HANDLE; mana_ib_destroy_eqs(mdev); } diff --git a/drivers/infiniband/hw/mana/mana_ib.h b/drivers/infiniband/hw/mana/mana_ib.h index a4b94ee..96454cf 100644 --- a/drivers/infiniband/hw/mana/mana_ib.h +++ b/drivers/infiniband/hw/mana/mana_ib.h @@ -48,6 +48,7 @@ struct mana_ib_adapter_caps { struct mana_ib_dev { struct ib_device ib_dev; struct gdma_dev *gdma_dev; + mana_handle_t adapter_handle; struct gdma_queue *fatal_err_eq; struct mana_ib_adapter_caps adapter_caps; }; @@ -115,6 +116,8 @@ struct mana_ib_rwq_ind_table { enum mana_ib_command_code { MANA_IB_GET_ADAPTER_CAP = 0x30001, + MANA_IB_CREATE_ADAPTER = 0x30002, + MANA_IB_DESTROY_ADAPTER = 0x30003, }; struct mana_ib_query_adapter_caps_req { @@ -143,6 +146,32 @@ struct mana_ib_query_adapter_caps_resp { u32 max_inline_data_size; }; /* HW Data */ +struct mana_rnic_create_adapter_req { + struct gdma_req_hdr hdr; + u32 notify_eq_id; + u32 reserved; + u64 feature_flags; +}; /*HW Data */ + +struct mana_rnic_create_adapter_resp { + struct gdma_resp_hdr hdr; + mana_handle_t adapter; +}; /* HW Data */ + +struct mana_rnic_destroy_adapter_req { + struct gdma_req_hdr hdr; + mana_handle_t adapter; +}; /*HW Data */ + +struct mana_rnic_destroy_adapter_resp { + struct gdma_resp_hdr hdr; +}; /* HW Data */ + +static inline bool rnic_is_enabled(struct mana_ib_dev *mdev) +{ + return mdev->adapter_handle != INVALID_MANA_HANDLE; +} + static inline struct gdma_context *mdev_to_gc(struct mana_ib_dev *mdev) { return mdev->gdma_dev->gdma_context; From patchwork Fri Feb 2 15:06:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Konstantin Taranov X-Patchwork-Id: 195884 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:9bc1:b0:106:209c:c626 with SMTP id op1csp492082dyc; Fri, 2 Feb 2024 07:09:47 -0800 (PST) X-Google-Smtp-Source: AGHT+IG8M2IAJHz68M71HdDge8ZhKtWzZDkj9hOvUdMqtAkSl1yNbeyod1Ops4p0t4DvNMEmRwq2 X-Received: by 2002:a17:907:20d9:b0:a35:fbc4:4c2f with SMTP id qq25-20020a17090720d900b00a35fbc44c2fmr5772609ejb.25.1706886586730; Fri, 02 Feb 2024 07:09:46 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706886586; cv=pass; d=google.com; s=arc-20160816; b=tu/aVSUI7FGte6mOeF8lhECfP0V/r2nLDoAbc5dcQuqfawYgjEJG+Bnb1G7/xZwNx9 Gqw+vPXfCl/BwDFvlY68ad9omkaxtYZNKb3PU122O+aiOXosHH+d+hAClWFKQZd/Rl0e Ywz4QuDjuZV38lWaElmqDXOLbdjtL8yzDGZ/uhJXgdtkP133vlLuCgOZbJfVLEFYkN3L TnBd5SLzPZyYPjvoVyawActSn8OEl43sQjt6JSaFqCXojnJaVtRNZI54ECPhGA6npJLH mt3xxOMJCzwNBlXSbQxpwMysc9m1H083PiO+ASDDLjQ32fQJNUcAH/DJu3gmqOc94jW1 1l0A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-id:precedence:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-filter; bh=5yXFaEp9vKBshRa+NgGc9Zh0uI+t9u7kQk3cCqXTndQ=; fh=cSpDgcXlNnGUw14PF0/7lYnjLOoeiS2hqTXl2Dx1yAQ=; b=gF10ikh49AfSo4udZHiqCmlQzRm4hm8VtQ/CkUdHUeeQTo2aqPTkPhtzofQZF40z20 9uNPoQZAEoqlZEJA9OGZ8Kr/X6ci9meQ9BkcWBcwj99eEuZLCrHHCEhY3qR3EosXz555 ZDHrosW0plQum1zkZ33tD+btj1nrc5Gjwyx2vdEt1Htf5Q6Ct5vCG4H1j/WSA5gRF8ug VolAdKLhD4AKVqF7xbEdbiS0jxBv1pgGHrDhEHBy4mHO+WBMaSBXUnTjnWais0waOo+z DWTZHRZBi1Kkl3IlW6C4e1LaEQJAq9tsrGMLrKIHa9GMMyHUshjbpDI5qZ8X59dP7u4g lG5w==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=nvzo2Fa2; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-49991-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-49991-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com X-Forwarded-Encrypted: i=1; AJvYcCVCCdYYjYdRpocXFShsLXB/6Y21x/DaYNg3A3aTLlWVmJO5RQpdG2EReeMYHBZIYzqYr6riYN5/fAUwO8n+JjFkSt6g8Q== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id cd21-20020a170906b35500b00a35ddad87e5si923709ejb.930.2024.02.02.07.09.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Feb 2024 07:09:46 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-49991-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=nvzo2Fa2; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-49991-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-49991-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 559EC1F2D49D for ; Fri, 2 Feb 2024 15:08:41 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 77D8F148307; Fri, 2 Feb 2024 15:06:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b="nvzo2Fa2" Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8BD3D145328; Fri, 2 Feb 2024 15:06:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=13.77.154.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706886415; cv=none; b=dw0GDQd4PEU4u7l7x31G2bVfkg+x2aQ8x4OknyKdPr8huxPhSpnNFuGuzCrTIksFV44Y2qhAPrZcFEtENsck15cMTi6P8CCfDsO0Ug1IxkKKTtbqHuy4gO8bhtmuqLjyzvPXrb9WR7Uosu1NVQUp9xhp/A/FG2V15U0KijeclY0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706886415; c=relaxed/simple; bh=lwkOSAnGEnMRxgE2eYpnXuty91szYnqPYwpQeHF0pik=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=g6rtHq2AojjGiR1ro85PYueNJv7xoQhUadm2xJBnApRmptJKWeyrwaYsNCWtbKMSPKZm/KhCfkNkBIrjJjPE2uShy69LdosKvyGliP3zChGLnewmYNOjZCgSlT7I0w1P4QqZy9imJ5MxmHUe4ibBQ5l2MwfApWL7vmXmaUJ2ir8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com; spf=pass smtp.mailfrom=linux.microsoft.com; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b=nvzo2Fa2; arc=none smtp.client-ip=13.77.154.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Received: from linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net (linux.microsoft.com [13.77.154.182]) by linux.microsoft.com (Postfix) with ESMTPSA id 844B920B2003; Fri, 2 Feb 2024 07:06:45 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 844B920B2003 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1706886405; bh=5yXFaEp9vKBshRa+NgGc9Zh0uI+t9u7kQk3cCqXTndQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nvzo2Fa2Rg1awmzVwTn0wIOcljM+JpU0PKlle5AKjZb6CxMHAGHFTNIrkzVcuuEtJ rW/WM72KzuHfZTLakJH+fKzIe2+0jzjVWJfqMJOfdI1ytTg/qEOdnsW5JEp6P6eGbn 2Qz2fHvakTs+x1g7fmiA//2l0PUfZ01E5ebca/H4= From: Konstantin Taranov To: kotaranov@microsoft.com, sharmaajay@microsoft.com, longli@microsoft.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH rdma-next v2 3/5] RDMA/mana_ib: Implement port parameters Date: Fri, 2 Feb 2024 07:06:35 -0800 Message-Id: <1706886397-16600-4-git-send-email-kotaranov@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1706886397-16600-1-git-send-email-kotaranov@linux.microsoft.com> References: <1706886397-16600-1-git-send-email-kotaranov@linux.microsoft.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789800309696006303 X-GMAIL-MSGID: 1789800309696006303 Implement port parameters for RNIC. Only port 1 is RoCEv2 capable. Signed-off-by: Konstantin Taranov --- drivers/infiniband/hw/mana/device.c | 2 ++ drivers/infiniband/hw/mana/main.c | 37 +++++++++++++++++++++++++++++++++++- drivers/infiniband/hw/mana/mana_ib.h | 4 ++++ 3 files changed, 42 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/mana/device.c b/drivers/infiniband/hw/mana/device.c index d8e8b10..11b0410 100644 --- a/drivers/infiniband/hw/mana/device.c +++ b/drivers/infiniband/hw/mana/device.c @@ -29,12 +29,14 @@ .destroy_rwq_ind_table = mana_ib_destroy_rwq_ind_table, .destroy_wq = mana_ib_destroy_wq, .disassociate_ucontext = mana_ib_disassociate_ucontext, + .get_link_layer = mana_ib_get_link_layer, .get_port_immutable = mana_ib_get_port_immutable, .mmap = mana_ib_mmap, .modify_qp = mana_ib_modify_qp, .modify_wq = mana_ib_modify_wq, .query_device = mana_ib_query_device, .query_gid = mana_ib_query_gid, + .query_pkey = mana_ib_query_pkey, .query_port = mana_ib_query_port, .reg_user_mr = mana_ib_reg_user_mr, diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c index 33cd69e..3e05a62 100644 --- a/drivers/infiniband/hw/mana/main.c +++ b/drivers/infiniband/hw/mana/main.c @@ -492,7 +492,42 @@ int mana_ib_query_device(struct ib_device *ibdev, struct ib_device_attr *props, int mana_ib_query_port(struct ib_device *ibdev, u32 port, struct ib_port_attr *props) { - /* This version doesn't return port properties */ + struct net_device *ndev = mana_ib_get_netdev(ibdev, port); + + if (!ndev) + return -EINVAL; + + memset(props, 0, sizeof(*props)); + props->max_mtu = IB_MTU_4096; + props->active_mtu = ib_mtu_int_to_enum(ndev->mtu); + + if (netif_carrier_ok(ndev) && netif_running(ndev)) { + props->state = IB_PORT_ACTIVE; + props->phys_state = IB_PORT_PHYS_STATE_LINK_UP; + } else { + props->state = IB_PORT_DOWN; + props->phys_state = IB_PORT_PHYS_STATE_DISABLED; + } + + props->active_width = IB_WIDTH_4X; + props->active_speed = IB_SPEED_EDR; + props->pkey_tbl_len = 1; + if (port == 1) + props->gid_tbl_len = 16; + + return 0; +} + +enum rdma_link_layer mana_ib_get_link_layer(struct ib_device *device, u32 port_num) +{ + return IB_LINK_LAYER_ETHERNET; +} + +int mana_ib_query_pkey(struct ib_device *ibdev, u32 port, u16 index, u16 *pkey) +{ + if (index != 0) + return -EINVAL; + *pkey = IB_DEFAULT_PKEY_FULL; return 0; } diff --git a/drivers/infiniband/hw/mana/mana_ib.h b/drivers/infiniband/hw/mana/mana_ib.h index 96454cf..196f3c8 100644 --- a/drivers/infiniband/hw/mana/mana_ib.h +++ b/drivers/infiniband/hw/mana/mana_ib.h @@ -262,4 +262,8 @@ int mana_ib_query_gid(struct ib_device *ibdev, u32 port, int index, void mana_ib_gd_create_rnic_adapter(struct mana_ib_dev *mdev); void mana_ib_gd_destroy_rnic_adapter(struct mana_ib_dev *mdev); + +int mana_ib_query_pkey(struct ib_device *ibdev, u32 port, u16 index, u16 *pkey); + +enum rdma_link_layer mana_ib_get_link_layer(struct ib_device *device, u32 port_num); #endif From patchwork Fri Feb 2 15:06:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Konstantin Taranov X-Patchwork-Id: 195901 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:9bc1:b0:106:209c:c626 with SMTP id op1csp503653dyc; Fri, 2 Feb 2024 07:27:41 -0800 (PST) X-Google-Smtp-Source: AGHT+IF51MPaOXSC0CBBBMR6o8PkZeAvkug9rMeMrQ2kpF0oR/DPqdtrOdnJRaBov+Z79LYp2jce X-Received: by 2002:a05:6a20:a622:b0:19c:60e7:a984 with SMTP id bb34-20020a056a20a62200b0019c60e7a984mr7228641pzb.27.1706887660816; Fri, 02 Feb 2024 07:27:40 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706887660; cv=pass; d=google.com; s=arc-20160816; b=FAQrmpOroTEfSDYOtnemmMSsx0GZxXreaKD1+9yAE1rFeSZxcckB2yOcmQYBgj9slS PwNyVG2J/EH00Bwj5hpY/QYhyQinbevKFZDPpLJePWi2XzmVzyrS9s6EwDzApekyjmpE DulMdlo02xbKEj1fkgCREEDwx3YX+YHb/B0+EfvqHuF5a/JCIttsACKBi7pxEMJ/lllg DS7pfVflYaAIsP6VWw5nWhSDVDTT6r+q+JZ7ZFrimiJdgDe/xY2goPsDW7BlejIYDSfn dmIExcXGK5f5op1ttlKMN22/UApzybvdcG3BETcDa5Gp4eNX4t5JWHFo66jSpVN7y7YX NUKQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-id:precedence:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-filter; bh=tbOBGmywCw1yErcHhceCuwXOroc2dZSPwsDCR0qMCyY=; fh=ZJeiJVmo+pSwNhzUhR6QCdI8QduT/Yu+7go1sxxTxaY=; b=qxcrP5I3BiXCV7a5aAEih00KNcpryybtyXUCdK84Bno1Q7UH9PuJy5p0iFqwfuW91G jyIhPq8xS8HlX+QZkGYCP+OrqMweZimrCy2gEMiq47t5+JvA2OsQyeOz6TOB3cqhm2OM HuveEaapxRbZDfWwL6mk7Y0qvcnK1Xbo+E00StxOVvFpC9AQ7pfalVh15xafJD8N0Bep HSLbM7P/h5IyAsYB6lXzi/ySA72WM4AJEKYCH5uDxmBN29OxpcLENUzS2GUptkPJ0svp GuGY3AboiReTIG+MoqskJs+GpXzT49phWIeCPjNXG2LdtKSkJEkpthn16dOHynEOFbFt LuWg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=L9d5b9x0; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-49988-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-49988-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com X-Forwarded-Encrypted: i=1; AJvYcCWmT/ox29Rca1KR/37uEAIH8W6l22bok62ha4FYJh7+87WGyt9DBKs+BqsFDMbpLJIbI9B4Cn9jSKh7DGzSCJw5JPTaqw== Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org. [2604:1380:40f1:3f00::1]) by mx.google.com with ESMTPS id y19-20020a62b513000000b006dbbae74070si1693820pfe.184.2024.02.02.07.27.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Feb 2024 07:27:40 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-49988-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=L9d5b9x0; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-49988-ouuuleilei=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-49988-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id E82F2B2B36A for ; Fri, 2 Feb 2024 15:08:04 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id F0A171474BF; Fri, 2 Feb 2024 15:06:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b="L9d5b9x0" Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8BE13145341; Fri, 2 Feb 2024 15:06:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=13.77.154.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706886413; cv=none; b=gyFGmRGkwA7jDklzCByPdf14flw277O315aB+qO4w8fmUTQEqeFJCAO+hWMJaUInvY4XX+h2XDlfyMew0hXW18HO6YWr313ZvVkqW3eC9VAKnhncApOmyDijGrnj+CuvmvenPpd2EJPTzPYtRnxKjTkXa4GHrL8syM6NeoNhsJ0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706886413; c=relaxed/simple; bh=xIgDQeQggRzU0YUbTvsi5ADzqvMrctEjnbNPqcT5orI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Vt62EdGECu97VUMiPV+cmUF60hMpw5M8FazOdRvOdYiVQu5kU2pjmmUJbKLVLmVFa35IFT3faQPtE77zbbN34hLvFDsNV7a01d+UE1eUW1XLdwnG8hSPHv+7YJoIejW3EGOhA6Mm6e12/V28jpgbfEJ5qWspJrRfG0iLTPFlQCw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com; spf=pass smtp.mailfrom=linux.microsoft.com; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b=L9d5b9x0; arc=none smtp.client-ip=13.77.154.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Received: from linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net (linux.microsoft.com [13.77.154.182]) by linux.microsoft.com (Postfix) with ESMTPSA id 9CAFF20B2004; Fri, 2 Feb 2024 07:06:45 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 9CAFF20B2004 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1706886405; bh=tbOBGmywCw1yErcHhceCuwXOroc2dZSPwsDCR0qMCyY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=L9d5b9x09yTEVlhqzmyDmgdBpcsfrZGnqOCy+kbh+bYb2ZKG8k/3CnwyuCrCJOOlX Mso9fdUN3BuWhJUuPOEvWa30dJ6YD/xgA9Ao7mXREHJNZY0nj2jX665hYRO7TjvhyH HS9nC6DExliRmNyLfA8f1u/QBzgLoo0x7s0j1Nx0= From: Konstantin Taranov To: kotaranov@microsoft.com, sharmaajay@microsoft.com, longli@microsoft.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH rdma-next v2 4/5] RDMA/mana_ib: Enable RoCE on port 1 Date: Fri, 2 Feb 2024 07:06:36 -0800 Message-Id: <1706886397-16600-5-git-send-email-kotaranov@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1706886397-16600-1-git-send-email-kotaranov@linux.microsoft.com> References: <1706886397-16600-1-git-send-email-kotaranov@linux.microsoft.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789801435795821764 X-GMAIL-MSGID: 1789801435795821764 Set netdev and RoCEv2 flag to be used in GID population. We need GIDs of the master netdev and mc->ports stores slave devices. Signed-off-by: Konstantin Taranov --- drivers/infiniband/hw/mana/device.c | 15 +++++++++++++++ drivers/infiniband/hw/mana/main.c | 16 ++++++++++++---- 2 files changed, 27 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/hw/mana/device.c b/drivers/infiniband/hw/mana/device.c index 11b0410..2b362f5 100644 --- a/drivers/infiniband/hw/mana/device.c +++ b/drivers/infiniband/hw/mana/device.c @@ -53,6 +53,7 @@ static int mana_ib_probe(struct auxiliary_device *adev, { struct mana_adev *madev = container_of(adev, struct mana_adev, adev); struct gdma_dev *mdev = madev->mdev; + struct net_device *upper_ndev; struct mana_context *mc; struct mana_ib_dev *dev; int ret; @@ -79,6 +80,20 @@ static int mana_ib_probe(struct auxiliary_device *adev, dev->ib_dev.num_comp_vectors = 1; dev->ib_dev.dev.parent = mdev->gdma_context->dev; + rcu_read_lock(); /* required to get upper dev */ + upper_ndev = netdev_master_upper_dev_get_rcu(mc->ports[0]); + if (!upper_ndev) { + rcu_read_unlock(); + ibdev_err(&dev->ib_dev, "Failed to get master netdev"); + goto free_ib_device; + } + ret = ib_device_set_netdev(&dev->ib_dev, upper_ndev, 1); + rcu_read_unlock(); + if (ret) { + ibdev_err(&dev->ib_dev, "Failed to set ib netdev, ret %d", ret); + goto free_ib_device; + } + ret = mana_gd_register_device(&mdev->gdma_context->mana_ib); if (ret) { ibdev_err(&dev->ib_dev, "Failed to register device, ret %d", diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c index 3e05a62..645abf3 100644 --- a/drivers/infiniband/hw/mana/main.c +++ b/drivers/infiniband/hw/mana/main.c @@ -462,11 +462,19 @@ int mana_ib_mmap(struct ib_ucontext *ibcontext, struct vm_area_struct *vma) int mana_ib_get_port_immutable(struct ib_device *ibdev, u32 port_num, struct ib_port_immutable *immutable) { - /* - * This version only support RAW_PACKET - * other values need to be filled for other types - */ + struct mana_ib_dev *mdev = container_of(ibdev, struct mana_ib_dev, ib_dev); + struct ib_port_attr attr; + int err; + + err = ib_query_port(ibdev, port_num, &attr); + if (err) + return err; + + immutable->pkey_tbl_len = attr.pkey_tbl_len; + immutable->gid_tbl_len = attr.gid_tbl_len; immutable->core_cap_flags = RDMA_CORE_PORT_RAW_PACKET; + if (port_num == 1 && rnic_is_enabled(mdev)) + immutable->core_cap_flags |= RDMA_CORE_PORT_IBA_ROCE_UDP_ENCAP; return 0; } From patchwork Fri Feb 2 15:06:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Konstantin Taranov X-Patchwork-Id: 195883 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7301:9bc1:b0:106:209c:c626 with SMTP id op1csp492071dyc; Fri, 2 Feb 2024 07:09:45 -0800 (PST) X-Google-Smtp-Source: AGHT+IHLv1z5JFJdqBjQUl4IyenntZtk1CI8Kiqf2/qceK5EoQ+zyoqiOea8tDGURm1YxJ9kjwNG X-Received: by 2002:a17:907:75cc:b0:a31:13e5:8fb3 with SMTP id jl12-20020a17090775cc00b00a3113e58fb3mr1524798ejc.38.1706886585669; Fri, 02 Feb 2024 07:09:45 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706886585; cv=pass; d=google.com; s=arc-20160816; b=bz39NWbxH37wDs49sKebl32gDptKyYFWTyFRncuWenwd6K+zQlui3zU6KWa5hccJea DnHAh8Cp37pLfvk5ggy6MhVximPqF8aaGl+sDPhPUtMYlhfO/ea/FqjkrKCCTmfaqyY4 V4trDRcKmnlUgm+lXP5YlkLIP3ANecdO2bDjV2HrFzh4R5iW6Ajk9/7fET5pIfj/ZDap JhNdsj3e2NCJKc+YCVBip8+EyQTq3epRQ34FdYyYa+SJdXz32w7XivQDLqnOjxdegD38 9RN648FKQkG6TTJ1yCXfcHp0yOe6ErpfXYgVizk44sLFzBfiR+JnW+L0sI+PHORJNB7n 55sg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-id:precedence:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-filter; bh=KFGjzl3gNK7i8STIUzlk+L5uIQAXSRDUsl59ComKeIk=; fh=1OKOUfJPR2H05qmmzGbnusmBt8GVABSQFOfJsu5HjpQ=; b=P557clZ1NYjoX1t1cyIsKw/vRvvE8/PnKJlql+1toN0xGjoREJd/ayH79QCycfH3at O1v4iHIclgIW8vVDzm0VqeFMaENNY4syLRrUH7QVjyWue/YgXayPgdX2nqj9sU9Yk09d KM7AsUiv8Kiuw/6uK4iUlGymuFi1Wgm7icGj6MF8gZkfhthLaIoi1jVCSXMSdFUECu52 bqS36hnDuVkN1CYJLfgIOI51qD4ftjn2ZCkojKm4dUUiEBchF6au/4O4XvwL/78rZx/c r3KGtmXqOWkRE9B3cbPh4x57+deeDXgbcb2j7kKbmb5BfRc2GAH0VOmRrewBDDXxIw0u q81w==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=DtJMLrQD; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-49990-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-49990-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com X-Forwarded-Encrypted: i=1; AJvYcCXSbcmxT/pgF2qCscQuYlXgBaWT3yMRNWoZvRGazPoZDXGPiNaVcjWCmr6B5G5rHcMKU3Iu3ifERp6Ol93Heq9pDrZyFw== Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id s6-20020a170906c30600b00a34a0d9bca7si902865ejz.109.2024.02.02.07.09.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Feb 2024 07:09:45 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-49990-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.microsoft.com header.s=default header.b=DtJMLrQD; arc=pass (i=1 spf=pass spfdomain=linux.microsoft.com dkim=pass dkdomain=linux.microsoft.com dmarc=pass fromdomain=linux.microsoft.com); spf=pass (google.com: domain of linux-kernel+bounces-49990-ouuuleilei=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-49990-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id AFF9D1F2D3E1 for ; Fri, 2 Feb 2024 15:08:38 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 25636148301; Fri, 2 Feb 2024 15:06:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b="DtJMLrQD" Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3D58D145B0A; Fri, 2 Feb 2024 15:06:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=13.77.154.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706886414; cv=none; b=KfgprhXRjLcUqyDEn5luWJvcDeMgG72zSspuTzdo3+xzlOoTclQhklX2/E5+fDbVk1f07HxRvMPGX5Z0bMCMjqxAoULh9vpcAW6NK8IvoAkNP148d0v+eqIQ3hWsGXqbR6pr+17//sXBd8JV/yQjS4mrP+S7Oen0I19Co2YpwQ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706886414; c=relaxed/simple; bh=9qFJEo1gG8jmdM1WliqaxN63JvvNL0jjXCrWGwgRnmg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=OddUFNQQA8VUleygSyuO/+nfhGvvOrouw+MlUXgPp8/K/lUKhZKa/wLCSwQkNXdCco1erCAj/KolEITzh7JYWWFdSMTFKfsMhIaXqA4V7NLd9/jmX+bOHXS9XQv8HZwvOO8fpG8g7sRa9GSTUkks6DiH/fQsjBrZljUk3ZpIAVs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com; spf=pass smtp.mailfrom=linux.microsoft.com; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b=DtJMLrQD; arc=none smtp.client-ip=13.77.154.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Received: from linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net (linux.microsoft.com [13.77.154.182]) by linux.microsoft.com (Postfix) with ESMTPSA id B47C320B2005; Fri, 2 Feb 2024 07:06:45 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com B47C320B2005 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1706886405; bh=KFGjzl3gNK7i8STIUzlk+L5uIQAXSRDUsl59ComKeIk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DtJMLrQDIHdR4PR5uueY/9AZnTSN6NF1ihRxYdQyffltLgof1sF8DHNHdPLC1FDPx op31XLCkKQtg3kJzYkRkNCky2jtrGXAWARr4icPGuajr4kzW89y7XT6xdqZ/DHUstx u4fFStqQ0PXwqj2RjMi6VqYYgBBzIILUbktKwhVM= From: Konstantin Taranov To: kotaranov@microsoft.com, sharmaajay@microsoft.com, longli@microsoft.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH rdma-next v2 5/5] RDMA/mana_ib: Adding and deleting GIDs Date: Fri, 2 Feb 2024 07:06:37 -0800 Message-Id: <1706886397-16600-6-git-send-email-kotaranov@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1706886397-16600-1-git-send-email-kotaranov@linux.microsoft.com> References: <1706886397-16600-1-git-send-email-kotaranov@linux.microsoft.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1789800308455953982 X-GMAIL-MSGID: 1789800308455953982 Implement add_gid and del_gid for RNIC. We support ipv4 and ipv6 addresses. Signed-off-by: Konstantin Taranov --- drivers/infiniband/hw/mana/device.c | 2 ++ drivers/infiniband/hw/mana/main.c | 66 ++++++++++++++++++++++++++++++++++++ drivers/infiniband/hw/mana/mana_ib.h | 37 ++++++++++++++++++++ 3 files changed, 105 insertions(+) diff --git a/drivers/infiniband/hw/mana/device.c b/drivers/infiniband/hw/mana/device.c index 2b362f5..9fb515b 100644 --- a/drivers/infiniband/hw/mana/device.c +++ b/drivers/infiniband/hw/mana/device.c @@ -15,6 +15,7 @@ .driver_id = RDMA_DRIVER_MANA, .uverbs_abi_ver = MANA_IB_UVERBS_ABI_VERSION, + .add_gid = mana_ib_gd_add_gid, .alloc_pd = mana_ib_alloc_pd, .alloc_ucontext = mana_ib_alloc_ucontext, .create_cq = mana_ib_create_cq, @@ -23,6 +24,7 @@ .create_wq = mana_ib_create_wq, .dealloc_pd = mana_ib_dealloc_pd, .dealloc_ucontext = mana_ib_dealloc_ucontext, + .del_gid = mana_ib_gd_del_gid, .dereg_mr = mana_ib_dereg_mr, .destroy_cq = mana_ib_destroy_cq, .destroy_qp = mana_ib_destroy_qp, diff --git a/drivers/infiniband/hw/mana/main.c b/drivers/infiniband/hw/mana/main.c index 645abf3..282c024 100644 --- a/drivers/infiniband/hw/mana/main.c +++ b/drivers/infiniband/hw/mana/main.c @@ -675,3 +675,69 @@ void mana_ib_gd_destroy_rnic_adapter(struct mana_ib_dev *mdev) mdev->adapter_handle = INVALID_MANA_HANDLE; mana_ib_destroy_eqs(mdev); } + +int mana_ib_gd_add_gid(const struct ib_gid_attr *attr, void **context) +{ + struct mana_ib_dev *mdev = container_of(attr->device, struct mana_ib_dev, ib_dev); + enum rdma_network_type ntype = rdma_gid_attr_network_type(attr); + struct mana_rnic_config_addr_resp resp = {}; + struct gdma_context *gc = mdev_to_gc(mdev); + struct mana_rnic_config_addr_req req = {}; + int err; + + if (!rnic_is_enabled(mdev)) + return -EINVAL; + + if (ntype != RDMA_NETWORK_IPV4 && ntype != RDMA_NETWORK_IPV6) { + ibdev_dbg(&mdev->ib_dev, "Unsupported rdma network type %d", ntype); + return -EINVAL; + } + + mana_gd_init_req_hdr(&req.hdr, MANA_IB_CONFIG_IP_ADDR, sizeof(req), sizeof(resp)); + req.hdr.dev_id = gc->mana_ib.dev_id; + req.adapter = mdev->adapter_handle; + req.op = ADDR_OP_ADD; + req.sgid_type = (ntype == RDMA_NETWORK_IPV6) ? SGID_TYPE_IPV6 : SGID_TYPE_IPV4; + copy_in_reverse(req.ip_addr, attr->gid.raw, sizeof(union ib_gid)); + + err = mana_gd_send_request(gc, sizeof(req), &req, sizeof(resp), &resp); + if (err) { + ibdev_err(&mdev->ib_dev, "Failed to config IP addr err %d\n", err); + return err; + } + + return 0; +} + +int mana_ib_gd_del_gid(const struct ib_gid_attr *attr, void **context) +{ + struct mana_ib_dev *mdev = container_of(attr->device, struct mana_ib_dev, ib_dev); + enum rdma_network_type ntype = rdma_gid_attr_network_type(attr); + struct mana_rnic_config_addr_resp resp = {}; + struct gdma_context *gc = mdev_to_gc(mdev); + struct mana_rnic_config_addr_req req = {}; + int err; + + if (!rnic_is_enabled(mdev)) + return -EINVAL; + + if (ntype != RDMA_NETWORK_IPV4 && ntype != RDMA_NETWORK_IPV6) { + ibdev_dbg(&mdev->ib_dev, "Unsupported rdma network type %d", ntype); + return -EINVAL; + } + + mana_gd_init_req_hdr(&req.hdr, MANA_IB_CONFIG_IP_ADDR, sizeof(req), sizeof(resp)); + req.hdr.dev_id = gc->mana_ib.dev_id; + req.adapter = mdev->adapter_handle; + req.op = ADDR_OP_REMOVE; + req.sgid_type = (ntype == RDMA_NETWORK_IPV6) ? SGID_TYPE_IPV6 : SGID_TYPE_IPV4; + copy_in_reverse(req.ip_addr, attr->gid.raw, sizeof(union ib_gid)); + + err = mana_gd_send_request(gc, sizeof(req), &req, sizeof(resp), &resp); + if (err) { + ibdev_err(&mdev->ib_dev, "Failed to config IP addr err %d\n", err); + return err; + } + + return 0; +} diff --git a/drivers/infiniband/hw/mana/mana_ib.h b/drivers/infiniband/hw/mana/mana_ib.h index 196f3c8..2a3e3b0 100644 --- a/drivers/infiniband/hw/mana/mana_ib.h +++ b/drivers/infiniband/hw/mana/mana_ib.h @@ -118,6 +118,7 @@ enum mana_ib_command_code { MANA_IB_GET_ADAPTER_CAP = 0x30001, MANA_IB_CREATE_ADAPTER = 0x30002, MANA_IB_DESTROY_ADAPTER = 0x30003, + MANA_IB_CONFIG_IP_ADDR = 0x30004, }; struct mana_ib_query_adapter_caps_req { @@ -167,6 +168,30 @@ struct mana_rnic_destroy_adapter_resp { struct gdma_resp_hdr hdr; }; /* HW Data */ +enum mana_ib_addr_op { + ADDR_OP_ADD = 1, + ADDR_OP_REMOVE, +}; + +enum sgid_entry_type { + SGID_TYPE_INVALID = 0, + SGID_TYPE_IPV4 = 1, + SGID_TYPE_IPV6 = 2, + SGID_TYPE_HYBRID = 3 +}; + +struct mana_rnic_config_addr_req { + struct gdma_req_hdr hdr; + mana_handle_t adapter; + enum mana_ib_addr_op op; + enum sgid_entry_type sgid_type; + u8 ip_addr[16]; +}; /* HW Data */ + +struct mana_rnic_config_addr_resp { + struct gdma_resp_hdr hdr; +}; /* HW Data */ + static inline bool rnic_is_enabled(struct mana_ib_dev *mdev) { return mdev->adapter_handle != INVALID_MANA_HANDLE; @@ -188,6 +213,14 @@ static inline struct net_device *mana_ib_get_netdev(struct ib_device *ibdev, u32 return mc->ports[port - 1]; } +static inline void copy_in_reverse(u8 *dst, const u8 *src, u32 size) +{ + u32 i; + + for (i = 0; i < size; i++) + dst[size - 1 - i] = src[i]; +} + int mana_ib_install_cq_cb(struct mana_ib_dev *mdev, struct mana_ib_cq *cq); int mana_ib_gd_create_dma_region(struct mana_ib_dev *dev, struct ib_umem *umem, @@ -266,4 +299,8 @@ int mana_ib_query_gid(struct ib_device *ibdev, u32 port, int index, int mana_ib_query_pkey(struct ib_device *ibdev, u32 port, u16 index, u16 *pkey); enum rdma_link_layer mana_ib_get_link_layer(struct ib_device *device, u32 port_num); + +int mana_ib_gd_add_gid(const struct ib_gid_attr *attr, void **context); + +int mana_ib_gd_del_gid(const struct ib_gid_attr *attr, void **context); #endif