From patchwork Tue Sep 19 14:41:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 141972 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp3482693vqi; Tue, 19 Sep 2023 08:39:45 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGw77JNZK65y8QKUyyNqcrsOWGk9ajEDy+GhQDRcRkVDZNtX8zHskDyB3ZJtfcO2UnWIG/q X-Received: by 2002:a05:6300:8002:b0:14c:d5e9:21b8 with SMTP id an2-20020a056300800200b0014cd5e921b8mr11105502pzc.30.1695137985184; Tue, 19 Sep 2023 08:39:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695137985; cv=none; d=google.com; s=arc-20160816; b=tGG4d2whMoPO1egGy09A5iyC9SuEoKF8TUxKZbNWyb4lDmp4afjGkOCy3CJThYvTtg OLCm4J9X30EeQCS9R5G73V44yWf7+yyFM+paxqKEtxI9P2pe6tOuLPA+aigA6+jsubv/ /kPFAr3KtqDqdz1vaVXauleO3q0uMUQCz0PY4GWhep7yxWG5tDU4sgYQPbc7U0xQf2TN tJeafyKzMI4EdcLa9Cc2HCSss+XWVSUp9gP59uRJoXLSkidZj14W7DJDe833Iff70qAU Bw1rTLzVX1mSTDQNT/WSJpNwWIJTVUw/ZfVaNmHCDKSp/SThD9OfZZcBMQ0O/SVfQuRp Nclw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=2pM1iPMxEbHhpM0cC4MR9TZN1sh+tiMrz6RmDDUzn8Y=; fh=2Ks5OhTjwS8gZt4F+VGxBODwUEQ0u2wJBF3lQY5ZqAM=; b=kru3u4Q8GRHWep4JOAbxRtd+Hg1XF/tZoXHqJq09jMnM7KOwTl2Y3ZH89X4o2pTeCH 1Lq3s3lSZNPINlEVji6mk1w1z7PS8qYRIyP2fuDYTcjKGgUsbbWxw0nuW7tULHKfgIlw KWNXUllz8eYsTyMmriMpdppTTkYX+PnMyo2KgnkBNHRENob5dsVFcb0Yl3/Pqt+bRXTY kqhIWBqFHHJYL0zwlMDO23XHPOt2cSG8UWe9Qq+ZESFAP2zrGfkcPjUOih65FM1sghOX 8D+cHGMQNWEi9skrd4r9zoxjoUA2x8WBxOkmQd51A5/dXJ91tkV5r3AG5TT5zmpj3awA jCSw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id 19-20020a631753000000b00578b4edf57esi1373535pgx.797.2023.09.19.08.39.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 08:39:45 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id F3DCD80D44DF; Tue, 19 Sep 2023 07:42:47 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232878AbjISOms (ORCPT + 26 others); Tue, 19 Sep 2023 10:42:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38636 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232316AbjISOmq (ORCPT ); Tue, 19 Sep 2023 10:42:46 -0400 Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com [115.124.30.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BFB4CBF; Tue, 19 Sep 2023 07:42:39 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046060;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0VsRsRjQ_1695134534; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsRsRjQ_1695134534) by smtp.aliyun-inc.com; Tue, 19 Sep 2023 22:42:36 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 01/18] net/smc: decouple ism_dev from SMC-D device dump Date: Tue, 19 Sep 2023 22:41:45 +0800 Message-Id: <1695134522-126655-2-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> References: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 19 Sep 2023 07:42:48 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777481007983736185 X-GMAIL-MSGID: 1777481007983736185 This patch helps to decouple ISM device from SMC-D device, allowing different underlying device forms, such as virtual ISM devices. Signed-off-by: Wen Gu --- net/smc/smc_ism.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c index fbee249..0045fee 100644 --- a/net/smc/smc_ism.c +++ b/net/smc/smc_ism.c @@ -230,12 +230,11 @@ static int smc_nl_handle_smcd_dev(struct smcd_dev *smcd, char smc_pnet[SMC_MAX_PNETID_LEN + 1]; struct smc_pci_dev smc_pci_dev; struct nlattr *port_attrs; + struct device *priv_dev; struct nlattr *attrs; - struct ism_dev *ism; int use_cnt = 0; void *nlh; - ism = smcd->priv; nlh = genlmsg_put(skb, NETLINK_CB(cb->skb).portid, cb->nlh->nlmsg_seq, &smc_gen_nl_family, NLM_F_MULTI, SMC_NETLINK_GET_DEV_SMCD); @@ -250,7 +249,10 @@ static int smc_nl_handle_smcd_dev(struct smcd_dev *smcd, if (nla_put_u8(skb, SMC_NLA_DEV_IS_CRIT, use_cnt > 0)) goto errattr; memset(&smc_pci_dev, 0, sizeof(smc_pci_dev)); - smc_set_pci_values(to_pci_dev(ism->dev.parent), &smc_pci_dev); + if (smcd->ops->get_dev) + priv_dev = smcd->ops->get_dev(smcd); + if (priv_dev->parent) + smc_set_pci_values(to_pci_dev(priv_dev->parent), &smc_pci_dev); if (nla_put_u32(skb, SMC_NLA_DEV_PCI_FID, smc_pci_dev.pci_fid)) goto errattr; if (nla_put_u16(skb, SMC_NLA_DEV_PCI_CHID, smc_pci_dev.pci_pchid)) From patchwork Tue Sep 19 14:41:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 141953 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp3461543vqi; Tue, 19 Sep 2023 08:10:44 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHpE9cxcYjfjcQU/tGPUVrW1RBEAJVDT3QknBslsriCw7OveJsrAXdY9bwokWn5NuTvH6ER X-Received: by 2002:a05:6a21:a103:b0:154:d3ac:2063 with SMTP id aq3-20020a056a21a10300b00154d3ac2063mr8872217pzc.27.1695136243920; Tue, 19 Sep 2023 08:10:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695136243; cv=none; d=google.com; s=arc-20160816; b=hDXHSD4Bn86JShDIC5nKeeEn8KKO/yDdOOalhirG+dxBVpWzBGd55VRnWrAF8bm1GL VAVo37rAvZlq6/WOv0FlQ9Knx/SWqv3lv/JU6Zi2DTyyeP/E6xBIxVJViDBsl1WZk07J uXctrLO827oyuI6U+NAomdFdVXVLS9d5iEX/GtFmRxspvH5wGcCdO8yFtkeHbv7TAUrU c9YBI1stQPjrkq/9JOYsc3ciwk5vT3dY0iTa9dRlWOiX8QWQtdDgo0DhzI+WIQdEsy8p USGfBBt+GMlimGxYBaurI0/vXvT7BiF0vr2NUnFQzjPrM2wHgfYYGel9YTqe+KhBE18P y3+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=1SX8HRS7EtRKka1R1GUwKo32lAfLYCYeawSzLWb8s6U=; fh=2Ks5OhTjwS8gZt4F+VGxBODwUEQ0u2wJBF3lQY5ZqAM=; b=DwbOvx/M8pVQ6symhkZiHypT52p8tnV8n7GjGn54StTfWTzYwW1fFsaqgDz3/cXT3n 9vB0gwMP0DG1CiVw8Mf7NP4O2KMLRLOkFCcYVSYW6glG8AVqgNIKpbiOK6o66PERF6mm fLkCZHmM+AKXfc6wuRwKFjw7ra9LdL9GhOCDJ7KcBFtUG0DtzQOMg1J0oOHBmATjs3KU ntZt9Eg8jYCsR8LwmbmPE6gmQXoHeRoenJwYJvufEDdRaTCLsatqHidNYygRkP1uJxYV h1YQsItdJJ8z3Vc/hclVP5UvMtJlmPnrqbOyy5rX9miMVM873BIHlQoqFKVuCjwWRQpv q4yQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id bz16-20020a056a02061000b005645bbe3ffbsi10619235pgb.577.2023.09.19.08.10.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 08:10:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id C7A86818D515; Tue, 19 Sep 2023 07:48:13 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232887AbjISOmx (ORCPT + 26 others); Tue, 19 Sep 2023 10:42:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38660 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232879AbjISOmt (ORCPT ); Tue, 19 Sep 2023 10:42:49 -0400 Received: from out30-113.freemail.mail.aliyun.com (out30-113.freemail.mail.aliyun.com [115.124.30.113]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A015BC6; Tue, 19 Sep 2023 07:42:42 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R211e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046060;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0VsRsRqi_1695134557; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsRsRqi_1695134557) by smtp.aliyun-inc.com; Tue, 19 Sep 2023 22:42:39 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 02/18] net/smc: decouple ism_dev from SMC-D DMB registration Date: Tue, 19 Sep 2023 22:41:46 +0800 Message-Id: <1695134522-126655-3-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> References: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Tue, 19 Sep 2023 07:48:13 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777479182381797318 X-GMAIL-MSGID: 1777479182381797318 This patch tries to decouple ISM device from SMC-D DMB registration, So that the register_dmb option is not restricted to ISM devices. Signed-off-by: Wen Gu --- drivers/s390/net/ism_drv.c | 2 +- include/net/smc.h | 4 ++-- net/smc/smc_ism.c | 7 ++----- 3 files changed, 5 insertions(+), 8 deletions(-) diff --git a/drivers/s390/net/ism_drv.c b/drivers/s390/net/ism_drv.c index 6df7f37..a34e913 100644 --- a/drivers/s390/net/ism_drv.c +++ b/drivers/s390/net/ism_drv.c @@ -781,7 +781,7 @@ static int smcd_query_rgid(struct smcd_dev *smcd, u64 rgid, u32 vid_valid, } static int smcd_register_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb, - struct ism_client *client) + void *client) { return ism_register_dmb(smcd->priv, (struct ism_dmb *)dmb, client); } diff --git a/include/net/smc.h b/include/net/smc.h index a002552..f75116e 100644 --- a/include/net/smc.h +++ b/include/net/smc.h @@ -50,13 +50,12 @@ struct smcd_dmb { #define ISM_ERROR 0xFFFF struct smcd_dev; -struct ism_client; struct smcd_ops { int (*query_remote_gid)(struct smcd_dev *dev, u64 rgid, u32 vid_valid, u32 vid); int (*register_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb, - struct ism_client *client); + void *client); int (*unregister_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb); int (*add_vlan_id)(struct smcd_dev *dev, u64 vlan_id); int (*del_vlan_id)(struct smcd_dev *dev, u64 vlan_id); @@ -77,6 +76,7 @@ struct smcd_ops { struct smcd_dev { const struct smcd_ops *ops; void *priv; + void *client; struct list_head list; spinlock_t lock; struct smc_connection **conn; diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c index 0045fee..9e53bcf 100644 --- a/net/smc/smc_ism.c +++ b/net/smc/smc_ism.c @@ -200,7 +200,6 @@ int smc_ism_unregister_dmb(struct smcd_dev *smcd, struct smc_buf_desc *dmb_desc) int smc_ism_register_dmb(struct smc_link_group *lgr, int dmb_len, struct smc_buf_desc *dmb_desc) { -#if IS_ENABLED(CONFIG_ISM) struct smcd_dmb dmb; int rc; @@ -209,7 +208,7 @@ int smc_ism_register_dmb(struct smc_link_group *lgr, int dmb_len, dmb.sba_idx = dmb_desc->sba_idx; dmb.vlan_id = lgr->vlan_id; dmb.rgid = lgr->peer_gid; - rc = lgr->smcd->ops->register_dmb(lgr->smcd, &dmb, &smc_ism_client); + rc = lgr->smcd->ops->register_dmb(lgr->smcd, &dmb, lgr->smcd->client); if (!rc) { dmb_desc->sba_idx = dmb.sba_idx; dmb_desc->token = dmb.dmb_tok; @@ -218,9 +217,6 @@ int smc_ism_register_dmb(struct smc_link_group *lgr, int dmb_len, dmb_desc->len = dmb.dmb_len; } return rc; -#else - return 0; -#endif } static int smc_nl_handle_smcd_dev(struct smcd_dev *smcd, @@ -422,6 +418,7 @@ static void smcd_register_dev(struct ism_dev *ism) if (!smcd) return; smcd->priv = ism; + smcd->client = &smc_ism_client; ism_set_priv(ism, &smc_ism_client, smcd); if (smc_pnetid_by_dev_port(&ism->pdev->dev, 0, smcd->pnetid)) smc_pnetid_by_table_smcd(smcd); From patchwork Tue Sep 19 14:41:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 141926 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp3445477vqi; Tue, 19 Sep 2023 07:49:22 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHiRUzv34sMrDBBAYDiB0F9yFkQy9OfVsTfQ8g+/RO+FNDrzhrereNrTwwkWnNwZ5DPtXop X-Received: by 2002:a17:902:b494:b0:1bc:6c8:cded with SMTP id y20-20020a170902b49400b001bc06c8cdedmr11721228plr.67.1695134961895; Tue, 19 Sep 2023 07:49:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695134961; cv=none; d=google.com; s=arc-20160816; b=yGjL3jRByM9zszOM3SRzZVgOGyPdjgARZAxjUXpDlPrB13c2TWRVpkm5o37Klj/3CC SND7W16D56SdhcFh+HlzjwDb8vpvWcK1ROOeEf7S30YVbop6VuOoZHVmlwDWlvSjpwf+ hdsK6B/H/twsmx9gMSWKqvNZ/PBRwrdqIjZyxiB/i0arUrZixUp9BNzIFM37jEe14mBW nj969ptewlLCoa+jHlvmO0K8N74lGbqo+7dDXbODZs2oSK5HACutNY9jkz5NY6pI19BZ XGiag0RgK6mg12cE9u2vi1JSXof0twq1P53WLLRLQBGLSBCPMSs5npB+YAe3/8WVQy9w /qgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=+47dTlC+Vj/P01OqcMc0YmlorliQR5W9SpHf6vBNwl8=; fh=2Ks5OhTjwS8gZt4F+VGxBODwUEQ0u2wJBF3lQY5ZqAM=; b=s+vsivhzbHiySnyQ78irv7hSx92QV/b3flA9Zr4A7bCYWw2kTyLMgX56b50Nw73eL9 /5+qkdskj3l/Z1GmmWH3CvzPaXa8iTojzo+ldGYOLp31eeHje/ygcwX9armWPomTdt6B FRgEocDJhmfO/BynSINqSEP9HKJQemG2Eqo7A7ZlZVQEK7h93J7gZXamirTTUeIhsUKC 5qNuUMev2DDWSlxLlqAeSchmfZCcQDhneY4so26PUMbKO69MnJUFcN1FAFshkidwkXL1 UYObB0GLwLJg82Cba7zqXs9+ungVhl1ml7vWqChTlTWDycKQO5FW+xDlkPJulGVQZAAi Xz1g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from agentk.vger.email (agentk.vger.email. [23.128.96.32]) by mx.google.com with ESMTPS id j6-20020a170903024600b001b9e31bda39si10415936plh.118.2023.09.19.07.49.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 07:49:21 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 5F11B81BDA47; Tue, 19 Sep 2023 07:43:44 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232918AbjISOnA (ORCPT + 26 others); Tue, 19 Sep 2023 10:43:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38716 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232877AbjISOmv (ORCPT ); Tue, 19 Sep 2023 10:42:51 -0400 Received: from out30-98.freemail.mail.aliyun.com (out30-98.freemail.mail.aliyun.com [115.124.30.98]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4A048D6; Tue, 19 Sep 2023 07:42:44 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R701e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046060;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0VsRxW5r_1695134559; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsRxW5r_1695134559) by smtp.aliyun-inc.com; Tue, 19 Sep 2023 22:42:41 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 03/18] net/smc: extract v2 check helper from SMC-D device registration Date: Tue, 19 Sep 2023 22:41:47 +0800 Message-Id: <1695134522-126655-4-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> References: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Tue, 19 Sep 2023 07:43:44 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777477837633440473 X-GMAIL-MSGID: 1777477837633440473 This patch extracts v2-capable logic from the process of registering the ISM device as an SMC-D device, so that the registration process of other underlying devices can reuse it. Signed-off-by: Wen Gu --- net/smc/smc_ism.c | 29 ++++++++++++++++++----------- net/smc/smc_ism.h | 1 + 2 files changed, 19 insertions(+), 11 deletions(-) diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c index 9e53bcf..6fa87d8 100644 --- a/net/smc/smc_ism.c +++ b/net/smc/smc_ism.c @@ -69,6 +69,22 @@ bool smc_ism_is_v2_capable(void) return smc_ism_v2_capable; } +/* must be called under smcd_dev_list.mutex lock */ +void smc_ism_check_v2_capable(struct smcd_dev *smcd) +{ + u8 *system_eid = NULL; + + if (smc_ism_v2_capable) + return; + + system_eid = smcd->ops->get_system_eid(); + if (smcd->ops->supports_v2()) { + smc_ism_v2_capable = true; + memcpy(smc_ism_v2_system_eid, system_eid, + SMC_MAX_EID_LEN); + } +} + /* Set a connection using this DMBE. */ void smc_ism_set_conn(struct smc_connection *conn) { @@ -424,16 +440,7 @@ static void smcd_register_dev(struct ism_dev *ism) smc_pnetid_by_table_smcd(smcd); mutex_lock(&smcd_dev_list.mutex); - if (list_empty(&smcd_dev_list.list)) { - u8 *system_eid = NULL; - - system_eid = smcd->ops->get_system_eid(); - if (smcd->ops->supports_v2()) { - smc_ism_v2_capable = true; - memcpy(smc_ism_v2_system_eid, system_eid, - SMC_MAX_EID_LEN); - } - } + smc_ism_check_v2_capable(smcd); /* sort list: devices without pnetid before devices with pnetid */ if (smcd->pnetid[0]) list_add_tail(&smcd->list, &smcd_dev_list.list); @@ -536,10 +543,10 @@ int smc_ism_init(void) { int rc = 0; -#if IS_ENABLED(CONFIG_ISM) smc_ism_v2_capable = false; memset(smc_ism_v2_system_eid, 0, SMC_MAX_EID_LEN); +#if IS_ENABLED(CONFIG_ISM) rc = ism_register_client(&smc_ism_client); #endif return rc; diff --git a/net/smc/smc_ism.h b/net/smc/smc_ism.h index 832b2f4..14d2e77 100644 --- a/net/smc/smc_ism.h +++ b/net/smc/smc_ism.h @@ -42,6 +42,7 @@ int smc_ism_register_dmb(struct smc_link_group *lgr, int buf_size, void smc_ism_get_system_eid(u8 **eid); u16 smc_ism_get_chid(struct smcd_dev *dev); bool smc_ism_is_v2_capable(void); +void smc_ism_check_v2_capable(struct smcd_dev *dev); int smc_ism_init(void); void smc_ism_exit(void); int smcd_nl_get_device(struct sk_buff *skb, struct netlink_callback *cb); From patchwork Tue Sep 19 14:41:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 141973 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp3482818vqi; Tue, 19 Sep 2023 08:39:55 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEu/iPlZddoSBasmRBxTA5KjFCcASzo0s8I8L1/m5V3Kw40FQI11PNpdVFx4fPwJDckuMFt X-Received: by 2002:a05:6a20:6a23:b0:159:c07d:66f0 with SMTP id p35-20020a056a206a2300b00159c07d66f0mr87960pzk.6.1695137995429; Tue, 19 Sep 2023 08:39:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695137995; cv=none; d=google.com; s=arc-20160816; b=mIqJfFOdiHEG/JVlsDJSbVDJ/AqR/LB9M/6LqembpZb+55TMQNZbJxVXv91nDaY4lZ LYzpQ6Tlqz056RmzpBc8YHof16ufJ+nD8nPKHBCP6uJ9kNbklrnlizTFGFT6LWaD9dE4 YBSSp5caTP6R7fkfFCIfkbEFvYjwkC1Fc9QsYTkzhXBcxdxZ11IfGtPsrj6/nuEldhA4 /iEtxsSKSpzFLLai77ECriyZ/0Org9itP7uAjRIXg4dDIttKuMYqzPlrCsgmEs9eeiFf zVNFYQW9sRQHI4E/Gpzbergmb3J1MSYY1VOu9SPIuziK++hl0fLcutUse1R9ZIFRNLd4 I8bQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=oN5acSSXbDjrT8x3z0Z+yJR6hbCRGn4dSXeLy93Dt94=; fh=2Ks5OhTjwS8gZt4F+VGxBODwUEQ0u2wJBF3lQY5ZqAM=; b=Y8TJiN6R0JGDNW8qjyLELB8CLct2vKqBbJuzCom5Q+kzYo9EVAg8Y+rry74V2cEr/M HwnIp29pqW2bSCk3PyZFE19F+eTiD1JJ+qipK1lzhKa0TdQB6E5UuaalUJH2najQ1XHQ xq21ElLGzjmsfZ0SSY5bkV4bd5i+ED95siic3PVdsZMn78d5L3FPOze+v60VzaABXmPJ Ja7P4RDL3a72lrWHHatgQSF+BpiiLcqME3j6rR0cGIHajcRmqymbGGT9sjwEyZVmNd1k gKx6xv/VYW6axlYijdiiuKhFHxhRyOqmJBg5xlUkvKLFEwVYX6Z+PE9khLUv68ghuqL6 voNQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id d10-20020a056a00198a00b006901f2f2577si10063362pfl.380.2023.09.19.08.39.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 08:39:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id D086A80D7E4E; Tue, 19 Sep 2023 07:43:17 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232925AbjISOnD (ORCPT + 26 others); Tue, 19 Sep 2023 10:43:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39812 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232896AbjISOmx (ORCPT ); Tue, 19 Sep 2023 10:42:53 -0400 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4AAF1BE; Tue, 19 Sep 2023 07:42:46 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R201e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046059;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0VsRqc5W_1695134561; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsRqc5W_1695134561) by smtp.aliyun-inc.com; Tue, 19 Sep 2023 22:42:42 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 04/18] net/smc: support SMCv2.x supplemental features negotiation Date: Tue, 19 Sep 2023 22:41:48 +0800 Message-Id: <1695134522-126655-5-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> References: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 19 Sep 2023 07:43:17 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777481018753284874 X-GMAIL-MSGID: 1777481018753284874 This patch adds SMCv2.1 supplemental features negotiation. Supported SMCv2.1 supplemental features are represented by feature_mask in FCE header of CLC messages. Server Client Proposal(features(c-mask bits)) <----------------------------------- Accept(features(s-mask bits)) -----------------------------------> Confirm(features(s&c-mask bits)) <----------------------------------- Signed-off-by: Wen Gu --- net/smc/smc_clc.c | 7 +++++++ net/smc/smc_clc.h | 6 ++++-- net/smc/smc_core.h | 1 + 3 files changed, 12 insertions(+), 2 deletions(-) diff --git a/net/smc/smc_clc.c b/net/smc/smc_clc.c index 8deb46c..125b0d2 100644 --- a/net/smc/smc_clc.c +++ b/net/smc/smc_clc.c @@ -426,6 +426,7 @@ static int smc_clc_fill_fce(struct smc_clc_first_contact_ext_v2x *fce, memset(fce, 0, sizeof(*fce)); fce->fce_v2_base.os_type = SMC_CLC_OS_LINUX; fce->fce_v2_base.release = ini->release_nr; + fce->fce_v2_base.feature_mask = htons(ini->feature_mask); memcpy(fce->fce_v2_base.hostname, smc_hostname, sizeof(smc_hostname)); if (ini->is_smcd && ini->release_nr < SMC_RELEASE_1) { ret = sizeof(struct smc_clc_first_contact_ext); @@ -906,6 +907,7 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini) pclc_smcd->v2_ext_offset = htons(v2_ext_offset); plen += sizeof(*v2_ext); + v2_ext->feature_mask = htons(ini->feature_mask); read_lock(&smc_clc_eid_table.lock); v2_ext->hdr.eid_cnt = smc_clc_eid_table.ueid_cnt; plen += smc_clc_eid_table.ueid_cnt * SMC_MAX_EID_LEN; @@ -1219,6 +1221,7 @@ int smc_clc_clnt_v2x_features_validate(struct smc_clc_first_contact_ext *fce, return SMC_CLC_DECL_MAXLINKERR; ini->max_links = fce_v2x->max_links; } + ini->feature_mask &= ntohs(fce->feature_mask); return 0; } @@ -1250,6 +1253,10 @@ int smc_clc_v2x_features_confirm_check(struct smc_clc_msg_accept_confirm *cclc, return SMC_CLC_DECL_MAXLINKERR; } + if (~(ini->feature_mask) & ntohs(fce->feature_mask)) + return SMC_CLC_DECL_FEATUNSUPP; + ini->feature_mask = ntohs(fce->feature_mask); + return 0; } diff --git a/net/smc/smc_clc.h b/net/smc/smc_clc.h index c5c8e7d..bcf37c8 100644 --- a/net/smc/smc_clc.h +++ b/net/smc/smc_clc.h @@ -48,6 +48,7 @@ #define SMC_CLC_DECL_RELEASEERR 0x03030009 /* release version negotiate failed */ #define SMC_CLC_DECL_MAXCONNERR 0x0303000a /* max connections negotiate failed */ #define SMC_CLC_DECL_MAXLINKERR 0x0303000b /* max links negotiate failed */ +#define SMC_CLC_DECL_FEATUNSUPP 0x0303000c /* supplemental features not supported */ #define SMC_CLC_DECL_MODEUNSUPP 0x03040000 /* smc modes do not match (R or D)*/ #define SMC_CLC_DECL_RMBE_EC 0x03050000 /* peer has eyecatcher in RMBE */ #define SMC_CLC_DECL_OPTUNSUPP 0x03060000 /* fastopen sockopt not supported */ @@ -138,7 +139,8 @@ struct smc_clc_v2_extension { u8 roce[16]; /* RoCEv2 GID */ u8 max_conns; u8 max_links; - u8 reserved[14]; + __be16 feature_mask; + u8 reserved[12]; u8 user_eids[][SMC_MAX_EID_LEN]; }; @@ -234,7 +236,7 @@ struct smc_clc_first_contact_ext { u8 release : 4, os_type : 4; #endif - u8 reserved2[2]; + __be16 feature_mask; u8 hostname[SMC_MAX_HOSTNAME_LEN]; }; diff --git a/net/smc/smc_core.h b/net/smc/smc_core.h index 120027d..9f65678 100644 --- a/net/smc/smc_core.h +++ b/net/smc/smc_core.h @@ -401,6 +401,7 @@ struct smc_init_info { u8 max_links; u8 first_contact_peer; u8 first_contact_local; + u16 feature_mask; unsigned short vlan_id; u32 rc; u8 negotiated_eid[SMC_MAX_EID_LEN]; From patchwork Tue Sep 19 14:41:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 141925 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp3445446vqi; Tue, 19 Sep 2023 07:49:19 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEl0JCZUnMvm6Io/7Fu1ndqJVFLOYDipYPwtqS6gyAabbKTWJT/gumY2+OjLA0T23wiUFkW X-Received: by 2002:a17:902:c40d:b0:1c4:44a0:5c03 with SMTP id k13-20020a170902c40d00b001c444a05c03mr11733592plk.9.1695134959138; Tue, 19 Sep 2023 07:49:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695134959; cv=none; d=google.com; s=arc-20160816; b=HgYxjeeuayvoMUR2RZl8S0xfJ0uBRMCUhZsZtS4rIDx0BfpSDn0c4aZ3JDUi2vKkcq JtF4AeyfJv+nHaRnJOOqJJY/jfRv3NNlT6wwzj0dNilN+4sMj0MH6U4Ny9ixDDgwjh5G kzUGlNJMh2pOMv1LvwJ02Vak6ZVQlMXWr1OyXhxC8XGvwxY7IZeDfhW5GvsOAiEyepVu IIN5KqrUw/cmDSQ+ySEaazDLQGezM1zSqkTbOocUZ9k96s04nnVwofD22QwPMo/ZtuDe F8tHJDB56NdAbX+wg/V46OgGicHrcKsO4RrZ2pbJAVswfa6907YDAR/JWMUzSr2i64Vn 45JA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=VOcJAd/GukUuNrekkdAuy9WsRxE6/ZRaYrN8C7EMKy4=; fh=2Ks5OhTjwS8gZt4F+VGxBODwUEQ0u2wJBF3lQY5ZqAM=; b=pJBAfMzIDJ4NE1tUr3LuQNeKG2XjXcFl7iCkuqITfDwmZ06hNCJnDJtoSo+yj0W/c6 fG03MZ/8hxtG83eb9IJJbWBMtDQVHQn/2xwRc6BgrYRhyak5xjfBOPblQOJiWFGw8ukj LeEYY/HwOxSoJJUESStzr35/ll15DwXsNas5pTsbsE6fLaZlc6Ot5xZVXw2oKb90ZY12 yIOqpf9+4oNeHJwcM8cIf/suFFjCJeFzsRtIDcE2O6E8JziFfpF136ulDBatPHyoZBtg p/lHP2VNr602x+IE6ABFC+MKpLc9i0l4eiBZewIqo94iC61JgFcRSTIDDtJ/1s50tIiY URUg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id b13-20020a170903228d00b001bbad3b063bsi10425720plh.345.2023.09.19.07.49.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 07:49:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 956B581BC2D3; Tue, 19 Sep 2023 07:43:24 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232944AbjISOnH (ORCPT + 26 others); Tue, 19 Sep 2023 10:43:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39832 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232907AbjISOm4 (ORCPT ); Tue, 19 Sep 2023 10:42:56 -0400 Received: from out30-99.freemail.mail.aliyun.com (out30-99.freemail.mail.aliyun.com [115.124.30.99]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1D47BDD; Tue, 19 Sep 2023 07:42:47 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R731e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0VsRs7CG_1695134563; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsRs7CG_1695134563) by smtp.aliyun-inc.com; Tue, 19 Sep 2023 22:42:45 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 05/18] net/smc: reserve CHID range for SMC-D virtual device Date: Tue, 19 Sep 2023 22:41:49 +0800 Message-Id: <1695134522-126655-6-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> References: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Tue, 19 Sep 2023 07:43:24 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777477835268073049 X-GMAIL-MSGID: 1777477835268073049 This patch reserve CHID range from 0xFF00 to 0xFFFF for SMC-D virtual device and introduces helpers to identify them. Signed-off-by: Wen Gu --- net/smc/smc_ism.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/net/smc/smc_ism.h b/net/smc/smc_ism.h index 14d2e77..2ecc8de 100644 --- a/net/smc/smc_ism.h +++ b/net/smc/smc_ism.h @@ -15,6 +15,9 @@ #include "smc.h" +#define SMC_VIRT_ISM_CHID_MAX 0xFFFF +#define SMC_VIRT_ISM_CHID_MIN 0xFF00 + struct smcd_dev_list { /* List of SMCD devices */ struct list_head list; struct mutex mutex; /* Protects list of devices */ @@ -57,4 +60,16 @@ static inline int smc_ism_write(struct smcd_dev *smcd, u64 dmb_tok, return rc < 0 ? rc : 0; } +static inline bool __smc_ism_is_virtdev(u16 chid) +{ + return (chid >= SMC_VIRT_ISM_CHID_MIN && chid <= SMC_VIRT_ISM_CHID_MAX); +} + +static inline bool smc_ism_is_virtdev(struct smcd_dev *smcd) +{ + u16 chid = smcd->ops->get_chid(smcd); + + return __smc_ism_is_virtdev(chid); +} + #endif From patchwork Tue Sep 19 14:41:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 141975 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp3483009vqi; Tue, 19 Sep 2023 08:40:13 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFCG892KK5aSCW0o1YTGhsqeOOKpPZyIWrtTEGaJq3l1E97cJFWW8GFZFXNBQbIStwBPrGF X-Received: by 2002:a05:6358:284:b0:143:61d:ffd3 with SMTP id w4-20020a056358028400b00143061dffd3mr212888rwj.4.1695138013263; Tue, 19 Sep 2023 08:40:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695138013; cv=none; d=google.com; s=arc-20160816; b=Ksr0XdKnnIgjTwe8RyfpytPo3bkmZEKd8Zqdao4HS1wlgfRBjjV5825a2N2sLAwhDc pxVzQnQxzbxQND9iOt7TYOpvyagF1QZ94LFu8KG5O9UfZw0E+kyE5iulKO9bn7LNxbRq aeSrRF635Kh/e/lJgKPLR8ZSFDqojgD6NADddq/5EY6S/NJ0MnQv2RRpZWEzVlnDz1fb Rml+vOJxt8kQX98RtkJiOT/9fQb3Cv8a7d1eqvS69cpRagKcSMqavyWAJ0e7x/2e0qSl VjclEN2ziVr3UIaNOWe1BKQaJCiGhGrmKed08m+vFQsNCPXHQUVD6cjD81aNWR/kX4Bo fXeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=xnJReWSfkquvPftM1701a8cBCEQmAqHj/YpebWV1W4M=; fh=2Ks5OhTjwS8gZt4F+VGxBODwUEQ0u2wJBF3lQY5ZqAM=; b=hrFdgdu6YBCxpJW0KtqL/derkw75xwFPeODDFikxhD/T+skCndhkZFXdfvWsBrNh8e 1CBE69LbEfsMkjO3bFTTrbqHucJvpnGDX8B3biGEmS2nunuJpAa4b7oac0rONrBEb4ti ntt6ZRcHC3wjoAJovPBJkIWzx9eRT862VdoZnvnSBcDTcmURPME0OgWt9bIsNsNunUcM HRLNkOQK58aA6XZ75r8tSHaYyqsuhrCmqu+dOZNpiYNPIgkjlwP6YaKt/RP3tgwbY8Q/ cUaRlgQWXjLc/gjEIjao+gmyMbNY4IoGK/jm6/fNsdQ4H4LXh/Po17udLi4ItYLprR3F LMqA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id w67-20020a636246000000b00578d0b600cdsi248442pgb.290.2023.09.19.08.40.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 08:40:13 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id D08FF801F744; Tue, 19 Sep 2023 07:43:17 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232984AbjISOnJ (ORCPT + 26 others); Tue, 19 Sep 2023 10:43:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49758 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232930AbjISOnG (ORCPT ); Tue, 19 Sep 2023 10:43:06 -0400 Received: from out30-97.freemail.mail.aliyun.com (out30-97.freemail.mail.aliyun.com [115.124.30.97]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D2B0EF3; Tue, 19 Sep 2023 07:42:49 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R151e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0VsRriw7_1695134565; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsRriw7_1695134565) by smtp.aliyun-inc.com; Tue, 19 Sep 2023 22:42:47 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 06/18] net/smc: extend GID to 128bits for virtual ISM device Date: Tue, 19 Sep 2023 22:41:50 +0800 Message-Id: <1695134522-126655-7-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> References: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 19 Sep 2023 07:43:17 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777481037360588007 X-GMAIL-MSGID: 1777481037360588007 As the SMC-Dv2 protocol introduces virtual ISM devices, whose GIDs are UUIDs generated by software, the maximum length of SMC-D GID has been extended to 128 bits. So this patch adapts the relevant code to make it compatible with 128 bits GID. Signed-off-by: Wen Gu --- drivers/s390/net/ism_drv.c | 11 ++++++----- include/net/smc.h | 11 ++++++++--- include/uapi/linux/smc.h | 3 +++ net/smc/af_smc.c | 45 +++++++++++++++++++++++++++++++++------------ net/smc/smc_clc.c | 35 ++++++++++++++++++++++++----------- net/smc/smc_clc.h | 4 ++-- net/smc/smc_core.c | 38 +++++++++++++++++++++++++++++--------- net/smc/smc_core.h | 7 ++++--- net/smc/smc_diag.c | 6 ++++-- net/smc/smc_ism.c | 13 +++++++++---- net/smc/smc_ism.h | 3 ++- net/smc/smc_pnet.c | 4 ++-- 12 files changed, 126 insertions(+), 54 deletions(-) diff --git a/drivers/s390/net/ism_drv.c b/drivers/s390/net/ism_drv.c index a34e913..1b714f6 100644 --- a/drivers/s390/net/ism_drv.c +++ b/drivers/s390/net/ism_drv.c @@ -774,10 +774,10 @@ static void __exit ism_exit(void) /*************************** SMC-D Implementation *****************************/ #if IS_ENABLED(CONFIG_SMC) -static int smcd_query_rgid(struct smcd_dev *smcd, u64 rgid, u32 vid_valid, - u32 vid) +static int smcd_query_rgid(struct smcd_dev *smcd, struct smcd_gid *rgid, + u32 vid_valid, u32 vid) { - return ism_query_rgid(smcd->priv, rgid, vid_valid, vid); + return ism_query_rgid(smcd->priv, rgid->gid, vid_valid, vid); } static int smcd_register_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb, @@ -830,9 +830,10 @@ static int smcd_supports_v2(void) SYSTEM_EID.type[0] != '0'; } -static u64 smcd_get_local_gid(struct smcd_dev *smcd) +static void smcd_get_local_gid(struct smcd_dev *smcd, + struct smcd_gid *smcd_gid) { - return ism_get_local_gid(smcd->priv); + smcd_gid->gid = ism_get_local_gid(smcd->priv); } static u16 smcd_get_chid(struct smcd_dev *smcd) diff --git a/include/net/smc.h b/include/net/smc.h index f75116e..a37fb38 100644 --- a/include/net/smc.h +++ b/include/net/smc.h @@ -51,9 +51,14 @@ struct smcd_dmb { struct smcd_dev; +struct smcd_gid { + u64 gid; + u64 gid_ext; +}; + struct smcd_ops { - int (*query_remote_gid)(struct smcd_dev *dev, u64 rgid, u32 vid_valid, - u32 vid); + int (*query_remote_gid)(struct smcd_dev *dev, struct smcd_gid *rgid, + u32 vid_valid, u32 vid); int (*register_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb, void *client); int (*unregister_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb); @@ -68,7 +73,7 @@ struct smcd_ops { unsigned int size); int (*supports_v2)(void); u8* (*get_system_eid)(void); - u64 (*get_local_gid)(struct smcd_dev *dev); + void (*get_local_gid)(struct smcd_dev *dev, struct smcd_gid *gid); u16 (*get_chid)(struct smcd_dev *dev); struct device* (*get_dev)(struct smcd_dev *dev); }; diff --git a/include/uapi/linux/smc.h b/include/uapi/linux/smc.h index 837fcd4..0d2f020 100644 --- a/include/uapi/linux/smc.h +++ b/include/uapi/linux/smc.h @@ -99,6 +99,9 @@ enum { SMC_NLA_LGR_V2_OS, /* u8 */ SMC_NLA_LGR_V2_NEG_EID, /* string */ SMC_NLA_LGR_V2_PEER_HOST, /* string */ + SMC_NLA_LGR_V2_PAD, /* flag */ + SMC_NLA_LGR_V2_GID_EXT, /* u64 */ + SMC_NLA_LGR_V2_PEER_GID_EXT, /* u64 */ __SMC_NLA_LGR_V2_MAX, SMC_NLA_LGR_V2_MAX = __SMC_NLA_LGR_V2_MAX - 1 }; diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index bacdd97..9e31033 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -1044,7 +1044,8 @@ static int smc_find_ism_v2_device_clnt(struct smc_sock *smc, { int rc = SMC_CLC_DECL_NOSMCDDEV; struct smcd_dev *smcd; - int i = 1; + int i = 1, slot = 1; + bool is_virtdev; u16 chid; if (smcd_indicated(ini->smc_type_v1)) @@ -1056,14 +1057,19 @@ static int smc_find_ism_v2_device_clnt(struct smc_sock *smc, chid = smc_ism_get_chid(smcd); if (!smc_find_ism_v2_is_unique_chid(chid, ini, i)) continue; + is_virtdev = __smc_ism_is_virtdev(chid); if (!smc_pnet_is_pnetid_set(smcd->pnetid) || smc_pnet_is_ndev_pnetid(sock_net(&smc->sk), smcd->pnetid)) { + /* GID-CHID array is not enough for a virtual GID (128bits) */ + if (is_virtdev && slot >= SMC_MAX_ISM_DEVS) + continue; ini->ism_dev[i] = smcd; ini->ism_chid[i] = chid; ini->is_smcd = true; rc = 0; i++; - if (i > SMC_MAX_ISM_DEVS) + slot = is_virtdev ? slot + 2 : slot + 1; + if (slot > SMC_MAX_ISM_DEVS) break; } } @@ -1409,8 +1415,9 @@ static int smc_connect_ism(struct smc_sock *smc, rc = smc_v2_determine_accepted_chid(aclc_v2, ini); if (rc) return rc; + ini->ism_peer_gid[ini->ism_selected].gid_ext = aclc_v2->d1.gid_ext; } - ini->ism_peer_gid[ini->ism_selected] = aclc->d0.gid; + ini->ism_peer_gid[ini->ism_selected].gid = aclc->d0.gid; /* there is only one lgr role for SMC-D; use server lock */ mutex_lock(&smc_server_lgr_pending); @@ -2101,7 +2108,8 @@ static bool smc_is_already_selected(struct smcd_dev *smcd, /* check for ISM devices matching proposed ISM devices */ static void smc_check_ism_v2_match(struct smc_init_info *ini, - u16 proposed_chid, u64 proposed_gid, + u16 proposed_chid, + struct smcd_gid *proposed_gid, unsigned int *matches) { struct smcd_dev *smcd; @@ -2113,7 +2121,10 @@ static void smc_check_ism_v2_match(struct smc_init_info *ini, continue; if (smc_ism_get_chid(smcd) == proposed_chid && !smc_ism_cantalk(proposed_gid, ISM_RESERVED_VLANID, smcd)) { - ini->ism_peer_gid[*matches] = proposed_gid; + ini->ism_peer_gid[*matches].gid = proposed_gid->gid; + if (__smc_ism_is_virtdev(proposed_chid)) + ini->ism_peer_gid[*matches].gid_ext = + proposed_gid->gid_ext; ini->ism_dev[*matches] = smcd; (*matches)++; break; @@ -2135,9 +2146,11 @@ static void smc_find_ism_v2_device_serv(struct smc_sock *new_smc, struct smc_clc_v2_extension *smc_v2_ext; struct smc_clc_msg_smcd *pclc_smcd; unsigned int matches = 0; + struct smcd_gid smcd_gid; u8 smcd_version; u8 *eid = NULL; int i, rc; + u16 chid; if (!(ini->smcd_version & SMC_V2) || !smcd_indicated(ini->smc_type_v2)) goto not_found; @@ -2147,18 +2160,26 @@ static void smc_find_ism_v2_device_serv(struct smc_sock *new_smc, smcd_v2_ext = smc_get_clc_smcd_v2_ext(smc_v2_ext); mutex_lock(&smcd_dev_list.mutex); - if (pclc_smcd->ism.chid) + if (pclc_smcd->ism.chid) { /* check for ISM device matching proposed native ISM device */ + smcd_gid.gid = ntohll(pclc_smcd->ism.gid); smc_check_ism_v2_match(ini, ntohs(pclc_smcd->ism.chid), - ntohll(pclc_smcd->ism.gid), &matches); + &smcd_gid, &matches); + } for (i = 1; i <= smc_v2_ext->hdr.ism_gid_cnt; i++) { /* check for ISM devices matching proposed non-native ISM * devices */ - smc_check_ism_v2_match(ini, - ntohs(smcd_v2_ext->gidchid[i - 1].chid), - ntohll(smcd_v2_ext->gidchid[i - 1].gid), - &matches); + smcd_gid.gid = ntohll(smcd_v2_ext->gidchid[i - 1].gid); + chid = ntohs(smcd_v2_ext->gidchid[i - 1].chid); + if (__smc_ism_is_virtdev(chid)) { + /* check if extended entry exists and is valid */ + if (i >= smc_v2_ext->hdr.ism_gid_cnt || + chid != ntohs(smcd_v2_ext->gidchid[i].chid)) + continue; + smcd_gid.gid_ext = ntohll(smcd_v2_ext->gidchid[i++].gid); + } + smc_check_ism_v2_match(ini, chid, &smcd_gid, &matches); } mutex_unlock(&smcd_dev_list.mutex); @@ -2207,7 +2228,7 @@ static void smc_find_ism_v1_device_serv(struct smc_sock *new_smc, if (!(ini->smcd_version & SMC_V1) || !smcd_indicated(ini->smc_type_v1)) goto not_found; ini->is_smcd = true; /* prepare ISM check */ - ini->ism_peer_gid[0] = ntohll(pclc_smcd->ism.gid); + ini->ism_peer_gid[0].gid = ntohll(pclc_smcd->ism.gid); rc = smc_find_ism_device(new_smc, ini); if (rc) goto not_found; diff --git a/net/smc/smc_clc.c b/net/smc/smc_clc.c index 125b0d2..c08e138 100644 --- a/net/smc/smc_clc.c +++ b/net/smc/smc_clc.c @@ -882,11 +882,13 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini) ETH_ALEN); } if (smcd_indicated(ini->smc_type_v1)) { + struct smcd_gid smcd_gid; + /* add SMC-D specifics */ if (ini->ism_dev[0]) { smcd = ini->ism_dev[0]; - pclc_smcd->ism.gid = - htonll(smcd->ops->get_local_gid(smcd)); + smcd->ops->get_local_gid(smcd, &smcd_gid); + pclc_smcd->ism.gid = htonll(smcd_gid.gid); pclc_smcd->ism.chid = htons(smc_ism_get_chid(ini->ism_dev[0])); } @@ -919,10 +921,11 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini) read_unlock(&smc_clc_eid_table.lock); } if (smcd_indicated(ini->smc_type_v2)) { + struct smcd_gid smcd_gid; u8 *eid = NULL; + int slot = 0; v2_ext->hdr.flag.seid = smc_clc_eid_table.seid_enabled; - v2_ext->hdr.ism_gid_cnt = ini->ism_offered_cnt; v2_ext->hdr.smcd_v2_ext_offset = htons(sizeof(*v2_ext) - offsetofend(struct smc_clnt_opts_area_hdr, smcd_v2_ext_offset) + @@ -934,14 +937,21 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini) if (ini->ism_offered_cnt) { for (i = 1; i <= ini->ism_offered_cnt; i++) { smcd = ini->ism_dev[i]; - gidchids[i - 1].gid = - htonll(smcd->ops->get_local_gid(smcd)); - gidchids[i - 1].chid = + smcd->ops->get_local_gid(smcd, &smcd_gid); + gidchids[slot].chid = htons(smc_ism_get_chid(ini->ism_dev[i])); + gidchids[slot].gid = htonll(smcd_gid.gid); + /* virtual-ism takes two slots */ + if (__smc_ism_is_virtdev(gidchids[slot].chid)) { + gidchids[slot + 1].chid = gidchids[slot].chid; + gidchids[slot + 1].gid = htonll(smcd_gid.gid_ext); + slot++; + } + slot++; } - plen += ini->ism_offered_cnt * - sizeof(struct smc_clc_smcd_gid_chid); + plen += slot * sizeof(struct smc_clc_smcd_gid_chid); } + v2_ext->hdr.ism_gid_cnt = slot; } if (smcr_indicated(ini->smc_type_v2)) { memcpy(v2_ext->roce, ini->smcrv2.ib_gid_v2, SMC_GID_SIZE); @@ -977,7 +987,7 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini) vec[i++].iov_len = sizeof(*smcd_v2_ext); if (ini->ism_offered_cnt) { vec[i].iov_base = gidchids; - vec[i++].iov_len = ini->ism_offered_cnt * + vec[i++].iov_len = v2_ext->hdr.ism_gid_cnt * sizeof(struct smc_clc_smcd_gid_chid); } } @@ -1019,12 +1029,14 @@ static int smc_clc_send_confirm_accept(struct smc_sock *smc, if (first_contact) clc->hdr.typev2 |= SMC_FIRST_CONTACT_MASK; if (conn->lgr->is_smcd) { + struct smcd_gid smcd_gid; + /* SMC-D specific settings */ memcpy(clc->hdr.eyecatcher, SMCD_EYECATCHER, sizeof(SMCD_EYECATCHER)); + conn->lgr->smcd->ops->get_local_gid(conn->lgr->smcd, &smcd_gid); clc->hdr.typev1 = SMC_TYPE_D; - clc->d0.gid = - conn->lgr->smcd->ops->get_local_gid(conn->lgr->smcd); + clc->d0.gid = smcd_gid.gid; clc->d0.token = conn->rmb_desc->token; clc->d0.dmbe_size = conn->rmbe_size_comp; clc->d0.dmbe_idx = 0; @@ -1036,6 +1048,7 @@ static int smc_clc_send_confirm_accept(struct smc_sock *smc, htons(smc_ism_get_chid(conn->lgr->smcd)); if (eid && eid[0]) memcpy(clc_v2->d1.eid, eid, SMC_MAX_EID_LEN); + clc_v2->d1.gid_ext = smcd_gid.gid_ext; len = SMCD_CLC_ACCEPT_CONFIRM_LEN_V2; if (first_contact) { fce_len = smc_clc_fill_fce(&fce, ini); diff --git a/net/smc/smc_clc.h b/net/smc/smc_clc.h index bcf37c8..611763a 100644 --- a/net/smc/smc_clc.h +++ b/net/smc/smc_clc.h @@ -281,8 +281,8 @@ struct smc_clc_msg_accept_confirm_v2 { /* clc accept / confirm message */ struct smcd_clc_msg_accept_confirm_common d0; __be16 chid; u8 eid[SMC_MAX_EID_LEN]; - u8 reserved5[8]; - } d1; + __be64 gid_ext; + } __packed d1; }; }; diff --git a/net/smc/smc_core.c b/net/smc/smc_core.c index d520ee6..c36500a 100644 --- a/net/smc/smc_core.c +++ b/net/smc/smc_core.c @@ -284,6 +284,9 @@ static int smc_nl_fill_lgr_v2_common(struct smc_link_group *lgr, { char smc_host[SMC_MAX_HOSTNAME_LEN + 1]; char smc_eid[SMC_MAX_EID_LEN + 1]; + struct smcd_dev *smcd = lgr->smcd; + struct smcd_gid smcd_gid; + bool is_virtdev; if (nla_put_u8(skb, SMC_NLA_LGR_V2_VER, lgr->smc_version)) goto errv2attr; @@ -299,6 +302,16 @@ static int smc_nl_fill_lgr_v2_common(struct smc_link_group *lgr, smc_eid[SMC_MAX_EID_LEN] = 0; if (nla_put_string(skb, SMC_NLA_LGR_V2_NEG_EID, smc_eid)) goto errv2attr; + smcd->ops->get_local_gid(smcd, &smcd_gid); + is_virtdev = smc_ism_is_virtdev(smcd); + if (nla_put_u64_64bit(skb, SMC_NLA_LGR_V2_GID_EXT, + is_virtdev ? smcd_gid.gid_ext : 0, + SMC_NLA_LGR_V2_PAD)) + goto errv2attr; + if (nla_put_u64_64bit(skb, SMC_NLA_LGR_V2_PEER_GID_EXT, + is_virtdev ? lgr->peer_gid.gid_ext : 0, + SMC_NLA_LGR_V2_PAD)) + goto errv2attr; nla_nest_end(skb, v2_attrs); return 0; @@ -506,6 +519,7 @@ static int smc_nl_fill_smcd_lgr(struct smc_link_group *lgr, { char smc_pnet[SMC_MAX_PNETID_LEN + 1]; struct smcd_dev *smcd = lgr->smcd; + struct smcd_gid smcd_gid; struct nlattr *attrs; void *nlh; @@ -521,11 +535,11 @@ static int smc_nl_fill_smcd_lgr(struct smc_link_group *lgr, if (nla_put_u32(skb, SMC_NLA_LGR_D_ID, *((u32 *)&lgr->id))) goto errattr; + smcd->ops->get_local_gid(smcd, &smcd_gid); if (nla_put_u64_64bit(skb, SMC_NLA_LGR_D_GID, - smcd->ops->get_local_gid(smcd), - SMC_NLA_LGR_D_PAD)) + smcd_gid.gid, SMC_NLA_LGR_D_PAD)) goto errattr; - if (nla_put_u64_64bit(skb, SMC_NLA_LGR_D_PEER_GID, lgr->peer_gid, + if (nla_put_u64_64bit(skb, SMC_NLA_LGR_D_PEER_GID, lgr->peer_gid.gid, SMC_NLA_LGR_D_PAD)) goto errattr; if (nla_put_u8(skb, SMC_NLA_LGR_D_VLAN_ID, lgr->vlan_id)) @@ -1514,7 +1528,8 @@ void smc_lgr_terminate_sched(struct smc_link_group *lgr) } /* Called when peer lgr shutdown (regularly or abnormally) is received */ -void smc_smcd_terminate(struct smcd_dev *dev, u64 peer_gid, unsigned short vlan) +void smc_smcd_terminate(struct smcd_dev *dev, struct smcd_gid *peer_gid, + unsigned short vlan) { struct smc_link_group *lgr, *l; LIST_HEAD(lgr_free_list); @@ -1522,7 +1537,10 @@ void smc_smcd_terminate(struct smcd_dev *dev, u64 peer_gid, unsigned short vlan) /* run common cleanup function and build free list */ spin_lock_bh(&dev->lgr_lock); list_for_each_entry_safe(lgr, l, &dev->lgr_list, list) { - if ((!peer_gid || lgr->peer_gid == peer_gid) && + if ((!peer_gid->gid || + (lgr->peer_gid.gid == peer_gid->gid && + !smc_ism_is_virtdev(dev) ? 1 : + lgr->peer_gid.gid_ext == peer_gid->gid_ext)) && (vlan == VLAN_VID_MASK || lgr->vlan_id == vlan)) { if (peer_gid) /* peer triggered termination */ lgr->peer_shutdown = 1; @@ -1859,10 +1877,12 @@ static bool smcr_lgr_match(struct smc_link_group *lgr, u8 smcr_version, return false; } -static bool smcd_lgr_match(struct smc_link_group *lgr, - struct smcd_dev *smcismdev, u64 peer_gid) +static bool smcd_lgr_match(struct smc_link_group *lgr, struct smcd_dev *smcismdev, + struct smcd_gid *peer_gid) { - return lgr->peer_gid == peer_gid && lgr->smcd == smcismdev; + return lgr->peer_gid.gid == peer_gid->gid && lgr->smcd == smcismdev && + smc_ism_is_virtdev(smcismdev) ? + (lgr->peer_gid.gid_ext == peer_gid->gid_ext) : 1; } /* create a new SMC connection (and a new link group if necessary) */ @@ -1892,7 +1912,7 @@ int smc_conn_create(struct smc_sock *smc, struct smc_init_info *ini) write_lock_bh(&lgr->conns_lock); if ((ini->is_smcd ? smcd_lgr_match(lgr, ini->ism_dev[ini->ism_selected], - ini->ism_peer_gid[ini->ism_selected]) : + &ini->ism_peer_gid[ini->ism_selected]) : smcr_lgr_match(lgr, ini->smcr_version, ini->peer_systemid, ini->peer_gid, ini->peer_mac, role, diff --git a/net/smc/smc_core.h b/net/smc/smc_core.h index 9f65678..d57eb9b 100644 --- a/net/smc/smc_core.h +++ b/net/smc/smc_core.h @@ -17,6 +17,7 @@ #include #include #include +#include #include "smc.h" #include "smc_ib.h" @@ -355,7 +356,7 @@ struct smc_link_group { /* max links can be added in lgr */ }; struct { /* SMC-D */ - u64 peer_gid; + struct smcd_gid peer_gid; /* Peer GID (remote) */ struct smcd_dev *smcd; /* ISM device for VLAN reg. */ @@ -417,7 +418,7 @@ struct smc_init_info { u32 ib_clcqpn; struct smc_init_info_smcrv2 smcrv2; /* SMC-D */ - u64 ism_peer_gid[SMC_MAX_ISM_DEVS + 1]; + struct smcd_gid ism_peer_gid[SMC_MAX_ISM_DEVS + 1]; struct smcd_dev *ism_dev[SMC_MAX_ISM_DEVS + 1]; u16 ism_chid[SMC_MAX_ISM_DEVS + 1]; u8 ism_offered_cnt; /* # of ISM devices offered */ @@ -545,7 +546,7 @@ static inline void smc_set_pci_values(struct pci_dev *pci_dev, void smc_lgr_put(struct smc_link_group *lgr); void smcr_port_add(struct smc_ib_device *smcibdev, u8 ibport); void smcr_port_err(struct smc_ib_device *smcibdev, u8 ibport); -void smc_smcd_terminate(struct smcd_dev *dev, u64 peer_gid, +void smc_smcd_terminate(struct smcd_dev *dev, struct smcd_gid *peer_gid, unsigned short vlan); void smc_smcd_terminate_all(struct smcd_dev *dev); void smc_smcr_terminate_all(struct smc_ib_device *smcibdev); diff --git a/net/smc/smc_diag.c b/net/smc/smc_diag.c index 7ff2152..9c465cc 100644 --- a/net/smc/smc_diag.c +++ b/net/smc/smc_diag.c @@ -168,12 +168,14 @@ static int __smc_diag_dump(struct sock *sk, struct sk_buff *skb, struct smc_connection *conn = &smc->conn; struct smcd_diag_dmbinfo dinfo; struct smcd_dev *smcd = conn->lgr->smcd; + struct smcd_gid smcd_gid; memset(&dinfo, 0, sizeof(dinfo)); dinfo.linkid = *((u32 *)conn->lgr->id); - dinfo.peer_gid = conn->lgr->peer_gid; - dinfo.my_gid = smcd->ops->get_local_gid(smcd); + dinfo.peer_gid = conn->lgr->peer_gid.gid; + smcd->ops->get_local_gid(smcd, &smcd_gid); + dinfo.my_gid = smcd_gid.gid; dinfo.token = conn->rmb_desc->token; dinfo.peer_token = conn->peer_token; diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c index 6fa87d8..ba37b27 100644 --- a/net/smc/smc_ism.c +++ b/net/smc/smc_ism.c @@ -44,7 +44,8 @@ static void smcd_handle_irq(struct ism_dev *ism, unsigned int dmbno, #endif /* Test if an ISM communication is possible - same CPC */ -int smc_ism_cantalk(u64 peer_gid, unsigned short vlan_id, struct smcd_dev *smcd) +int smc_ism_cantalk(struct smcd_gid *peer_gid, unsigned short vlan_id, + struct smcd_dev *smcd) { return smcd->ops->query_remote_gid(smcd, peer_gid, vlan_id ? 1 : 0, vlan_id); @@ -223,7 +224,7 @@ int smc_ism_register_dmb(struct smc_link_group *lgr, int dmb_len, dmb.dmb_len = dmb_len; dmb.sba_idx = dmb_desc->sba_idx; dmb.vlan_id = lgr->vlan_id; - dmb.rgid = lgr->peer_gid; + dmb.rgid = lgr->peer_gid.gid; rc = lgr->smcd->ops->register_dmb(lgr->smcd, &dmb, lgr->smcd->client); if (!rc) { dmb_desc->sba_idx = dmb.sba_idx; @@ -355,11 +356,13 @@ struct smc_ism_event_work { static void smcd_handle_sw_event(struct smc_ism_event_work *wrk) { union smcd_sw_event_info ev_info; + struct smcd_gid peer_gid; ev_info.info = wrk->event.info; switch (wrk->event.code) { case ISM_EVENT_CODE_SHUTDOWN: /* Peer shut down DMBs */ - smc_smcd_terminate(wrk->smcd, wrk->event.tok, ev_info.vlan_id); + peer_gid.gid = wrk->event.tok; + smc_smcd_terminate(wrk->smcd, &peer_gid, ev_info.vlan_id); break; case ISM_EVENT_CODE_TESTLINK: /* Activity timer */ if (ev_info.code == ISM_EVENT_REQUEST) { @@ -379,10 +382,12 @@ static void smc_ism_event_work(struct work_struct *work) { struct smc_ism_event_work *wrk = container_of(work, struct smc_ism_event_work, work); + struct smcd_gid smcd_gid; switch (wrk->event.type) { case ISM_EVENT_GID: /* GID event, token is peer GID */ - smc_smcd_terminate(wrk->smcd, wrk->event.tok, VLAN_VID_MASK); + smcd_gid.gid = wrk->event.tok; + smc_smcd_terminate(wrk->smcd, &smcd_gid, VLAN_VID_MASK); break; case ISM_EVENT_DMB: break; diff --git a/net/smc/smc_ism.h b/net/smc/smc_ism.h index 2ecc8de..e6ea08c 100644 --- a/net/smc/smc_ism.h +++ b/net/smc/smc_ism.h @@ -33,7 +33,8 @@ struct smc_ism_vlanid { /* VLAN id set on ISM device */ struct smcd_dev; -int smc_ism_cantalk(u64 peer_gid, unsigned short vlan_id, struct smcd_dev *dev); +int smc_ism_cantalk(struct smcd_gid *peer_gid, unsigned short vlan_id, + struct smcd_dev *dev); void smc_ism_set_conn(struct smc_connection *conn); void smc_ism_unset_conn(struct smc_connection *conn); int smc_ism_get_vlan(struct smcd_dev *dev, unsigned short vlan_id); diff --git a/net/smc/smc_pnet.c b/net/smc/smc_pnet.c index 1177540..9f2c58c 100644 --- a/net/smc/smc_pnet.c +++ b/net/smc/smc_pnet.c @@ -1103,8 +1103,8 @@ static void smc_pnet_find_ism_by_pnetid(struct net_device *ndev, list_for_each_entry(ismdev, &smcd_dev_list.list, list) { if (smc_pnet_match(ismdev->pnetid, ndev_pnetid) && !ismdev->going_away && - (!ini->ism_peer_gid[0] || - !smc_ism_cantalk(ini->ism_peer_gid[0], ini->vlan_id, + (!ini->ism_peer_gid[0].gid || + !smc_ism_cantalk(&ini->ism_peer_gid[0], ini->vlan_id, ismdev))) { ini->ism_dev[0] = ismdev; break; From patchwork Tue Sep 19 14:41:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 141974 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp3482950vqi; Tue, 19 Sep 2023 08:40:08 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGd73Sb7jVvzBqKnM2oeDWYCSksSLPRX2aFvseMqpzvJw3CCq5ydfKWIgYUInsmehjvOklF X-Received: by 2002:a17:90a:410e:b0:276:7907:ecf with SMTP id u14-20020a17090a410e00b0027679070ecfmr56474pjf.36.1695138008477; Tue, 19 Sep 2023 08:40:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695138008; cv=none; d=google.com; s=arc-20160816; b=Jtt5mhy5MnoAhD6VzjyYkP/XXkRcmtflfCik8qU67+P8hDBKeZFO7ys0+FweksZsBY HRN9PomswkSIn5p6UL4GK/F8/d2/fti3DG/649PNC3UQ0G+okLCy6YlkGntw/lacBExp OpZ77Kh3+gbktymc3JWtWpcpxmIyHQtjjtcr6nq3TGslvcwpxJc0T/+t4gz4Go3TmXVv Vzhfx1wLawyWZCAEoSdZ88SV/tvUxg9QjoGy8c28bet/u6Rihf5bgimt47QnKvgdUcki Qizf78ZMZSWjU4ih18plx0ykQVCO7Q5eA2XCiBndsmFGZXr0lamq7x5PzdJZz9ooVI/B 3yLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=QM5jWKqujrJ6N/9s0YdsuBov0QHoba6spTSFi4QNLGk=; fh=2Ks5OhTjwS8gZt4F+VGxBODwUEQ0u2wJBF3lQY5ZqAM=; b=keMyorcAlRodWVvW89NIbdQiBgt14OT17VzSC53FTsMa7qhCDELV4XuBJQC9gE+MuV ERzwqxtc5gkEsGfo0Ru60foaeb0a19BpCXuvi+sU0n0FkCJoT1dLzPvHAA9SOWQKsZOh N/JritBSzzHyH7fMjCtp173bIEg3ooYexKX5aXC4GzG/UGY3i3174siNzMn2V3KNNwYx qhDJzDZDf8JVNkdkaapKJdUx1NroC8JImDWAWzVc+9Dy7yUlQEuisIRBWVmXtrnBf49q DQzBylZiIPZgHxMoQDaiH0WlLnCMMZFPBKsiKNzBr7kyDitIZvoChU8R4r2/FtIW1hce TymA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id pj2-20020a17090b4f4200b00276b35bc424si688854pjb.26.2023.09.19.08.40.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 08:40:08 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id B644480FA86C; Tue, 19 Sep 2023 07:43:25 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233011AbjISOnP (ORCPT + 26 others); Tue, 19 Sep 2023 10:43:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49784 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232943AbjISOnG (ORCPT ); Tue, 19 Sep 2023 10:43:06 -0400 Received: from out30-124.freemail.mail.aliyun.com (out30-124.freemail.mail.aliyun.com [115.124.30.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CFAB5BF; Tue, 19 Sep 2023 07:42:52 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R201e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0VsRsRuO_1695134567; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsRsRuO_1695134567) by smtp.aliyun-inc.com; Tue, 19 Sep 2023 22:42:49 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 07/18] net/smc: disable SEID on non-s390 architecture Date: Tue, 19 Sep 2023 22:41:51 +0800 Message-Id: <1695134522-126655-8-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> References: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 19 Sep 2023 07:43:25 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777481031988267661 X-GMAIL-MSGID: 1777481031988267661 On s390 architecture SMC-Dv2 SEID is generated by ISMv2 device with partial cpuid. It is predefined and unique to represents the physical machine (CPC), the maximum communication space of SMC-D. On non-s390 archs like x86/64 or ARM, there is no similar information to identify physical machine, especially in virtualization scenarios like KVM, etc. In such cases, SEID is forcibly disabled and the user- defined UEID will be used to represent the communicable space. Signed-off-by: Wen Gu --- net/smc/smc_clc.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/net/smc/smc_clc.c b/net/smc/smc_clc.c index c08e138..be56a7d 100644 --- a/net/smc/smc_clc.c +++ b/net/smc/smc_clc.c @@ -155,10 +155,12 @@ static int smc_clc_ueid_remove(char *ueid) rc = 0; } } +#if IS_ENABLED(CONFIG_S390) if (!rc && !smc_clc_eid_table.ueid_cnt) { smc_clc_eid_table.seid_enabled = 1; rc = -EAGAIN; /* indicate success and enabling of seid */ } +#endif write_unlock(&smc_clc_eid_table.lock); return rc; } @@ -273,22 +275,30 @@ int smc_nl_dump_seid(struct sk_buff *skb, struct netlink_callback *cb) int smc_nl_enable_seid(struct sk_buff *skb, struct genl_info *info) { +#if IS_ENABLED(CONFIG_S390) write_lock(&smc_clc_eid_table.lock); smc_clc_eid_table.seid_enabled = 1; write_unlock(&smc_clc_eid_table.lock); return 0; +#else + return -EOPNOTSUPP; +#endif } int smc_nl_disable_seid(struct sk_buff *skb, struct genl_info *info) { int rc = 0; +#if IS_ENABLED(CONFIG_S390) write_lock(&smc_clc_eid_table.lock); if (!smc_clc_eid_table.ueid_cnt) rc = -ENOENT; else smc_clc_eid_table.seid_enabled = 0; write_unlock(&smc_clc_eid_table.lock); +#else + rc = -EOPNOTSUPP; +#endif return rc; } @@ -1290,7 +1300,11 @@ void __init smc_clc_init(void) INIT_LIST_HEAD(&smc_clc_eid_table.list); rwlock_init(&smc_clc_eid_table.lock); smc_clc_eid_table.ueid_cnt = 0; +#if IS_ENABLED(CONFIG_S390) smc_clc_eid_table.seid_enabled = 1; +#else + smc_clc_eid_table.seid_enabled = 0; +#endif } void smc_clc_exit(void) From patchwork Tue Sep 19 14:41:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 141933 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp3447220vqi; Tue, 19 Sep 2023 07:52:15 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHiJ8xz9Hyepf6WLxzwejHkkGTl9eo5BSbEHfhwMmuGm+zEpvur6l3zpM3Ja2GvPNKrl5l9 X-Received: by 2002:a17:902:cec4:b0:1c4:21f3:4be9 with SMTP id d4-20020a170902cec400b001c421f34be9mr11977827plg.62.1695135135399; Tue, 19 Sep 2023 07:52:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695135135; cv=none; d=google.com; s=arc-20160816; b=OkFCPk2gW1743TF6gVvlIZW+ZPcb7k+EC2dpOPNNZSwfC/3hwbwvHt9NqnXfypGwMh DUkY5jCUxGwNveLm8lJAVsGd8TczUYHcs0g7DNbLD2JQR8xIuPb/ygyVZEWyWZ+WdeB2 GKJUo7Il2lezdvShTGmJDEiqqK720YvHN/QToxL0TsO3VQIMhrab6L45lnD9XuHxzqZa jnWTBxKDrKk5MPfILFV5rsGw5BLta4ThBGlDTGKTVBXRiHnZb9bop3g2ku7WkBvDEKyq 7RVaqThGT318k0DL7G5UMXh/rTsfV1z5jzWVNzD6xf2MYwwwFp/cePIk6Op1ZKZxlNaJ hLbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=pP9s9r8HD7ySucjT4KcOXsTqAgAMIkkPNmugiGe+8G4=; fh=2Ks5OhTjwS8gZt4F+VGxBODwUEQ0u2wJBF3lQY5ZqAM=; b=DIW/eLWK0+Uggjtp0SbGbYxgMd5aodm0WDGpAm6Sybb4Nk3Hf097ScanlPu9VoSSMs K2ohvlm76NRHkOW3ahkWuCpmwSaTcZQbSPqnOvOiCXi1Cnf1Tqc5qIvgg+ZckcoPcHJS kFl8B435GdOuv8JU9rfiupDzy+TW7f/dPsqTAwNFnf3zEvrDU0HWzgejg3gK0idIKt6b mfX0tmI2sHVqhY1FNC69AfuEu3jLtH8hmIk6SAF38NOtxYdREFdQOILVR88ojmdq9a0w 0a1SBg4FUv3o/iNiDpK01pDXux1QaUNJEuAVjd2FoUREBh4ARtZck631EYdjxc0zseDB hd9w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from fry.vger.email (fry.vger.email. [23.128.96.38]) by mx.google.com with ESMTPS id ik26-20020a170902ab1a00b001c0ab540e62si9786974plb.277.2023.09.19.07.52.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 07:52:15 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id AAA0F8244C97; Tue, 19 Sep 2023 07:43:39 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233024AbjISOnS (ORCPT + 26 others); Tue, 19 Sep 2023 10:43:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49736 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232922AbjISOnH (ORCPT ); Tue, 19 Sep 2023 10:43:07 -0400 Received: from out30-99.freemail.mail.aliyun.com (out30-99.freemail.mail.aliyun.com [115.124.30.99]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D6256130; Tue, 19 Sep 2023 07:42:53 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R181e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046056;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0VsRxW9Y_1695134569; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsRxW9Y_1695134569) by smtp.aliyun-inc.com; Tue, 19 Sep 2023 22:42:51 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 08/18] net/smc: enable virtual ISM device feature bit Date: Tue, 19 Sep 2023 22:41:52 +0800 Message-Id: <1695134522-126655-9-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> References: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Tue, 19 Sep 2023 07:43:39 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777478019722794789 X-GMAIL-MSGID: 1777478019722794789 This patch enables the first supplement feature of SMCv2.1, that is the support for virtual ISM devices. Signed-off-by: Wen Gu --- net/smc/af_smc.c | 2 ++ net/smc/smc.h | 7 +++++++ 2 files changed, 9 insertions(+) diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index 9e31033..7eab600 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -1522,6 +1522,7 @@ static int __smc_connect(struct smc_sock *smc) ini->smcr_version = SMC_V1 | SMC_V2; ini->smc_type_v1 = SMC_TYPE_B; ini->smc_type_v2 = SMC_TYPE_B; + ini->feature_mask = SMC_FEATURE_MASK; /* get vlan id from IP device */ if (smc_vlan_by_tcpsk(smc->clcsock, ini)) { @@ -1987,6 +1988,7 @@ static int smc_listen_v2_check(struct smc_sock *new_smc, ini->smc_type_v2 = pclc->hdr.typev2; ini->smcd_version = smcd_indicated(ini->smc_type_v1) ? SMC_V1 : 0; ini->smcr_version = smcr_indicated(ini->smc_type_v1) ? SMC_V1 : 0; + ini->feature_mask = SMC_FEATURE_MASK; if (pclc->hdr.version > SMC_V1) { if (smcd_indicated(ini->smc_type_v2)) ini->smcd_version |= SMC_V2; diff --git a/net/smc/smc.h b/net/smc/smc.h index 24745fd..71b9640 100644 --- a/net/smc/smc.h +++ b/net/smc/smc.h @@ -58,6 +58,13 @@ enum smc_state { /* possible states of an SMC socket */ SMC_PROCESSABORT = 27, }; +enum smc_supp_feature { /* supplemental features supported by current version */ + SMC_SPF_VIRT_ISM_DEV = 0, +}; + +#define SMC_FEATURE_MASK \ + (BIT(SMC_SPF_VIRT_ISM_DEV)) + struct smc_link_group; struct smc_wr_rx_hdr { /* common prefix part of LLC and CDC to demultiplex */ From patchwork Tue Sep 19 14:41:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 141922 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp3443634vqi; Tue, 19 Sep 2023 07:46:31 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFWYEQgTQ6UWwC8UmweMYKi5PW/fHA/DKKX52QkIfeTdcdO4D2jVueQNNPW4K5rX7IRwXSe X-Received: by 2002:a17:902:f544:b0:1bb:e74b:39ff with SMTP id h4-20020a170902f54400b001bbe74b39ffmr12941777plf.0.1695134791000; Tue, 19 Sep 2023 07:46:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695134790; cv=none; d=google.com; s=arc-20160816; b=e160NXiot36i9hIznEafV/GLS3XB9uPq+RjXjLBdFKOZWEysbAx0/wijWB0s4nSnet vCaK0bksSeEhCmj8SaG8LTn+fJ2vlWOb7KwlCFcu4T2keBYMiJWgFFHyrgfx5P9xtLZw 8b3Xjk3WLnuimBO+jpAUVatv1pV9oc5t0uq8MuGFB/XaqILt19wGyCeYmg3hyWyX84N6 eGuCHIH7Yj4TvLgld8WN6OeBdl3pCP3rIKQLpAErmYMznQdvxzEhVK6m0j2xz6a9tLJv WiN6RTBWppY6bF5IUzvgpL8gvCPxtnzI0v8J307WanLPMvejlZZF442MLB24wnHBS1fV qEvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=76PXCm2Rj8itN3Rg/CAAoPFc7YiTqk5BvMObomY4MRE=; fh=2Ks5OhTjwS8gZt4F+VGxBODwUEQ0u2wJBF3lQY5ZqAM=; b=IzLt30JvP8blWzDpT6B2TOO+jlxpamj4xwlsATDxnoIC8ABUCUlpuI3rZZ1UygsIXm x9pIOEu+wSXGXbOiqX54mKVk2JS5VpW15dogdHSmA0Pn53tj2BEEYx4F0pz7pBW8f8BS VOVhYglv1BgGREpNvO/daMe30tJ2m8boZirklNxSg/0yPOk5+Wc+hEz0jg5E+5PScIy1 31YHf/pCbTwuiOSKtuUPPjPKmSPtOxi3360pY+k8aY8oC0N+hVOG2z3rfHiQrd7xoLCj o4W3sHkma22wAsedJ3z40k63O5WoX5PPIqM1YWPFjAkoA+9kRBHGCyViYA0GkMWLO8Ft KBmA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from groat.vger.email (groat.vger.email. [2620:137:e000::3:5]) by mx.google.com with ESMTPS id i2-20020a170902c94200b001bbad1883d5si4150953pla.293.2023.09.19.07.46.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 07:46:30 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id A74E38266256; Tue, 19 Sep 2023 07:43:59 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232955AbjISOnU (ORCPT + 26 others); Tue, 19 Sep 2023 10:43:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49736 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232909AbjISOnI (ORCPT ); Tue, 19 Sep 2023 10:43:08 -0400 Received: from out30-111.freemail.mail.aliyun.com (out30-111.freemail.mail.aliyun.com [115.124.30.111]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D383D194; Tue, 19 Sep 2023 07:42:57 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R151e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045192;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0VsRxWAF_1695134571; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsRxWAF_1695134571) by smtp.aliyun-inc.com; Tue, 19 Sep 2023 22:42:53 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 09/18] net/smc: introduce SMC-D loopback device Date: Tue, 19 Sep 2023 22:41:53 +0800 Message-Id: <1695134522-126655-10-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> References: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Tue, 19 Sep 2023 07:43:59 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777477658652476654 X-GMAIL-MSGID: 1777477658652476654 This patch introduces a kind of loopback device for SMC-D. The device is created when SMC module is loaded and destroyed when the SMC module is unloaded. The loopback device is a kernel device used only by the SMC module and is not restricted by net namespace, so it can be used for local inter-process or inter-container communication. Signed-off-by: Wen Gu --- net/smc/Makefile | 2 +- net/smc/af_smc.c | 12 +++- net/smc/smc_loopback.c | 157 +++++++++++++++++++++++++++++++++++++++++++++++++ net/smc/smc_loopback.h | 31 ++++++++++ 4 files changed, 200 insertions(+), 2 deletions(-) create mode 100644 net/smc/smc_loopback.c create mode 100644 net/smc/smc_loopback.h diff --git a/net/smc/Makefile b/net/smc/Makefile index 875efcd..a8c3711 100644 --- a/net/smc/Makefile +++ b/net/smc/Makefile @@ -4,5 +4,5 @@ obj-$(CONFIG_SMC) += smc.o obj-$(CONFIG_SMC_DIAG) += smc_diag.o smc-y := af_smc.o smc_pnet.o smc_ib.o smc_clc.o smc_core.o smc_wr.o smc_llc.o smc-y += smc_cdc.o smc_tx.o smc_rx.o smc_close.o smc_ism.o smc_netlink.o smc_stats.o -smc-y += smc_tracepoint.o +smc-y += smc_tracepoint.o smc_loopback.o smc-$(CONFIG_SYSCTL) += smc_sysctl.o diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index 7eab600..bc4300e 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -53,6 +53,7 @@ #include "smc_stats.h" #include "smc_tracepoint.h" #include "smc_sysctl.h" +#include "smc_loopback.h" static DEFINE_MUTEX(smc_server_lgr_pending); /* serialize link group * creation on server @@ -3552,15 +3553,23 @@ static int __init smc_init(void) goto out_sock; } + rc = smc_loopback_init(); + if (rc) { + pr_err("%s: smc_loopback_init fails with %d\n", __func__, rc); + goto out_ib; + } + rc = tcp_register_ulp(&smc_ulp_ops); if (rc) { pr_err("%s: tcp_ulp_register fails with %d\n", __func__, rc); - goto out_ib; + goto out_lo; } static_branch_enable(&tcp_have_smc); return 0; +out_lo: + smc_loopback_exit(); out_ib: smc_ib_unregister_client(); out_sock: @@ -3598,6 +3607,7 @@ static void __exit smc_exit(void) tcp_unregister_ulp(&smc_ulp_ops); sock_unregister(PF_SMC); smc_core_exit(); + smc_loopback_exit(); smc_ib_unregister_client(); smc_ism_exit(); destroy_workqueue(smc_close_wq); diff --git a/net/smc/smc_loopback.c b/net/smc/smc_loopback.c new file mode 100644 index 0000000..7d88856 --- /dev/null +++ b/net/smc/smc_loopback.c @@ -0,0 +1,157 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Shared Memory Communications Direct over loopback device. + * + * Provide a SMC-D loopback dummy device. + * + * Copyright (c) 2022, Alibaba Inc. + * + * Author: Wen Gu + * Tony Lu + * + */ + +#include +#include +#include + +#include "smc_ism.h" +#include "smc_loopback.h" + +static const char smc_lo_dev_name[] = "smc_lo"; +static struct smc_lo_dev *lo_dev; + +static const struct smcd_ops lo_ops = { + .query_remote_gid = NULL, + .register_dmb = NULL, + .unregister_dmb = NULL, + .add_vlan_id = NULL, + .del_vlan_id = NULL, + .set_vlan_required = NULL, + .reset_vlan_required = NULL, + .signal_event = NULL, + .move_data = NULL, + .supports_v2 = NULL, + .get_system_eid = NULL, + .get_local_gid = NULL, + .get_chid = NULL, + .get_dev = NULL, +}; + +static struct smcd_dev *smcd_lo_alloc_dev(const struct smcd_ops *ops, + int max_dmbs) +{ + struct smcd_dev *smcd; + + smcd = kzalloc(sizeof(*smcd), GFP_KERNEL); + if (!smcd) + return NULL; + + smcd->conn = kcalloc(max_dmbs, sizeof(struct smc_connection *), + GFP_KERNEL); + if (!smcd->conn) + goto out_smcd; + + smcd->ops = ops; + + spin_lock_init(&smcd->lock); + spin_lock_init(&smcd->lgr_lock); + INIT_LIST_HEAD(&smcd->vlan); + INIT_LIST_HEAD(&smcd->lgr_list); + init_waitqueue_head(&smcd->lgrs_deleted); + return smcd; + +out_smcd: + kfree(smcd); + return NULL; +} + +static int smcd_lo_register_dev(struct smc_lo_dev *ldev) +{ + struct smcd_dev *smcd; + + smcd = smcd_lo_alloc_dev(&lo_ops, SMC_LODEV_MAX_DMBS); + if (!smcd) + return -ENOMEM; + + ldev->smcd = smcd; + smcd->priv = ldev; + + /* TODO: + * register smc_lo to smcd_dev list. + */ + return 0; +} + +static void smcd_lo_unregister_dev(struct smc_lo_dev *ldev) +{ + /* TODO: + * unregister smc_lo from smcd_dev list. + */ +} + +static void smc_lo_dev_release(struct device *dev) +{ + struct smc_lo_dev *ldev = + container_of(dev, struct smc_lo_dev, dev); + struct smcd_dev *smcd = ldev->smcd; + + kfree(smcd->conn); + kfree(smcd); + kfree(ldev); +} + +static int smc_lo_dev_init(struct smc_lo_dev *ldev) +{ + return smcd_lo_register_dev(ldev); +} + +static int smc_lo_dev_probe(void) +{ + struct smc_lo_dev *ldev; + int ret; + + ldev = kzalloc(sizeof(*ldev), GFP_KERNEL); + if (!ldev) + return -ENOMEM; + + ldev->dev.parent = NULL; + ldev->dev.release = smc_lo_dev_release; + device_initialize(&ldev->dev); + dev_set_name(&ldev->dev, smc_lo_dev_name); + + ret = smc_lo_dev_init(ldev); + if (ret) + goto free_dev; + + lo_dev = ldev; /* global loopback device */ + return 0; + +free_dev: + kfree(ldev); + return ret; +} + +static void smc_lo_dev_exit(struct smc_lo_dev *ldev) +{ + smcd_lo_unregister_dev(ldev); +} + +static void smc_lo_dev_remove(void) +{ + if (!lo_dev) + return; + + smc_lo_dev_exit(lo_dev); + put_device(&lo_dev->dev); /* device_initialize in smc_lo_dev_probe */ +} + +int smc_loopback_init(void) +{ + return smc_lo_dev_probe(); +} + +void smc_loopback_exit(void) +{ + smc_lo_dev_remove(); +} diff --git a/net/smc/smc_loopback.h b/net/smc/smc_loopback.h new file mode 100644 index 0000000..0f7583c --- /dev/null +++ b/net/smc/smc_loopback.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Shared Memory Communications Direct over loopback device. + * + * Provide a SMC-D loopback dummy device. + * + * Copyright (c) 2022, Alibaba Inc. + * + * Author: Wen Gu + * Tony Lu + * + */ + +#ifndef _SMC_LOOPBACK_H +#define _SMC_LOOPBACK_H + +#include +#include +#include + +#define SMC_LODEV_MAX_DMBS 5000 + +struct smc_lo_dev { + struct smcd_dev *smcd; + struct device dev; +}; + +int smc_loopback_init(void); +void smc_loopback_exit(void); + +#endif /* _SMC_LOOPBACK_H */ From patchwork Tue Sep 19 14:41:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 141924 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp3445368vqi; Tue, 19 Sep 2023 07:49:12 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEbg0a3IVSZeh9CuGoyy41GbKm0eTj4H6mEaGzeKYGieE8G8149DWt4Nb3vw207fGCZb4tL X-Received: by 2002:a17:90a:c385:b0:274:7fbc:3f62 with SMTP id h5-20020a17090ac38500b002747fbc3f62mr3597451pjt.16.1695134952470; Tue, 19 Sep 2023 07:49:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695134952; cv=none; d=google.com; s=arc-20160816; b=x5NxXD54782M54Hu+SdmMEn0TZk4+8ZJZ/XogSuZZOhaD4ut2JFfVYYmQSzR+PCoPf D/ee71MFAKWqGnsumqQblZQ61uH2c9reCve4+6ZTsbCMiIVTG2i+925Cs2lf0vPY0h/8 2X/efA7Ocu7auPcdtRDZRKuXcr6lR+fJedHNr0F2DKIiIK0Xf74kbNAZ/I93gPf0WXNq goCHFM/Bl0lpkJBtbHjkibpLdDIcnzK9p/coqFmn3NmsPe/X0JlvWnI4Zx7YRhmXcQ0G pQOdLADeJezCrIVWUFkR7JBbL2gANkjH2EaGmuURbl1e6kRbFPPvWIPAermxW1MIfmGW McBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=tzVfMqREXEGgYsRKpciIU5Rozu/a3ThpSHh6M7t8GsI=; fh=2Ks5OhTjwS8gZt4F+VGxBODwUEQ0u2wJBF3lQY5ZqAM=; b=Bh1lwjyE29dDeC7rXPhEGIJoFCsg2KWIvqmul/PPiSVA3aUOn1kPSZ7gG/ylmK8CNG MHp7bK6a8sWu1vVBDMvjLTG6B9BaEDUqnY+nq1ZRGcfe7IpsK6LWDY7HnjlU8+dg3OD+ l4SBW/Epy72QmKeu/Bnx/3xgKUPXuRqrw34N/4N0kblf/T6m/PmWFXPAXayYLQS4YMrC HmCqZS7paMTTStxYnTHmgiyu9vz0ZZOwfyqpqq5QcfyCbvDqxIjQ/jUDouUyIgtR5aYw XIBmEXdbaxzZ6aJyNxDVOHlyhiScJe5XGo2x3PzBN/jDrxL73TY0ZbCmmco12QVX9GIi S0Lw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id kk9-20020a17090b4a0900b00258996c09e4si1210771pjb.22.2023.09.19.07.49.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 07:49:12 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 6E2EF811F920; Tue, 19 Sep 2023 07:43:40 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233042AbjISOnY (ORCPT + 26 others); Tue, 19 Sep 2023 10:43:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49990 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232982AbjISOnJ (ORCPT ); Tue, 19 Sep 2023 10:43:09 -0400 Received: from out30-119.freemail.mail.aliyun.com (out30-119.freemail.mail.aliyun.com [115.124.30.119]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B15471A8; Tue, 19 Sep 2023 07:42:59 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R161e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045168;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0VsRxWB5_1695134574; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsRxWB5_1695134574) by smtp.aliyun-inc.com; Tue, 19 Sep 2023 22:42:56 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 10/18] net/smc: implement ID-related operations of loopback Date: Tue, 19 Sep 2023 22:41:54 +0800 Message-Id: <1695134522-126655-11-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> References: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 19 Sep 2023 07:43:40 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777477828186516602 X-GMAIL-MSGID: 1777477828186516602 This patch implements GID/CHID/SEID related operations of SMC-D loopback device. In loopback device, GID is generated by UUIDv4 algorithm, CHID is reserved 0xFFFF, SEID is generated using the same algorithm as ISM device under s390 architecture, and is 0 and disabled under non-s390 architecture. Signed-off-by: Wen Gu --- include/net/smc.h | 6 ++++ net/smc/smc_ism.h | 2 ++ net/smc/smc_loopback.c | 95 ++++++++++++++++++++++++++++++++++++++++++++++---- net/smc/smc_loopback.h | 3 ++ 4 files changed, 100 insertions(+), 6 deletions(-) diff --git a/include/net/smc.h b/include/net/smc.h index a37fb38..389d26d 100644 --- a/include/net/smc.h +++ b/include/net/smc.h @@ -41,6 +41,12 @@ struct smcd_dmb { dma_addr_t dma_addr; }; +struct smcd_seid { + u8 seid_string[24]; + u8 serial_number[4]; + u8 type[4]; +}; + #define ISM_EVENT_DMB 0 #define ISM_EVENT_GID 1 #define ISM_EVENT_SWR 2 diff --git a/net/smc/smc_ism.h b/net/smc/smc_ism.h index e6ea08c..7ab82dd 100644 --- a/net/smc/smc_ism.h +++ b/net/smc/smc_ism.h @@ -15,6 +15,8 @@ #include "smc.h" +#define S390_ISM_IDENT_MASK 0x00FFFF + #define SMC_VIRT_ISM_CHID_MAX 0xFFFF #define SMC_VIRT_ISM_CHID_MIN 0xFF00 diff --git a/net/smc/smc_loopback.c b/net/smc/smc_loopback.c index 7d88856..6b8ff65 100644 --- a/net/smc/smc_loopback.c +++ b/net/smc/smc_loopback.c @@ -18,11 +18,92 @@ #include "smc_ism.h" #include "smc_loopback.h" +#define SMC_LO_SUPPORTS_V2 0x1 /* SMC-D loopback supports SMC-Dv2 */ + static const char smc_lo_dev_name[] = "smc_lo"; +static struct smcd_seid SMC_LO_SEID = { + .seid_string = "IBM-SYSZ-ISMSEID00000000", + .serial_number = "0000", + .type = "0000", +}; + static struct smc_lo_dev *lo_dev; +static void smc_lo_create_seid(struct smcd_seid *seid) +{ +#if IS_ENABLED(CONFIG_S390) + struct cpuid id; + u16 ident_tail; + char tmp[5]; + + get_cpu_id(&id); + ident_tail = (u16)(id.ident & S390_ISM_IDENT_MASK); + snprintf(tmp, 5, "%04X", ident_tail); + memcpy(&seid->serial_number, tmp, 4); + snprintf(tmp, 5, "%04X", id.machine); + memcpy(&seid->type, tmp, 4); +#else + memset(seid, 0, SMC_MAX_EID_LEN); +#endif +} + +static void smc_lo_generate_id(struct smc_lo_dev *ldev) +{ + struct smcd_gid *lgid = &ldev->local_gid; + uuid_t uuid; + + uuid_gen(&uuid); + memcpy(&lgid->gid, &uuid, sizeof(lgid->gid)); + memcpy(&lgid->gid_ext, (u8 *)&uuid + sizeof(lgid->gid), + sizeof(lgid->gid_ext)); + + ldev->chid = SMC_LO_CHID; + smc_lo_create_seid(&SMC_LO_SEID); +} + +static int smc_lo_query_rgid(struct smcd_dev *smcd, struct smcd_gid *rgid, + u32 vid_valid, u32 vid) +{ + struct smc_lo_dev *ldev = smcd->priv; + + /* rgid should equal to lgid in loopback */ + if (!ldev || rgid->gid != ldev->local_gid.gid || + rgid->gid_ext != ldev->local_gid.gid_ext) + return -ENETUNREACH; + return 0; +} + +static int smc_lo_supports_v2(void) +{ + return SMC_LO_SUPPORTS_V2; +} + +static u8 *smc_lo_get_system_eid(void) +{ + return SMC_LO_SEID.seid_string; +} + +static void smc_lo_get_local_gid(struct smcd_dev *smcd, + struct smcd_gid *smcd_gid) +{ + struct smc_lo_dev *ldev = smcd->priv; + + smcd_gid->gid = ldev->local_gid.gid; + smcd_gid->gid_ext = ldev->local_gid.gid_ext; +} + +static u16 smc_lo_get_chid(struct smcd_dev *smcd) +{ + return ((struct smc_lo_dev *)smcd->priv)->chid; +} + +static struct device *smc_lo_get_dev(struct smcd_dev *smcd) +{ + return &((struct smc_lo_dev *)smcd->priv)->dev; +} + static const struct smcd_ops lo_ops = { - .query_remote_gid = NULL, + .query_remote_gid = smc_lo_query_rgid, .register_dmb = NULL, .unregister_dmb = NULL, .add_vlan_id = NULL, @@ -31,11 +112,11 @@ .reset_vlan_required = NULL, .signal_event = NULL, .move_data = NULL, - .supports_v2 = NULL, - .get_system_eid = NULL, - .get_local_gid = NULL, - .get_chid = NULL, - .get_dev = NULL, + .supports_v2 = smc_lo_supports_v2, + .get_system_eid = smc_lo_get_system_eid, + .get_local_gid = smc_lo_get_local_gid, + .get_chid = smc_lo_get_chid, + .get_dev = smc_lo_get_dev, }; static struct smcd_dev *smcd_lo_alloc_dev(const struct smcd_ops *ops, @@ -103,6 +184,8 @@ static void smc_lo_dev_release(struct device *dev) static int smc_lo_dev_init(struct smc_lo_dev *ldev) { + smc_lo_generate_id(ldev); + return smcd_lo_register_dev(ldev); } diff --git a/net/smc/smc_loopback.h b/net/smc/smc_loopback.h index 0f7583c..2156f22 100644 --- a/net/smc/smc_loopback.h +++ b/net/smc/smc_loopback.h @@ -18,11 +18,14 @@ #include #include +#define SMC_LO_CHID 0xFFFF #define SMC_LODEV_MAX_DMBS 5000 struct smc_lo_dev { struct smcd_dev *smcd; struct device dev; + u16 chid; + struct smcd_gid local_gid; }; int smc_loopback_init(void); From patchwork Tue Sep 19 14:41:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 141934 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp3447596vqi; Tue, 19 Sep 2023 07:52:53 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGnsrlTiQTLST8ILPR/lMNX/Wt/EDZ6gRy+W8EomFi5RVv6RGsbgbqaiqHOHV4Idi1qzSwg X-Received: by 2002:a05:6a20:8e02:b0:141:cc5f:7d28 with SMTP id y2-20020a056a208e0200b00141cc5f7d28mr6491447pzj.50.1695135172988; Tue, 19 Sep 2023 07:52:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695135172; cv=none; d=google.com; s=arc-20160816; b=dy3vQOKsOr0tA9fuoNz6/F7hKakfrj4TbBGmzS9CuLdtX4ZuQ8C1SeKVQTCEdb/sQe lc/wNMyRctuQlk0+vZpOO3LI2XPbLlDzSisQ5Jp2rqIJ2hlfj5oRfGLXNC8LpVVnCR15 N/42eY7MDLKz5GG73bP9qcstYm1z8y/C4WBHY6LbTOsDrwFC7+6KyBB+n1I55hCO5UGg 7mRn4nueebK925AkhBFNAks9SpdPUobKNGGjhc//yejgqi0v4KyBczXytSYZXNxDpnMf trJcYcOT7R28CWU6mhOpEMqutz9LPaQe5gq1AHwTsc96XWwfpx9VEVQW2a8TnKxD5NtQ 4xXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=757vBKA+aPUcQjvHtDQDkXhjjWNpZNpazhiORavJmyw=; fh=2Ks5OhTjwS8gZt4F+VGxBODwUEQ0u2wJBF3lQY5ZqAM=; b=vdm/xD0fwLvGCkM8jN9JMDCy89QIhlgw1biYdTUtKoGUztLJUx4wHQLbX87pvOM7J+ lCSDyngry6zgoD43Xxy4Le35H1uDulXI0y6fqsUY8sBiGyq263TnWNir92GHRHFfhjfb iYP0X1f1EsHidCJ6O8Qt89VRbtphEkb0ZdBvuAR3Ybe67+VgVsqzEt+kDSwW+cJWNiMI +n5DPF3Prj3jJBCpvA54jS9hQscRqX8r+2YcOYzuaM6QFuwzMBf+7CkYGZCpJqwYiBKy C6KfViRlOYfnA8Z1nJV05LzCMeImZlVTMxXwtbl4g8a/DwOIZ4FW4EtkdlwW937Qe+fq yPXQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id cq24-20020a056a00331800b0068a6f6d9f7dsi9885745pfb.57.2023.09.19.07.52.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 07:52:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 2E76F8023918; Tue, 19 Sep 2023 07:43:57 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233106AbjISOnm (ORCPT + 26 others); Tue, 19 Sep 2023 10:43:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34234 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233050AbjISOna (ORCPT ); Tue, 19 Sep 2023 10:43:30 -0400 Received: from out30-118.freemail.mail.aliyun.com (out30-118.freemail.mail.aliyun.com [115.124.30.118]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0FD22CDC; Tue, 19 Sep 2023 07:43:07 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R731e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045168;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0VsRqc92_1695134576; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsRqc92_1695134576) by smtp.aliyun-inc.com; Tue, 19 Sep 2023 22:42:58 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 11/18] net/smc: implement some unsupported operations of loopback Date: Tue, 19 Sep 2023 22:41:55 +0800 Message-Id: <1695134522-126655-12-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> References: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Tue, 19 Sep 2023 07:43:57 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777478059122309258 X-GMAIL-MSGID: 1777478059122309258 Vlan operations are not supported currently since the need for vlan in loopback does not seem to be strong. Signal_event operation is not supported since no event now needs to be processed by loopback device. Signed-off-by: Wen Gu --- net/smc/smc_loopback.c | 36 +++++++++++++++++++++++++++++++----- 1 file changed, 31 insertions(+), 5 deletions(-) diff --git a/net/smc/smc_loopback.c b/net/smc/smc_loopback.c index 6b8ff65..fe61260 100644 --- a/net/smc/smc_loopback.c +++ b/net/smc/smc_loopback.c @@ -73,6 +73,32 @@ static int smc_lo_query_rgid(struct smcd_dev *smcd, struct smcd_gid *rgid, return 0; } +static int smc_lo_add_vlan_id(struct smcd_dev *smcd, u64 vlan_id) +{ + return -EOPNOTSUPP; +} + +static int smc_lo_del_vlan_id(struct smcd_dev *smcd, u64 vlan_id) +{ + return -EOPNOTSUPP; +} + +static int smc_lo_set_vlan_required(struct smcd_dev *smcd) +{ + return -EOPNOTSUPP; +} + +static int smc_lo_reset_vlan_required(struct smcd_dev *smcd) +{ + return -EOPNOTSUPP; +} + +static int smc_lo_signal_event(struct smcd_dev *dev, u64 rgid, u32 trigger_irq, + u32 event_code, u64 info) +{ + return 0; +} + static int smc_lo_supports_v2(void) { return SMC_LO_SUPPORTS_V2; @@ -106,11 +132,11 @@ static struct device *smc_lo_get_dev(struct smcd_dev *smcd) .query_remote_gid = smc_lo_query_rgid, .register_dmb = NULL, .unregister_dmb = NULL, - .add_vlan_id = NULL, - .del_vlan_id = NULL, - .set_vlan_required = NULL, - .reset_vlan_required = NULL, - .signal_event = NULL, + .add_vlan_id = smc_lo_add_vlan_id, + .del_vlan_id = smc_lo_del_vlan_id, + .set_vlan_required = smc_lo_set_vlan_required, + .reset_vlan_required = smc_lo_reset_vlan_required, + .signal_event = smc_lo_signal_event, .move_data = NULL, .supports_v2 = smc_lo_supports_v2, .get_system_eid = smc_lo_get_system_eid, From patchwork Tue Sep 19 14:41:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 141951 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp3461336vqi; Tue, 19 Sep 2023 08:10:28 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEOxLDGWlaVS/9KYIfIm4G0Y0+XIcgH9IoXL5OhdSMe0Ps6KLCoYKVjQ4z/G8zOG7mNhryC X-Received: by 2002:a05:6a00:2eaa:b0:68f:b5cb:ced0 with SMTP id fd42-20020a056a002eaa00b0068fb5cbced0mr15026144pfb.34.1695136228339; Tue, 19 Sep 2023 08:10:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695136228; cv=none; d=google.com; s=arc-20160816; b=sn601A8pkHQLlG+GLh2ko2hHt4xf117Ui9VoFS9eQH1dE0sPwhx5GCIienjNb4qRIE CHc8yM6NCQ2PJ3ATaoqYE6wXFSSTotmWA6NWYFUYV0/zWXJ387BrdrUcKdXfO+5W/4UH hibuI4U1Ql/H823VwXfvwBCpqW5EueUoN20PKSvb8/i7v3YjDZAKEue5wTqBXCYj+fRq ZH/gbbcgCSYkYEjWuCsaH5kujzFYHmoE3Mf1NRs9uwjzt4k2PeiHxtWkiiWxCqKkRiea TcrpHx25PYklHfIdA5xsLztaL+18Q6hsUDO9Ji8OMWyZ9gKE5uVWxbCqLAYaODPu+JvY cGyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=4c3NFccSGYrbIx+lnALaSoCVQDE87yNB0px+UjdNolQ=; fh=2Ks5OhTjwS8gZt4F+VGxBODwUEQ0u2wJBF3lQY5ZqAM=; b=DLrRUQgUvFzw+Bb6bSgLwh1gB4KeoA2FjB2LnwfMJl/i9uQnoiaNNBEGRWNRXxpfgb On6FWQ0kkH9zxU/WWtOLtuSw05JTA64JYZDqwYNdZp8sF0D36S12GBdobBqcQd9vB0oe +9707b9B1S6lq3SX7gxgGWtpspkO2Uim9UT0mQnYM69IlQf5H0syGr1dhmX9z/tmQXdy mXbl6s+aC+SD10NpoKCXDlZtjpKF+P0ykHFw7QdJEsTfmeqSHqlikQGkd6vWxR+G1S4h GdLSli0zx/ruVrbw91wMf7wxKT+fxqxIjMa0pLS67lb68qvzG+pe8mVWmg15Az99XmEM 0Z7Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id s196-20020a632ccd000000b00573ffd25b44si9831569pgs.182.2023.09.19.08.10.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 08:10:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 00A1F8047462; Tue, 19 Sep 2023 07:45:44 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233013AbjISOnd (ORCPT + 26 others); Tue, 19 Sep 2023 10:43:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47844 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232939AbjISOnO (ORCPT ); Tue, 19 Sep 2023 10:43:14 -0400 Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D96CCEA; Tue, 19 Sep 2023 07:43:03 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R581e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0VsRx7eE_1695134578; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsRx7eE_1695134578) by smtp.aliyun-inc.com; Tue, 19 Sep 2023 22:43:00 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 12/18] net/smc: implement DMB-related operations of loopback Date: Tue, 19 Sep 2023 22:41:56 +0800 Message-Id: <1695134522-126655-13-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> References: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Tue, 19 Sep 2023 07:45:45 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777479165773720899 X-GMAIL-MSGID: 1777479165773720899 This patch implements DMB registration, unregistration and data move operations of SMC-D loopback. Signed-off-by: Wen Gu --- net/smc/smc_cdc.c | 6 +++ net/smc/smc_cdc.h | 1 + net/smc/smc_loopback.c | 128 +++++++++++++++++++++++++++++++++++++++++++++++-- net/smc/smc_loopback.h | 13 +++++ 4 files changed, 145 insertions(+), 3 deletions(-) diff --git a/net/smc/smc_cdc.c b/net/smc/smc_cdc.c index 89105e9..2641800 100644 --- a/net/smc/smc_cdc.c +++ b/net/smc/smc_cdc.c @@ -411,6 +411,12 @@ static void smc_cdc_msg_recv(struct smc_sock *smc, struct smc_cdc_msg *cdc) static void smcd_cdc_rx_tsklet(struct tasklet_struct *t) { struct smc_connection *conn = from_tasklet(conn, t, rx_tsklet); + + smcd_cdc_rx_handler(conn); +} + +void smcd_cdc_rx_handler(struct smc_connection *conn) +{ struct smcd_cdc_msg *data_cdc; struct smcd_cdc_msg cdc; struct smc_sock *smc; diff --git a/net/smc/smc_cdc.h b/net/smc/smc_cdc.h index 696cc11..11559d4 100644 --- a/net/smc/smc_cdc.h +++ b/net/smc/smc_cdc.h @@ -301,5 +301,6 @@ int smcr_cdc_msg_send_validation(struct smc_connection *conn, struct smc_wr_buf *wr_buf); int smc_cdc_init(void) __init; void smcd_cdc_rx_init(struct smc_connection *conn); +void smcd_cdc_rx_handler(struct smc_connection *conn); #endif /* SMC_CDC_H */ diff --git a/net/smc/smc_loopback.c b/net/smc/smc_loopback.c index fe61260..7807b38 100644 --- a/net/smc/smc_loopback.c +++ b/net/smc/smc_loopback.c @@ -15,6 +15,7 @@ #include #include +#include "smc_cdc.h" #include "smc_ism.h" #include "smc_loopback.h" @@ -73,6 +74,93 @@ static int smc_lo_query_rgid(struct smcd_dev *smcd, struct smcd_gid *rgid, return 0; } +static int smc_lo_register_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb, + void *client_priv) +{ + struct smc_lo_dmb_node *dmb_node, *tmp_node; + struct smc_lo_dev *ldev = smcd->priv; + int sba_idx, rc; + + /* check space for new dmb */ + for_each_clear_bit(sba_idx, ldev->sba_idx_mask, SMC_LODEV_MAX_DMBS) { + if (!test_and_set_bit(sba_idx, ldev->sba_idx_mask)) + break; + } + if (sba_idx == SMC_LODEV_MAX_DMBS) + return -ENOSPC; + + dmb_node = kzalloc(sizeof(*dmb_node), GFP_KERNEL); + if (!dmb_node) { + rc = -ENOMEM; + goto err_bit; + } + + dmb_node->sba_idx = sba_idx; + dmb_node->cpu_addr = kzalloc(dmb->dmb_len, GFP_KERNEL | + __GFP_NOWARN | __GFP_NORETRY | + __GFP_NOMEMALLOC); + if (!dmb_node->cpu_addr) { + rc = -ENOMEM; + goto err_node; + } + dmb_node->len = dmb->dmb_len; + dmb_node->dma_addr = (dma_addr_t)dmb_node->cpu_addr; + +again: + /* add new dmb into hash table */ + get_random_bytes(&dmb_node->token, sizeof(dmb_node->token)); + write_lock(&ldev->dmb_ht_lock); + hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb_node->token) { + if (tmp_node->token == dmb_node->token) { + write_unlock(&ldev->dmb_ht_lock); + goto again; + } + } + hash_add(ldev->dmb_ht, &dmb_node->list, dmb_node->token); + write_unlock(&ldev->dmb_ht_lock); + + dmb->sba_idx = dmb_node->sba_idx; + dmb->dmb_tok = dmb_node->token; + dmb->cpu_addr = dmb_node->cpu_addr; + dmb->dma_addr = dmb_node->dma_addr; + dmb->dmb_len = dmb_node->len; + + return 0; + +err_node: + kfree(dmb_node); +err_bit: + clear_bit(sba_idx, ldev->sba_idx_mask); + return rc; +} + +static int smc_lo_unregister_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb) +{ + struct smc_lo_dmb_node *dmb_node = NULL, *tmp_node; + struct smc_lo_dev *ldev = smcd->priv; + + /* remove dmb from hash table */ + write_lock(&ldev->dmb_ht_lock); + hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb->dmb_tok) { + if (tmp_node->token == dmb->dmb_tok) { + dmb_node = tmp_node; + break; + } + } + if (!dmb_node) { + write_unlock(&ldev->dmb_ht_lock); + return -EINVAL; + } + hash_del(&dmb_node->list); + write_unlock(&ldev->dmb_ht_lock); + + clear_bit(dmb_node->sba_idx, ldev->sba_idx_mask); + kfree(dmb_node->cpu_addr); + kfree(dmb_node); + + return 0; +} + static int smc_lo_add_vlan_id(struct smcd_dev *smcd, u64 vlan_id) { return -EOPNOTSUPP; @@ -99,6 +187,38 @@ static int smc_lo_signal_event(struct smcd_dev *dev, u64 rgid, u32 trigger_irq, return 0; } +static int smc_lo_move_data(struct smcd_dev *smcd, u64 dmb_tok, unsigned int idx, + bool sf, unsigned int offset, void *data, + unsigned int size) +{ + struct smc_lo_dmb_node *rmb_node = NULL, *tmp_node; + struct smc_lo_dev *ldev = smcd->priv; + + read_lock(&ldev->dmb_ht_lock); + hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb_tok) { + if (tmp_node->token == dmb_tok) { + rmb_node = tmp_node; + break; + } + } + if (!rmb_node) { + read_unlock(&ldev->dmb_ht_lock); + return -EINVAL; + } + read_unlock(&ldev->dmb_ht_lock); + + memcpy((char *)rmb_node->cpu_addr + offset, data, size); + + if (sf) { + struct smc_connection *conn = + smcd->conn[rmb_node->sba_idx]; + + if (conn && !conn->killed) + smcd_cdc_rx_handler(conn); + } + return 0; +} + static int smc_lo_supports_v2(void) { return SMC_LO_SUPPORTS_V2; @@ -130,14 +250,14 @@ static struct device *smc_lo_get_dev(struct smcd_dev *smcd) static const struct smcd_ops lo_ops = { .query_remote_gid = smc_lo_query_rgid, - .register_dmb = NULL, - .unregister_dmb = NULL, + .register_dmb = smc_lo_register_dmb, + .unregister_dmb = smc_lo_unregister_dmb, .add_vlan_id = smc_lo_add_vlan_id, .del_vlan_id = smc_lo_del_vlan_id, .set_vlan_required = smc_lo_set_vlan_required, .reset_vlan_required = smc_lo_reset_vlan_required, .signal_event = smc_lo_signal_event, - .move_data = NULL, + .move_data = smc_lo_move_data, .supports_v2 = smc_lo_supports_v2, .get_system_eid = smc_lo_get_system_eid, .get_local_gid = smc_lo_get_local_gid, @@ -211,6 +331,8 @@ static void smc_lo_dev_release(struct device *dev) static int smc_lo_dev_init(struct smc_lo_dev *ldev) { smc_lo_generate_id(ldev); + rwlock_init(&ldev->dmb_ht_lock); + hash_init(ldev->dmb_ht); return smcd_lo_register_dev(ldev); } diff --git a/net/smc/smc_loopback.h b/net/smc/smc_loopback.h index 2156f22..943424f 100644 --- a/net/smc/smc_loopback.h +++ b/net/smc/smc_loopback.h @@ -20,12 +20,25 @@ #define SMC_LO_CHID 0xFFFF #define SMC_LODEV_MAX_DMBS 5000 +#define SMC_LODEV_DMBS_HASH_BITS 12 + +struct smc_lo_dmb_node { + struct hlist_node list; + u64 token; + u32 len; + u32 sba_idx; + void *cpu_addr; + dma_addr_t dma_addr; +}; struct smc_lo_dev { struct smcd_dev *smcd; struct device dev; u16 chid; struct smcd_gid local_gid; + DECLARE_BITMAP(sba_idx_mask, SMC_LODEV_MAX_DMBS); + rwlock_t dmb_ht_lock; + DECLARE_HASHTABLE(dmb_ht, SMC_LODEV_DMBS_HASH_BITS); }; int smc_loopback_init(void); From patchwork Tue Sep 19 14:41:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 141927 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp3445476vqi; Tue, 19 Sep 2023 07:49:22 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGNW7d+9VMmhfcGsCjxdUJ4Al5+Se5iLdT+Fvi6hHrg7Ud3fElhph2rzd+PssfxUXxmclP3 X-Received: by 2002:a17:90a:fe07:b0:274:8ef2:b251 with SMTP id ck7-20020a17090afe0700b002748ef2b251mr9274614pjb.3.1695134961626; Tue, 19 Sep 2023 07:49:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695134961; cv=none; d=google.com; s=arc-20160816; b=OqYz1sRADxSSGXxK8nmJW9pXA7I2oqf9aHZ4jW827sUud90bdAjQF1kaNIcRUyyKNN Q8rKAXH61mZOTac9+ifjXKVvfX7PNt/3x664YJ96AAGLRpSoZd9kl41SytIzp0up1lfk ZzzNKMmfErFNr8mZ67du4r7eOU/EpdlubWezkxhSe4il/SpXlgQS6v7JGQgGMCVIUceU MfKWSqnCnEb3pL3U+PZWgafUBQlSOnuz7kf7mUlC90110dcIL+55gbIhC3uaeMomWspI AdBV1R0GhRy2xecPlv03Qn3BasGFfV2iAgOFV+63HqvdVbhP+cq0DiDr7cIcEaePy69j UD1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=GF8zCI998f1CPyCWzYeVkXiAxwOQQZhc57GiZIMUel8=; fh=2Ks5OhTjwS8gZt4F+VGxBODwUEQ0u2wJBF3lQY5ZqAM=; b=Wr6FhXtRDFR2J/scNUMBfRFzsXsju8B/Bk65zU1T5pse9MDRTSEQG/2kl6hlsyvRby dlnNdaMgLcdP73dKccMyuzWVd156IJWVIcDtshAxHp/EJ8ZC9jaEw24A5wnc1cOmC0Ld horN0+ZwJbN8VJoNknwr69laipxr82fW+3GxSjijP+T2MNDACF+642Ic+0EI+A1Oo7sU lQfzNLCfI+WISng+XN9xKKCTJ22Y9zQHkUmUOYEP5Aqa5tCHT2C2Kx6topJPxmhop/Ji s2dCUR1TsUBD8wWoU+604whpdxOY+xUwQr92kW3OySs/SH4qS6SvF01JDNbJsZUf6J+p k40g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id gn2-20020a17090ac78200b0026b09e2e3eesi10693350pjb.47.2023.09.19.07.49.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 07:49:21 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 3A5448129AC3; Tue, 19 Sep 2023 07:43:46 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233097AbjISOnk (ORCPT + 26 others); Tue, 19 Sep 2023 10:43:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232958AbjISOnV (ORCPT ); Tue, 19 Sep 2023 10:43:21 -0400 Received: from out30-112.freemail.mail.aliyun.com (out30-112.freemail.mail.aliyun.com [115.124.30.112]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD7AC18F; Tue, 19 Sep 2023 07:43:06 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R151e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0VsRxWDi_1695134580; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsRxWDi_1695134580) by smtp.aliyun-inc.com; Tue, 19 Sep 2023 22:43:02 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 13/18] net/smc: register loopback device as SMC-Dv2 device Date: Tue, 19 Sep 2023 22:41:57 +0800 Message-Id: <1695134522-126655-14-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> References: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 19 Sep 2023 07:43:46 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777477837501940419 X-GMAIL-MSGID: 1777477837501940419 After loopback device gets ready, add it to the smcd_dev list as an SMC-Dv2 device for use by SMC-D protocol. Signed-off-by: Wen Gu --- net/smc/smc_loopback.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/net/smc/smc_loopback.c b/net/smc/smc_loopback.c index 7807b38..650561b 100644 --- a/net/smc/smc_loopback.c +++ b/net/smc/smc_loopback.c @@ -303,18 +303,27 @@ static int smcd_lo_register_dev(struct smc_lo_dev *ldev) ldev->smcd = smcd; smcd->priv = ldev; - - /* TODO: - * register smc_lo to smcd_dev list. - */ + mutex_lock(&smcd_dev_list.mutex); + smc_ism_check_v2_capable(smcd); + list_add(&smcd->list, &smcd_dev_list.list); + mutex_unlock(&smcd_dev_list.mutex); + pr_warn_ratelimited("smc: adding smcd device %s with pnetid %.16s%s\n", + smc_lo_dev_name, smcd->pnetid, + smcd->pnetid_by_user ? " (user defined)" : ""); return 0; } static void smcd_lo_unregister_dev(struct smc_lo_dev *ldev) { - /* TODO: - * unregister smc_lo from smcd_dev list. - */ + struct smcd_dev *smcd = ldev->smcd; + + pr_warn_ratelimited("smc: removing smcd device %s\n", + smc_lo_dev_name); + smcd->going_away = 1; + smc_smcd_terminate_all(smcd); + mutex_lock(&smcd_dev_list.mutex); + list_del_init(&smcd->list); + mutex_unlock(&smcd_dev_list.mutex); } static void smc_lo_dev_release(struct device *dev) From patchwork Tue Sep 19 14:41:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 141928 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp3445499vqi; Tue, 19 Sep 2023 07:49:23 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFvLn3LgjuclT6tWLQ+xeB75Jifw/SWebOr+J1sRcFaVv2K6Or1ipEzLSOczgII2ytLdXKC X-Received: by 2002:a05:6a21:33a1:b0:15c:9ed0:b1a7 with SMTP id yy33-20020a056a2133a100b0015c9ed0b1a7mr171917pzb.25.1695134963724; Tue, 19 Sep 2023 07:49:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695134963; cv=none; d=google.com; s=arc-20160816; b=EM+RoeigH4r62fI09K2Anotu66cxnD8jQ5D+aThctTITqOe8DIosNTtIy6fdKYTCXN 5Mq9n4+b/iRBTlW0ateVr52hN56Rez3GjY27HhozEV7sdQ/MlDUkrTH+b8Aj9nRTuh4X Mtzvb19XhpIdGQ7U3TtmS0XpQ3axnymK9DRDYIR/AMOg0742uiLOy5hA+qkuy8cAIw2M 69da8uuWdA1myP5KxfvUPEYkRG5rwgVZAW/kWMEH8qo9HCC1sxcAD9VPiiZiZOa3VSUZ ODXexaZiaPLj8oxyb4mHeHQyc+vYTh6h4/RQzooAqCn06+ajdYeiEaUSQ92A+pnUA4Tk dvIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=FFcdt3tSi5VEvwuNXlGgsQTDRbQOqV1EM+p8fHV+qd8=; fh=2Ks5OhTjwS8gZt4F+VGxBODwUEQ0u2wJBF3lQY5ZqAM=; b=z9oJci8AQfY5B2ZTOVjxbxh3DSBDjNLeZPdJ4C3EGw7RbodTYrxxxbbgDkvaN99z/h uPp3BWh4YJiWvos/t1RvKxqYAT83ntgCxKhq5urbZ3H/4Wxxn+AMC29eQF/ENY3mqb2I XLrVRcqJK9b89h0MvAgzXtKijT7xTs40Oy/ZkR8tHd0kR+gPKBLrAEkpDzN/Bg3cV793 1Og02ChlXaTFA7APHpB3feCfwI9bZKzMcynoIX0K7gacaNy65hfal2DckvlOHSZfZWGI 5vUWxtK2Yrtw4mCeNOsnRnf0DpNWVpubFlSavI5j9d/Odz0x98BCrMoAg3z34hCClOC6 mv+Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from agentk.vger.email (agentk.vger.email. [23.128.96.32]) by mx.google.com with ESMTPS id r15-20020a632b0f000000b00578b0afaee3si1540112pgr.626.2023.09.19.07.49.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 07:49:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id EFFF981BE2D5; Tue, 19 Sep 2023 07:44:13 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233112AbjISOno (ORCPT + 26 others); Tue, 19 Sep 2023 10:43:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49906 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233056AbjISOna (ORCPT ); Tue, 19 Sep 2023 10:43:30 -0400 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14170CE0; Tue, 19 Sep 2023 07:43:08 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R141e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045192;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0VsRxWEI_1695134583; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsRxWEI_1695134583) by smtp.aliyun-inc.com; Tue, 19 Sep 2023 22:43:04 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 14/18] net/smc: add operation for getting DMB attribute Date: Tue, 19 Sep 2023 22:41:58 +0800 Message-Id: <1695134522-126655-15-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> References: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Tue, 19 Sep 2023 07:44:14 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777477839663292620 X-GMAIL-MSGID: 1777477839663292620 On s390, ISM devices are used on machine level hypervisors which is a partitioning hypervisor without paging. The sndbufs and peer DMBs in such case can't be mapped to the same physical memory. However in other cases, such as communication within the same OS instance with loopback, the sndbufs and peer DMBs can be mapped to the same physical memory to avoid memory copy from sndbufs to peer DMBs. So this patch introduces an operation to smcd_ops to judge whether the sndbufs-DMB map is available. And for reuse, the interface is designed to return DMB attribute, not only the sndbuf-DMB map feature. Signed-off-by: Wen Gu --- include/net/smc.h | 5 +++++ net/smc/smc_ism.c | 8 ++++++++ net/smc/smc_ism.h | 1 + 3 files changed, 14 insertions(+) diff --git a/include/net/smc.h b/include/net/smc.h index 389d26d..6930c8a 100644 --- a/include/net/smc.h +++ b/include/net/smc.h @@ -55,6 +55,10 @@ struct smcd_seid { #define ISM_ERROR 0xFFFF +enum { + ISM_ATTR_DMB_MAP = 0, +}; + struct smcd_dev; struct smcd_gid { @@ -82,6 +86,7 @@ struct smcd_ops { void (*get_local_gid)(struct smcd_dev *dev, struct smcd_gid *gid); u16 (*get_chid)(struct smcd_dev *dev); struct device* (*get_dev)(struct smcd_dev *dev); + int (*get_dev_attr)(struct smcd_dev *dev); }; struct smcd_dev { diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c index ba37b27..14e018c 100644 --- a/net/smc/smc_ism.c +++ b/net/smc/smc_ism.c @@ -214,6 +214,14 @@ int smc_ism_unregister_dmb(struct smcd_dev *smcd, struct smc_buf_desc *dmb_desc) return rc; } +bool smc_ism_dmb_mappable(struct smcd_dev *smcd) +{ + if (smcd->ops->get_dev_attr && + (smcd->ops->get_dev_attr(smcd) & BIT(ISM_ATTR_DMB_MAP))) + return true; + return false; +} + int smc_ism_register_dmb(struct smc_link_group *lgr, int dmb_len, struct smc_buf_desc *dmb_desc) { diff --git a/net/smc/smc_ism.h b/net/smc/smc_ism.h index 7ab82dd..cef212c 100644 --- a/net/smc/smc_ism.h +++ b/net/smc/smc_ism.h @@ -44,6 +44,7 @@ int smc_ism_cantalk(struct smcd_gid *peer_gid, unsigned short vlan_id, int smc_ism_register_dmb(struct smc_link_group *lgr, int buf_size, struct smc_buf_desc *dmb_desc); int smc_ism_unregister_dmb(struct smcd_dev *dev, struct smc_buf_desc *dmb_desc); +bool smc_ism_dmb_mappable(struct smcd_dev *smcd); int smc_ism_signal_shutdown(struct smc_link_group *lgr); void smc_ism_get_system_eid(u8 **eid); u16 smc_ism_get_chid(struct smcd_dev *dev); From patchwork Tue Sep 19 14:41:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 141929 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp3445553vqi; Tue, 19 Sep 2023 07:49:28 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEh/a0613ftnF+Cw5B0EFnR7AYc65w3S9afHUzkyA1aXZuJD8R44iFBQbfmWp1tcxSxoEfg X-Received: by 2002:a17:903:1251:b0:1c3:8230:30d8 with SMTP id u17-20020a170903125100b001c3823030d8mr13674881plh.38.1695134968196; Tue, 19 Sep 2023 07:49:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695134968; cv=none; d=google.com; s=arc-20160816; b=WLyYJtECPTpFQ15Uw5H9QRpgqkxvqdkhxpBfEau73NmPerVLhrerIZvpb0Kh9WVAOL x6gl7FUUiMlPCskU6F4oOD32lkE2rjJu6KV1Tn8/WLbFT+bmPfvZnDXRy6s0szjhsmuk ngwV6uIWSfAnOok5rvqtq7fuhditjeWBv8WO/8TdT8EN6MvvQdxbHnYbL2pZ22wVBSof THp5IebmZappm9TsIAGv5I/Ri7v2TA6Mgv2hFglyqVvALNtR0CkWZtHsRBVMHHNSoz7n sIAETHcQlQmU2wDoepsj1iQLsgOJ07mCxoh1uTwGlix8uGbYXpZ28aoY+6bH7QuQAlai e7DQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=daSWE+IW/WzvGnCAh7q+krnO9dbiWzvL4hxNiV8ZcfU=; fh=2Ks5OhTjwS8gZt4F+VGxBODwUEQ0u2wJBF3lQY5ZqAM=; b=S35OH+3RR584m4Ys7G7YheqX57r0YhZiopj9F+04a391FQUM391Qh6O9D7BOCUiWbm qIlbWBp80mfqFjbsybxMUlbwUciDu33CTTSA6uVVT4Vp/o4nN+SmjvIAMwLRzOKa/09+ 1GnYTrKBch2ImRHD4p2Gl93+A7mF8+2FDrw7LGecdKshqW5YHqCPj61IHjSFoInj4snh J79Ugs28rVbCL4rgkcYca9bznRY/Hdue4yHeML45CMl/Y6zK8/BwZst6GzI+iY4JSTop hyK4CRlwaUXiZJy3C3QIHY+ywgj/jzM/DYfnbF06PUszyem/EZVvpKMeMYGZ2LZ+0jgq vKSA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id y4-20020a17090322c400b001b86ddfd49bsi10078189plg.6.2023.09.19.07.49.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 07:49:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id B45D9812DBF6; Tue, 19 Sep 2023 07:43:57 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233003AbjISOnw (ORCPT + 26 others); Tue, 19 Sep 2023 10:43:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47860 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233076AbjISOnd (ORCPT ); Tue, 19 Sep 2023 10:43:33 -0400 Received: from out30-111.freemail.mail.aliyun.com (out30-111.freemail.mail.aliyun.com [115.124.30.111]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9A409E47; Tue, 19 Sep 2023 07:43:12 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R111e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045176;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0VsRsS-6_1695134585; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsRsS-6_1695134585) by smtp.aliyun-inc.com; Tue, 19 Sep 2023 22:43:06 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 15/18] net/smc: add operations for DMB attach and detach Date: Tue, 19 Sep 2023 22:41:59 +0800 Message-Id: <1695134522-126655-16-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> References: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 19 Sep 2023 07:43:57 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777477844461552802 X-GMAIL-MSGID: 1777477844461552802 This patch extends smcd_ops, adding two more semantic for SMC-D DMB: - attach_dmb: Attach an already registered DMB to a specific buf_desc, so that we can refer to the DMB through this buf_desc. - detach_dmb: Reverse operation of attach_dmb. detach the DMB from the buf_desc. This interface extension is to prepare for the avoidance of memory copy from sndbuf to RMB with SMC-D device whose DMBs has ISM_ATTR_DMB_MAP attribute. Signed-off-by: Wen Gu --- include/net/smc.h | 2 ++ net/smc/smc_ism.c | 31 +++++++++++++++++++++++++++++++ net/smc/smc_ism.h | 2 ++ 3 files changed, 35 insertions(+) diff --git a/include/net/smc.h b/include/net/smc.h index 6930c8a..04dbaa4 100644 --- a/include/net/smc.h +++ b/include/net/smc.h @@ -72,6 +72,8 @@ struct smcd_ops { int (*register_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb, void *client); int (*unregister_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb); + int (*attach_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb); + int (*detach_dmb)(struct smcd_dev *dev, u64 token); int (*add_vlan_id)(struct smcd_dev *dev, u64 vlan_id); int (*del_vlan_id)(struct smcd_dev *dev, u64 vlan_id); int (*set_vlan_required)(struct smcd_dev *dev); diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c index 14e018c..0c20745 100644 --- a/net/smc/smc_ism.c +++ b/net/smc/smc_ism.c @@ -244,6 +244,37 @@ int smc_ism_register_dmb(struct smc_link_group *lgr, int dmb_len, return rc; } +int smc_ism_attach_dmb(struct smcd_dev *dev, u64 token, + struct smc_buf_desc *dmb_desc) +{ + struct smcd_dmb dmb; + int rc = 0; + + memset(&dmb, 0, sizeof(dmb)); + dmb.dmb_tok = token; + + if (!dev->ops->attach_dmb) + return -EINVAL; + + rc = dev->ops->attach_dmb(dev, &dmb); + if (!rc) { + dmb_desc->sba_idx = dmb.sba_idx; + dmb_desc->token = dmb.dmb_tok; + dmb_desc->cpu_addr = dmb.cpu_addr; + dmb_desc->dma_addr = dmb.dma_addr; + dmb_desc->len = dmb.dmb_len; + } + return rc; +} + +int smc_ism_detach_dmb(struct smcd_dev *dev, u64 token) +{ + if (!dev->ops->detach_dmb) + return -EINVAL; + + return dev->ops->detach_dmb(dev, token); +} + static int smc_nl_handle_smcd_dev(struct smcd_dev *smcd, struct sk_buff *skb, struct netlink_callback *cb) diff --git a/net/smc/smc_ism.h b/net/smc/smc_ism.h index cef212c..f30aae4 100644 --- a/net/smc/smc_ism.h +++ b/net/smc/smc_ism.h @@ -45,6 +45,8 @@ int smc_ism_register_dmb(struct smc_link_group *lgr, int buf_size, struct smc_buf_desc *dmb_desc); int smc_ism_unregister_dmb(struct smcd_dev *dev, struct smc_buf_desc *dmb_desc); bool smc_ism_dmb_mappable(struct smcd_dev *smcd); +int smc_ism_attach_dmb(struct smcd_dev *dev, u64 token, struct smc_buf_desc *dmb_desc); +int smc_ism_detach_dmb(struct smcd_dev *dev, u64 token); int smc_ism_signal_shutdown(struct smc_link_group *lgr); void smc_ism_get_system_eid(u8 **eid); u16 smc_ism_get_chid(struct smcd_dev *dev); From patchwork Tue Sep 19 14:42:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 141935 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp3447661vqi; Tue, 19 Sep 2023 07:52:59 -0700 (PDT) X-Google-Smtp-Source: AGHT+IG+C23/zWT2RTFrujbkZ8SmZT5dcUVzOOcu+cBHHceXLfjfz8kO4iDnB/QYDFhkIGVJD285 X-Received: by 2002:a17:90a:9f93:b0:274:7db1:f50f with SMTP id o19-20020a17090a9f9300b002747db1f50fmr3575180pjp.15.1695135179340; Tue, 19 Sep 2023 07:52:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695135179; cv=none; d=google.com; s=arc-20160816; b=i76ZW+620nxvcmInK+6ElGrDtLMn8u3YYGoGVHLL4gneLrqZu635z0rHUkumx3TPBZ jkJdWRW+GQETmr9ghBaYCAMIs3g6ub2Qd+cwTMgaciz/52EU3AhjogRmqRSFFZlS7cBo U705U1TLCi9MVCgqPOe/xA95jQwp0M6PUCW7waqM0KVmViOFibrsOHqoeMW7+2sgoLVE enxaM2M/+gNEobdjh9lIsCrpfa8pKpF/nesGaqJgJ7xzjNbKqoqv5N4ZMz/jEJ4s2CtV I6fZZP9+C66GYtle6aSFsHCHdYdvA2BqpmTSK/hMDGXezsPuFWX9SzrmIhj557v1t57N ZKFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=FSf6NCZjvjXKvrH40pKOuJ/UrT6ORR+B+kehPXCpuRM=; fh=2Ks5OhTjwS8gZt4F+VGxBODwUEQ0u2wJBF3lQY5ZqAM=; b=wvotLxW6AFZC5XtXOV4UZVXyBA+phb1M3zUgp71pE1/ShT5gW42uze0TrdGyj8E5Vt SEywW1DbL0J154hc3nALdjKIWSnSuGt4LjjEH4oau60/8/19gMZ/PMN6y12MBwt+0Qlk 27yTSqS1RkYa49zXt9fg2KyWD5aik2CX1tn98GfFLNJjTiCHusJuYTNcdOkmqDWge0Wa Mqy30C9K4ecwOAFcVZk3tnTRSVml2edoWZOzHJzd+xoyKPxaSYaoFmV5FS0DbXlzG6KI IjPCvipVgOz4pOlsho34/Vj9ahbbX/an32nFVL7QJi3zZaByemP624GJpm0Y25ImVZiy V2qQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from howler.vger.email (howler.vger.email. [2620:137:e000::3:4]) by mx.google.com with ESMTPS id s4-20020a17090a948400b00269584b6a10si3243661pjo.15.2023.09.19.07.52.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 07:52:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 5F43D80ECFBB; Tue, 19 Sep 2023 07:44:10 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233014AbjISOn5 (ORCPT + 26 others); Tue, 19 Sep 2023 10:43:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39872 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232912AbjISOnd (ORCPT ); Tue, 19 Sep 2023 10:43:33 -0400 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4596E49; Tue, 19 Sep 2023 07:43:12 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R181e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046060;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0VsRxWFd_1695134587; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsRxWFd_1695134587) by smtp.aliyun-inc.com; Tue, 19 Sep 2023 22:43:08 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 16/18] net/smc: avoid data copy from sndbuf to peer RMB in SMC-D Date: Tue, 19 Sep 2023 22:42:00 +0800 Message-Id: <1695134522-126655-17-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> References: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Tue, 19 Sep 2023 07:44:10 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777478066178981089 X-GMAIL-MSGID: 1777478066178981089 This patch aims to avoid data copy from local sndbuf to peer RMB by mapping local sndbuf to peer RMB when DMBs have ISM_ATTR_DMB_MAP attribute. After this, local sndbuf and peer RMB share the same physical memory. +----------+ +----------+ | socket A | | socket B | +----------+ +----------+ | ^ | +---------+ | regard as | | regard as local sndbuf | B's | local RMB | | RMB | | |-------> | |-----------| +---------+ 1. From the perspective of RMB: a. Created or reused when connection is created. b. Unused and recycled to lgr buffer pool when connection is freed. c. Freed when link group is freed. 2. From the perspective of sndbuf: a. Mapped to peer RMB by the rtoken exchanged through CLC message. Then accessing local sndbuf is equivalent to accessing peer RMB. c. Unmapped from peer RMB and freed when connection is freed. Won't be recycled to lgr buffer pool. Therefore, the data written to local sndbuf will directly reach peer RMB. Signed-off-by: Wen Gu --- net/smc/af_smc.c | 14 +++++++++++ net/smc/smc_core.c | 70 +++++++++++++++++++++++++++++++++++++++++++++++++++++- net/smc/smc_core.h | 1 + 3 files changed, 84 insertions(+), 1 deletion(-) diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index bc4300e..fd0b91f 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -1436,6 +1436,12 @@ static int smc_connect_ism(struct smc_sock *smc, } smc_conn_save_peer_info(smc, aclc); + + if (smc_ism_dmb_mappable(smc->conn.lgr->smcd)) { + rc = smcd_buf_attach(smc); + if (rc) + goto connect_abort; + } smc_close_init(smc); smc_rx_init(smc); smc_tx_init(smc); @@ -2537,6 +2543,14 @@ static void smc_listen_work(struct work_struct *work) mutex_unlock(&smc_server_lgr_pending); } smc_conn_save_peer_info(new_smc, cclc); + + if (ini->is_smcd && + smc_ism_dmb_mappable(new_smc->conn.lgr->smcd)) { + rc = smcd_buf_attach(new_smc); + if (rc) + goto out_decl; + } + smc_listen_out_connected(new_smc); SMC_STAT_SERV_SUCC_INC(sock_net(newclcsock->sk), ini); goto out_free; diff --git a/net/smc/smc_core.c b/net/smc/smc_core.c index c36500a..bae2116 100644 --- a/net/smc/smc_core.c +++ b/net/smc/smc_core.c @@ -1153,6 +1153,20 @@ static void smcr_buf_unuse(struct smc_buf_desc *buf_desc, bool is_rmb, } } +static void smcd_buf_detach(struct smc_connection *conn) +{ + struct smcd_dev *smcd = conn->lgr->smcd; + u64 peer_token = conn->peer_token; + + if (!conn->sndbuf_desc) + return; + + smc_ism_detach_dmb(smcd, peer_token); + + kfree(conn->sndbuf_desc); + conn->sndbuf_desc = NULL; +} + static void smc_buf_unuse(struct smc_connection *conn, struct smc_link_group *lgr) { @@ -1197,6 +1211,10 @@ void smc_conn_free(struct smc_connection *conn) if (!list_empty(&lgr->list)) smc_ism_unset_conn(conn); tasklet_kill(&conn->rx_tsklet); + + /* detach sndbuf from peer RMB */ + if (smc_ism_dmb_mappable(lgr->smcd)) + smcd_buf_detach(conn); } else { smc_cdc_wait_pend_tx_wr(conn); if (current_work() != &conn->abort_work) @@ -2458,15 +2476,23 @@ void smc_rmb_sync_sg_for_cpu(struct smc_connection *conn) */ int smc_buf_create(struct smc_sock *smc, bool is_smcd) { + bool sndbuf_created = false; int rc; + if (is_smcd && + smc_ism_dmb_mappable(smc->conn.lgr->smcd)) + goto create_rmb; + /* create send buffer */ rc = __smc_buf_create(smc, is_smcd, false); if (rc) return rc; + sndbuf_created = true; + +create_rmb: /* create rmb */ rc = __smc_buf_create(smc, is_smcd, true); - if (rc) { + if (rc && sndbuf_created) { down_write(&smc->conn.lgr->sndbufs_lock); list_del(&smc->conn.sndbuf_desc->list); up_write(&smc->conn.lgr->sndbufs_lock); @@ -2476,6 +2502,48 @@ int smc_buf_create(struct smc_sock *smc, bool is_smcd) return rc; } +int smcd_buf_attach(struct smc_sock *smc) +{ + struct smc_connection *conn = &smc->conn; + struct smcd_dev *smcd = conn->lgr->smcd; + u64 peer_token = conn->peer_token; + struct smc_buf_desc *buf_desc; + int rc; + + buf_desc = kzalloc(sizeof(*buf_desc), GFP_KERNEL); + if (!buf_desc) + return -ENOMEM; + + /* map local sndbuf desc to peer RMB, so operations on local + * sndbuf are equivalent to operations on peer RMB. + */ + rc = smc_ism_attach_dmb(smcd, peer_token, buf_desc); + if (rc) { + rc = SMC_CLC_DECL_MEM; + goto free; + } + + smc->sk.sk_sndbuf = buf_desc->len; + buf_desc->cpu_addr = (u8 *)buf_desc->cpu_addr + sizeof(struct smcd_cdc_msg); + buf_desc->len -= sizeof(struct smcd_cdc_msg); + conn->sndbuf_desc = buf_desc; + conn->sndbuf_desc->used = 1; + atomic_set(&conn->sndbuf_space, conn->sndbuf_desc->len); + return 0; + +free: + if (conn->rmb_desc) { + /* free local RMB as well */ + down_write(&conn->lgr->rmbs_lock); + list_del(&conn->rmb_desc->list); + up_write(&conn->lgr->rmbs_lock); + smc_buf_free(conn->lgr, true, conn->rmb_desc); + conn->rmb_desc = NULL; + } + kfree(buf_desc); + return rc; +} + static inline int smc_rmb_reserve_rtoken_idx(struct smc_link_group *lgr) { int i; diff --git a/net/smc/smc_core.h b/net/smc/smc_core.h index d57eb9b..2cba119 100644 --- a/net/smc/smc_core.h +++ b/net/smc/smc_core.h @@ -551,6 +551,7 @@ void smc_smcd_terminate(struct smcd_dev *dev, struct smcd_gid *peer_gid, void smc_smcd_terminate_all(struct smcd_dev *dev); void smc_smcr_terminate_all(struct smc_ib_device *smcibdev); int smc_buf_create(struct smc_sock *smc, bool is_smcd); +int smcd_buf_attach(struct smc_sock *smc); int smc_uncompress_bufsize(u8 compressed); int smc_rmb_rtoken_handling(struct smc_connection *conn, struct smc_link *link, struct smc_clc_msg_accept_confirm *clc); From patchwork Tue Sep 19 14:42:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 141930 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp3445593vqi; Tue, 19 Sep 2023 07:49:33 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF28D4yJWPaa7p7FudDNylG85wyK5eMg4vmc6/NHWJuyI+6X8zrVeAaJ8HFS9ync//ahF4d X-Received: by 2002:a05:6a20:734c:b0:159:cf93:9b50 with SMTP id v12-20020a056a20734c00b00159cf939b50mr12558475pzc.46.1695134972602; Tue, 19 Sep 2023 07:49:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695134972; cv=none; d=google.com; s=arc-20160816; b=yBcswpgHfTQQaGNm5r0XFvQrEQAU+7fnf5HNeNtE2wAQIznzHiNx6kk4nf7DjfOQAz Imq0MzezHN+fW0XlaAXBhJeCPz+I37eJcaYeuimuktO1H3Mc4vfHi1bQH+hTX69QQtkB dIx/dgdnLQMGl4rmwkFvHzJ9qKbZfPPF8BKiiuQM36krb60FHOcsb3sTKo2d6lMh9pVe P6dusHAub8d+T4VhCTVuzlg5EVUjt/HaiEmlq2advbHJNKvlvV/j4uHqe3dZiAuDDEMk PADpDZvIAXrqlpNtt8B1nSEgb0NIyLU7ujmtAvnMpMVRJ+Q9VIkXRBo8VIDHW1YAH6Tt dJXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=y07OQ04D2+O4jhr1Pt9VUQzgrtDGr1vB/DMBgXymiNU=; fh=2Ks5OhTjwS8gZt4F+VGxBODwUEQ0u2wJBF3lQY5ZqAM=; b=weYCXgG5x87zfXTU/jL1Ars39t3gEq5tUP6ZLW4m0K/0JaUSC8bweHp4jvkRmym3k/ mdP6cI9mRo66NE52yEhpLAZu4JhT3HsQBnxGpFQ4TMvAJTgLRtO4mvgBIqxX6miSVQ/V qorBckaYGvFEzVkK89nEqkik/G+qmfLw2W58VqA+u52B+leIzHEF/98deM6fuN++aBF0 jNKWHWtIDD9Wf3g0Ah2V9/QlfEUDyHfiU6mpYqPPGDhQjh52CuhCt4fSFdxgxCWkqhYz kL++nFNCGv2OfS0QMrh1CVWtXqH9WEzYVbUtA7IDbs6v7FXVDKSaAqkaEim5f15qjGvs svAA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id b25-20020a630c19000000b005656cec81f5si9723753pgl.268.2023.09.19.07.49.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 07:49:32 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 927268135294; Tue, 19 Sep 2023 07:44:09 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232999AbjISOoG (ORCPT + 26 others); Tue, 19 Sep 2023 10:44:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50036 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232956AbjISOni (ORCPT ); Tue, 19 Sep 2023 10:43:38 -0400 Received: from out30-112.freemail.mail.aliyun.com (out30-112.freemail.mail.aliyun.com [115.124.30.112]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EB12E119; Tue, 19 Sep 2023 07:43:16 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R131e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0VsRsS.Z_1695134589; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsRsS.Z_1695134589) by smtp.aliyun-inc.com; Tue, 19 Sep 2023 22:43:10 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 17/18] net/smc: modify cursor update logic when sndbuf mapped to RMB Date: Tue, 19 Sep 2023 22:42:01 +0800 Message-Id: <1695134522-126655-18-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> References: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Tue, 19 Sep 2023 07:44:09 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777477848988768051 X-GMAIL-MSGID: 1777477848988768051 Since local sndbuf shares the same physical memory with peer RMB after mapping, the logic related to cursor update needs to be adapted to ensure that the data written by local won't overwrite the data that has not been consumed by the peer. So in this case, the fin_curs and sndbuf_space that were originally updated after sending the CDC message should be modified to not be update until the peer updates cons_curs. Signed-off-by: Wen Gu --- net/smc/smc_cdc.c | 50 +++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 39 insertions(+), 11 deletions(-) diff --git a/net/smc/smc_cdc.c b/net/smc/smc_cdc.c index 2641800..7cc677d 100644 --- a/net/smc/smc_cdc.c +++ b/net/smc/smc_cdc.c @@ -18,6 +18,7 @@ #include "smc_tx.h" #include "smc_rx.h" #include "smc_close.h" +#include "smc_ism.h" /********************************** send *************************************/ @@ -256,17 +257,24 @@ int smcd_cdc_msg_send(struct smc_connection *conn) return rc; smc_curs_copy(&conn->rx_curs_confirmed, &curs, conn); conn->local_rx_ctrl.prod_flags.cons_curs_upd_req = 0; - /* Calculate transmitted data and increment free send buffer space */ - diff = smc_curs_diff(conn->sndbuf_desc->len, &conn->tx_curs_fin, - &conn->tx_curs_sent); - /* increased by confirmed number of bytes */ - smp_mb__before_atomic(); - atomic_add(diff, &conn->sndbuf_space); - /* guarantee 0 <= sndbuf_space <= sndbuf_desc->len */ - smp_mb__after_atomic(); - smc_curs_copy(&conn->tx_curs_fin, &conn->tx_curs_sent, conn); + if (!smc_ism_dmb_mappable(conn->lgr->smcd)) { + /* If local sndbuf has been mapped to peer RMB, then + * don't update the tx_curs_fin and sndbuf_space until + * peer has consumed the data in RMB. + */ - smc_tx_sndbuf_nonfull(smc); + /* Calculate transmitted data and increment free send buffer space */ + diff = smc_curs_diff(conn->sndbuf_desc->len, &conn->tx_curs_fin, + &conn->tx_curs_sent); + /* increased by confirmed number of bytes */ + smp_mb__before_atomic(); + atomic_add(diff, &conn->sndbuf_space); + /* guarantee 0 <= sndbuf_space <= sndbuf_desc->len */ + smp_mb__after_atomic(); + smc_curs_copy(&conn->tx_curs_fin, &conn->tx_curs_sent, conn); + + smc_tx_sndbuf_nonfull(smc); + } return rc; } @@ -324,7 +332,7 @@ static void smc_cdc_msg_recv_action(struct smc_sock *smc, { union smc_host_cursor cons_old, prod_old; struct smc_connection *conn = &smc->conn; - int diff_cons, diff_prod; + int diff_cons, diff_prod, diff_tx; smc_curs_copy(&prod_old, &conn->local_rx_ctrl.prod, conn); smc_curs_copy(&cons_old, &conn->local_rx_ctrl.cons, conn); @@ -340,6 +348,26 @@ static void smc_cdc_msg_recv_action(struct smc_sock *smc, atomic_add(diff_cons, &conn->peer_rmbe_space); /* guarantee 0 <= peer_rmbe_space <= peer_rmbe_size */ smp_mb__after_atomic(); + + if (conn->lgr->is_smcd && + smc_ism_dmb_mappable(conn->lgr->smcd)) { + /* If local sndbuf has been mapped to peer RMB, then + * update tx_curs_fin and sndbuf_space when peer has + * consumed the data in it's RMB. + */ + + /* calculate peer rmb consumed data */ + diff_tx = smc_curs_diff(conn->sndbuf_desc->len, &conn->tx_curs_fin, + &conn->local_rx_ctrl.cons); + /* increase local sndbuf space and fin_curs */ + smp_mb__before_atomic(); + atomic_add(diff_tx, &conn->sndbuf_space); + /* guarantee 0 <= sndbuf_space <= sndbuf_desc->len */ + smp_mb__after_atomic(); + smc_curs_copy(&conn->tx_curs_fin, &conn->local_rx_ctrl.cons, conn); + + smc_tx_sndbuf_nonfull(smc); + } } diff_prod = smc_curs_diff(conn->rmb_desc->len, &prod_old, From patchwork Tue Sep 19 14:42:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 141947 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:612c:172:b0:3f2:4152:657d with SMTP id h50csp3459878vqi; Tue, 19 Sep 2023 08:08:30 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFjzgRRtFPJosz0k/36qd3sGApO8hk9yTpdg52FIYdhBYt5qWJeMqGB2VEjwiM4fJxwnKSF X-Received: by 2002:a05:6a00:1d29:b0:68e:16ce:f2e3 with SMTP id a41-20020a056a001d2900b0068e16cef2e3mr3661931pfx.0.1695136110621; Tue, 19 Sep 2023 08:08:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695136110; cv=none; d=google.com; s=arc-20160816; b=ENkyaSn84pYeXWCW0gXbYn58XavadmeesWEJSW7GaqPMrLD1DT2zbbsZFDHZwUYSR0 2WtqQA/8eiWot2xtbxcyZz8tH5Z1qj54tEYLJ1R1bi2n6oj1+2xc1IpfjwOdkeZ2itVd HhoLqn4BYK3ofY9AxwpJySHXZyJ5pmTPvf8cIqDZuc5KMc0TrAdD0+huFpL9qh42MfcK L/Dd2sVHnodJh12JGzldVolv9KMuIt5lSzxgMKU54XeDQoJzf/1raknw70g8qQocHkKR qt6P4f8UkWYudlIUqE1cXuL6uBpUdWuYDx4QBTEwpQLgHBlo8IbK67Kq+zawVPBffcLL vwXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=zyYT9SGzB13l2kzbyVZBbEGsJLBbjawKmFWwWIXO/+8=; fh=2Ks5OhTjwS8gZt4F+VGxBODwUEQ0u2wJBF3lQY5ZqAM=; b=TYjQSLkY4T2Xvy7aGT5wXJGDWZHpf3cltS0oqOQdlmWHKww8sIKanL5ftw2su/EU1k RT2/1KzOMuFbFQ58qjThAI5Re2kMRfERqI2NkfWwRyRPu6bR9PwORGqy+fEh5FOO0E7s ZXq4UdZAxZ3t0v7JdfNQGNgBBpNSbmeIni/XrahX+fcfA/3RV12qkFpTaq65Rdy7l7Sk Jc8QPaQwmPgcfbSi9WTPCGVRpkvmYNThDt4Gna2IObmmDP3Bs6J9vX7+4CgZsXvWIkvf WnEgDj1X4abXi9WvP40rATLQyHb3R02uuN9dCbaICcDkGMT1G1/o7gxmmaae5XU/ggsN iCJg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id o10-20020a056a001bca00b00666c9148d03si9790306pfw.6.2023.09.19.08.08.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 08:08:30 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 54E97812E37F; Tue, 19 Sep 2023 07:44:57 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233101AbjISOoI (ORCPT + 26 others); Tue, 19 Sep 2023 10:44:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232964AbjISOnk (ORCPT ); Tue, 19 Sep 2023 10:43:40 -0400 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F37BE64; Tue, 19 Sep 2023 07:43:19 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R151e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0VsRqcDd_1695134591; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsRqcDd_1695134591) by smtp.aliyun-inc.com; Tue, 19 Sep 2023 22:43:13 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 18/18] net/smc: add interface implementation of loopback device Date: Tue, 19 Sep 2023 22:42:02 +0800 Message-Id: <1695134522-126655-19-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> References: <1695134522-126655-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Tue, 19 Sep 2023 07:44:57 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777479042604544981 X-GMAIL-MSGID: 1777479042604544981 This patch completes the specific implementation of loopback device for the newly added SMC-D DMB-related interface. The loopback device always provides mappable DMB because the device users are in the same OS instance. Signed-off-by: Wen Gu --- net/smc/smc_loopback.c | 105 ++++++++++++++++++++++++++++++++++++++++++++----- net/smc/smc_loopback.h | 5 +++ 2 files changed, 100 insertions(+), 10 deletions(-) diff --git a/net/smc/smc_loopback.c b/net/smc/smc_loopback.c index 650561b..611998b 100644 --- a/net/smc/smc_loopback.c +++ b/net/smc/smc_loopback.c @@ -105,6 +105,7 @@ static int smc_lo_register_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb, } dmb_node->len = dmb->dmb_len; dmb_node->dma_addr = (dma_addr_t)dmb_node->cpu_addr; + refcount_set(&dmb_node->refcnt, 1); again: /* add new dmb into hash table */ @@ -118,6 +119,7 @@ static int smc_lo_register_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb, } hash_add(ldev->dmb_ht, &dmb_node->list, dmb_node->token); write_unlock(&ldev->dmb_ht_lock); + atomic_inc(&ldev->dmb_cnt); dmb->sba_idx = dmb_node->sba_idx; dmb->dmb_tok = dmb_node->token; @@ -139,28 +141,98 @@ static int smc_lo_unregister_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb) struct smc_lo_dmb_node *dmb_node = NULL, *tmp_node; struct smc_lo_dev *ldev = smcd->priv; - /* remove dmb from hash table */ - write_lock(&ldev->dmb_ht_lock); + /* find dmb from hash table */ + read_lock(&ldev->dmb_ht_lock); hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb->dmb_tok) { if (tmp_node->token == dmb->dmb_tok) { dmb_node = tmp_node; + dmb_node->freeing = 1; break; } } if (!dmb_node) { - write_unlock(&ldev->dmb_ht_lock); + read_unlock(&ldev->dmb_ht_lock); return -EINVAL; } + read_unlock(&ldev->dmb_ht_lock); + + /* wait for dmb refcnt to be 0 */ + if (!refcount_dec_and_test(&dmb_node->refcnt)) + wait_event(ldev->dmbs_release, !refcount_read(&dmb_node->refcnt)); + + /* remove dmb from hash table */ + write_lock(&ldev->dmb_ht_lock); hash_del(&dmb_node->list); write_unlock(&ldev->dmb_ht_lock); clear_bit(dmb_node->sba_idx, ldev->sba_idx_mask); + kfree(dmb_node->cpu_addr); kfree(dmb_node); + if (atomic_dec_and_test(&ldev->dmb_cnt)) + wake_up(&ldev->ldev_release); + return 0; +} + +static int smc_lo_attach_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb) +{ + struct smc_lo_dmb_node *dmb_node = NULL, *tmp_node; + struct smc_lo_dev *ldev = smcd->priv; + + /* find dmb_node according to dmb->dmb_tok */ + read_lock(&ldev->dmb_ht_lock); + hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb->dmb_tok) { + if (tmp_node->token == dmb->dmb_tok && !tmp_node->freeing) { + dmb_node = tmp_node; + break; + } + } + if (!dmb_node) { + read_unlock(&ldev->dmb_ht_lock); + return -EINVAL; + } + refcount_inc(&dmb_node->refcnt); + read_unlock(&ldev->dmb_ht_lock); + + /* provide dmb information */ + dmb->sba_idx = dmb_node->sba_idx; + dmb->dmb_tok = dmb_node->token; + dmb->cpu_addr = dmb_node->cpu_addr; + dmb->dma_addr = dmb_node->dma_addr; + dmb->dmb_len = dmb_node->len; return 0; } +static int smc_lo_detach_dmb(struct smcd_dev *smcd, u64 token) +{ + struct smc_lo_dmb_node *dmb_node = NULL, *tmp_node; + struct smc_lo_dev *ldev = smcd->priv; + + /* find dmb_node according to dmb->dmb_tok */ + read_lock(&ldev->dmb_ht_lock); + hash_for_each_possible(ldev->dmb_ht, tmp_node, list, token) { + if (tmp_node->token == token) { + dmb_node = tmp_node; + break; + } + } + if (!dmb_node) { + read_unlock(&ldev->dmb_ht_lock); + return -EINVAL; + } + read_unlock(&ldev->dmb_ht_lock); + + if (refcount_dec_and_test(&dmb_node->refcnt)) + wake_up_all(&ldev->dmbs_release); + return 0; +} + +static int smc_lo_get_dev_attr(struct smcd_dev *smcd) +{ + return BIT(ISM_ATTR_DMB_MAP); +} + static int smc_lo_add_vlan_id(struct smcd_dev *smcd, u64 vlan_id) { return -EOPNOTSUPP; @@ -193,7 +265,15 @@ static int smc_lo_move_data(struct smcd_dev *smcd, u64 dmb_tok, unsigned int idx { struct smc_lo_dmb_node *rmb_node = NULL, *tmp_node; struct smc_lo_dev *ldev = smcd->priv; - + struct smc_connection *conn; + + if (!sf) { + /* local sndbuf shares the same physical memory with + * peer RMB, so no need to copy data from local sndbuf + * to peer RMB. + */ + return 0; + } read_lock(&ldev->dmb_ht_lock); hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb_tok) { if (tmp_node->token == dmb_tok) { @@ -209,13 +289,10 @@ static int smc_lo_move_data(struct smcd_dev *smcd, u64 dmb_tok, unsigned int idx memcpy((char *)rmb_node->cpu_addr + offset, data, size); - if (sf) { - struct smc_connection *conn = - smcd->conn[rmb_node->sba_idx]; + conn = smcd->conn[rmb_node->sba_idx]; + if (conn && !conn->killed) + smcd_cdc_rx_handler(conn); - if (conn && !conn->killed) - smcd_cdc_rx_handler(conn); - } return 0; } @@ -252,6 +329,8 @@ static struct device *smc_lo_get_dev(struct smcd_dev *smcd) .query_remote_gid = smc_lo_query_rgid, .register_dmb = smc_lo_register_dmb, .unregister_dmb = smc_lo_unregister_dmb, + .attach_dmb = smc_lo_attach_dmb, + .detach_dmb = smc_lo_detach_dmb, .add_vlan_id = smc_lo_add_vlan_id, .del_vlan_id = smc_lo_del_vlan_id, .set_vlan_required = smc_lo_set_vlan_required, @@ -263,6 +342,7 @@ static struct device *smc_lo_get_dev(struct smcd_dev *smcd) .get_local_gid = smc_lo_get_local_gid, .get_chid = smc_lo_get_chid, .get_dev = smc_lo_get_dev, + .get_dev_attr = smc_lo_get_dev_attr, }; static struct smcd_dev *smcd_lo_alloc_dev(const struct smcd_ops *ops, @@ -342,6 +422,9 @@ static int smc_lo_dev_init(struct smc_lo_dev *ldev) smc_lo_generate_id(ldev); rwlock_init(&ldev->dmb_ht_lock); hash_init(ldev->dmb_ht); + atomic_set(&ldev->dmb_cnt, 0); + init_waitqueue_head(&ldev->dmbs_release); + init_waitqueue_head(&ldev->ldev_release); return smcd_lo_register_dev(ldev); } @@ -375,6 +458,8 @@ static int smc_lo_dev_probe(void) static void smc_lo_dev_exit(struct smc_lo_dev *ldev) { smcd_lo_unregister_dev(ldev); + if (atomic_read(&ldev->dmb_cnt)) + wait_event(ldev->ldev_release, !atomic_read(&ldev->dmb_cnt)); } static void smc_lo_dev_remove(void) diff --git a/net/smc/smc_loopback.h b/net/smc/smc_loopback.h index 943424f..506e524 100644 --- a/net/smc/smc_loopback.h +++ b/net/smc/smc_loopback.h @@ -29,6 +29,8 @@ struct smc_lo_dmb_node { u32 sba_idx; void *cpu_addr; dma_addr_t dma_addr; + refcount_t refcnt; + u8 freeing : 1; }; struct smc_lo_dev { @@ -39,6 +41,9 @@ struct smc_lo_dev { DECLARE_BITMAP(sba_idx_mask, SMC_LODEV_MAX_DMBS); rwlock_t dmb_ht_lock; DECLARE_HASHTABLE(dmb_ht, SMC_LODEV_DMBS_HASH_BITS); + atomic_t dmb_cnt; + wait_queue_head_t dmbs_release; + wait_queue_head_t ldev_release; }; int smc_loopback_init(void);