From patchwork Sun Sep 24 15:16:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 144094 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp713466vqu; Sun, 24 Sep 2023 08:17:52 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHzPUCJz0QjynonoYg6Alt55MLE5+mLDWibOkWHnwtPCHvuT18Tcce7Uuja8YS+fpAF9aBm X-Received: by 2002:a17:902:c40a:b0:1c1:dbd6:9bf6 with SMTP id k10-20020a170902c40a00b001c1dbd69bf6mr5239514plk.41.1695568672197; Sun, 24 Sep 2023 08:17:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695568672; cv=none; d=google.com; s=arc-20160816; b=vCzPhr56Qzw1U+cV18ac/RZDl3MSwp0sIXh2+Q+OW5kpJ1luxBjyPApfrnbL7BY6dd Bi6RXcEdLgDe/YWNeu7y70TTm/T7cSX5+b0oDVfmFkipcP0q7ic2YRx3flmAfw3BGf7F SnG4Iba8hKEeCpUNGUcpqSPpjd54yN2VIeF0rdFtqzVlyZGv9RKiRb5FVHWr5fYW7atE Of03phGs//sZKN5FjGTJasdAGetqMi+CCecpo1J9ffoEtqQNkxRrUraF8LVXO3WYzIAn 7PFCebknqF22ZYhenYkzpkC2Nugu4n3wiJ19qV7N8GU4xu1tg4hiIRh+h0ffyujxuZ3m uiCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=psNL2GUiWI5oiG8MLygt16knT30svwUAro/x7ltt/MY=; fh=kTBstCixiri9Ahdk8UNuw7wmcx8Ry+mqafUdDsc8Vd0=; b=ZsKRll6zJg8+cYFtsMpTIi9UONbTsbjNO8hvcGPQcaDkm9KeRDXA3v4cxW2gWxZf2w Xk+6eDOYmaClAn0OvSnoGCGLwc9VGxR6HWrkltqHXJspIhbujq+gO7H7AgV8b6UUQWbg RXDbqGSEYUSY0U1ehuiwbOs0zB1CxYsEdh26WzoNKykmfcIA8a8Wko1WIzTZFpoJhuCI M2iv1L9vsKX26n7Btwc16lNj9CEOcDOyo70bKgTDhvs6EIGVyXj0hTPBHvfMfX+j9OhF sr+TInCnnAzmaiqJ+VD5dPt1hHm8mRz1n/ZbDNFTn9I8QnqDdbcdWBoBQHh3fjbhza5p WhGg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from fry.vger.email (fry.vger.email. [23.128.96.38]) by mx.google.com with ESMTPS id lf6-20020a170902fb4600b001b878f9e11csi7977774plb.54.2023.09.24.08.17.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 08:17:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id 70E0580D7889; Sun, 24 Sep 2023 08:17:28 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229608AbjIXPRQ (ORCPT + 30 others); Sun, 24 Sep 2023 11:17:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47146 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229574AbjIXPRN (ORCPT ); Sun, 24 Sep 2023 11:17:13 -0400 Received: from out30-99.freemail.mail.aliyun.com (out30-99.freemail.mail.aliyun.com [115.124.30.99]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8412FCF; Sun, 24 Sep 2023 08:17:05 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R901e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046056;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0VsilXe4_1695568619; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsilXe4_1695568619) by smtp.aliyun-inc.com; Sun, 24 Sep 2023 23:17:02 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: wintera@linux.ibm.com, schnelle@linux.ibm.com, gbayer@linux.ibm.com, pasic@linux.ibm.com, alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, dust.li@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v4 01/18] net/smc: decouple ism_dev from SMC-D device dump Date: Sun, 24 Sep 2023 23:16:36 +0800 Message-Id: <1695568613-125057-2-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> References: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Sun, 24 Sep 2023 08:17:28 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777932615616728978 X-GMAIL-MSGID: 1777932615616728978 This patch helps to decouple ISM device from SMC-D device, allowing different underlying device forms, such as virtual ISM devices. Signed-off-by: Wen Gu --- net/smc/smc_ism.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c index fbee249..fc551f4 100644 --- a/net/smc/smc_ism.c +++ b/net/smc/smc_ism.c @@ -230,12 +230,11 @@ static int smc_nl_handle_smcd_dev(struct smcd_dev *smcd, char smc_pnet[SMC_MAX_PNETID_LEN + 1]; struct smc_pci_dev smc_pci_dev; struct nlattr *port_attrs; + struct device *priv_dev; struct nlattr *attrs; - struct ism_dev *ism; int use_cnt = 0; void *nlh; - ism = smcd->priv; nlh = genlmsg_put(skb, NETLINK_CB(cb->skb).portid, cb->nlh->nlmsg_seq, &smc_gen_nl_family, NLM_F_MULTI, SMC_NETLINK_GET_DEV_SMCD); @@ -250,7 +249,9 @@ static int smc_nl_handle_smcd_dev(struct smcd_dev *smcd, if (nla_put_u8(skb, SMC_NLA_DEV_IS_CRIT, use_cnt > 0)) goto errattr; memset(&smc_pci_dev, 0, sizeof(smc_pci_dev)); - smc_set_pci_values(to_pci_dev(ism->dev.parent), &smc_pci_dev); + priv_dev = smcd->ops->get_dev(smcd); + if (priv_dev && priv_dev->parent) + smc_set_pci_values(to_pci_dev(priv_dev->parent), &smc_pci_dev); if (nla_put_u32(skb, SMC_NLA_DEV_PCI_FID, smc_pci_dev.pci_fid)) goto errattr; if (nla_put_u16(skb, SMC_NLA_DEV_PCI_CHID, smc_pci_dev.pci_pchid)) From patchwork Sun Sep 24 15:16:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 144096 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp713486vqu; Sun, 24 Sep 2023 08:17:55 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGiFE3Lh4CcVbzPNNBvCg61ysYiJGWJ7/9vX6p5ryqjvB6JXsqNPYkAjOZ/1UPJwd2rMrEW X-Received: by 2002:a17:90b:d81:b0:274:6251:3d4c with SMTP id bg1-20020a17090b0d8100b0027462513d4cmr4058809pjb.15.1695568674842; Sun, 24 Sep 2023 08:17:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695568674; cv=none; d=google.com; s=arc-20160816; b=P16iti3fM2HUpiBbdYpLjnZRVQBXkQqCLQD1lrr3djFXPohxoo32yKlNte8WwBNUUi Tvg2F8VR5/NH+xmCkZZcAkyl4AZwyEulMibT0+8yh1Mz9SB0T+4Hffq2ewc6HsuaRZKE 8dz2yyMZbeLbXjjcllG+pOS009obLqwWUAdb68AJzBEInKWdstFS+ttIOHaQJnzKUwn1 6CTXD/3uGvJS6qZTa6/UH6M8SSCKmXYsyGuqVtwrC8hwJVx6JO01uBrRAHkyszBb65yn fu2kTuRvuCrrK1FuRgslw4vowiMiyiHHfSjIOjx0anKe5DP7T9D5/bSAkjSII1V6N+tZ /LqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=f0EV04ehMJ2ZfdISjMdLCTjHv5suxyB+NCCn7ClG20k=; fh=kTBstCixiri9Ahdk8UNuw7wmcx8Ry+mqafUdDsc8Vd0=; b=aqvuKajAqV3fvclIHU7t9FAeV7u10bxkrK48jIR3DeJkR8P7Sm4+xqcGvzlbLaLlM9 GElsREtSX7VGTFR5xUCQzI0F+RH0aRRwAva4q5CqggjAGJOzp9F+AIZij50P0el8VStT /guNFXhJgyrQegYhf2fnLpXhSn9p0k8pYjM9VdWt9d7FEUrOiMsmjHjIICAD7evjE31M mTvCscH2AHf4koXDPwU0pGiv7ziq+ToqzxiRwOq7dO+guWzll6zUkXycIxXRjR6xh/JG fX7TE2yB+riNAJ9xke0hJAw6hFWgCnT6CiPuwyova3aEjI9nnqC1f4bou2jrnO/syJhe RDRQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from pete.vger.email (pete.vger.email. [2620:137:e000::3:6]) by mx.google.com with ESMTPS id j14-20020a170903024e00b001b9d800b487si8981294plh.87.2023.09.24.08.17.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 08:17:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id AFD9C8022C24; Sun, 24 Sep 2023 08:17:40 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229652AbjIXPRU (ORCPT + 30 others); Sun, 24 Sep 2023 11:17:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47170 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229596AbjIXPRP (ORCPT ); Sun, 24 Sep 2023 11:17:15 -0400 Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com [115.124.30.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C2118FD; Sun, 24 Sep 2023 08:17:08 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R131e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0VsinOY6_1695568622; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsinOY6_1695568622) by smtp.aliyun-inc.com; Sun, 24 Sep 2023 23:17:04 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: wintera@linux.ibm.com, schnelle@linux.ibm.com, gbayer@linux.ibm.com, pasic@linux.ibm.com, alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, dust.li@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v4 02/18] net/smc: decouple ism_dev from SMC-D DMB registration Date: Sun, 24 Sep 2023 23:16:37 +0800 Message-Id: <1695568613-125057-3-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> References: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Sun, 24 Sep 2023 08:17:40 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777932619166095567 X-GMAIL-MSGID: 1777932619166095567 This patch tries to decouple ISM device from SMC-D DMB registration, So that the register_dmb option is not restricted to ISM devices. Signed-off-by: Wen Gu --- drivers/s390/net/ism_drv.c | 2 +- include/net/smc.h | 4 ++-- net/smc/smc_ism.c | 7 ++----- 3 files changed, 5 insertions(+), 8 deletions(-) diff --git a/drivers/s390/net/ism_drv.c b/drivers/s390/net/ism_drv.c index 6df7f37..a34e913 100644 --- a/drivers/s390/net/ism_drv.c +++ b/drivers/s390/net/ism_drv.c @@ -781,7 +781,7 @@ static int smcd_query_rgid(struct smcd_dev *smcd, u64 rgid, u32 vid_valid, } static int smcd_register_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb, - struct ism_client *client) + void *client) { return ism_register_dmb(smcd->priv, (struct ism_dmb *)dmb, client); } diff --git a/include/net/smc.h b/include/net/smc.h index a002552..f75116e 100644 --- a/include/net/smc.h +++ b/include/net/smc.h @@ -50,13 +50,12 @@ struct smcd_dmb { #define ISM_ERROR 0xFFFF struct smcd_dev; -struct ism_client; struct smcd_ops { int (*query_remote_gid)(struct smcd_dev *dev, u64 rgid, u32 vid_valid, u32 vid); int (*register_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb, - struct ism_client *client); + void *client); int (*unregister_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb); int (*add_vlan_id)(struct smcd_dev *dev, u64 vlan_id); int (*del_vlan_id)(struct smcd_dev *dev, u64 vlan_id); @@ -77,6 +76,7 @@ struct smcd_ops { struct smcd_dev { const struct smcd_ops *ops; void *priv; + void *client; struct list_head list; spinlock_t lock; struct smc_connection **conn; diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c index fc551f4..455ae0a 100644 --- a/net/smc/smc_ism.c +++ b/net/smc/smc_ism.c @@ -200,7 +200,6 @@ int smc_ism_unregister_dmb(struct smcd_dev *smcd, struct smc_buf_desc *dmb_desc) int smc_ism_register_dmb(struct smc_link_group *lgr, int dmb_len, struct smc_buf_desc *dmb_desc) { -#if IS_ENABLED(CONFIG_ISM) struct smcd_dmb dmb; int rc; @@ -209,7 +208,7 @@ int smc_ism_register_dmb(struct smc_link_group *lgr, int dmb_len, dmb.sba_idx = dmb_desc->sba_idx; dmb.vlan_id = lgr->vlan_id; dmb.rgid = lgr->peer_gid; - rc = lgr->smcd->ops->register_dmb(lgr->smcd, &dmb, &smc_ism_client); + rc = lgr->smcd->ops->register_dmb(lgr->smcd, &dmb, lgr->smcd->client); if (!rc) { dmb_desc->sba_idx = dmb.sba_idx; dmb_desc->token = dmb.dmb_tok; @@ -218,9 +217,6 @@ int smc_ism_register_dmb(struct smc_link_group *lgr, int dmb_len, dmb_desc->len = dmb.dmb_len; } return rc; -#else - return 0; -#endif } static int smc_nl_handle_smcd_dev(struct smcd_dev *smcd, @@ -421,6 +417,7 @@ static void smcd_register_dev(struct ism_dev *ism) if (!smcd) return; smcd->priv = ism; + smcd->client = &smc_ism_client; ism_set_priv(ism, &smc_ism_client, smcd); if (smc_pnetid_by_dev_port(&ism->pdev->dev, 0, smcd->pnetid)) smc_pnetid_by_table_smcd(smcd); From patchwork Sun Sep 24 15:16:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 144104 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp713816vqu; Sun, 24 Sep 2023 08:18:36 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGMcaSd0mi+Xo/rj4BZHZTwPRkIej4JlZ/qGGkn7T5Fm3/HwIxNYEn9SNIXduQvYaEea8/o X-Received: by 2002:a17:902:bc4a:b0:1c3:4565:22f with SMTP id t10-20020a170902bc4a00b001c34565022fmr4431495plz.21.1695568716132; Sun, 24 Sep 2023 08:18:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695568716; cv=none; d=google.com; s=arc-20160816; b=TBP05AvhKpEReeZi/VRV7IVDCGkWz4ZBc/BiyN9FZv35zLj4KdNn9a5H/0NlOEhphe q7L8msaPb8svNLUVracYggCZzvhPqm57PbAFmzFr4jHfixKiZbWHDXuUYmhvD37oJa7K 44taI+C9liLilAmLi4EVGW0MuDG0QwkZcRCPJqJialYR+5eqi9sEuDOo/Jnf7FzGRQX1 Dgd6ArhHSn0QT6YXBB71wsKcy3moTGyTH5gCMWVSDH74M4XjF72bYDyuRX/wcbixP8JE 4N76Lo7n5v/YYyhvVm5OhUHCPCJ8bYd9m5sATWBttMsCQoT9Cqcb7IJ+iqh6Z7L5/X7A DcOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=iR8HOI5VcAO+ZlkVjKGr1OV1kwtVFGV8rsOoOl5zJ48=; fh=kTBstCixiri9Ahdk8UNuw7wmcx8Ry+mqafUdDsc8Vd0=; b=LSxc3ppaPREpFqN+yA4wnfau4VlRfxyPXCg3hW6JM03g04tIUVXHvQ5lSQpQ5kZNNd /oDa3esscJ0prCXyoNaKQFDiFqPKamTOts68gi8/CQY1yxedrIx/taEZRvlvJGTTB5Q8 Bsh8P+SiiZXqCf7wunBdVbN2hXNZVhl/90Ee6SyzuKDEPcAY3sUDIiZlkxA36DfzBTH7 7s0H/DLtMgEZBBuUstV+8bXNmXdHH3QsllayzvdFMpT6tAOym91sW+o+8IFfi/5ZJKMn ni+eIvS+/VUKR7aTdXfqYn9FABLgBnQF4NP/I7E/oQpGU6JooNT5ckSyWBD4tfy5GuV4 XAeg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from fry.vger.email (fry.vger.email. [23.128.96.38]) by mx.google.com with ESMTPS id j3-20020a170903028300b001c5fd2a28desi3599522plr.222.2023.09.24.08.18.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 08:18:36 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by fry.vger.email (Postfix) with ESMTP id 8F77780D7887; Sun, 24 Sep 2023 08:18:29 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at fry.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229803AbjIXPRY (ORCPT + 30 others); Sun, 24 Sep 2023 11:17:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47216 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229634AbjIXPRR (ORCPT ); Sun, 24 Sep 2023 11:17:17 -0400 Received: from out30-118.freemail.mail.aliyun.com (out30-118.freemail.mail.aliyun.com [115.124.30.118]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C165CF; Sun, 24 Sep 2023 08:17:11 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R201e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046049;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0Vsipe5h_1695568624; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0Vsipe5h_1695568624) by smtp.aliyun-inc.com; Sun, 24 Sep 2023 23:17:06 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: wintera@linux.ibm.com, schnelle@linux.ibm.com, gbayer@linux.ibm.com, pasic@linux.ibm.com, alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, dust.li@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v4 03/18] net/smc: extract v2 check helper from SMC-D device registration Date: Sun, 24 Sep 2023 23:16:38 +0800 Message-Id: <1695568613-125057-4-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> References: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on fry.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (fry.vger.email [0.0.0.0]); Sun, 24 Sep 2023 08:18:29 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777932662105519868 X-GMAIL-MSGID: 1777932662105519868 This patch extracts v2-capable logic from the process of registering the ISM device as an SMC-D device, so that the registration process of other underlying devices can reuse it. Signed-off-by: Wen Gu --- net/smc/smc_ism.c | 29 ++++++++++++++++++----------- net/smc/smc_ism.h | 1 + 2 files changed, 19 insertions(+), 11 deletions(-) diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c index 455ae0a..8f1ba74 100644 --- a/net/smc/smc_ism.c +++ b/net/smc/smc_ism.c @@ -69,6 +69,22 @@ bool smc_ism_is_v2_capable(void) return smc_ism_v2_capable; } +/* must be called under smcd_dev_list.mutex lock */ +void smc_ism_check_v2_capable(struct smcd_dev *smcd) +{ + u8 *system_eid = NULL; + + if (smc_ism_v2_capable) + return; + + system_eid = smcd->ops->get_system_eid(); + if (smcd->ops->supports_v2()) { + smc_ism_v2_capable = true; + memcpy(smc_ism_v2_system_eid, system_eid, + SMC_MAX_EID_LEN); + } +} + /* Set a connection using this DMBE. */ void smc_ism_set_conn(struct smc_connection *conn) { @@ -423,16 +439,7 @@ static void smcd_register_dev(struct ism_dev *ism) smc_pnetid_by_table_smcd(smcd); mutex_lock(&smcd_dev_list.mutex); - if (list_empty(&smcd_dev_list.list)) { - u8 *system_eid = NULL; - - system_eid = smcd->ops->get_system_eid(); - if (smcd->ops->supports_v2()) { - smc_ism_v2_capable = true; - memcpy(smc_ism_v2_system_eid, system_eid, - SMC_MAX_EID_LEN); - } - } + smc_ism_check_v2_capable(smcd); /* sort list: devices without pnetid before devices with pnetid */ if (smcd->pnetid[0]) list_add_tail(&smcd->list, &smcd_dev_list.list); @@ -535,10 +542,10 @@ int smc_ism_init(void) { int rc = 0; -#if IS_ENABLED(CONFIG_ISM) smc_ism_v2_capable = false; memset(smc_ism_v2_system_eid, 0, SMC_MAX_EID_LEN); +#if IS_ENABLED(CONFIG_ISM) rc = ism_register_client(&smc_ism_client); #endif return rc; diff --git a/net/smc/smc_ism.h b/net/smc/smc_ism.h index 832b2f4..14d2e77 100644 --- a/net/smc/smc_ism.h +++ b/net/smc/smc_ism.h @@ -42,6 +42,7 @@ int smc_ism_register_dmb(struct smc_link_group *lgr, int buf_size, void smc_ism_get_system_eid(u8 **eid); u16 smc_ism_get_chid(struct smcd_dev *dev); bool smc_ism_is_v2_capable(void); +void smc_ism_check_v2_capable(struct smcd_dev *dev); int smc_ism_init(void); void smc_ism_exit(void); int smcd_nl_get_device(struct sk_buff *skb, struct netlink_callback *cb); From patchwork Sun Sep 24 15:16:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 144099 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp713657vqu; Sun, 24 Sep 2023 08:18:18 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFgb/vvum1toQQIXVDBnXAeubMTcNeA27HFsWrqZS0POwqKtztHWDR9az6Uo6wTShD4z4pk X-Received: by 2002:a05:6358:341b:b0:133:a8e:6feb with SMTP id h27-20020a056358341b00b001330a8e6febmr6057044rwd.12.1695568698671; Sun, 24 Sep 2023 08:18:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695568698; cv=none; d=google.com; s=arc-20160816; b=qHqvPwtIoTtpHnzs9tKFX0hQqOmqqKkTNHdAFbR8DDpWF1P5600XD0QNGAASxJLFx7 54mHrw6HDHwdr/Wm83BE73mLiMDj+TzcdXACzkx2xJ8g0/Xx6jvq+tCTjRFzoNiz+69g +CxniXmCMcGwB6+Xd40OSAVxyNXF+hkkeOGFGd1YpIJ3PlJwLwbWnDNGAAFDcik7UGU1 DC7Bjj+HBZNkh/lVYpt/v09SABVjKHYtjBv3niQngUEtvEvFLdRXbHr7t6R6E1gZ0nRV TVS1ARDCLn8PT47LC7Py70kNOAiMGbkoywcCI08/046p86m8+aF0SFmfN2iLB3IRINtA 0GYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=oN5acSSXbDjrT8x3z0Z+yJR6hbCRGn4dSXeLy93Dt94=; fh=kTBstCixiri9Ahdk8UNuw7wmcx8Ry+mqafUdDsc8Vd0=; b=GAvg2peE7Wll/GtsloyCviEZKYKkJg3TLHOz3q5ns9rZBDBJsB2SfXTvFpy4fowdZh 7ouqtu6I310jJCcOV11RLKIIvvodp8/ltnr3RwFlJRtoHy5r3xbzykbb7MXH63xiCIfr SjOYIk3GpPJsGbzLoKG4xdDp2A4WTTg4o3CiOLFjvIDQ9ksYIMFSvsdmif6x9et4WZ0B gjQyOLI7stF5LOhbL7/4Tw3eOxoHvb0yHTRzYpfJuFUIbzLllth8oZ9FTkRnIdJRVjHQ rqT/s1D2gpTISjOD9ZecEXmhj22wnPDzLR6SQGanOesznrwf656xl4V9e2jz12a+RhbD KGbQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from morse.vger.email (morse.vger.email. [23.128.96.31]) by mx.google.com with ESMTPS id g38-20020a631126000000b00565f0e9cfbbsi8228023pgl.382.2023.09.24.08.18.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 08:18:18 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id C113A801F9A9; Sun, 24 Sep 2023 08:18:11 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229596AbjIXPRW (ORCPT + 30 others); Sun, 24 Sep 2023 11:17:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47232 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229637AbjIXPRR (ORCPT ); Sun, 24 Sep 2023 11:17:17 -0400 Received: from out30-131.freemail.mail.aliyun.com (out30-131.freemail.mail.aliyun.com [115.124.30.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 13C8CB8; Sun, 24 Sep 2023 08:17:10 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R401e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046056;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0VsimeNt_1695568626; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsimeNt_1695568626) by smtp.aliyun-inc.com; Sun, 24 Sep 2023 23:17:08 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: wintera@linux.ibm.com, schnelle@linux.ibm.com, gbayer@linux.ibm.com, pasic@linux.ibm.com, alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, dust.li@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v4 04/18] net/smc: support SMCv2.x supplemental features negotiation Date: Sun, 24 Sep 2023 23:16:39 +0800 Message-Id: <1695568613-125057-5-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> References: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Sun, 24 Sep 2023 08:18:11 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777932643699890424 X-GMAIL-MSGID: 1777932643699890424 This patch adds SMCv2.1 supplemental features negotiation. Supported SMCv2.1 supplemental features are represented by feature_mask in FCE header of CLC messages. Server Client Proposal(features(c-mask bits)) <----------------------------------- Accept(features(s-mask bits)) -----------------------------------> Confirm(features(s&c-mask bits)) <----------------------------------- Signed-off-by: Wen Gu --- net/smc/smc_clc.c | 7 +++++++ net/smc/smc_clc.h | 6 ++++-- net/smc/smc_core.h | 1 + 3 files changed, 12 insertions(+), 2 deletions(-) diff --git a/net/smc/smc_clc.c b/net/smc/smc_clc.c index 8deb46c..125b0d2 100644 --- a/net/smc/smc_clc.c +++ b/net/smc/smc_clc.c @@ -426,6 +426,7 @@ static int smc_clc_fill_fce(struct smc_clc_first_contact_ext_v2x *fce, memset(fce, 0, sizeof(*fce)); fce->fce_v2_base.os_type = SMC_CLC_OS_LINUX; fce->fce_v2_base.release = ini->release_nr; + fce->fce_v2_base.feature_mask = htons(ini->feature_mask); memcpy(fce->fce_v2_base.hostname, smc_hostname, sizeof(smc_hostname)); if (ini->is_smcd && ini->release_nr < SMC_RELEASE_1) { ret = sizeof(struct smc_clc_first_contact_ext); @@ -906,6 +907,7 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini) pclc_smcd->v2_ext_offset = htons(v2_ext_offset); plen += sizeof(*v2_ext); + v2_ext->feature_mask = htons(ini->feature_mask); read_lock(&smc_clc_eid_table.lock); v2_ext->hdr.eid_cnt = smc_clc_eid_table.ueid_cnt; plen += smc_clc_eid_table.ueid_cnt * SMC_MAX_EID_LEN; @@ -1219,6 +1221,7 @@ int smc_clc_clnt_v2x_features_validate(struct smc_clc_first_contact_ext *fce, return SMC_CLC_DECL_MAXLINKERR; ini->max_links = fce_v2x->max_links; } + ini->feature_mask &= ntohs(fce->feature_mask); return 0; } @@ -1250,6 +1253,10 @@ int smc_clc_v2x_features_confirm_check(struct smc_clc_msg_accept_confirm *cclc, return SMC_CLC_DECL_MAXLINKERR; } + if (~(ini->feature_mask) & ntohs(fce->feature_mask)) + return SMC_CLC_DECL_FEATUNSUPP; + ini->feature_mask = ntohs(fce->feature_mask); + return 0; } diff --git a/net/smc/smc_clc.h b/net/smc/smc_clc.h index c5c8e7d..bcf37c8 100644 --- a/net/smc/smc_clc.h +++ b/net/smc/smc_clc.h @@ -48,6 +48,7 @@ #define SMC_CLC_DECL_RELEASEERR 0x03030009 /* release version negotiate failed */ #define SMC_CLC_DECL_MAXCONNERR 0x0303000a /* max connections negotiate failed */ #define SMC_CLC_DECL_MAXLINKERR 0x0303000b /* max links negotiate failed */ +#define SMC_CLC_DECL_FEATUNSUPP 0x0303000c /* supplemental features not supported */ #define SMC_CLC_DECL_MODEUNSUPP 0x03040000 /* smc modes do not match (R or D)*/ #define SMC_CLC_DECL_RMBE_EC 0x03050000 /* peer has eyecatcher in RMBE */ #define SMC_CLC_DECL_OPTUNSUPP 0x03060000 /* fastopen sockopt not supported */ @@ -138,7 +139,8 @@ struct smc_clc_v2_extension { u8 roce[16]; /* RoCEv2 GID */ u8 max_conns; u8 max_links; - u8 reserved[14]; + __be16 feature_mask; + u8 reserved[12]; u8 user_eids[][SMC_MAX_EID_LEN]; }; @@ -234,7 +236,7 @@ struct smc_clc_first_contact_ext { u8 release : 4, os_type : 4; #endif - u8 reserved2[2]; + __be16 feature_mask; u8 hostname[SMC_MAX_HOSTNAME_LEN]; }; diff --git a/net/smc/smc_core.h b/net/smc/smc_core.h index 120027d..9f65678 100644 --- a/net/smc/smc_core.h +++ b/net/smc/smc_core.h @@ -401,6 +401,7 @@ struct smc_init_info { u8 max_links; u8 first_contact_peer; u8 first_contact_local; + u16 feature_mask; unsigned short vlan_id; u32 rc; u8 negotiated_eid[SMC_MAX_EID_LEN]; From patchwork Sun Sep 24 15:16:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 144095 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp713478vqu; Sun, 24 Sep 2023 08:17:54 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH/GZFamGfMifrJ3oAUp4lp7NGWcl/TUD6MtrQJ67LnyyTEPV29Z5YDXEQEXwblUEW9RQRj X-Received: by 2002:a17:903:32d1:b0:1c4:335:b06d with SMTP id i17-20020a17090332d100b001c40335b06dmr3786500plr.32.1695568673827; Sun, 24 Sep 2023 08:17:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695568673; cv=none; d=google.com; s=arc-20160816; b=RKiPRKHPzS+mjeETvIOitjKNS+Y3Roi7hEbUiSmNWbrtU1/NdcQ+Glv81z4pRli+G6 ZdGX2lZVB0ae3t3kPbHt9emqgZKeliKUrZjNfYcSbrIqZvn8KO2DhVm88X+2oYWRvgeH peGadAqauxfKXd+B+rXYmHq7Afif0lul9Hx8sliHFwnw626bhRndxgr1+2EWGq1c57L3 WNbWPXHNhj5AZV4CdSS1eeTh/WSqKxu0TvFAMa23F3bCBTcgE71zqczHC8V6zjiULSGe u2PDkE9gJ85rbPaXCBaPoCMxVNmueHnJ0VY8rodiADHBiHzZND45GU/v9lpKNHFpXinZ QP4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=VOcJAd/GukUuNrekkdAuy9WsRxE6/ZRaYrN8C7EMKy4=; fh=kTBstCixiri9Ahdk8UNuw7wmcx8Ry+mqafUdDsc8Vd0=; b=IKBFoPCjWde9HuT+qc2z3/NLWYJde8njN0GA5qpX1s4c4hyA4AkjrHEsRblclDXgxK 3nyhJHeYEuOrkSVzlS/j3yoQlUvrMXrQYEKvBWBCxFvQUyevTTxOTesnUzkMsI+pXi6r 7jW4KplkE3xxQ0Jj3kOZ+Bsra/JU6lxfnrBOqToykjiHd/9Xfi3wKFAW4sZJr4W5lMNi 6MIZ4dE3My8oin+y0AK8818UYlOwB3+Ib6xsbgI1u8Wh0t1TFg9yAcfQ4TuxWQxuk+dH j+F80dnvR4ep9VuhU06sk2V4hMj/I+p7RGtiaZ5oFr32Z9eIYw2xB3fwqClSGrWCfowS BYiA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from agentk.vger.email (agentk.vger.email. [23.128.96.32]) by mx.google.com with ESMTPS id q17-20020a17090311d100b001c36018fdaasi8508003plh.219.2023.09.24.08.17.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 08:17:53 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id D74B9819D1E7; Sun, 24 Sep 2023 08:17:47 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229852AbjIXPR1 (ORCPT + 30 others); Sun, 24 Sep 2023 11:17:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42718 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229708AbjIXPRV (ORCPT ); Sun, 24 Sep 2023 11:17:21 -0400 Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BC60EFD; Sun, 24 Sep 2023 08:17:13 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R221e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046059;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0Vsio7sa_1695568628; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0Vsio7sa_1695568628) by smtp.aliyun-inc.com; Sun, 24 Sep 2023 23:17:10 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: wintera@linux.ibm.com, schnelle@linux.ibm.com, gbayer@linux.ibm.com, pasic@linux.ibm.com, alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, dust.li@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v4 05/18] net/smc: reserve CHID range for SMC-D virtual device Date: Sun, 24 Sep 2023 23:16:40 +0800 Message-Id: <1695568613-125057-6-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> References: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Sun, 24 Sep 2023 08:17:48 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777932617762654615 X-GMAIL-MSGID: 1777932617762654615 This patch reserve CHID range from 0xFF00 to 0xFFFF for SMC-D virtual device and introduces helpers to identify them. Signed-off-by: Wen Gu --- net/smc/smc_ism.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/net/smc/smc_ism.h b/net/smc/smc_ism.h index 14d2e77..2ecc8de 100644 --- a/net/smc/smc_ism.h +++ b/net/smc/smc_ism.h @@ -15,6 +15,9 @@ #include "smc.h" +#define SMC_VIRT_ISM_CHID_MAX 0xFFFF +#define SMC_VIRT_ISM_CHID_MIN 0xFF00 + struct smcd_dev_list { /* List of SMCD devices */ struct list_head list; struct mutex mutex; /* Protects list of devices */ @@ -57,4 +60,16 @@ static inline int smc_ism_write(struct smcd_dev *smcd, u64 dmb_tok, return rc < 0 ? rc : 0; } +static inline bool __smc_ism_is_virtdev(u16 chid) +{ + return (chid >= SMC_VIRT_ISM_CHID_MIN && chid <= SMC_VIRT_ISM_CHID_MAX); +} + +static inline bool smc_ism_is_virtdev(struct smcd_dev *smcd) +{ + u16 chid = smcd->ops->get_chid(smcd); + + return __smc_ism_is_virtdev(chid); +} + #endif From patchwork Sun Sep 24 15:16:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 144108 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp713867vqu; Sun, 24 Sep 2023 08:18:45 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEKqix1/D4p8mRSdaJHyIa5mBxAZB0YfzUDuM6pDI11J+GCxKukSHB+NXfll43/Ph5zuPEh X-Received: by 2002:a05:6808:1a13:b0:3a4:4b42:612b with SMTP id bk19-20020a0568081a1300b003a44b42612bmr6644695oib.42.1695568725114; Sun, 24 Sep 2023 08:18:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695568725; cv=none; d=google.com; s=arc-20160816; b=cs24NDWLWm+z/rupaZZtHrrnXPuQ6Pl0v9jNpxK49mg6Mr4CUqOfjoyEUICJcfh8wX Du/Q4kQ3aI7o4dpcvrjKNPk9fFV6gEmz2sLzPGaqYkhXESLOUxu4m8N4MN2OWaaWm4gu GgaNXNebfFNkz7OgjXgOPawzz7q/c6BTyDUzzG7trNiUijrIGn9m2sQC5FUlCoIvBKpp +TXFRa4LVerAOhJiz+VKY4ZY1L39xrpdOfIgZf5EwM9G6UgqLoQ859Ogui7V6yUoxYoY sYHuw+cOuJGeFFW8ck/mFqjpVHKj8OHFhMVoh3sCaCOdH1wb7MpoRIAuhypSRKtA1a9C fDnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=1GFF9LqxBVigUDwQqHF9EV/A29W5d7UK4V3hWnQE8bY=; fh=kTBstCixiri9Ahdk8UNuw7wmcx8Ry+mqafUdDsc8Vd0=; b=pzoq1+vrDOry3fxMA1/vl8zkq9q1jV3K5IXMnjmBPBfV+WzRMLD6P6wU6tGwvGKkOa 8GnxjXkGiF0RczPaalvSMvHXFkmJ61a2MrefMw5w2WGffdZ8Oktuc4z7Tx6QW9HssS7H YIvZHFaJFIQxLu7SoTkDgbCVdLjeyhs6cpn6S/8VCQf0ZTI2l6At/TVuQhe1R+CH9hLB ebMhARXBfU09dwcS6bsgiSzBOAwiK2K3xsHlrvhTSuyKBAoGlM68H3qVNcP6kValWmcL VueubuLQb9SNWExgi9MIHq2LUTlzrMDZm+ebNzGJSwqYGnYQ/pjDYpjQ3PDYhmB+muIE 2jIQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from pete.vger.email (pete.vger.email. [23.128.96.36]) by mx.google.com with ESMTPS id bx22-20020a056a02051600b00565eccbc52fsi9106509pgb.303.2023.09.24.08.18.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 08:18:45 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 2F746801B65B; Sun, 24 Sep 2023 08:18:31 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230170AbjIXPRk (ORCPT + 30 others); Sun, 24 Sep 2023 11:17:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46792 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230028AbjIXPRf (ORCPT ); Sun, 24 Sep 2023 11:17:35 -0400 Received: from out30-112.freemail.mail.aliyun.com (out30-112.freemail.mail.aliyun.com [115.124.30.112]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9900D11B; Sun, 24 Sep 2023 08:17:16 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R211e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0VsimePF_1695568631; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsimePF_1695568631) by smtp.aliyun-inc.com; Sun, 24 Sep 2023 23:17:13 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: wintera@linux.ibm.com, schnelle@linux.ibm.com, gbayer@linux.ibm.com, pasic@linux.ibm.com, alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, dust.li@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v4 06/18] net/smc: extend GID to 128bits only for virtual ISM device Date: Sun, 24 Sep 2023 23:16:41 +0800 Message-Id: <1695568613-125057-7-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> References: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Sun, 24 Sep 2023 08:18:31 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777932671532689271 X-GMAIL-MSGID: 1777932671532689271 Virtual ISM devices are introduced to SMC-Dv2.1 protocal, whose GIDs are 128-bits UUIDs as defined by RFC4122. And note that the GIDs of ISM devices still remain 64-bits. This patch adapts the relevant codes, such as CLC handshake, to make it compatible with 128 bits GID. Signed-off-by: Wen Gu --- drivers/s390/net/ism_drv.c | 18 +++++++------ include/net/smc.h | 15 +++++++---- include/uapi/linux/smc.h | 3 +++ include/uapi/linux/smc_diag.h | 2 ++ net/smc/af_smc.c | 60 +++++++++++++++++++++++++++++++++---------- net/smc/smc_clc.c | 43 +++++++++++++++++++++---------- net/smc/smc_clc.h | 4 +-- net/smc/smc_core.c | 41 +++++++++++++++++++++-------- net/smc/smc_core.h | 7 ++--- net/smc/smc_diag.c | 11 ++++++-- net/smc/smc_ism.c | 18 ++++++++----- net/smc/smc_ism.h | 3 ++- net/smc/smc_pnet.c | 4 +-- 13 files changed, 163 insertions(+), 66 deletions(-) diff --git a/drivers/s390/net/ism_drv.c b/drivers/s390/net/ism_drv.c index a34e913..8e42eb2 100644 --- a/drivers/s390/net/ism_drv.c +++ b/drivers/s390/net/ism_drv.c @@ -774,10 +774,10 @@ static void __exit ism_exit(void) /*************************** SMC-D Implementation *****************************/ #if IS_ENABLED(CONFIG_SMC) -static int smcd_query_rgid(struct smcd_dev *smcd, u64 rgid, u32 vid_valid, - u32 vid) +static int smcd_query_rgid(struct smcd_dev *smcd, struct smcd_gid *rgid, + u32 vid_valid, u32 vid) { - return ism_query_rgid(smcd->priv, rgid, vid_valid, vid); + return ism_query_rgid(smcd->priv, rgid->gid, vid_valid, vid); } static int smcd_register_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb, @@ -811,10 +811,10 @@ static int smcd_reset_vlan_required(struct smcd_dev *smcd) return ism_cmd_simple(smcd->priv, ISM_RESET_VLAN); } -static int smcd_signal_ieq(struct smcd_dev *smcd, u64 rgid, u32 trigger_irq, - u32 event_code, u64 info) +static int smcd_signal_ieq(struct smcd_dev *smcd, struct smcd_gid *rgid, + u32 trigger_irq, u32 event_code, u64 info) { - return ism_signal_ieq(smcd->priv, rgid, trigger_irq, event_code, info); + return ism_signal_ieq(smcd->priv, rgid->gid, trigger_irq, event_code, info); } static int smcd_move(struct smcd_dev *smcd, u64 dmb_tok, unsigned int idx, @@ -830,9 +830,11 @@ static int smcd_supports_v2(void) SYSTEM_EID.type[0] != '0'; } -static u64 smcd_get_local_gid(struct smcd_dev *smcd) +static void smcd_get_local_gid(struct smcd_dev *smcd, + struct smcd_gid *smcd_gid) { - return ism_get_local_gid(smcd->priv); + smcd_gid->gid = ism_get_local_gid(smcd->priv); + smcd_gid->gid_ext = 0; } static u16 smcd_get_chid(struct smcd_dev *smcd) diff --git a/include/net/smc.h b/include/net/smc.h index f75116e..d8db5e1 100644 --- a/include/net/smc.h +++ b/include/net/smc.h @@ -51,9 +51,14 @@ struct smcd_dmb { struct smcd_dev; +struct smcd_gid { + u64 gid; + u64 gid_ext; +}; + struct smcd_ops { - int (*query_remote_gid)(struct smcd_dev *dev, u64 rgid, u32 vid_valid, - u32 vid); + int (*query_remote_gid)(struct smcd_dev *dev, struct smcd_gid *rgid, + u32 vid_valid, u32 vid); int (*register_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb, void *client); int (*unregister_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb); @@ -61,14 +66,14 @@ struct smcd_ops { int (*del_vlan_id)(struct smcd_dev *dev, u64 vlan_id); int (*set_vlan_required)(struct smcd_dev *dev); int (*reset_vlan_required)(struct smcd_dev *dev); - int (*signal_event)(struct smcd_dev *dev, u64 rgid, u32 trigger_irq, - u32 event_code, u64 info); + int (*signal_event)(struct smcd_dev *dev, struct smcd_gid *rgid, + u32 trigger_irq, u32 event_code, u64 info); int (*move_data)(struct smcd_dev *dev, u64 dmb_tok, unsigned int idx, bool sf, unsigned int offset, void *data, unsigned int size); int (*supports_v2)(void); u8* (*get_system_eid)(void); - u64 (*get_local_gid)(struct smcd_dev *dev); + void (*get_local_gid)(struct smcd_dev *dev, struct smcd_gid *gid); u16 (*get_chid)(struct smcd_dev *dev); struct device* (*get_dev)(struct smcd_dev *dev); }; diff --git a/include/uapi/linux/smc.h b/include/uapi/linux/smc.h index 837fcd4..0d2f020 100644 --- a/include/uapi/linux/smc.h +++ b/include/uapi/linux/smc.h @@ -99,6 +99,9 @@ enum { SMC_NLA_LGR_V2_OS, /* u8 */ SMC_NLA_LGR_V2_NEG_EID, /* string */ SMC_NLA_LGR_V2_PEER_HOST, /* string */ + SMC_NLA_LGR_V2_PAD, /* flag */ + SMC_NLA_LGR_V2_GID_EXT, /* u64 */ + SMC_NLA_LGR_V2_PEER_GID_EXT, /* u64 */ __SMC_NLA_LGR_V2_MAX, SMC_NLA_LGR_V2_MAX = __SMC_NLA_LGR_V2_MAX - 1 }; diff --git a/include/uapi/linux/smc_diag.h b/include/uapi/linux/smc_diag.h index 8cb3a6f..78b7dfb 100644 --- a/include/uapi/linux/smc_diag.h +++ b/include/uapi/linux/smc_diag.h @@ -107,6 +107,8 @@ struct smcd_diag_dmbinfo { /* SMC-D Socket internals */ __aligned_u64 my_gid; /* My GID */ __aligned_u64 token; /* Token of DMB */ __aligned_u64 peer_token; /* Token of remote DMBE */ + __aligned_u64 peer_gid_ext; /* Peer GID extension */ + __aligned_u64 my_gid_ext; /* My GID extension */ }; #endif /* _UAPI_SMC_DIAG_H_ */ diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index bacdd97..5bb41404 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -719,7 +719,7 @@ static void smcd_conn_save_peer_info(struct smc_sock *smc, int bufsize = smc_uncompress_bufsize(clc->d0.dmbe_size); smc->conn.peer_rmbe_idx = clc->d0.dmbe_idx; - smc->conn.peer_token = clc->d0.token; + smc->conn.peer_token = ntohll(clc->d0.token); /* msg header takes up space in the buffer */ smc->conn.peer_rmbe_size = bufsize - sizeof(struct smcd_cdc_msg); atomic_set(&smc->conn.peer_rmbe_space, smc->conn.peer_rmbe_size); @@ -1044,7 +1044,8 @@ static int smc_find_ism_v2_device_clnt(struct smc_sock *smc, { int rc = SMC_CLC_DECL_NOSMCDDEV; struct smcd_dev *smcd; - int i = 1; + int i = 1, slot = 1; + bool is_virtdev; u16 chid; if (smcd_indicated(ini->smc_type_v1)) @@ -1056,14 +1057,21 @@ static int smc_find_ism_v2_device_clnt(struct smc_sock *smc, chid = smc_ism_get_chid(smcd); if (!smc_find_ism_v2_is_unique_chid(chid, ini, i)) continue; + is_virtdev = __smc_ism_is_virtdev(chid); if (!smc_pnet_is_pnetid_set(smcd->pnetid) || smc_pnet_is_ndev_pnetid(sock_net(&smc->sk), smcd->pnetid)) { + if (is_virtdev && slot == SMC_MAX_ISM_DEVS) + /* no enough space for virtual ISM devices, whose GID + * takes 2 slots. Try the next potential ISM device. + */ + continue; ini->ism_dev[i] = smcd; ini->ism_chid[i] = chid; ini->is_smcd = true; rc = 0; i++; - if (i > SMC_MAX_ISM_DEVS) + slot = is_virtdev ? slot + 2 : slot + 1; + if (slot > SMC_MAX_ISM_DEVS) break; } } @@ -1409,8 +1417,13 @@ static int smc_connect_ism(struct smc_sock *smc, rc = smc_v2_determine_accepted_chid(aclc_v2, ini); if (rc) return rc; + + if (__smc_ism_is_virtdev(ini->ism_chid[ini->ism_selected])) + ini->ism_peer_gid[ini->ism_selected].gid_ext = + ntohll(aclc_v2->d1.gid_ext); + /* for non-virtual ISM devices, peer gid_ext remains 0. */ } - ini->ism_peer_gid[ini->ism_selected] = aclc->d0.gid; + ini->ism_peer_gid[ini->ism_selected].gid = ntohll(aclc->d0.gid); /* there is only one lgr role for SMC-D; use server lock */ mutex_lock(&smc_server_lgr_pending); @@ -2101,7 +2114,8 @@ static bool smc_is_already_selected(struct smcd_dev *smcd, /* check for ISM devices matching proposed ISM devices */ static void smc_check_ism_v2_match(struct smc_init_info *ini, - u16 proposed_chid, u64 proposed_gid, + u16 proposed_chid, + struct smcd_gid *proposed_gid, unsigned int *matches) { struct smcd_dev *smcd; @@ -2113,7 +2127,11 @@ static void smc_check_ism_v2_match(struct smc_init_info *ini, continue; if (smc_ism_get_chid(smcd) == proposed_chid && !smc_ism_cantalk(proposed_gid, ISM_RESERVED_VLANID, smcd)) { - ini->ism_peer_gid[*matches] = proposed_gid; + ini->ism_peer_gid[*matches].gid = proposed_gid->gid; + if (__smc_ism_is_virtdev(proposed_chid)) + ini->ism_peer_gid[*matches].gid_ext = + proposed_gid->gid_ext; + /* non-virtual ISM's peer gid_ext remains 0. */ ini->ism_dev[*matches] = smcd; (*matches)++; break; @@ -2135,9 +2153,11 @@ static void smc_find_ism_v2_device_serv(struct smc_sock *new_smc, struct smc_clc_v2_extension *smc_v2_ext; struct smc_clc_msg_smcd *pclc_smcd; unsigned int matches = 0; + struct smcd_gid smcd_gid; u8 smcd_version; u8 *eid = NULL; int i, rc; + u16 chid; if (!(ini->smcd_version & SMC_V2) || !smcd_indicated(ini->smc_type_v2)) goto not_found; @@ -2147,18 +2167,31 @@ static void smc_find_ism_v2_device_serv(struct smc_sock *new_smc, smcd_v2_ext = smc_get_clc_smcd_v2_ext(smc_v2_ext); mutex_lock(&smcd_dev_list.mutex); - if (pclc_smcd->ism.chid) + if (pclc_smcd->ism.chid) { /* check for ISM device matching proposed native ISM device */ + smcd_gid.gid = ntohll(pclc_smcd->ism.gid); + smcd_gid.gid_ext = 0; smc_check_ism_v2_match(ini, ntohs(pclc_smcd->ism.chid), - ntohll(pclc_smcd->ism.gid), &matches); + &smcd_gid, &matches); + } for (i = 1; i <= smc_v2_ext->hdr.ism_gid_cnt; i++) { /* check for ISM devices matching proposed non-native ISM * devices */ - smc_check_ism_v2_match(ini, - ntohs(smcd_v2_ext->gidchid[i - 1].chid), - ntohll(smcd_v2_ext->gidchid[i - 1].gid), - &matches); + smcd_gid.gid = ntohll(smcd_v2_ext->gidchid[i - 1].gid); + smcd_gid.gid_ext = 0; + chid = ntohs(smcd_v2_ext->gidchid[i - 1].chid); + if (__smc_ism_is_virtdev(chid)) { + if (i >= smc_v2_ext->hdr.ism_gid_cnt || + chid != ntohs(smcd_v2_ext->gidchid[i].chid)) + /* check if next slot exists and the next slot's + * chid is consistent with the current slot. + */ + continue; + + smcd_gid.gid_ext = ntohll(smcd_v2_ext->gidchid[i++].gid); + } + smc_check_ism_v2_match(ini, chid, &smcd_gid, &matches); } mutex_unlock(&smcd_dev_list.mutex); @@ -2207,7 +2240,8 @@ static void smc_find_ism_v1_device_serv(struct smc_sock *new_smc, if (!(ini->smcd_version & SMC_V1) || !smcd_indicated(ini->smc_type_v1)) goto not_found; ini->is_smcd = true; /* prepare ISM check */ - ini->ism_peer_gid[0] = ntohll(pclc_smcd->ism.gid); + ini->ism_peer_gid[0].gid = ntohll(pclc_smcd->ism.gid); + ini->ism_peer_gid[0].gid_ext = 0; rc = smc_find_ism_device(new_smc, ini); if (rc) goto not_found; diff --git a/net/smc/smc_clc.c b/net/smc/smc_clc.c index 125b0d2..aeb9643 100644 --- a/net/smc/smc_clc.c +++ b/net/smc/smc_clc.c @@ -882,11 +882,13 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini) ETH_ALEN); } if (smcd_indicated(ini->smc_type_v1)) { + struct smcd_gid smcd_gid; + /* add SMC-D specifics */ if (ini->ism_dev[0]) { smcd = ini->ism_dev[0]; - pclc_smcd->ism.gid = - htonll(smcd->ops->get_local_gid(smcd)); + smcd->ops->get_local_gid(smcd, &smcd_gid); + pclc_smcd->ism.gid = htonll(smcd_gid.gid); pclc_smcd->ism.chid = htons(smc_ism_get_chid(ini->ism_dev[0])); } @@ -919,10 +921,11 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini) read_unlock(&smc_clc_eid_table.lock); } if (smcd_indicated(ini->smc_type_v2)) { + struct smcd_gid smcd_gid; u8 *eid = NULL; + int slot = 0; v2_ext->hdr.flag.seid = smc_clc_eid_table.seid_enabled; - v2_ext->hdr.ism_gid_cnt = ini->ism_offered_cnt; v2_ext->hdr.smcd_v2_ext_offset = htons(sizeof(*v2_ext) - offsetofend(struct smc_clnt_opts_area_hdr, smcd_v2_ext_offset) + @@ -934,14 +937,21 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini) if (ini->ism_offered_cnt) { for (i = 1; i <= ini->ism_offered_cnt; i++) { smcd = ini->ism_dev[i]; - gidchids[i - 1].gid = - htonll(smcd->ops->get_local_gid(smcd)); - gidchids[i - 1].chid = + smcd->ops->get_local_gid(smcd, &smcd_gid); + gidchids[slot].chid = htons(smc_ism_get_chid(ini->ism_dev[i])); + gidchids[slot].gid = htonll(smcd_gid.gid); + if (__smc_ism_is_virtdev(gidchids[slot].chid)) { + /* virtual-ism takes two slots */ + gidchids[slot + 1].chid = gidchids[slot].chid; + gidchids[slot + 1].gid = htonll(smcd_gid.gid_ext); + slot++; + } + slot++; } - plen += ini->ism_offered_cnt * - sizeof(struct smc_clc_smcd_gid_chid); + plen += slot * sizeof(struct smc_clc_smcd_gid_chid); } + v2_ext->hdr.ism_gid_cnt = slot; } if (smcr_indicated(ini->smc_type_v2)) { memcpy(v2_ext->roce, ini->smcrv2.ib_gid_v2, SMC_GID_SIZE); @@ -977,7 +987,7 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini) vec[i++].iov_len = sizeof(*smcd_v2_ext); if (ini->ism_offered_cnt) { vec[i].iov_base = gidchids; - vec[i++].iov_len = ini->ism_offered_cnt * + vec[i++].iov_len = v2_ext->hdr.ism_gid_cnt * sizeof(struct smc_clc_smcd_gid_chid); } } @@ -1019,23 +1029,28 @@ static int smc_clc_send_confirm_accept(struct smc_sock *smc, if (first_contact) clc->hdr.typev2 |= SMC_FIRST_CONTACT_MASK; if (conn->lgr->is_smcd) { + struct smcd_gid smcd_gid; + u16 chid; + /* SMC-D specific settings */ memcpy(clc->hdr.eyecatcher, SMCD_EYECATCHER, sizeof(SMCD_EYECATCHER)); + conn->lgr->smcd->ops->get_local_gid(conn->lgr->smcd, &smcd_gid); clc->hdr.typev1 = SMC_TYPE_D; - clc->d0.gid = - conn->lgr->smcd->ops->get_local_gid(conn->lgr->smcd); - clc->d0.token = conn->rmb_desc->token; + clc->d0.gid = htonll(smcd_gid.gid); + clc->d0.token = htonll(conn->rmb_desc->token); clc->d0.dmbe_size = conn->rmbe_size_comp; clc->d0.dmbe_idx = 0; memcpy(&clc->d0.linkid, conn->lgr->id, SMC_LGR_ID_SIZE); if (version == SMC_V1) { clc->hdr.length = htons(SMCD_CLC_ACCEPT_CONFIRM_LEN); } else { - clc_v2->d1.chid = - htons(smc_ism_get_chid(conn->lgr->smcd)); + chid = smc_ism_get_chid(conn->lgr->smcd); + clc_v2->d1.chid = htons(chid); if (eid && eid[0]) memcpy(clc_v2->d1.eid, eid, SMC_MAX_EID_LEN); + if (__smc_ism_is_virtdev(chid)) + clc_v2->d1.gid_ext = htonll(smcd_gid.gid_ext); len = SMCD_CLC_ACCEPT_CONFIRM_LEN_V2; if (first_contact) { fce_len = smc_clc_fill_fce(&fce, ini); diff --git a/net/smc/smc_clc.h b/net/smc/smc_clc.h index bcf37c8..611763a 100644 --- a/net/smc/smc_clc.h +++ b/net/smc/smc_clc.h @@ -281,8 +281,8 @@ struct smc_clc_msg_accept_confirm_v2 { /* clc accept / confirm message */ struct smcd_clc_msg_accept_confirm_common d0; __be16 chid; u8 eid[SMC_MAX_EID_LEN]; - u8 reserved5[8]; - } d1; + __be64 gid_ext; + } __packed d1; }; }; diff --git a/net/smc/smc_core.c b/net/smc/smc_core.c index d520ee6..6d7c738 100644 --- a/net/smc/smc_core.c +++ b/net/smc/smc_core.c @@ -284,6 +284,9 @@ static int smc_nl_fill_lgr_v2_common(struct smc_link_group *lgr, { char smc_host[SMC_MAX_HOSTNAME_LEN + 1]; char smc_eid[SMC_MAX_EID_LEN + 1]; + struct smcd_dev *smcd = lgr->smcd; + struct smcd_gid smcd_gid; + bool is_virtdev; if (nla_put_u8(skb, SMC_NLA_LGR_V2_VER, lgr->smc_version)) goto errv2attr; @@ -299,6 +302,16 @@ static int smc_nl_fill_lgr_v2_common(struct smc_link_group *lgr, smc_eid[SMC_MAX_EID_LEN] = 0; if (nla_put_string(skb, SMC_NLA_LGR_V2_NEG_EID, smc_eid)) goto errv2attr; + smcd->ops->get_local_gid(smcd, &smcd_gid); + is_virtdev = smc_ism_is_virtdev(smcd); + if (nla_put_u64_64bit(skb, SMC_NLA_LGR_V2_GID_EXT, + is_virtdev ? smcd_gid.gid_ext : 0, + SMC_NLA_LGR_V2_PAD)) + goto errv2attr; + if (nla_put_u64_64bit(skb, SMC_NLA_LGR_V2_PEER_GID_EXT, + is_virtdev ? lgr->peer_gid.gid_ext : 0, + SMC_NLA_LGR_V2_PAD)) + goto errv2attr; nla_nest_end(skb, v2_attrs); return 0; @@ -506,6 +519,7 @@ static int smc_nl_fill_smcd_lgr(struct smc_link_group *lgr, { char smc_pnet[SMC_MAX_PNETID_LEN + 1]; struct smcd_dev *smcd = lgr->smcd; + struct smcd_gid smcd_gid; struct nlattr *attrs; void *nlh; @@ -521,11 +535,11 @@ static int smc_nl_fill_smcd_lgr(struct smc_link_group *lgr, if (nla_put_u32(skb, SMC_NLA_LGR_D_ID, *((u32 *)&lgr->id))) goto errattr; + smcd->ops->get_local_gid(smcd, &smcd_gid); if (nla_put_u64_64bit(skb, SMC_NLA_LGR_D_GID, - smcd->ops->get_local_gid(smcd), - SMC_NLA_LGR_D_PAD)) + smcd_gid.gid, SMC_NLA_LGR_D_PAD)) goto errattr; - if (nla_put_u64_64bit(skb, SMC_NLA_LGR_D_PEER_GID, lgr->peer_gid, + if (nla_put_u64_64bit(skb, SMC_NLA_LGR_D_PEER_GID, lgr->peer_gid.gid, SMC_NLA_LGR_D_PAD)) goto errattr; if (nla_put_u8(skb, SMC_NLA_LGR_D_VLAN_ID, lgr->vlan_id)) @@ -876,7 +890,8 @@ static int smc_lgr_create(struct smc_sock *smc, struct smc_init_info *ini) /* SMC-D specific settings */ smcd = ini->ism_dev[ini->ism_selected]; get_device(smcd->ops->get_dev(smcd)); - lgr->peer_gid = ini->ism_peer_gid[ini->ism_selected]; + memcpy(&lgr->peer_gid, &ini->ism_peer_gid[ini->ism_selected], + sizeof(struct smcd_gid)); lgr->smcd = ini->ism_dev[ini->ism_selected]; lgr_list = &ini->ism_dev[ini->ism_selected]->lgr_list; lgr_lock = &lgr->smcd->lgr_lock; @@ -1514,7 +1529,8 @@ void smc_lgr_terminate_sched(struct smc_link_group *lgr) } /* Called when peer lgr shutdown (regularly or abnormally) is received */ -void smc_smcd_terminate(struct smcd_dev *dev, u64 peer_gid, unsigned short vlan) +void smc_smcd_terminate(struct smcd_dev *dev, struct smcd_gid *peer_gid, + unsigned short vlan) { struct smc_link_group *lgr, *l; LIST_HEAD(lgr_free_list); @@ -1522,7 +1538,10 @@ void smc_smcd_terminate(struct smcd_dev *dev, u64 peer_gid, unsigned short vlan) /* run common cleanup function and build free list */ spin_lock_bh(&dev->lgr_lock); list_for_each_entry_safe(lgr, l, &dev->lgr_list, list) { - if ((!peer_gid || lgr->peer_gid == peer_gid) && + if ((!peer_gid->gid || + (lgr->peer_gid.gid == peer_gid->gid && + !smc_ism_is_virtdev(dev) ? 1 : + lgr->peer_gid.gid_ext == peer_gid->gid_ext)) && (vlan == VLAN_VID_MASK || lgr->vlan_id == vlan)) { if (peer_gid) /* peer triggered termination */ lgr->peer_shutdown = 1; @@ -1859,10 +1878,12 @@ static bool smcr_lgr_match(struct smc_link_group *lgr, u8 smcr_version, return false; } -static bool smcd_lgr_match(struct smc_link_group *lgr, - struct smcd_dev *smcismdev, u64 peer_gid) +static bool smcd_lgr_match(struct smc_link_group *lgr, struct smcd_dev *smcismdev, + struct smcd_gid *peer_gid) { - return lgr->peer_gid == peer_gid && lgr->smcd == smcismdev; + return lgr->peer_gid.gid == peer_gid->gid && lgr->smcd == smcismdev && + smc_ism_is_virtdev(smcismdev) ? + (lgr->peer_gid.gid_ext == peer_gid->gid_ext) : 1; } /* create a new SMC connection (and a new link group if necessary) */ @@ -1892,7 +1913,7 @@ int smc_conn_create(struct smc_sock *smc, struct smc_init_info *ini) write_lock_bh(&lgr->conns_lock); if ((ini->is_smcd ? smcd_lgr_match(lgr, ini->ism_dev[ini->ism_selected], - ini->ism_peer_gid[ini->ism_selected]) : + &ini->ism_peer_gid[ini->ism_selected]) : smcr_lgr_match(lgr, ini->smcr_version, ini->peer_systemid, ini->peer_gid, ini->peer_mac, role, diff --git a/net/smc/smc_core.h b/net/smc/smc_core.h index 9f65678..d57eb9b 100644 --- a/net/smc/smc_core.h +++ b/net/smc/smc_core.h @@ -17,6 +17,7 @@ #include #include #include +#include #include "smc.h" #include "smc_ib.h" @@ -355,7 +356,7 @@ struct smc_link_group { /* max links can be added in lgr */ }; struct { /* SMC-D */ - u64 peer_gid; + struct smcd_gid peer_gid; /* Peer GID (remote) */ struct smcd_dev *smcd; /* ISM device for VLAN reg. */ @@ -417,7 +418,7 @@ struct smc_init_info { u32 ib_clcqpn; struct smc_init_info_smcrv2 smcrv2; /* SMC-D */ - u64 ism_peer_gid[SMC_MAX_ISM_DEVS + 1]; + struct smcd_gid ism_peer_gid[SMC_MAX_ISM_DEVS + 1]; struct smcd_dev *ism_dev[SMC_MAX_ISM_DEVS + 1]; u16 ism_chid[SMC_MAX_ISM_DEVS + 1]; u8 ism_offered_cnt; /* # of ISM devices offered */ @@ -545,7 +546,7 @@ static inline void smc_set_pci_values(struct pci_dev *pci_dev, void smc_lgr_put(struct smc_link_group *lgr); void smcr_port_add(struct smc_ib_device *smcibdev, u8 ibport); void smcr_port_err(struct smc_ib_device *smcibdev, u8 ibport); -void smc_smcd_terminate(struct smcd_dev *dev, u64 peer_gid, +void smc_smcd_terminate(struct smcd_dev *dev, struct smcd_gid *peer_gid, unsigned short vlan); void smc_smcd_terminate_all(struct smcd_dev *dev); void smc_smcr_terminate_all(struct smc_ib_device *smcibdev); diff --git a/net/smc/smc_diag.c b/net/smc/smc_diag.c index 7ff2152..a7a48b6 100644 --- a/net/smc/smc_diag.c +++ b/net/smc/smc_diag.c @@ -21,6 +21,7 @@ #include "smc.h" #include "smc_core.h" +#include "smc_ism.h" struct smc_diag_dump_ctx { int pos[2]; @@ -168,12 +169,18 @@ static int __smc_diag_dump(struct sock *sk, struct sk_buff *skb, struct smc_connection *conn = &smc->conn; struct smcd_diag_dmbinfo dinfo; struct smcd_dev *smcd = conn->lgr->smcd; + struct smcd_gid smcd_gid; memset(&dinfo, 0, sizeof(dinfo)); dinfo.linkid = *((u32 *)conn->lgr->id); - dinfo.peer_gid = conn->lgr->peer_gid; - dinfo.my_gid = smcd->ops->get_local_gid(smcd); + dinfo.peer_gid = conn->lgr->peer_gid.gid; + smcd->ops->get_local_gid(smcd, &smcd_gid); + dinfo.my_gid = smcd_gid.gid; + if (smc_ism_is_virtdev(smcd)) { + dinfo.peer_gid_ext = conn->lgr->peer_gid.gid_ext; + dinfo.my_gid_ext = smcd_gid.gid_ext; + } dinfo.token = conn->rmb_desc->token; dinfo.peer_token = conn->peer_token; diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c index 8f1ba74..0922fab 100644 --- a/net/smc/smc_ism.c +++ b/net/smc/smc_ism.c @@ -44,7 +44,8 @@ static void smcd_handle_irq(struct ism_dev *ism, unsigned int dmbno, #endif /* Test if an ISM communication is possible - same CPC */ -int smc_ism_cantalk(u64 peer_gid, unsigned short vlan_id, struct smcd_dev *smcd) +int smc_ism_cantalk(struct smcd_gid *peer_gid, unsigned short vlan_id, + struct smcd_dev *smcd) { return smcd->ops->query_remote_gid(smcd, peer_gid, vlan_id ? 1 : 0, vlan_id); @@ -223,7 +224,7 @@ int smc_ism_register_dmb(struct smc_link_group *lgr, int dmb_len, dmb.dmb_len = dmb_len; dmb.sba_idx = dmb_desc->sba_idx; dmb.vlan_id = lgr->vlan_id; - dmb.rgid = lgr->peer_gid; + dmb.rgid = lgr->peer_gid.gid; rc = lgr->smcd->ops->register_dmb(lgr->smcd, &dmb, lgr->smcd->client); if (!rc) { dmb_desc->sba_idx = dmb.sba_idx; @@ -353,18 +354,20 @@ struct smc_ism_event_work { static void smcd_handle_sw_event(struct smc_ism_event_work *wrk) { + struct smcd_gid peer_gid = { .gid = wrk->event.tok, + .gid_ext = 0 }; union smcd_sw_event_info ev_info; ev_info.info = wrk->event.info; switch (wrk->event.code) { case ISM_EVENT_CODE_SHUTDOWN: /* Peer shut down DMBs */ - smc_smcd_terminate(wrk->smcd, wrk->event.tok, ev_info.vlan_id); + smc_smcd_terminate(wrk->smcd, &peer_gid, ev_info.vlan_id); break; case ISM_EVENT_CODE_TESTLINK: /* Activity timer */ if (ev_info.code == ISM_EVENT_REQUEST) { ev_info.code = ISM_EVENT_RESPONSE; wrk->smcd->ops->signal_event(wrk->smcd, - wrk->event.tok, + &peer_gid, ISM_EVENT_REQUEST_IR, ISM_EVENT_CODE_TESTLINK, ev_info.info); @@ -378,10 +381,13 @@ static void smc_ism_event_work(struct work_struct *work) { struct smc_ism_event_work *wrk = container_of(work, struct smc_ism_event_work, work); + struct smcd_gid smcd_gid = { .gid = wrk->event.tok, + .gid_ext = 0 }; switch (wrk->event.type) { case ISM_EVENT_GID: /* GID event, token is peer GID */ - smc_smcd_terminate(wrk->smcd, wrk->event.tok, VLAN_VID_MASK); + smcd_gid.gid = wrk->event.tok; + smc_smcd_terminate(wrk->smcd, &smcd_gid, VLAN_VID_MASK); break; case ISM_EVENT_DMB: break; @@ -530,7 +536,7 @@ int smc_ism_signal_shutdown(struct smc_link_group *lgr) memcpy(ev_info.uid, lgr->id, SMC_LGR_ID_SIZE); ev_info.vlan_id = lgr->vlan_id; ev_info.code = ISM_EVENT_REQUEST; - rc = lgr->smcd->ops->signal_event(lgr->smcd, lgr->peer_gid, + rc = lgr->smcd->ops->signal_event(lgr->smcd, &lgr->peer_gid, ISM_EVENT_REQUEST_IR, ISM_EVENT_CODE_SHUTDOWN, ev_info.info); diff --git a/net/smc/smc_ism.h b/net/smc/smc_ism.h index 2ecc8de..e6ea08c 100644 --- a/net/smc/smc_ism.h +++ b/net/smc/smc_ism.h @@ -33,7 +33,8 @@ struct smc_ism_vlanid { /* VLAN id set on ISM device */ struct smcd_dev; -int smc_ism_cantalk(u64 peer_gid, unsigned short vlan_id, struct smcd_dev *dev); +int smc_ism_cantalk(struct smcd_gid *peer_gid, unsigned short vlan_id, + struct smcd_dev *dev); void smc_ism_set_conn(struct smc_connection *conn); void smc_ism_unset_conn(struct smc_connection *conn); int smc_ism_get_vlan(struct smcd_dev *dev, unsigned short vlan_id); diff --git a/net/smc/smc_pnet.c b/net/smc/smc_pnet.c index 1177540..9f2c58c 100644 --- a/net/smc/smc_pnet.c +++ b/net/smc/smc_pnet.c @@ -1103,8 +1103,8 @@ static void smc_pnet_find_ism_by_pnetid(struct net_device *ndev, list_for_each_entry(ismdev, &smcd_dev_list.list, list) { if (smc_pnet_match(ismdev->pnetid, ndev_pnetid) && !ismdev->going_away && - (!ini->ism_peer_gid[0] || - !smc_ism_cantalk(ini->ism_peer_gid[0], ini->vlan_id, + (!ini->ism_peer_gid[0].gid || + !smc_ism_cantalk(&ini->ism_peer_gid[0], ini->vlan_id, ismdev))) { ini->ism_dev[0] = ismdev; break; From patchwork Sun Sep 24 15:16:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 144098 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp713603vqu; Sun, 24 Sep 2023 08:18:10 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHDDT1SzrexB/iTYwOS8p41MF0rnklY9r2jBHFIusLuEqVDQ+nMKSvK9ouiU5vyjmIEVVVR X-Received: by 2002:a17:902:9896:b0:1c3:73aa:618b with SMTP id s22-20020a170902989600b001c373aa618bmr4691261plp.9.1695568690295; Sun, 24 Sep 2023 08:18:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695568690; cv=none; d=google.com; s=arc-20160816; b=dqqjSIn4BZboI9Xc7rr/NfKssfeLAwxRc39pkFYtI8D/myAe3JkpgFgCw83A2xJLOw CLM0xhIaPxcIZ7yqF8PbKITb8bdY/bj5cqgzrCt2SFEsGQp+MPRUtPggCp/XA/U2P9WM pKlDY4TwocERhPJzFyWR370jldEcBRTiWV//ilXkpgfEBS8xJi8+5rAVGhpVOivKRneu iWAeKr495i8/T22vbJcVdIG+2SvI8DSbqBVtdeD2OIOknKpCES79GSnKKwoD1ot/anFf J9+5TSq8cy3JIspCxhrrdF5fSA1Jgw42UZfzUYM9VYbMTTzPDG2fTwYVzGoOHUd7JG7X KEAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=vaRV/SHmo38xfGxi/oWEK8KYykRR6d7p/u9gPXM7bTA=; fh=kTBstCixiri9Ahdk8UNuw7wmcx8Ry+mqafUdDsc8Vd0=; b=DwZATF2guYYUApl0RcdRE1WcGzvJ+oEUBrSMQKtcGiNAoJVl/xp6KKykVtNOv3HU1/ 1XqN2/EXbU6RNWSsyO/jt9/u+xKcNGk/f5MKkD6gs7bIoBke3bpdut79Am12NPGyBQ8L 1eySIPfUrGhjRCqDsK523KI7IrLDgvb+IWMTENFvG1Wyjatm4L+PVTp+lKW+BvLk5ppF gY8M9DITCrZRHdwWavduv+plLO0XYJbagzNl9l91PTsWiFUEciTMCLotNNz99Pg+7XVJ TXF+Yn8HYd9yB6nui1TCl4E62DgnLyTSl62Rl8tIw2CRteFHLzO0XEYuj3bR+6LnHfor Gegw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from morse.vger.email (morse.vger.email. [2620:137:e000::3:1]) by mx.google.com with ESMTPS id p17-20020a170902e75100b001c32d285db8si8393210plf.308.2023.09.24.08.18.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 08:18:10 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) client-ip=2620:137:e000::3:1; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id 44D9D8020B5D; Sun, 24 Sep 2023 08:18:05 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230137AbjIXPRi (ORCPT + 30 others); Sun, 24 Sep 2023 11:17:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46722 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229850AbjIXPRe (ORCPT ); Sun, 24 Sep 2023 11:17:34 -0400 Received: from out30-112.freemail.mail.aliyun.com (out30-112.freemail.mail.aliyun.com [115.124.30.112]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D9862180; Sun, 24 Sep 2023 08:17:17 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R131e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045176;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0VsinOab_1695568633; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsinOab_1695568633) by smtp.aliyun-inc.com; Sun, 24 Sep 2023 23:17:15 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: wintera@linux.ibm.com, schnelle@linux.ibm.com, gbayer@linux.ibm.com, pasic@linux.ibm.com, alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, dust.li@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v4 07/18] net/smc: disable SEID on non-s390 architecture Date: Sun, 24 Sep 2023 23:16:42 +0800 Message-Id: <1695568613-125057-8-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> References: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Sun, 24 Sep 2023 08:18:05 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777932634973605969 X-GMAIL-MSGID: 1777932634973605969 On s390 architecture SMC-Dv2 SEID is generated by ISMv2 device with partial cpuid. It is predefined and unique to represents the physical machine (CPC), the maximum communication space of SMC-D. On non-s390 archs like x86/64 or ARM, there is no similar information to identify physical machine, especially in virtualization scenarios like KVM, etc. In such cases, SEID is forcibly disabled and the user- defined UEID will be used to represent the communicable space. Signed-off-by: Wen Gu --- net/smc/smc_clc.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/net/smc/smc_clc.c b/net/smc/smc_clc.c index aeb9643..deb43db 100644 --- a/net/smc/smc_clc.c +++ b/net/smc/smc_clc.c @@ -155,10 +155,12 @@ static int smc_clc_ueid_remove(char *ueid) rc = 0; } } +#if IS_ENABLED(CONFIG_S390) if (!rc && !smc_clc_eid_table.ueid_cnt) { smc_clc_eid_table.seid_enabled = 1; rc = -EAGAIN; /* indicate success and enabling of seid */ } +#endif write_unlock(&smc_clc_eid_table.lock); return rc; } @@ -273,22 +275,30 @@ int smc_nl_dump_seid(struct sk_buff *skb, struct netlink_callback *cb) int smc_nl_enable_seid(struct sk_buff *skb, struct genl_info *info) { +#if IS_ENABLED(CONFIG_S390) write_lock(&smc_clc_eid_table.lock); smc_clc_eid_table.seid_enabled = 1; write_unlock(&smc_clc_eid_table.lock); return 0; +#else + return -EOPNOTSUPP; +#endif } int smc_nl_disable_seid(struct sk_buff *skb, struct genl_info *info) { int rc = 0; +#if IS_ENABLED(CONFIG_S390) write_lock(&smc_clc_eid_table.lock); if (!smc_clc_eid_table.ueid_cnt) rc = -ENOENT; else smc_clc_eid_table.seid_enabled = 0; write_unlock(&smc_clc_eid_table.lock); +#else + rc = -EOPNOTSUPP; +#endif return rc; } @@ -1292,7 +1302,11 @@ void __init smc_clc_init(void) INIT_LIST_HEAD(&smc_clc_eid_table.list); rwlock_init(&smc_clc_eid_table.lock); smc_clc_eid_table.ueid_cnt = 0; +#if IS_ENABLED(CONFIG_S390) smc_clc_eid_table.seid_enabled = 1; +#else + smc_clc_eid_table.seid_enabled = 0; +#endif } void smc_clc_exit(void) From patchwork Sun Sep 24 15:16:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 144097 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp713599vqu; Sun, 24 Sep 2023 08:18:10 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFAnMVoa0brTHO8OIT3KAqziuLpIRt2AvzuktZvnNQA6T3P0Cmbg7VGQqPx0L13vthfDgXX X-Received: by 2002:a17:90b:3d1:b0:268:3f2d:66e4 with SMTP id go17-20020a17090b03d100b002683f2d66e4mr2583543pjb.37.1695568689868; Sun, 24 Sep 2023 08:18:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695568689; cv=none; d=google.com; s=arc-20160816; b=uxVEfoCcpwx4V8rH7KiYF9ud7Plt8t9J0+tjyNx39bLpumFFZZ/Ph+qPVx4HLjnJLU M0bW4PQ1CNjYbllmallE1DRtc4nowIbDrBZbbfOvvM6Ii+EcTABo5z/xrlsMjMEIssX0 PUsPrMZH38zD9gCDG3CdZhCGuadI6GUxiyNP1QEISam3sQhqMT9XHIaHObWkAWHddN5f ikm2Nnufdruv6S9lBFypjcCPsZqY6l2yPMlngZoekKpEl66gTMTyNIWZQJVMoDedlWv2 lIk1s/UW3EfBjSCU7pwxaWeaP8NgVAZH6DNksw+y485//CdRzWrJ8bwUTz8etk62YTAh ZWKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=EYcGpfMcAYSbZ7LiIeLnw9Do+HzrOWthPw6TqqTf33M=; fh=kTBstCixiri9Ahdk8UNuw7wmcx8Ry+mqafUdDsc8Vd0=; b=XwK8oUW1PgC5RS6tawVzHd+wG69KJPEV6QEp2od17atF9EwljAUnMlMg3Exme9/OxV XUkz71CqNUOXuYGeM4WtUyO0n6P/Zmlr0BpLdyqlaMChpcZskh2Z/utrwo0q0wTXNAZn hC0hpT+zkNL5KEgXbVRIJ7psxaxwarXSSBDkh+yJVYnDMz+294lEanQk7JnXKTJxx3IO 2BFNK6Ugb/wRU7W4kqQouMn12iziQNHUhHHQRDYgdXv13SQ6ZT0if0usaBRJLO1bitur X2otMRIrKrzHgntgSOALCQdchn6ekmg88um/FPD18NjnBlSFH2i+RZfeQoxelpZYvo8d Y9lg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from lipwig.vger.email (lipwig.vger.email. [2620:137:e000::3:3]) by mx.google.com with ESMTPS id om2-20020a17090b3a8200b0026d0f7c6b17si11390774pjb.30.2023.09.24.08.18.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 08:18:09 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 5D88080309BB; Sun, 24 Sep 2023 08:18:06 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230314AbjIXPRx (ORCPT + 30 others); Sun, 24 Sep 2023 11:17:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46818 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230024AbjIXPRi (ORCPT ); Sun, 24 Sep 2023 11:17:38 -0400 Received: from out30-97.freemail.mail.aliyun.com (out30-97.freemail.mail.aliyun.com [115.124.30.97]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DF2E4CE5; Sun, 24 Sep 2023 08:17:23 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R771e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0Vsir4-C_1695568635; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0Vsir4-C_1695568635) by smtp.aliyun-inc.com; Sun, 24 Sep 2023 23:17:17 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: wintera@linux.ibm.com, schnelle@linux.ibm.com, gbayer@linux.ibm.com, pasic@linux.ibm.com, alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, dust.li@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v4 08/18] net/smc: enable virtual ISM device feature bit Date: Sun, 24 Sep 2023 23:16:43 +0800 Message-Id: <1695568613-125057-9-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> References: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Sun, 24 Sep 2023 08:18:06 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777932634836385534 X-GMAIL-MSGID: 1777932634836385534 This patch enables the first supplement feature of SMCv2.1, that is the support for virtual ISM devices. Signed-off-by: Wen Gu --- net/smc/af_smc.c | 2 ++ net/smc/smc.h | 7 +++++++ 2 files changed, 9 insertions(+) diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index 5bb41404..c5b7716 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -1528,6 +1528,7 @@ static int __smc_connect(struct smc_sock *smc) ini->smcr_version = SMC_V1 | SMC_V2; ini->smc_type_v1 = SMC_TYPE_B; ini->smc_type_v2 = SMC_TYPE_B; + ini->feature_mask = SMC_FEATURE_MASK; /* get vlan id from IP device */ if (smc_vlan_by_tcpsk(smc->clcsock, ini)) { @@ -1993,6 +1994,7 @@ static int smc_listen_v2_check(struct smc_sock *new_smc, ini->smc_type_v2 = pclc->hdr.typev2; ini->smcd_version = smcd_indicated(ini->smc_type_v1) ? SMC_V1 : 0; ini->smcr_version = smcr_indicated(ini->smc_type_v1) ? SMC_V1 : 0; + ini->feature_mask = SMC_FEATURE_MASK; if (pclc->hdr.version > SMC_V1) { if (smcd_indicated(ini->smc_type_v2)) ini->smcd_version |= SMC_V2; diff --git a/net/smc/smc.h b/net/smc/smc.h index 24745fd..71b9640 100644 --- a/net/smc/smc.h +++ b/net/smc/smc.h @@ -58,6 +58,13 @@ enum smc_state { /* possible states of an SMC socket */ SMC_PROCESSABORT = 27, }; +enum smc_supp_feature { /* supplemental features supported by current version */ + SMC_SPF_VIRT_ISM_DEV = 0, +}; + +#define SMC_FEATURE_MASK \ + (BIT(SMC_SPF_VIRT_ISM_DEV)) + struct smc_link_group; struct smc_wr_rx_hdr { /* common prefix part of LLC and CDC to demultiplex */ From patchwork Sun Sep 24 15:16:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 144102 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp713782vqu; Sun, 24 Sep 2023 08:18:33 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFpHShkyWik+o9y2gizFnEQAlBXVY/0A4Y0NRP0AYkzNv5hgt9TalcH0rCOTljsHlGfpBjj X-Received: by 2002:a05:6358:618c:b0:141:162:b0cd with SMTP id w12-20020a056358618c00b001410162b0cdmr6599863rww.19.1695568713389; Sun, 24 Sep 2023 08:18:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695568713; cv=none; d=google.com; s=arc-20160816; b=Z7xM7u7JnsFeByjW2NwTEWiQAEel1F1vxB6t0n/qrcOTGG5wD4OXSI1O07cxxIunGe Q2EC5sQciTp5LGzi0VPoBrxBjlsVEWSQ6oD6ycP8NqgLlijzyKlB5/vRCryQZ0ipLs15 q6kWeCgDECGvaZ3zBgN8EFB+ug6gTt1kB+vnKD6DGNL6qHwVmb3nYm4SUlS+w3jwSMu8 Dyou7qGSQM9yyJlU02xxRrZ3u/4IV1yGKxAm40TX7CXrLEUS3e51O0RRwj9GV4x/PnAX ep+Wo+8DVzLKKW/thnAMMBOJnAlMvKTfMzH0h29SKiEile2ccTbd5u6qkP85ui/wfy77 XW6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=8GkqB492CzHIKkUsqleJGC8HiGJS1kdyiDmCLmYvO9I=; fh=kTBstCixiri9Ahdk8UNuw7wmcx8Ry+mqafUdDsc8Vd0=; b=ppKTDEnGerTmonY9R6Yl72hAggDv6kcjoMv2P0m8OpDkXBN4eGjTYsCOmmNBMV2Eb8 eDoLpuytuGtmrhcwhnEpFICPLzjiBYJPC+bTj2USxk1Gcf01XVOOxFN3z2bBfiO+CiMW HvUklu8D4Hu++F8XF8uI22x7mdVx39NNeHr7R2UbN8cLEfHOo9chCVK9O3cqlnUI2FfA cF0t6SkbuWeljvLYzQoqo4vg6hagn22Ulw07JGNV5GwyIy8YHLPzcYyV0C9S/LX0MRMQ yVW/CWN1tMTDShD+5g7uMPF8OW7vSgZaajQbudejrU6G8wtIgMRyMTmI3onV9p5C0if9 Fx0Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from agentk.vger.email (agentk.vger.email. [2620:137:e000::3:2]) by mx.google.com with ESMTPS id n67-20020a632746000000b0057759a5b7ccsi8529195pgn.305.2023.09.24.08.18.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 08:18:33 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by agentk.vger.email (Postfix) with ESMTP id 7D201819E4F6; Sun, 24 Sep 2023 08:18:25 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at agentk.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230188AbjIXPR4 (ORCPT + 30 others); Sun, 24 Sep 2023 11:17:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47010 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229775AbjIXPRi (ORCPT ); Sun, 24 Sep 2023 11:17:38 -0400 Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 65D3ECD7; Sun, 24 Sep 2023 08:17:22 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R131e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045168;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0VsilXj7_1695568638; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsilXj7_1695568638) by smtp.aliyun-inc.com; Sun, 24 Sep 2023 23:17:20 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: wintera@linux.ibm.com, schnelle@linux.ibm.com, gbayer@linux.ibm.com, pasic@linux.ibm.com, alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, dust.li@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v4 09/18] net/smc: introduce SMC-D loopback device Date: Sun, 24 Sep 2023 23:16:44 +0800 Message-Id: <1695568613-125057-10-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> References: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Sun, 24 Sep 2023 08:18:25 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777932659124052986 X-GMAIL-MSGID: 1777932659124052986 This patch introduces a kind of loopback device for SMC-D. The device is created when SMC module is loaded and destroyed when the SMC module is unloaded. The loopback device is a kernel device used only by the SMC module and is not restricted by net namespace, so it can be used for local inter-process or inter-container communication. Signed-off-by: Wen Gu --- net/smc/Kconfig | 13 ++++ net/smc/Makefile | 2 +- net/smc/af_smc.c | 12 +++- net/smc/smc_loopback.c | 165 +++++++++++++++++++++++++++++++++++++++++++++++++ net/smc/smc_loopback.h | 33 ++++++++++ 5 files changed, 223 insertions(+), 2 deletions(-) create mode 100644 net/smc/smc_loopback.c create mode 100644 net/smc/smc_loopback.h diff --git a/net/smc/Kconfig b/net/smc/Kconfig index 1ab3c5a..3e9e03e 100644 --- a/net/smc/Kconfig +++ b/net/smc/Kconfig @@ -19,3 +19,16 @@ config SMC_DIAG smcss. if unsure, say Y. + +config SMC_LO + bool "SMC_LO: virtual loopback-ism for SMC" + depends on SMC + default n + help + SMC_LO provides a kind of virtual ISM device called loopback-ism + for SMC-D to upgrade AF_INET TCP connections whose ends share the + same kernel. + loopback-ism is a software-implemented simulation device that does + not depend on a specific architecture or hardware. + + if unsure, say N. diff --git a/net/smc/Makefile b/net/smc/Makefile index 875efcd..a8c3711 100644 --- a/net/smc/Makefile +++ b/net/smc/Makefile @@ -4,5 +4,5 @@ obj-$(CONFIG_SMC) += smc.o obj-$(CONFIG_SMC_DIAG) += smc_diag.o smc-y := af_smc.o smc_pnet.o smc_ib.o smc_clc.o smc_core.o smc_wr.o smc_llc.o smc-y += smc_cdc.o smc_tx.o smc_rx.o smc_close.o smc_ism.o smc_netlink.o smc_stats.o -smc-y += smc_tracepoint.o +smc-y += smc_tracepoint.o smc_loopback.o smc-$(CONFIG_SYSCTL) += smc_sysctl.o diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index c5b7716..6435659 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -53,6 +53,7 @@ #include "smc_stats.h" #include "smc_tracepoint.h" #include "smc_sysctl.h" +#include "smc_loopback.h" static DEFINE_MUTEX(smc_server_lgr_pending); /* serialize link group * creation on server @@ -3565,15 +3566,23 @@ static int __init smc_init(void) goto out_sock; } + rc = smc_loopback_init(); + if (rc) { + pr_err("%s: smc_loopback_init fails with %d\n", __func__, rc); + goto out_ib; + } + rc = tcp_register_ulp(&smc_ulp_ops); if (rc) { pr_err("%s: tcp_ulp_register fails with %d\n", __func__, rc); - goto out_ib; + goto out_lo; } static_branch_enable(&tcp_have_smc); return 0; +out_lo: + smc_loopback_exit(); out_ib: smc_ib_unregister_client(); out_sock: @@ -3611,6 +3620,7 @@ static void __exit smc_exit(void) tcp_unregister_ulp(&smc_ulp_ops); sock_unregister(PF_SMC); smc_core_exit(); + smc_loopback_exit(); smc_ib_unregister_client(); smc_ism_exit(); destroy_workqueue(smc_close_wq); diff --git a/net/smc/smc_loopback.c b/net/smc/smc_loopback.c new file mode 100644 index 0000000..4631236 --- /dev/null +++ b/net/smc/smc_loopback.c @@ -0,0 +1,165 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Shared Memory Communications Direct over loopback device. + * + * Provide a SMC-D loopback dummy device. + * + * Copyright (c) 2022, Alibaba Inc. + * + * Author: Wen Gu + * Tony Lu + * + */ + +#include +#include +#include + +#include "smc_ism.h" +#include "smc_loopback.h" + +#if IS_ENABLED(CONFIG_SMC_LO) +static const char smc_lo_dev_name[] = "smc_lo"; +static struct smc_lo_dev *lo_dev; + +static const struct smcd_ops lo_ops = { + .query_remote_gid = NULL, + .register_dmb = NULL, + .unregister_dmb = NULL, + .add_vlan_id = NULL, + .del_vlan_id = NULL, + .set_vlan_required = NULL, + .reset_vlan_required = NULL, + .signal_event = NULL, + .move_data = NULL, + .supports_v2 = NULL, + .get_system_eid = NULL, + .get_local_gid = NULL, + .get_chid = NULL, + .get_dev = NULL, +}; + +static struct smcd_dev *smcd_lo_alloc_dev(const struct smcd_ops *ops, + int max_dmbs) +{ + struct smcd_dev *smcd; + + smcd = kzalloc(sizeof(*smcd), GFP_KERNEL); + if (!smcd) + return NULL; + + smcd->conn = kcalloc(max_dmbs, sizeof(struct smc_connection *), + GFP_KERNEL); + if (!smcd->conn) + goto out_smcd; + + smcd->ops = ops; + + spin_lock_init(&smcd->lock); + spin_lock_init(&smcd->lgr_lock); + INIT_LIST_HEAD(&smcd->vlan); + INIT_LIST_HEAD(&smcd->lgr_list); + init_waitqueue_head(&smcd->lgrs_deleted); + return smcd; + +out_smcd: + kfree(smcd); + return NULL; +} + +static int smcd_lo_register_dev(struct smc_lo_dev *ldev) +{ + struct smcd_dev *smcd; + + smcd = smcd_lo_alloc_dev(&lo_ops, SMC_LODEV_MAX_DMBS); + if (!smcd) + return -ENOMEM; + + ldev->smcd = smcd; + smcd->priv = ldev; + + /* TODO: + * register smc_lo to smcd_dev list. + */ + return 0; +} + +static void smcd_lo_unregister_dev(struct smc_lo_dev *ldev) +{ + /* TODO: + * unregister smc_lo from smcd_dev list. + */ +} + +static void smc_lo_dev_release(struct device *dev) +{ + struct smc_lo_dev *ldev = + container_of(dev, struct smc_lo_dev, dev); + struct smcd_dev *smcd = ldev->smcd; + + kfree(smcd->conn); + kfree(smcd); + kfree(ldev); +} + +static int smc_lo_dev_init(struct smc_lo_dev *ldev) +{ + return smcd_lo_register_dev(ldev); +} + +static int smc_lo_dev_probe(void) +{ + struct smc_lo_dev *ldev; + int ret; + + ldev = kzalloc(sizeof(*ldev), GFP_KERNEL); + if (!ldev) + return -ENOMEM; + + ldev->dev.parent = NULL; + ldev->dev.release = smc_lo_dev_release; + device_initialize(&ldev->dev); + dev_set_name(&ldev->dev, smc_lo_dev_name); + + ret = smc_lo_dev_init(ldev); + if (ret) + goto free_dev; + + lo_dev = ldev; /* global loopback device */ + return 0; + +free_dev: + kfree(ldev); + return ret; +} + +static void smc_lo_dev_exit(struct smc_lo_dev *ldev) +{ + smcd_lo_unregister_dev(ldev); +} + +static void smc_lo_dev_remove(void) +{ + if (!lo_dev) + return; + + smc_lo_dev_exit(lo_dev); + put_device(&lo_dev->dev); /* device_initialize in smc_lo_dev_probe */ +} +#endif + +int smc_loopback_init(void) +{ +#if IS_ENABLED(CONFIG_SMC_LO) + return smc_lo_dev_probe(); +#else + return 0; +#endif +} + +void smc_loopback_exit(void) +{ +#if IS_ENABLED(CONFIG_SMC_LO) + smc_lo_dev_remove(); +#endif +} diff --git a/net/smc/smc_loopback.h b/net/smc/smc_loopback.h new file mode 100644 index 0000000..39aa1dc --- /dev/null +++ b/net/smc/smc_loopback.h @@ -0,0 +1,33 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Shared Memory Communications Direct over loopback device. + * + * Provide a SMC-D loopback dummy device. + * + * Copyright (c) 2022, Alibaba Inc. + * + * Author: Wen Gu + * Tony Lu + * + */ + +#ifndef _SMC_LOOPBACK_H +#define _SMC_LOOPBACK_H + +#include +#include +#include + +#if IS_ENABLED(CONFIG_SMC_LO) +#define SMC_LODEV_MAX_DMBS 5000 + +struct smc_lo_dev { + struct smcd_dev *smcd; + struct device dev; +}; +#endif + +int smc_loopback_init(void); +void smc_loopback_exit(void); + +#endif /* _SMC_LOOPBACK_H */ From patchwork Sun Sep 24 15:16:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 144101 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp713764vqu; Sun, 24 Sep 2023 08:18:32 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF+T0/1oS05ztcSPxIzuEs22rO5n33Z0HYwWc87GEZst2msv06AU0kCI/XA/bKHvmSu9sfj X-Received: by 2002:a05:6a00:2d08:b0:68f:c6f8:144a with SMTP id fa8-20020a056a002d0800b0068fc6f8144amr3477212pfb.22.1695568711796; Sun, 24 Sep 2023 08:18:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695568711; cv=none; d=google.com; s=arc-20160816; b=Qrkqy8npA12Jm5GhyRlgpM+bYjkrxzcpPwkT7S0a6aN0yp9SLKKxr7s4cxuiZc7hb+ FoqHIK+hLIuRS8qVMkOniLXED5tqi6hfJADhBUfh0qqipOmE84325qL130ZwQbIvz7GG qhco/7lEJ+6mJvHsvdyjnP+lBO6xbU9O5e2YpAiDLjVz+7HrA1Ae2l6OVMvyhHcFklVF IOeMPCtrQYIYddtHqUeiCio5dF4aG8hDum55Sv2Fq9L3MWfmb5m6fItF712qPwfxz+UU 3oib/jZGLi3bjlH6kDi/f9D5+9Tj75vt1iHV76ik7H04askw5ayvlFbbEN7GuUPRccHE FN7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=J3CC/dxNK9rOK9zCkmn8/gsWsupUw3smyWvw/3/8tiU=; fh=kTBstCixiri9Ahdk8UNuw7wmcx8Ry+mqafUdDsc8Vd0=; b=px5Wh0/bNTvFuS+JqNrUhywcopJhHGuNVZ+uJRzr6HFbDyllbQ0ozUHSEynG7SI6wW YQcj3hHv1KbdJkZ77koNIVioGucK+6fO4Jcb1geXMzY2AGKCzsU7YTreqfQuX2oUdc4s hgBP0WZWZiPeqegcwPqh8QRIAe+klbGoFVHXiuj2wJgymr9hpnoqQbv1Zp7tD8SFyon5 XTrb5DJflp78FVoljOjYRGyr+qqkdLBwNDv5cZtHVbxyZs6yvWyT4oDXcn4MPMfLTlkN YO7z9dPbJmKCWk7EtVuEz4aWrri+cLFJjESYb3J1gS30xI3myoW/xumD3ANC3bT9wXaH +2yw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from morse.vger.email (morse.vger.email. [23.128.96.31]) by mx.google.com with ESMTPS id bq8-20020a056a000e0800b0068e3950c705si7926881pfb.24.2023.09.24.08.18.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 08:18:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id 3171E8022453; Sun, 24 Sep 2023 08:18:27 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229459AbjIXPR7 (ORCPT + 30 others); Sun, 24 Sep 2023 11:17:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47128 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230163AbjIXPRk (ORCPT ); Sun, 24 Sep 2023 11:17:40 -0400 Received: from out30-112.freemail.mail.aliyun.com (out30-112.freemail.mail.aliyun.com [115.124.30.112]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3A9F9CF4; Sun, 24 Sep 2023 08:17:24 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R211e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046059;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0Vsio7wU_1695568640; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0Vsio7wU_1695568640) by smtp.aliyun-inc.com; Sun, 24 Sep 2023 23:17:22 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: wintera@linux.ibm.com, schnelle@linux.ibm.com, gbayer@linux.ibm.com, pasic@linux.ibm.com, alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, dust.li@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v4 10/18] net/smc: implement ID-related operations of loopback Date: Sun, 24 Sep 2023 23:16:45 +0800 Message-Id: <1695568613-125057-11-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> References: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Sun, 24 Sep 2023 08:18:27 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777932657668791988 X-GMAIL-MSGID: 1777932657668791988 This patch implements GID/CHID/SEID related operations of SMC-D loopback device. In loopback device, GID is generated by UUIDv4 algorithm, CHID is reserved 0xFFFF, SEID is generated using the same algorithm as ISM device under s390 architecture, and is 0 and disabled under non-s390 architecture. Signed-off-by: Wen Gu --- include/net/smc.h | 6 ++++ net/smc/smc_ism.h | 2 ++ net/smc/smc_loopback.c | 96 ++++++++++++++++++++++++++++++++++++++++++++++---- net/smc/smc_loopback.h | 3 ++ 4 files changed, 101 insertions(+), 6 deletions(-) diff --git a/include/net/smc.h b/include/net/smc.h index d8db5e1..7c2d35c 100644 --- a/include/net/smc.h +++ b/include/net/smc.h @@ -41,6 +41,12 @@ struct smcd_dmb { dma_addr_t dma_addr; }; +struct smcd_seid { + u8 seid_string[24]; + u8 serial_number[4]; + u8 type[4]; +}; + #define ISM_EVENT_DMB 0 #define ISM_EVENT_GID 1 #define ISM_EVENT_SWR 2 diff --git a/net/smc/smc_ism.h b/net/smc/smc_ism.h index e6ea08c..7ab82dd 100644 --- a/net/smc/smc_ism.h +++ b/net/smc/smc_ism.h @@ -15,6 +15,8 @@ #include "smc.h" +#define S390_ISM_IDENT_MASK 0x00FFFF + #define SMC_VIRT_ISM_CHID_MAX 0xFFFF #define SMC_VIRT_ISM_CHID_MIN 0xFF00 diff --git a/net/smc/smc_loopback.c b/net/smc/smc_loopback.c index 4631236..610af99 100644 --- a/net/smc/smc_loopback.c +++ b/net/smc/smc_loopback.c @@ -13,17 +13,99 @@ #include #include +#include #include #include "smc_ism.h" #include "smc_loopback.h" #if IS_ENABLED(CONFIG_SMC_LO) +#define SMC_LO_SUPPORTS_V2 0x1 /* SMC-D loopback supports SMC-Dv2 */ + static const char smc_lo_dev_name[] = "smc_lo"; +static struct smcd_seid SMC_LO_SEID = { + .seid_string = "IBM-SYSZ-ISMSEID00000000", + .serial_number = "0000", + .type = "0000", +}; + static struct smc_lo_dev *lo_dev; +static void smc_lo_create_seid(struct smcd_seid *seid) +{ +#if IS_ENABLED(CONFIG_S390) + struct cpuid id; + u16 ident_tail; + char tmp[5]; + + get_cpu_id(&id); + ident_tail = (u16)(id.ident & S390_ISM_IDENT_MASK); + snprintf(tmp, 5, "%04X", ident_tail); + memcpy(&seid->serial_number, tmp, 4); + snprintf(tmp, 5, "%04X", id.machine); + memcpy(&seid->type, tmp, 4); +#else + memset(seid, 0, SMC_MAX_EID_LEN); +#endif +} + +static void smc_lo_generate_id(struct smc_lo_dev *ldev) +{ + struct smcd_gid *lgid = &ldev->local_gid; + uuid_t uuid; + + uuid_gen(&uuid); + memcpy(&lgid->gid, &uuid, sizeof(lgid->gid)); + memcpy(&lgid->gid_ext, (u8 *)&uuid + sizeof(lgid->gid), + sizeof(lgid->gid_ext)); + + ldev->chid = SMC_LO_CHID; + smc_lo_create_seid(&SMC_LO_SEID); +} + +static int smc_lo_query_rgid(struct smcd_dev *smcd, struct smcd_gid *rgid, + u32 vid_valid, u32 vid) +{ + struct smc_lo_dev *ldev = smcd->priv; + + /* rgid should equal to lgid in loopback */ + if (!ldev || rgid->gid != ldev->local_gid.gid || + rgid->gid_ext != ldev->local_gid.gid_ext) + return -ENETUNREACH; + return 0; +} + +static int smc_lo_supports_v2(void) +{ + return SMC_LO_SUPPORTS_V2; +} + +static u8 *smc_lo_get_system_eid(void) +{ + return SMC_LO_SEID.seid_string; +} + +static void smc_lo_get_local_gid(struct smcd_dev *smcd, + struct smcd_gid *smcd_gid) +{ + struct smc_lo_dev *ldev = smcd->priv; + + smcd_gid->gid = ldev->local_gid.gid; + smcd_gid->gid_ext = ldev->local_gid.gid_ext; +} + +static u16 smc_lo_get_chid(struct smcd_dev *smcd) +{ + return ((struct smc_lo_dev *)smcd->priv)->chid; +} + +static struct device *smc_lo_get_dev(struct smcd_dev *smcd) +{ + return &((struct smc_lo_dev *)smcd->priv)->dev; +} + static const struct smcd_ops lo_ops = { - .query_remote_gid = NULL, + .query_remote_gid = smc_lo_query_rgid, .register_dmb = NULL, .unregister_dmb = NULL, .add_vlan_id = NULL, @@ -32,11 +114,11 @@ .reset_vlan_required = NULL, .signal_event = NULL, .move_data = NULL, - .supports_v2 = NULL, - .get_system_eid = NULL, - .get_local_gid = NULL, - .get_chid = NULL, - .get_dev = NULL, + .supports_v2 = smc_lo_supports_v2, + .get_system_eid = smc_lo_get_system_eid, + .get_local_gid = smc_lo_get_local_gid, + .get_chid = smc_lo_get_chid, + .get_dev = smc_lo_get_dev, }; static struct smcd_dev *smcd_lo_alloc_dev(const struct smcd_ops *ops, @@ -104,6 +186,8 @@ static void smc_lo_dev_release(struct device *dev) static int smc_lo_dev_init(struct smc_lo_dev *ldev) { + smc_lo_generate_id(ldev); + return smcd_lo_register_dev(ldev); } diff --git a/net/smc/smc_loopback.h b/net/smc/smc_loopback.h index 39aa1dc..71a52da 100644 --- a/net/smc/smc_loopback.h +++ b/net/smc/smc_loopback.h @@ -19,11 +19,14 @@ #include #if IS_ENABLED(CONFIG_SMC_LO) +#define SMC_LO_CHID 0xFFFF #define SMC_LODEV_MAX_DMBS 5000 struct smc_lo_dev { struct smcd_dev *smcd; struct device dev; + u16 chid; + struct smcd_gid local_gid; }; #endif From patchwork Sun Sep 24 15:16:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 144103 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp713808vqu; Sun, 24 Sep 2023 08:18:35 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFSwoyohnOvzbo/bpSseH53gg/+UoS2ecmYMEAuZohMvPT6fUaeDSVzFDepftw+6cnGwI3R X-Received: by 2002:a17:903:1cf:b0:1c2:1068:1f4f with SMTP id e15-20020a17090301cf00b001c210681f4fmr4461041plh.17.1695568715336; Sun, 24 Sep 2023 08:18:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695568715; cv=none; d=google.com; s=arc-20160816; b=BWNInnu4zfA3jQQ0cRgWLWHchajjA4VKs3Sbeeqrl1VNWFCjqrwLtypFNz96IMlqsh ay2kOjUdmfLzyM4qf7FqHYw+vWVRGvQB9ls15mKlc//bIomc+enBdlpHr6eQCiZbwgPe +ef/YZ/ZNWMNFzdHbaNRlKLVtsM3UlPsJCGvF6+pagMDFGgtNpDiLjaYYvIQpQJSdfJu +lkE61Jr9gI2M7oBTLs3vJuEOukdMGGQ1iEXiSEwc4vW0fp52LUt827h07zAl/7TdVYT tKT74OycRNrzZYKNFZ+eRMCFpRfxiW1rpClo0JXLyd/oCwEnl6X6OT2o3UO738R5Nfe3 oBnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=5h9h2mmOafXNsXHkeoa/34jwI5r0oX1R8ItoaaA4Wtc=; fh=kTBstCixiri9Ahdk8UNuw7wmcx8Ry+mqafUdDsc8Vd0=; b=E/PM2me7sdLkhjgBUIeymVBQ0zOy6yKVvSMtNmQFCIK0T5Pco0hZPIkZs0YYNsOViP zE66FcU+t/86qhUdXQ27M9E8WRfkCLCNGSlbYnyQMUtmP5h+dJXe6QSzGLe6puZjsY0P SjxM0N49TYF1D3pBUMnVEM49OpVCwMNXo9JcZUI9Ifn2qLPpZzlPU4s+NQugEmJMmUz8 P+BGC81hGe+a/79kc7+qJ+aMlzojUkSz4OrGhZsB0Kj3hZtx4FmUFvBEGcEA3ReCXjXu B7nsWbdbv8HuY+WV20Zdt/7BFaqqxuNfoL96sB8a2AjmLAtXzWu0MGkNUwPUT4ok9NuV bzyg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from morse.vger.email (morse.vger.email. [2620:137:e000::3:1]) by mx.google.com with ESMTPS id be8-20020a170902aa0800b001bb0ba81053si7619585plb.50.2023.09.24.08.18.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 08:18:35 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) client-ip=2620:137:e000::3:1; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id 7CF21802246C; Sun, 24 Sep 2023 08:18:31 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230329AbjIXPSI (ORCPT + 30 others); Sun, 24 Sep 2023 11:18:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59062 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230226AbjIXPRu (ORCPT ); Sun, 24 Sep 2023 11:17:50 -0400 Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com [115.124.30.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C008CE5F; Sun, 24 Sep 2023 08:17:28 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R181e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0VsinOcr_1695568642; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsinOcr_1695568642) by smtp.aliyun-inc.com; Sun, 24 Sep 2023 23:17:24 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: wintera@linux.ibm.com, schnelle@linux.ibm.com, gbayer@linux.ibm.com, pasic@linux.ibm.com, alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, dust.li@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v4 11/18] net/smc: implement some unsupported operations of loopback Date: Sun, 24 Sep 2023 23:16:46 +0800 Message-Id: <1695568613-125057-12-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> References: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Sun, 24 Sep 2023 08:18:31 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777932661181315798 X-GMAIL-MSGID: 1777932661181315798 Vlan operations are not supported currently since the need for vlan in loopback does not seem to be strong. Signal_event operation is not supported since no event now needs to be processed by loopback device. Signed-off-by: Wen Gu --- net/smc/smc_loopback.c | 36 +++++++++++++++++++++++++++++++----- 1 file changed, 31 insertions(+), 5 deletions(-) diff --git a/net/smc/smc_loopback.c b/net/smc/smc_loopback.c index 610af99..4341803 100644 --- a/net/smc/smc_loopback.c +++ b/net/smc/smc_loopback.c @@ -75,6 +75,32 @@ static int smc_lo_query_rgid(struct smcd_dev *smcd, struct smcd_gid *rgid, return 0; } +static int smc_lo_add_vlan_id(struct smcd_dev *smcd, u64 vlan_id) +{ + return -EOPNOTSUPP; +} + +static int smc_lo_del_vlan_id(struct smcd_dev *smcd, u64 vlan_id) +{ + return -EOPNOTSUPP; +} + +static int smc_lo_set_vlan_required(struct smcd_dev *smcd) +{ + return -EOPNOTSUPP; +} + +static int smc_lo_reset_vlan_required(struct smcd_dev *smcd) +{ + return -EOPNOTSUPP; +} + +static int smc_lo_signal_event(struct smcd_dev *dev, struct smcd_gid *rgid, + u32 trigger_irq, u32 event_code, u64 info) +{ + return 0; +} + static int smc_lo_supports_v2(void) { return SMC_LO_SUPPORTS_V2; @@ -108,11 +134,11 @@ static struct device *smc_lo_get_dev(struct smcd_dev *smcd) .query_remote_gid = smc_lo_query_rgid, .register_dmb = NULL, .unregister_dmb = NULL, - .add_vlan_id = NULL, - .del_vlan_id = NULL, - .set_vlan_required = NULL, - .reset_vlan_required = NULL, - .signal_event = NULL, + .add_vlan_id = smc_lo_add_vlan_id, + .del_vlan_id = smc_lo_del_vlan_id, + .set_vlan_required = smc_lo_set_vlan_required, + .reset_vlan_required = smc_lo_reset_vlan_required, + .signal_event = smc_lo_signal_event, .move_data = NULL, .supports_v2 = smc_lo_supports_v2, .get_system_eid = smc_lo_get_system_eid, From patchwork Sun Sep 24 15:16:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 144100 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp713713vqu; Sun, 24 Sep 2023 08:18:25 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFtHnaftRPE+eMZs9jRr76USZTzJZCwJYnD8GhmuKwDs46DhCe0eiTOm1qVWb2DAoXBpFQ1 X-Received: by 2002:a17:902:a415:b0:1c5:fab7:70ac with SMTP id p21-20020a170902a41500b001c5fab770acmr4052873plq.27.1695568705461; Sun, 24 Sep 2023 08:18:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695568705; cv=none; d=google.com; s=arc-20160816; b=JXwkUjGENj1hpRSKFNpodDcYl+fC999kL3Jb3a5IwVTprrZmh9E/fv5C14V/DDbX/k HmDTnLWG77N9lyF81Xiciajut9iIrZeelygFNXBz6j/0F9eRMisMpPcNYrYtpXGyhYXU OIwSPKN2F4QVQnZdmSeqG59/j3iC++JgzKCxLTwBc61yCpIik1x+Mm5CMWuHFb0hiSUp 26P4eWj+9gio9ksVVU0Dbk5XFqOdsDI6RYen5XnHmrqa3d4lhzqSnGcT3xck66oZz0xP CfVVhWB4AWGvNkmJo4ANNgcVZQpJ9i937xo+nJ8tGqeCJZPO7pQT0PD5WKTHsrV+hVsC 0kyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=VPl8EozI8ujlUxx79dluXkHLtjDHFV/jRlRABELJYfI=; fh=kTBstCixiri9Ahdk8UNuw7wmcx8Ry+mqafUdDsc8Vd0=; b=uas2uBNI9bV3dH8SLQhaxaWnveLavhmmrCaNQUhzxmxXnTbbNRVS8q0IkNi+Ttexzx EOmIGqJ7A0n6IdGo3FR0b7my8eRxejBjvfXzPBd01nUz5JP3m7S1TC74RWvMMMPy3vUg VPcyM0sLsxEO4/JunE4JjROsaGaroZIkWcrwrEznaKL4o59jWVymTZv6HdOctEPU1ic5 OjdRD4mTPigjdOX4hPWn9TwMORNCFHHSLP8pu1jukz5+ENt0/hKUgzSiacjrUptIzMlT ZiEaP8SeNrWDNGeuFaezRbQNTWp4LDhQBkb54nOAr19278BTgWvGaB3zfNqMpNcKW2iS IUCg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id p2-20020a170902e74200b001c613091af5si1168533plf.563.2023.09.24.08.18.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 08:18:25 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id C57A0816038D; Sun, 24 Sep 2023 08:18:21 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230143AbjIXPSL (ORCPT + 30 others); Sun, 24 Sep 2023 11:18:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230241AbjIXPRu (ORCPT ); Sun, 24 Sep 2023 11:17:50 -0400 Received: from out30-98.freemail.mail.aliyun.com (out30-98.freemail.mail.aliyun.com [115.124.30.98]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D5999E69; Sun, 24 Sep 2023 08:17:29 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R181e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046049;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0VsinOdJ_1695568645; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsinOdJ_1695568645) by smtp.aliyun-inc.com; Sun, 24 Sep 2023 23:17:26 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: wintera@linux.ibm.com, schnelle@linux.ibm.com, gbayer@linux.ibm.com, pasic@linux.ibm.com, alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, dust.li@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v4 12/18] net/smc: implement DMB-related operations of loopback Date: Sun, 24 Sep 2023 23:16:47 +0800 Message-Id: <1695568613-125057-13-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> References: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Sun, 24 Sep 2023 08:18:21 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777932650855901988 X-GMAIL-MSGID: 1777932650855901988 This patch implements DMB registration, unregistration and data move operations of SMC-D loopback. Signed-off-by: Wen Gu --- net/smc/smc_cdc.c | 6 +++ net/smc/smc_cdc.h | 1 + net/smc/smc_loopback.c | 127 +++++++++++++++++++++++++++++++++++++++++++++++-- net/smc/smc_loopback.h | 13 +++++ 4 files changed, 144 insertions(+), 3 deletions(-) diff --git a/net/smc/smc_cdc.c b/net/smc/smc_cdc.c index 89105e9..2641800 100644 --- a/net/smc/smc_cdc.c +++ b/net/smc/smc_cdc.c @@ -411,6 +411,12 @@ static void smc_cdc_msg_recv(struct smc_sock *smc, struct smc_cdc_msg *cdc) static void smcd_cdc_rx_tsklet(struct tasklet_struct *t) { struct smc_connection *conn = from_tasklet(conn, t, rx_tsklet); + + smcd_cdc_rx_handler(conn); +} + +void smcd_cdc_rx_handler(struct smc_connection *conn) +{ struct smcd_cdc_msg *data_cdc; struct smcd_cdc_msg cdc; struct smc_sock *smc; diff --git a/net/smc/smc_cdc.h b/net/smc/smc_cdc.h index 696cc11..11559d4 100644 --- a/net/smc/smc_cdc.h +++ b/net/smc/smc_cdc.h @@ -301,5 +301,6 @@ int smcr_cdc_msg_send_validation(struct smc_connection *conn, struct smc_wr_buf *wr_buf); int smc_cdc_init(void) __init; void smcd_cdc_rx_init(struct smc_connection *conn); +void smcd_cdc_rx_handler(struct smc_connection *conn); #endif /* SMC_CDC_H */ diff --git a/net/smc/smc_loopback.c b/net/smc/smc_loopback.c index 4341803..eb13291 100644 --- a/net/smc/smc_loopback.c +++ b/net/smc/smc_loopback.c @@ -16,11 +16,13 @@ #include #include +#include "smc_cdc.h" #include "smc_ism.h" #include "smc_loopback.h" #if IS_ENABLED(CONFIG_SMC_LO) #define SMC_LO_SUPPORTS_V2 0x1 /* SMC-D loopback supports SMC-Dv2 */ +#define SMC_DMA_ADDR_INVALID (~(dma_addr_t)0) static const char smc_lo_dev_name[] = "smc_lo"; static struct smcd_seid SMC_LO_SEID = { @@ -75,6 +77,91 @@ static int smc_lo_query_rgid(struct smcd_dev *smcd, struct smcd_gid *rgid, return 0; } +static int smc_lo_register_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb, + void *client_priv) +{ + struct smc_lo_dmb_node *dmb_node, *tmp_node; + struct smc_lo_dev *ldev = smcd->priv; + int sba_idx, rc; + + /* check space for new dmb */ + for_each_clear_bit(sba_idx, ldev->sba_idx_mask, SMC_LODEV_MAX_DMBS) { + if (!test_and_set_bit(sba_idx, ldev->sba_idx_mask)) + break; + } + if (sba_idx == SMC_LODEV_MAX_DMBS) + return -ENOSPC; + + dmb_node = kzalloc(sizeof(*dmb_node), GFP_KERNEL); + if (!dmb_node) { + rc = -ENOMEM; + goto err_bit; + } + + dmb_node->sba_idx = sba_idx; + dmb_node->cpu_addr = vzalloc(dmb->dmb_len); + if (!dmb_node->cpu_addr) { + rc = -ENOMEM; + goto err_node; + } + dmb_node->len = dmb->dmb_len; + dmb_node->dma_addr = SMC_DMA_ADDR_INVALID; + +again: + /* add new dmb into hash table */ + get_random_bytes(&dmb_node->token, sizeof(dmb_node->token)); + write_lock(&ldev->dmb_ht_lock); + hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb_node->token) { + if (tmp_node->token == dmb_node->token) { + write_unlock(&ldev->dmb_ht_lock); + goto again; + } + } + hash_add(ldev->dmb_ht, &dmb_node->list, dmb_node->token); + write_unlock(&ldev->dmb_ht_lock); + + dmb->sba_idx = dmb_node->sba_idx; + dmb->dmb_tok = dmb_node->token; + dmb->cpu_addr = dmb_node->cpu_addr; + dmb->dma_addr = dmb_node->dma_addr; + dmb->dmb_len = dmb_node->len; + + return 0; + +err_node: + kfree(dmb_node); +err_bit: + clear_bit(sba_idx, ldev->sba_idx_mask); + return rc; +} + +static int smc_lo_unregister_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb) +{ + struct smc_lo_dmb_node *dmb_node = NULL, *tmp_node; + struct smc_lo_dev *ldev = smcd->priv; + + /* remove dmb from hash table */ + write_lock(&ldev->dmb_ht_lock); + hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb->dmb_tok) { + if (tmp_node->token == dmb->dmb_tok) { + dmb_node = tmp_node; + break; + } + } + if (!dmb_node) { + write_unlock(&ldev->dmb_ht_lock); + return -EINVAL; + } + hash_del(&dmb_node->list); + write_unlock(&ldev->dmb_ht_lock); + + clear_bit(dmb_node->sba_idx, ldev->sba_idx_mask); + vfree(dmb_node->cpu_addr); + kfree(dmb_node); + + return 0; +} + static int smc_lo_add_vlan_id(struct smcd_dev *smcd, u64 vlan_id) { return -EOPNOTSUPP; @@ -101,6 +188,38 @@ static int smc_lo_signal_event(struct smcd_dev *dev, struct smcd_gid *rgid, return 0; } +static int smc_lo_move_data(struct smcd_dev *smcd, u64 dmb_tok, unsigned int idx, + bool sf, unsigned int offset, void *data, + unsigned int size) +{ + struct smc_lo_dmb_node *rmb_node = NULL, *tmp_node; + struct smc_lo_dev *ldev = smcd->priv; + + read_lock(&ldev->dmb_ht_lock); + hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb_tok) { + if (tmp_node->token == dmb_tok) { + rmb_node = tmp_node; + break; + } + } + if (!rmb_node) { + read_unlock(&ldev->dmb_ht_lock); + return -EINVAL; + } + read_unlock(&ldev->dmb_ht_lock); + + memcpy((char *)rmb_node->cpu_addr + offset, data, size); + + if (sf) { + struct smc_connection *conn = + smcd->conn[rmb_node->sba_idx]; + + if (conn && !conn->killed) + smcd_cdc_rx_handler(conn); + } + return 0; +} + static int smc_lo_supports_v2(void) { return SMC_LO_SUPPORTS_V2; @@ -132,14 +251,14 @@ static struct device *smc_lo_get_dev(struct smcd_dev *smcd) static const struct smcd_ops lo_ops = { .query_remote_gid = smc_lo_query_rgid, - .register_dmb = NULL, - .unregister_dmb = NULL, + .register_dmb = smc_lo_register_dmb, + .unregister_dmb = smc_lo_unregister_dmb, .add_vlan_id = smc_lo_add_vlan_id, .del_vlan_id = smc_lo_del_vlan_id, .set_vlan_required = smc_lo_set_vlan_required, .reset_vlan_required = smc_lo_reset_vlan_required, .signal_event = smc_lo_signal_event, - .move_data = NULL, + .move_data = smc_lo_move_data, .supports_v2 = smc_lo_supports_v2, .get_system_eid = smc_lo_get_system_eid, .get_local_gid = smc_lo_get_local_gid, @@ -213,6 +332,8 @@ static void smc_lo_dev_release(struct device *dev) static int smc_lo_dev_init(struct smc_lo_dev *ldev) { smc_lo_generate_id(ldev); + rwlock_init(&ldev->dmb_ht_lock); + hash_init(ldev->dmb_ht); return smcd_lo_register_dev(ldev); } diff --git a/net/smc/smc_loopback.h b/net/smc/smc_loopback.h index 71a52da..a5b501b 100644 --- a/net/smc/smc_loopback.h +++ b/net/smc/smc_loopback.h @@ -21,12 +21,25 @@ #if IS_ENABLED(CONFIG_SMC_LO) #define SMC_LO_CHID 0xFFFF #define SMC_LODEV_MAX_DMBS 5000 +#define SMC_LODEV_DMBS_HASH_BITS 12 + +struct smc_lo_dmb_node { + struct hlist_node list; + u64 token; + u32 len; + u32 sba_idx; + void *cpu_addr; + dma_addr_t dma_addr; +}; struct smc_lo_dev { struct smcd_dev *smcd; struct device dev; u16 chid; struct smcd_gid local_gid; + DECLARE_BITMAP(sba_idx_mask, SMC_LODEV_MAX_DMBS); + rwlock_t dmb_ht_lock; + DECLARE_HASHTABLE(dmb_ht, SMC_LODEV_DMBS_HASH_BITS); }; #endif From patchwork Sun Sep 24 15:16:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 144106 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp713853vqu; Sun, 24 Sep 2023 08:18:44 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGu1uC8TUMys+b13OW4a6Md+5Kh/aEA/qjKmTB7NqN1F5fuKjHMJOftacc5wcfzIUWF0i0J X-Received: by 2002:a05:6870:f583:b0:1be:ceca:c666 with SMTP id eh3-20020a056870f58300b001bececac666mr6705725oab.14.1695568723778; Sun, 24 Sep 2023 08:18:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695568723; cv=none; d=google.com; s=arc-20160816; b=gbYSk30no3IzckvXI9AGGBYzb7QpedVEaqFHKtHc1WLXcFaCCtSYEOyEHe1NM+ejMV OINrbk96HtJQ9P6Lh9u7HaDTiw7wr1noDxedbPuBRRn/c+aaLnAsob4MbOWoPlFk21Hp aodqkdqALQnJlzOA1OMAW5dHQz3FiK8+7mt1g5p86yMS2rSIcNdppLcLNZ1wHZK6H89C oINiygH4CyD6OFhwWLdiYyNfSRLSyfAJrQT+7FiQItTk7XGIY3MBnJ2pmB2vpwYsI1Ex ZGl1xWI9AAeEh34SvvHBuoV5nd2lFhJ9a0Jfjd0AFvxDeTkht6tSEkKGEsxtKDPW7Wwu hFAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=xa98sDZioUnrket/O/63THRFBCzmBafowWrtgRLMRAQ=; fh=kTBstCixiri9Ahdk8UNuw7wmcx8Ry+mqafUdDsc8Vd0=; b=Y75DyyjMjRaoc1dv00d14mN2TYt8IWB+pGzGFIgA4Rai9TH6BOI3rxvg8giqj6sL4j ZK4R6hmJODaQQvI/hBUnJsbtxAXK8noH9GhGRx1VqaAY97OOaRpIrZoZws75MFZ/PRS+ 2lwg1o87bATfYgoqfML7c33u6v72ZTJhC8aIGUO9nmUdpmKYUgK3YFlcQTnnB2UhKJZc OD4kgNNPrUkC5VJf0RhucW+6AsCH5EvNt5euenoPWxn2gtcSIhBotaBVxjC8u/qzH2kP 6xjy+Kx+5k0DcB2aPk1JXtRL6ptt/qKJejD+3MRXTCbshG6c2AgtUG4bHduTG17xI8hR OZUg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id k195-20020a636fcc000000b00569374c7c71si1042778pgc.857.2023.09.24.08.18.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 08:18:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id 3228581603BC; Sun, 24 Sep 2023 08:18:40 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229732AbjIXPSP (ORCPT + 30 others); Sun, 24 Sep 2023 11:18:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46860 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229559AbjIXPRx (ORCPT ); Sun, 24 Sep 2023 11:17:53 -0400 Received: from out30-111.freemail.mail.aliyun.com (out30-111.freemail.mail.aliyun.com [115.124.30.111]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 369601AD; Sun, 24 Sep 2023 08:17:33 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R871e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045192;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0VsinOdy_1695568647; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsinOdy_1695568647) by smtp.aliyun-inc.com; Sun, 24 Sep 2023 23:17:29 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: wintera@linux.ibm.com, schnelle@linux.ibm.com, gbayer@linux.ibm.com, pasic@linux.ibm.com, alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, dust.li@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v4 13/18] net/smc: register loopback device as SMC-Dv2 device Date: Sun, 24 Sep 2023 23:16:48 +0800 Message-Id: <1695568613-125057-14-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> References: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Sun, 24 Sep 2023 08:18:40 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777932670037995277 X-GMAIL-MSGID: 1777932670037995277 After loopback device gets ready, add it to the smcd_dev list as an SMC-Dv2 device for use by SMC-D protocol. Signed-off-by: Wen Gu --- net/smc/smc_loopback.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/net/smc/smc_loopback.c b/net/smc/smc_loopback.c index eb13291..8375575 100644 --- a/net/smc/smc_loopback.c +++ b/net/smc/smc_loopback.c @@ -304,18 +304,27 @@ static int smcd_lo_register_dev(struct smc_lo_dev *ldev) ldev->smcd = smcd; smcd->priv = ldev; - - /* TODO: - * register smc_lo to smcd_dev list. - */ + mutex_lock(&smcd_dev_list.mutex); + smc_ism_check_v2_capable(smcd); + list_add(&smcd->list, &smcd_dev_list.list); + mutex_unlock(&smcd_dev_list.mutex); + pr_warn_ratelimited("smc: adding smcd device %s with pnetid %.16s%s\n", + smc_lo_dev_name, smcd->pnetid, + smcd->pnetid_by_user ? " (user defined)" : ""); return 0; } static void smcd_lo_unregister_dev(struct smc_lo_dev *ldev) { - /* TODO: - * unregister smc_lo from smcd_dev list. - */ + struct smcd_dev *smcd = ldev->smcd; + + pr_warn_ratelimited("smc: removing smcd device %s\n", + smc_lo_dev_name); + smcd->going_away = 1; + smc_smcd_terminate_all(smcd); + mutex_lock(&smcd_dev_list.mutex); + list_del_init(&smcd->list); + mutex_unlock(&smcd_dev_list.mutex); } static void smc_lo_dev_release(struct device *dev) From patchwork Sun Sep 24 15:16:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 144110 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp713958vqu; Sun, 24 Sep 2023 08:19:02 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGETnPj669kuaYutLymBKkQfOw+BRRuI45dA2yPgIY7eeUKwa0kdK90+3oY80bM2UU8ppuH X-Received: by 2002:a17:90a:3de1:b0:268:b7a2:62e8 with SMTP id i88-20020a17090a3de100b00268b7a262e8mr3676837pjc.7.1695568742006; Sun, 24 Sep 2023 08:19:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695568741; cv=none; d=google.com; s=arc-20160816; b=rIOKT3P9d26si1d6YFSVOSX3zgv+gtDQ8wKsw74z5ARNjBWAyxY1paHRsmbV2/DFJP oOf+qW1A8r+Aj4QyDTz4W7gt6G6lieQe5jcGWYzIjf3Bq3eGSJXWJCWQwCSaJv71EIsl I7NIZKdEJavuL8wu8Pycihin0dn9cRmdQlnGI4MejEyGgobR8PWvNXMSRsyB6WmoR5RJ XpWwkKpZCkmrH1nwwvzQShnoGU8oFRcyOOw2QhJvDhci1cqDEFiUS/QlSp12nMFBOG8u G3LfBsKNewGiJ8OqEuKuSOVq1OnxbBFjv0r46RlV6YO3Tr+niF3J+s+P4ECUb7rbNAg5 WCVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=km7+MH/uplDfRrTYczBdO8VBKCN2dKEvxBb7zJjCRIE=; fh=kTBstCixiri9Ahdk8UNuw7wmcx8Ry+mqafUdDsc8Vd0=; b=mS4gKA26yiJS30/I2qKdXisp0e/GVv/UOkFhMSbvt/LSsv6mNPzjAP3LZFTehOREOo By/rBUU3o2unWfJk58PF7IusdP8Y/iTtE8SXLFrzD4VhVL88wjT8duqntog+qt2yJ+Yv iRySX0NmCvf3xXK0QG+N4SwVi1GF8d6fa3RVOXyLFd65eQMYBSsFTtsIP6FaC38ZIMic EO6s3iBaNmjLCAWlGMF+K59lFYlbx8wjuTzHkj3Y6c+o0jgahqucCH6qKn6o7Tdx+qhS 6ECLapBOiTiLLGqzqOuiyWGtg50hdH/h/8eM/QI1Qp4x9xAGM8zj6tpy3THapEdM66TP C8pw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from morse.vger.email (morse.vger.email. [23.128.96.31]) by mx.google.com with ESMTPS id nv9-20020a17090b1b4900b0026b365096e5si11194203pjb.53.2023.09.24.08.19.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 08:19:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by morse.vger.email (Postfix) with ESMTP id B3821801F3F0; Sun, 24 Sep 2023 08:18:59 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at morse.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230249AbjIXPSS (ORCPT + 30 others); Sun, 24 Sep 2023 11:18:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46762 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230162AbjIXPRx (ORCPT ); Sun, 24 Sep 2023 11:17:53 -0400 Received: from out30-101.freemail.mail.aliyun.com (out30-101.freemail.mail.aliyun.com [115.124.30.101]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3363719C; Sun, 24 Sep 2023 08:17:33 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R151e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046056;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0Vsir41W_1695568649; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0Vsir41W_1695568649) by smtp.aliyun-inc.com; Sun, 24 Sep 2023 23:17:31 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: wintera@linux.ibm.com, schnelle@linux.ibm.com, gbayer@linux.ibm.com, pasic@linux.ibm.com, alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, dust.li@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v4 14/18] net/smc: add operation for getting DMB attribute Date: Sun, 24 Sep 2023 23:16:49 +0800 Message-Id: <1695568613-125057-15-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> References: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Sun, 24 Sep 2023 08:18:59 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777932689346159274 X-GMAIL-MSGID: 1777932689346159274 On s390, ISM devices are used on machine level hypervisors which is a partitioning hypervisor without paging. The sndbufs and peer DMBs in such case can't be mapped to the same physical memory. However in other cases, such as communication within the same OS instance with loopback, the sndbufs and peer DMBs can be mapped to the same physical memory to avoid memory copy from sndbufs to peer DMBs. So this patch introduces an operation to smcd_ops to judge whether the sndbufs-DMB map is available. And for reuse, the interface is designed to return DMB attribute, not only the sndbuf-DMB map feature. Signed-off-by: Wen Gu --- include/net/smc.h | 5 +++++ net/smc/smc_ism.c | 8 ++++++++ net/smc/smc_ism.h | 1 + 3 files changed, 14 insertions(+) diff --git a/include/net/smc.h b/include/net/smc.h index 7c2d35c..917572fb 100644 --- a/include/net/smc.h +++ b/include/net/smc.h @@ -55,6 +55,10 @@ struct smcd_seid { #define ISM_ERROR 0xFFFF +enum { + ISM_ATTR_DMB_MAP = 0, +}; + struct smcd_dev; struct smcd_gid { @@ -82,6 +86,7 @@ struct smcd_ops { void (*get_local_gid)(struct smcd_dev *dev, struct smcd_gid *gid); u16 (*get_chid)(struct smcd_dev *dev); struct device* (*get_dev)(struct smcd_dev *dev); + int (*get_dev_attr)(struct smcd_dev *dev); }; struct smcd_dev { diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c index 0922fab..9b31d00 100644 --- a/net/smc/smc_ism.c +++ b/net/smc/smc_ism.c @@ -214,6 +214,14 @@ int smc_ism_unregister_dmb(struct smcd_dev *smcd, struct smc_buf_desc *dmb_desc) return rc; } +bool smc_ism_dmb_mappable(struct smcd_dev *smcd) +{ + if (smcd->ops->get_dev_attr && + (smcd->ops->get_dev_attr(smcd) & BIT(ISM_ATTR_DMB_MAP))) + return true; + return false; +} + int smc_ism_register_dmb(struct smc_link_group *lgr, int dmb_len, struct smc_buf_desc *dmb_desc) { diff --git a/net/smc/smc_ism.h b/net/smc/smc_ism.h index 7ab82dd..cef212c 100644 --- a/net/smc/smc_ism.h +++ b/net/smc/smc_ism.h @@ -44,6 +44,7 @@ int smc_ism_cantalk(struct smcd_gid *peer_gid, unsigned short vlan_id, int smc_ism_register_dmb(struct smc_link_group *lgr, int buf_size, struct smc_buf_desc *dmb_desc); int smc_ism_unregister_dmb(struct smcd_dev *dev, struct smc_buf_desc *dmb_desc); +bool smc_ism_dmb_mappable(struct smcd_dev *smcd); int smc_ism_signal_shutdown(struct smc_link_group *lgr); void smc_ism_get_system_eid(u8 **eid); u16 smc_ism_get_chid(struct smcd_dev *dev); From patchwork Sun Sep 24 15:16:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 144105 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp713844vqu; Sun, 24 Sep 2023 08:18:41 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFGE7FbAauRZcaQK6BdJYopU8TBBXenLGsFn7a5hAV+8I3TFbH9lXMvYslZYrCzBtKtglf7 X-Received: by 2002:a05:6a00:1783:b0:690:3f29:a874 with SMTP id s3-20020a056a00178300b006903f29a874mr3924625pfg.23.1695568720952; Sun, 24 Sep 2023 08:18:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695568720; cv=none; d=google.com; s=arc-20160816; b=YvFx7MaPnFSDbbtOMDIiRyIfgwbdB1eb6CgMEEskcU2aOzqF0tARtpaKnRGytYGFII 3AWSYILuC/t9aCRMkI67xb2xQXxXdm6iiSms9MxrSYZvKuk2ZaSHUuB4V90Yumrtim/x Kfv/PEnVndz0MVtm8X2XZq1h6PD5+VOPjuyrQ/nEeHd2Fo9aX/kBLpj+l+Llz6mrm5ci Z4LbfqvCVVrCPes345cAIuODlRZfaKmzOq7Cv0RxxummOx8X7Hi0RwVg+h0GKtl1WvKn QJIThekrO7DuRUnCEcByZsNrqlGLaF8DOAEQA2dvGuch8PrrPNazzwTbrs3MsYMfsBhk q15w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=0qOiCvainaNcdtZKyF4Y6WFfKvyMXdWNzKozxRKr+Bg=; fh=kTBstCixiri9Ahdk8UNuw7wmcx8Ry+mqafUdDsc8Vd0=; b=HwmEo/sHUz4D/YlSMnBIf3J3lEejVZhaRJG+Kj2d0pWZ9smJ+/ODnpvfra/qj2sjlu Sdtm83Ua8oJGeSPOD7GkJ1/tflZf9UtpwcgzqyIviOoWOgpLS0UDgM7CDi4Uqg40tqt+ 9h7eTv7Uv4vSP9q+7HLc/TxpLidZAQaoGGwclDT34qufLZ3PrcLwwQ/7Y22yZ2ALGOE7 FKZApqeRucaTxAI8U9jI3wfL2QM0hvnjGOwUsUrZ+AOgA9yvGFZG1RthAi12MZMgn89Z sOgqGF7EPWw5TdumYdK25dPgd3UkGnxhm42nCSeaehWQnxuJkVBxZVTZy3Z0QB6CnAsC 4IuQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id 126-20020a630284000000b00565f611a1f8si7742688pgc.263.2023.09.24.08.18.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 08:18:40 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id DDED080C0388; Sun, 24 Sep 2023 08:18:39 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230070AbjIXPSZ (ORCPT + 30 others); Sun, 24 Sep 2023 11:18:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58938 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229593AbjIXPR6 (ORCPT ); Sun, 24 Sep 2023 11:17:58 -0400 Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com [115.124.30.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 51A3410D0; Sun, 24 Sep 2023 08:17:37 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R121e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045176;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0VsinOet_1695568651; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsinOet_1695568651) by smtp.aliyun-inc.com; Sun, 24 Sep 2023 23:17:33 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: wintera@linux.ibm.com, schnelle@linux.ibm.com, gbayer@linux.ibm.com, pasic@linux.ibm.com, alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, dust.li@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v4 15/18] net/smc: add operations for DMB attach and detach Date: Sun, 24 Sep 2023 23:16:50 +0800 Message-Id: <1695568613-125057-16-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> References: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sun, 24 Sep 2023 08:18:39 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777932667405125380 X-GMAIL-MSGID: 1777932667405125380 This patch extends smcd_ops, adding two more semantic for SMC-D DMB: - attach_dmb: Attach an already registered DMB to a specific buf_desc, so that we can refer to the DMB through this buf_desc. - detach_dmb: Reverse operation of attach_dmb. detach the DMB from the buf_desc. This interface extension is to prepare for the avoidance of memory copy from sndbuf to RMB with SMC-D device whose DMBs has ISM_ATTR_DMB_MAP attribute. Signed-off-by: Wen Gu --- include/net/smc.h | 2 ++ net/smc/smc_ism.c | 31 +++++++++++++++++++++++++++++++ net/smc/smc_ism.h | 2 ++ 3 files changed, 35 insertions(+) diff --git a/include/net/smc.h b/include/net/smc.h index 917572fb..39a21ed 100644 --- a/include/net/smc.h +++ b/include/net/smc.h @@ -72,6 +72,8 @@ struct smcd_ops { int (*register_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb, void *client); int (*unregister_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb); + int (*attach_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb); + int (*detach_dmb)(struct smcd_dev *dev, u64 token); int (*add_vlan_id)(struct smcd_dev *dev, u64 vlan_id); int (*del_vlan_id)(struct smcd_dev *dev, u64 vlan_id); int (*set_vlan_required)(struct smcd_dev *dev); diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c index 9b31d00..32d5d96 100644 --- a/net/smc/smc_ism.c +++ b/net/smc/smc_ism.c @@ -244,6 +244,37 @@ int smc_ism_register_dmb(struct smc_link_group *lgr, int dmb_len, return rc; } +int smc_ism_attach_dmb(struct smcd_dev *dev, u64 token, + struct smc_buf_desc *dmb_desc) +{ + struct smcd_dmb dmb; + int rc = 0; + + memset(&dmb, 0, sizeof(dmb)); + dmb.dmb_tok = token; + + if (!dev->ops->attach_dmb) + return -EINVAL; + + rc = dev->ops->attach_dmb(dev, &dmb); + if (!rc) { + dmb_desc->sba_idx = dmb.sba_idx; + dmb_desc->token = dmb.dmb_tok; + dmb_desc->cpu_addr = dmb.cpu_addr; + dmb_desc->dma_addr = dmb.dma_addr; + dmb_desc->len = dmb.dmb_len; + } + return rc; +} + +int smc_ism_detach_dmb(struct smcd_dev *dev, u64 token) +{ + if (!dev->ops->detach_dmb) + return -EINVAL; + + return dev->ops->detach_dmb(dev, token); +} + static int smc_nl_handle_smcd_dev(struct smcd_dev *smcd, struct sk_buff *skb, struct netlink_callback *cb) diff --git a/net/smc/smc_ism.h b/net/smc/smc_ism.h index cef212c..f30aae4 100644 --- a/net/smc/smc_ism.h +++ b/net/smc/smc_ism.h @@ -45,6 +45,8 @@ int smc_ism_register_dmb(struct smc_link_group *lgr, int buf_size, struct smc_buf_desc *dmb_desc); int smc_ism_unregister_dmb(struct smcd_dev *dev, struct smc_buf_desc *dmb_desc); bool smc_ism_dmb_mappable(struct smcd_dev *smcd); +int smc_ism_attach_dmb(struct smcd_dev *dev, u64 token, struct smc_buf_desc *dmb_desc); +int smc_ism_detach_dmb(struct smcd_dev *dev, u64 token); int smc_ism_signal_shutdown(struct smc_link_group *lgr); void smc_ism_get_system_eid(u8 **eid); u16 smc_ism_get_chid(struct smcd_dev *dev); From patchwork Sun Sep 24 15:16:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 144107 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp713855vqu; Sun, 24 Sep 2023 08:18:44 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEkdJ2fBSHK8eJgWLYCJS8RfpZSwPQC3SlM/tPztme+CctXPjzosSxXa6HDRiHXQzA+mRIb X-Received: by 2002:a17:90a:f510:b0:274:99e8:1670 with SMTP id cs16-20020a17090af51000b0027499e81670mr4120578pjb.43.1695568723930; Sun, 24 Sep 2023 08:18:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695568723; cv=none; d=google.com; s=arc-20160816; b=ec4F6KZxGpPapRawrgSIyLv4jJa7fa9urjaJrjSz8TK7fjxhajdlmxuvHS4RhK4gT0 tcvAIWSsOMnFsbEfoJVPiJjCPT1IdjnAfi39B/rn3KQ7hLYQoTkSA0z5LDmTJpV9e/A+ 50pHlZnysk37glG6dHYG09HnczvmpVDqOQ18W/ZKRKogadtstqNABO/8ToaANo/FVoc8 RsqLvc6E1VpR7IPVQRbQ1I/Mh5rsFvqULP4eUhZgBIqoxxffS8veTH3EG+sgZXMxUDcu CTQymNaQraUYOFYkCnpXQqPZvMVnMIr1CzP7oMaIRT9TuXvRgnG6gGNqrOiQ5sg55FeX tkbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=g5oeTc7bcil+OQOTu0SrSmFHrSRQzogjjiK6Mdo/bNs=; fh=kTBstCixiri9Ahdk8UNuw7wmcx8Ry+mqafUdDsc8Vd0=; b=bziriIpYEn6GhKXp4UuIqt5TnsKiFt7bE2QDNGgbu8mgunotbUgrvLkMr0zt98U4mu S16AWKOIy+kuxXZvwDpYiD00LZA9Tg8nL0qyfjFxSbbR9DQh7BS4bMe9tLQFt5Zl+ex7 4sF8aUUgdl/pRnlTMzG0HQlzOKNlyYnf4NuDwTRF1nSUIVcegYIvyYSOKQ1xPLuL6ztH Pm+RBujpcDVNQTDh+lDDrgTZ2Dri181HdkFOHM6oYoAsaBK0gka+MKc0D4WIdyjWHalN wYtSWgCyxkjshm9m7w4tq6KeqNhI1sTj0EhS/UMFGxRp3MVS+Z6bj4InOzZQlqWJ3k8q uPxA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from snail.vger.email (snail.vger.email. [23.128.96.37]) by mx.google.com with ESMTPS id l17-20020a170902f69100b001bb9f190bafsi8550994plg.526.2023.09.24.08.18.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 08:18:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 41E6780C0383; Sun, 24 Sep 2023 08:18:43 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230256AbjIXPS3 (ORCPT + 30 others); Sun, 24 Sep 2023 11:18:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46832 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230339AbjIXPSI (ORCPT ); Sun, 24 Sep 2023 11:18:08 -0400 Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4E4B010FA; Sun, 24 Sep 2023 08:17:39 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R611e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046059;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0VsilXnY_1695568653; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsilXnY_1695568653) by smtp.aliyun-inc.com; Sun, 24 Sep 2023 23:17:36 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: wintera@linux.ibm.com, schnelle@linux.ibm.com, gbayer@linux.ibm.com, pasic@linux.ibm.com, alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, dust.li@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v4 16/18] net/smc: avoid data copy from sndbuf to peer RMB in SMC-D Date: Sun, 24 Sep 2023 23:16:51 +0800 Message-Id: <1695568613-125057-17-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> References: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Sun, 24 Sep 2023 08:18:43 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777932669780035194 X-GMAIL-MSGID: 1777932669780035194 This patch aims to avoid data copy from local sndbuf to peer RMB by mapping local sndbuf to peer RMB when DMBs have ISM_ATTR_DMB_MAP attribute. After this, local sndbuf and peer RMB share the same physical memory. +----------+ +----------+ | socket A | | socket B | +----------+ +----------+ | ^ | +---------+ | regard as | | regard as local sndbuf | B's | local RMB | | RMB | | |-------> | |-----------| +---------+ 1. From the perspective of RMB: a. Created or reused when connection is created. b. Unused and recycled to lgr buffer pool when connection is freed. c. Freed when link group is freed. 2. From the perspective of sndbuf: a. Mapped to peer RMB by the rtoken exchanged through CLC message. Then accessing local sndbuf is equivalent to accessing peer RMB. c. Unmapped from peer RMB and freed when connection is freed. Won't be recycled to lgr buffer pool. Therefore, the data written to local sndbuf will directly reach peer RMB. Signed-off-by: Wen Gu --- net/smc/af_smc.c | 14 +++++++++++ net/smc/smc_core.c | 70 +++++++++++++++++++++++++++++++++++++++++++++++++++++- net/smc/smc_core.h | 1 + 3 files changed, 84 insertions(+), 1 deletion(-) diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index 6435659..47556c3 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -1442,6 +1442,12 @@ static int smc_connect_ism(struct smc_sock *smc, } smc_conn_save_peer_info(smc, aclc); + + if (smc_ism_dmb_mappable(smc->conn.lgr->smcd)) { + rc = smcd_buf_attach(smc); + if (rc) + goto connect_abort; + } smc_close_init(smc); smc_rx_init(smc); smc_tx_init(smc); @@ -2550,6 +2556,14 @@ static void smc_listen_work(struct work_struct *work) mutex_unlock(&smc_server_lgr_pending); } smc_conn_save_peer_info(new_smc, cclc); + + if (ini->is_smcd && + smc_ism_dmb_mappable(new_smc->conn.lgr->smcd)) { + rc = smcd_buf_attach(new_smc); + if (rc) + goto out_decl; + } + smc_listen_out_connected(new_smc); SMC_STAT_SERV_SUCC_INC(sock_net(newclcsock->sk), ini); goto out_free; diff --git a/net/smc/smc_core.c b/net/smc/smc_core.c index 6d7c738..96b1def 100644 --- a/net/smc/smc_core.c +++ b/net/smc/smc_core.c @@ -1154,6 +1154,20 @@ static void smcr_buf_unuse(struct smc_buf_desc *buf_desc, bool is_rmb, } } +static void smcd_buf_detach(struct smc_connection *conn) +{ + struct smcd_dev *smcd = conn->lgr->smcd; + u64 peer_token = conn->peer_token; + + if (!conn->sndbuf_desc) + return; + + smc_ism_detach_dmb(smcd, peer_token); + + kfree(conn->sndbuf_desc); + conn->sndbuf_desc = NULL; +} + static void smc_buf_unuse(struct smc_connection *conn, struct smc_link_group *lgr) { @@ -1198,6 +1212,10 @@ void smc_conn_free(struct smc_connection *conn) if (!list_empty(&lgr->list)) smc_ism_unset_conn(conn); tasklet_kill(&conn->rx_tsklet); + + /* detach sndbuf from peer RMB */ + if (smc_ism_dmb_mappable(lgr->smcd)) + smcd_buf_detach(conn); } else { smc_cdc_wait_pend_tx_wr(conn); if (current_work() != &conn->abort_work) @@ -2459,15 +2477,23 @@ void smc_rmb_sync_sg_for_cpu(struct smc_connection *conn) */ int smc_buf_create(struct smc_sock *smc, bool is_smcd) { + bool sndbuf_created = false; int rc; + if (is_smcd && + smc_ism_dmb_mappable(smc->conn.lgr->smcd)) + goto create_rmb; + /* create send buffer */ rc = __smc_buf_create(smc, is_smcd, false); if (rc) return rc; + sndbuf_created = true; + +create_rmb: /* create rmb */ rc = __smc_buf_create(smc, is_smcd, true); - if (rc) { + if (rc && sndbuf_created) { down_write(&smc->conn.lgr->sndbufs_lock); list_del(&smc->conn.sndbuf_desc->list); up_write(&smc->conn.lgr->sndbufs_lock); @@ -2477,6 +2503,48 @@ int smc_buf_create(struct smc_sock *smc, bool is_smcd) return rc; } +int smcd_buf_attach(struct smc_sock *smc) +{ + struct smc_connection *conn = &smc->conn; + struct smcd_dev *smcd = conn->lgr->smcd; + u64 peer_token = conn->peer_token; + struct smc_buf_desc *buf_desc; + int rc; + + buf_desc = kzalloc(sizeof(*buf_desc), GFP_KERNEL); + if (!buf_desc) + return -ENOMEM; + + /* map local sndbuf desc to peer RMB, so operations on local + * sndbuf are equivalent to operations on peer RMB. + */ + rc = smc_ism_attach_dmb(smcd, peer_token, buf_desc); + if (rc) { + rc = SMC_CLC_DECL_MEM; + goto free; + } + + smc->sk.sk_sndbuf = buf_desc->len; + buf_desc->cpu_addr = (u8 *)buf_desc->cpu_addr + sizeof(struct smcd_cdc_msg); + buf_desc->len -= sizeof(struct smcd_cdc_msg); + conn->sndbuf_desc = buf_desc; + conn->sndbuf_desc->used = 1; + atomic_set(&conn->sndbuf_space, conn->sndbuf_desc->len); + return 0; + +free: + if (conn->rmb_desc) { + /* free local RMB as well */ + down_write(&conn->lgr->rmbs_lock); + list_del(&conn->rmb_desc->list); + up_write(&conn->lgr->rmbs_lock); + smc_buf_free(conn->lgr, true, conn->rmb_desc); + conn->rmb_desc = NULL; + } + kfree(buf_desc); + return rc; +} + static inline int smc_rmb_reserve_rtoken_idx(struct smc_link_group *lgr) { int i; diff --git a/net/smc/smc_core.h b/net/smc/smc_core.h index d57eb9b..2cba119 100644 --- a/net/smc/smc_core.h +++ b/net/smc/smc_core.h @@ -551,6 +551,7 @@ void smc_smcd_terminate(struct smcd_dev *dev, struct smcd_gid *peer_gid, void smc_smcd_terminate_all(struct smcd_dev *dev); void smc_smcr_terminate_all(struct smc_ib_device *smcibdev); int smc_buf_create(struct smc_sock *smc, bool is_smcd); +int smcd_buf_attach(struct smc_sock *smc); int smc_uncompress_bufsize(u8 compressed); int smc_rmb_rtoken_handling(struct smc_connection *conn, struct smc_link *link, struct smc_clc_msg_accept_confirm *clc); From patchwork Sun Sep 24 15:16:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 144109 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp713895vqu; Sun, 24 Sep 2023 08:18:50 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFwpoT+G8JMhPdGtE2yTe484/Pt/nxHrC/8o3Bo2VgmPcTKIWnEGH7q6qik3/T3EoTocJgT X-Received: by 2002:a05:6a00:1488:b0:68f:e121:b37c with SMTP id v8-20020a056a00148800b0068fe121b37cmr6712702pfu.4.1695568730176; Sun, 24 Sep 2023 08:18:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695568730; cv=none; d=google.com; s=arc-20160816; b=0t/8NBrlvPo/talO/+iM+vpApcx+oyIjp7ADG2DXpcuBU0iU5kgK/VWaQsPtRwOEkM QPJ5/9S6lMTfggZR4jlGBiWcjwNvaBdEbT3thb/i4nH7pUyPOmsv5NYIqwELj/sAo56U NxpXMf8jnQCj9kXgCNwu2kBrIVcW4s+DFn3HYwW6Hr3lvleQpU/la4+68SCK2w+CyVwu RzLVIWLR8RC5JzEHCuPD3qW7pYQcWsp+ayBRx7otB2RTpNrnuakk3oaWWxRd1hmCZRsg Mtpg3SPG66HCOLarpQb871JyXOTltpJkK2MyeB+FzBoL8zro0/XI32ICVJRux2eHWAmz zing== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=y07OQ04D2+O4jhr1Pt9VUQzgrtDGr1vB/DMBgXymiNU=; fh=kTBstCixiri9Ahdk8UNuw7wmcx8Ry+mqafUdDsc8Vd0=; b=bwcJitxCdunOmbtuDPs8h4Kt+6qGcMQS+7TDVfGxqi+VY1cYGXKsmsx2gSvWA0vuMD 7yK9CUqFTLEj84jHP2FA6LlcGqC7sphPG9q+c3lfxYuaad7Kb7lT43DyCDSLy62SaBNc sGsZE+y+UhQa2h6XTe5ywms7wn13NxkEZ5vRBDzoWCcFoH8CvJC1WqML20fWxaEe/C93 lE1II4PMZKz2G4VqHpoQEHo/m+tz1gGjSLOj/IKBMt4Ciq3KRRR+NUhngjbaLF0ctdKL jTnt+k9XzBvX331lWx5HDcFI9FEWbmyXvkgIkbTuj/ZuSUpYsfoynOQHEHG3XDogxgsE agNQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from howler.vger.email (howler.vger.email. [23.128.96.34]) by mx.google.com with ESMTPS id x191-20020a6386c8000000b0057cf9be76c2si5744420pgd.580.2023.09.24.08.18.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 08:18:50 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by howler.vger.email (Postfix) with ESMTP id 766D0824929F; Sun, 24 Sep 2023 08:18:47 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at howler.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230155AbjIXPSl (ORCPT + 30 others); Sun, 24 Sep 2023 11:18:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47162 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230241AbjIXPSN (ORCPT ); Sun, 24 Sep 2023 11:18:13 -0400 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2B72ACC9; Sun, 24 Sep 2023 08:17:42 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R131e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046056;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0VsimeWz_1695568657; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VsimeWz_1695568657) by smtp.aliyun-inc.com; Sun, 24 Sep 2023 23:17:39 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: wintera@linux.ibm.com, schnelle@linux.ibm.com, gbayer@linux.ibm.com, pasic@linux.ibm.com, alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, dust.li@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v4 17/18] net/smc: modify cursor update logic when sndbuf mapped to RMB Date: Sun, 24 Sep 2023 23:16:52 +0800 Message-Id: <1695568613-125057-18-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> References: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Sun, 24 Sep 2023 08:18:48 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777932676876609493 X-GMAIL-MSGID: 1777932676876609493 Since local sndbuf shares the same physical memory with peer RMB after mapping, the logic related to cursor update needs to be adapted to ensure that the data written by local won't overwrite the data that has not been consumed by the peer. So in this case, the fin_curs and sndbuf_space that were originally updated after sending the CDC message should be modified to not be update until the peer updates cons_curs. Signed-off-by: Wen Gu --- net/smc/smc_cdc.c | 50 +++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 39 insertions(+), 11 deletions(-) diff --git a/net/smc/smc_cdc.c b/net/smc/smc_cdc.c index 2641800..7cc677d 100644 --- a/net/smc/smc_cdc.c +++ b/net/smc/smc_cdc.c @@ -18,6 +18,7 @@ #include "smc_tx.h" #include "smc_rx.h" #include "smc_close.h" +#include "smc_ism.h" /********************************** send *************************************/ @@ -256,17 +257,24 @@ int smcd_cdc_msg_send(struct smc_connection *conn) return rc; smc_curs_copy(&conn->rx_curs_confirmed, &curs, conn); conn->local_rx_ctrl.prod_flags.cons_curs_upd_req = 0; - /* Calculate transmitted data and increment free send buffer space */ - diff = smc_curs_diff(conn->sndbuf_desc->len, &conn->tx_curs_fin, - &conn->tx_curs_sent); - /* increased by confirmed number of bytes */ - smp_mb__before_atomic(); - atomic_add(diff, &conn->sndbuf_space); - /* guarantee 0 <= sndbuf_space <= sndbuf_desc->len */ - smp_mb__after_atomic(); - smc_curs_copy(&conn->tx_curs_fin, &conn->tx_curs_sent, conn); + if (!smc_ism_dmb_mappable(conn->lgr->smcd)) { + /* If local sndbuf has been mapped to peer RMB, then + * don't update the tx_curs_fin and sndbuf_space until + * peer has consumed the data in RMB. + */ - smc_tx_sndbuf_nonfull(smc); + /* Calculate transmitted data and increment free send buffer space */ + diff = smc_curs_diff(conn->sndbuf_desc->len, &conn->tx_curs_fin, + &conn->tx_curs_sent); + /* increased by confirmed number of bytes */ + smp_mb__before_atomic(); + atomic_add(diff, &conn->sndbuf_space); + /* guarantee 0 <= sndbuf_space <= sndbuf_desc->len */ + smp_mb__after_atomic(); + smc_curs_copy(&conn->tx_curs_fin, &conn->tx_curs_sent, conn); + + smc_tx_sndbuf_nonfull(smc); + } return rc; } @@ -324,7 +332,7 @@ static void smc_cdc_msg_recv_action(struct smc_sock *smc, { union smc_host_cursor cons_old, prod_old; struct smc_connection *conn = &smc->conn; - int diff_cons, diff_prod; + int diff_cons, diff_prod, diff_tx; smc_curs_copy(&prod_old, &conn->local_rx_ctrl.prod, conn); smc_curs_copy(&cons_old, &conn->local_rx_ctrl.cons, conn); @@ -340,6 +348,26 @@ static void smc_cdc_msg_recv_action(struct smc_sock *smc, atomic_add(diff_cons, &conn->peer_rmbe_space); /* guarantee 0 <= peer_rmbe_space <= peer_rmbe_size */ smp_mb__after_atomic(); + + if (conn->lgr->is_smcd && + smc_ism_dmb_mappable(conn->lgr->smcd)) { + /* If local sndbuf has been mapped to peer RMB, then + * update tx_curs_fin and sndbuf_space when peer has + * consumed the data in it's RMB. + */ + + /* calculate peer rmb consumed data */ + diff_tx = smc_curs_diff(conn->sndbuf_desc->len, &conn->tx_curs_fin, + &conn->local_rx_ctrl.cons); + /* increase local sndbuf space and fin_curs */ + smp_mb__before_atomic(); + atomic_add(diff_tx, &conn->sndbuf_space); + /* guarantee 0 <= sndbuf_space <= sndbuf_desc->len */ + smp_mb__after_atomic(); + smc_curs_copy(&conn->tx_curs_fin, &conn->local_rx_ctrl.cons, conn); + + smc_tx_sndbuf_nonfull(smc); + } } diff_prod = smc_curs_diff(conn->rmb_desc->len, &prod_old, From patchwork Sun Sep 24 15:16:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 144111 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a59:cae8:0:b0:403:3b70:6f57 with SMTP id r8csp714040vqu; Sun, 24 Sep 2023 08:19:14 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGokoya0STg0hp9Br5QCLi4xpR/0I57JNDXZGK8A6UPEtWWisFcc43ngAXpx2duBV9IFJFU X-Received: by 2002:a05:6a20:728c:b0:140:61f8:53f3 with SMTP id o12-20020a056a20728c00b0014061f853f3mr5737322pzk.21.1695568754010; Sun, 24 Sep 2023 08:19:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695568753; cv=none; d=google.com; s=arc-20160816; b=MTRR3dQUpfVglQ3/KZ7n3BotDP+qtFzx/wOUfpbAx0idT3aFsNO3e4L6kcjq7K5isB F0jyeuJxZOalQckLcg9FDDNtUDHNclKMw9N06tR/OdcHUWxz0d0yqKZwgfk2yWbHkRrd jGQIljuq/Vn6k8Ie3qu4JAbo4JvuKmhOvAcegI53B6ZcR/3dLBJ5DoDTnyXAp8WRBMjm 8akzVNgQRQGcGhZ/G8ADuPzQ6Z6L9cYW1pqUhuNcQ5021M37XO5P51bk3Q8rLuVhonsV Z1sgis36zC/nbBVQWv0asKNFpfXowMcdsg3r1vndAPYd5rQb/EJimUBGcZV1U2gDstYc guQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=IxdXmoaC+GPnC7bijU8XTIX91MASs7FrHNLId6M1JXs=; fh=kTBstCixiri9Ahdk8UNuw7wmcx8Ry+mqafUdDsc8Vd0=; b=Lup+GFcXaTiPokcettkz+tJJXFqWwpDW57jwBJx6vCa6F3y18ed01Hspg9SvNe7IBy hstHnMdfk4SQLGf8zBIGtoNFB+oB7grV1ljX8EJ9WZLsI8sPYlYDNdlciF84MMGBKhaK Z13KFsHnSKeLQAHttneRMtdHqsLUHY/qwJpqQw0/g/FDb1R7v7AWpNVT4HZHeHRbN74X cRAuNFwxm+F6H13I4rCmXNqrrQlSaIaSX5nbYkzMDlLs15fAmukBmOtzX+TfnxTSut/K wkLwZ4ALdI4nLH/Nly8swcDwB7z2hBb9THvZF3ho9S1lcqb3vzo9UB9yU+trJCagm7ov UOog== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from pete.vger.email (pete.vger.email. [2620:137:e000::3:6]) by mx.google.com with ESMTPS id l184-20020a6325c1000000b00578a9ea5686si7992790pgl.660.2023.09.24.08.19.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Sep 2023 08:19:13 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id 57A8F801BEA3; Sun, 24 Sep 2023 08:19:08 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230360AbjIXPSj (ORCPT + 30 others); Sun, 24 Sep 2023 11:18:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46860 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230147AbjIXPSN (ORCPT ); Sun, 24 Sep 2023 11:18:13 -0400 Received: from out30-99.freemail.mail.aliyun.com (out30-99.freemail.mail.aliyun.com [115.124.30.99]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 339F3CD8; Sun, 24 Sep 2023 08:17:43 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R191e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046060;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0Vsio80D_1695568659; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0Vsio80D_1695568659) by smtp.aliyun-inc.com; Sun, 24 Sep 2023 23:17:41 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: wintera@linux.ibm.com, schnelle@linux.ibm.com, gbayer@linux.ibm.com, pasic@linux.ibm.com, alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, dust.li@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v4 18/18] net/smc: add interface implementation of loopback device Date: Sun, 24 Sep 2023 23:16:53 +0800 Message-Id: <1695568613-125057-19-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> References: <1695568613-125057-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Sun, 24 Sep 2023 08:19:08 -0700 (PDT) X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1777932701860638703 X-GMAIL-MSGID: 1777932701860638703 This patch completes the specific implementation of loopback device for the newly added SMC-D DMB-related interface. The loopback device always provides mappable DMB because the device users are in the same OS instance. Signed-off-by: Wen Gu --- net/smc/smc_loopback.c | 104 ++++++++++++++++++++++++++++++++++++++++++++----- net/smc/smc_loopback.h | 5 +++ 2 files changed, 99 insertions(+), 10 deletions(-) diff --git a/net/smc/smc_loopback.c b/net/smc/smc_loopback.c index 8375575..e9d2d62 100644 --- a/net/smc/smc_loopback.c +++ b/net/smc/smc_loopback.c @@ -106,6 +106,7 @@ static int smc_lo_register_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb, } dmb_node->len = dmb->dmb_len; dmb_node->dma_addr = SMC_DMA_ADDR_INVALID; + refcount_set(&dmb_node->refcnt, 1); again: /* add new dmb into hash table */ @@ -119,6 +120,7 @@ static int smc_lo_register_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb, } hash_add(ldev->dmb_ht, &dmb_node->list, dmb_node->token); write_unlock(&ldev->dmb_ht_lock); + atomic_inc(&ldev->dmb_cnt); dmb->sba_idx = dmb_node->sba_idx; dmb->dmb_tok = dmb_node->token; @@ -140,18 +142,27 @@ static int smc_lo_unregister_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb) struct smc_lo_dmb_node *dmb_node = NULL, *tmp_node; struct smc_lo_dev *ldev = smcd->priv; - /* remove dmb from hash table */ - write_lock(&ldev->dmb_ht_lock); + /* find dmb from hash table */ + read_lock(&ldev->dmb_ht_lock); hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb->dmb_tok) { if (tmp_node->token == dmb->dmb_tok) { dmb_node = tmp_node; + dmb_node->freeing = 1; break; } } if (!dmb_node) { - write_unlock(&ldev->dmb_ht_lock); + read_unlock(&ldev->dmb_ht_lock); return -EINVAL; } + read_unlock(&ldev->dmb_ht_lock); + + /* wait for dmb refcnt to be 0 */ + if (!refcount_dec_and_test(&dmb_node->refcnt)) + wait_event(ldev->dmbs_release, !refcount_read(&dmb_node->refcnt)); + + /* remove dmb from hash table */ + write_lock(&ldev->dmb_ht_lock); hash_del(&dmb_node->list); write_unlock(&ldev->dmb_ht_lock); @@ -159,9 +170,69 @@ static int smc_lo_unregister_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb) vfree(dmb_node->cpu_addr); kfree(dmb_node); + if (atomic_dec_and_test(&ldev->dmb_cnt)) + wake_up(&ldev->ldev_release); return 0; } +static int smc_lo_attach_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb) +{ + struct smc_lo_dmb_node *dmb_node = NULL, *tmp_node; + struct smc_lo_dev *ldev = smcd->priv; + + /* find dmb_node according to dmb->dmb_tok */ + read_lock(&ldev->dmb_ht_lock); + hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb->dmb_tok) { + if (tmp_node->token == dmb->dmb_tok && !tmp_node->freeing) { + dmb_node = tmp_node; + break; + } + } + if (!dmb_node) { + read_unlock(&ldev->dmb_ht_lock); + return -EINVAL; + } + refcount_inc(&dmb_node->refcnt); + read_unlock(&ldev->dmb_ht_lock); + + /* provide dmb information */ + dmb->sba_idx = dmb_node->sba_idx; + dmb->dmb_tok = dmb_node->token; + dmb->cpu_addr = dmb_node->cpu_addr; + dmb->dma_addr = dmb_node->dma_addr; + dmb->dmb_len = dmb_node->len; + return 0; +} + +static int smc_lo_detach_dmb(struct smcd_dev *smcd, u64 token) +{ + struct smc_lo_dmb_node *dmb_node = NULL, *tmp_node; + struct smc_lo_dev *ldev = smcd->priv; + + /* find dmb_node according to dmb->dmb_tok */ + read_lock(&ldev->dmb_ht_lock); + hash_for_each_possible(ldev->dmb_ht, tmp_node, list, token) { + if (tmp_node->token == token) { + dmb_node = tmp_node; + break; + } + } + if (!dmb_node) { + read_unlock(&ldev->dmb_ht_lock); + return -EINVAL; + } + read_unlock(&ldev->dmb_ht_lock); + + if (refcount_dec_and_test(&dmb_node->refcnt)) + wake_up_all(&ldev->dmbs_release); + return 0; +} + +static int smc_lo_get_dev_attr(struct smcd_dev *smcd) +{ + return BIT(ISM_ATTR_DMB_MAP); +} + static int smc_lo_add_vlan_id(struct smcd_dev *smcd, u64 vlan_id) { return -EOPNOTSUPP; @@ -194,7 +265,15 @@ static int smc_lo_move_data(struct smcd_dev *smcd, u64 dmb_tok, unsigned int idx { struct smc_lo_dmb_node *rmb_node = NULL, *tmp_node; struct smc_lo_dev *ldev = smcd->priv; - + struct smc_connection *conn; + + if (!sf) { + /* local sndbuf shares the same physical memory with + * peer RMB, so no need to copy data from local sndbuf + * to peer RMB. + */ + return 0; + } read_lock(&ldev->dmb_ht_lock); hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb_tok) { if (tmp_node->token == dmb_tok) { @@ -210,13 +289,10 @@ static int smc_lo_move_data(struct smcd_dev *smcd, u64 dmb_tok, unsigned int idx memcpy((char *)rmb_node->cpu_addr + offset, data, size); - if (sf) { - struct smc_connection *conn = - smcd->conn[rmb_node->sba_idx]; + conn = smcd->conn[rmb_node->sba_idx]; + if (conn && !conn->killed) + smcd_cdc_rx_handler(conn); - if (conn && !conn->killed) - smcd_cdc_rx_handler(conn); - } return 0; } @@ -253,6 +329,8 @@ static struct device *smc_lo_get_dev(struct smcd_dev *smcd) .query_remote_gid = smc_lo_query_rgid, .register_dmb = smc_lo_register_dmb, .unregister_dmb = smc_lo_unregister_dmb, + .attach_dmb = smc_lo_attach_dmb, + .detach_dmb = smc_lo_detach_dmb, .add_vlan_id = smc_lo_add_vlan_id, .del_vlan_id = smc_lo_del_vlan_id, .set_vlan_required = smc_lo_set_vlan_required, @@ -264,6 +342,7 @@ static struct device *smc_lo_get_dev(struct smcd_dev *smcd) .get_local_gid = smc_lo_get_local_gid, .get_chid = smc_lo_get_chid, .get_dev = smc_lo_get_dev, + .get_dev_attr = smc_lo_get_dev_attr, }; static struct smcd_dev *smcd_lo_alloc_dev(const struct smcd_ops *ops, @@ -343,6 +422,9 @@ static int smc_lo_dev_init(struct smc_lo_dev *ldev) smc_lo_generate_id(ldev); rwlock_init(&ldev->dmb_ht_lock); hash_init(ldev->dmb_ht); + atomic_set(&ldev->dmb_cnt, 0); + init_waitqueue_head(&ldev->dmbs_release); + init_waitqueue_head(&ldev->ldev_release); return smcd_lo_register_dev(ldev); } @@ -376,6 +458,8 @@ static int smc_lo_dev_probe(void) static void smc_lo_dev_exit(struct smc_lo_dev *ldev) { smcd_lo_unregister_dev(ldev); + if (atomic_read(&ldev->dmb_cnt)) + wait_event(ldev->ldev_release, !atomic_read(&ldev->dmb_cnt)); } static void smc_lo_dev_remove(void) diff --git a/net/smc/smc_loopback.h b/net/smc/smc_loopback.h index a5b501b..e42c807 100644 --- a/net/smc/smc_loopback.h +++ b/net/smc/smc_loopback.h @@ -30,6 +30,8 @@ struct smc_lo_dmb_node { u32 sba_idx; void *cpu_addr; dma_addr_t dma_addr; + refcount_t refcnt; + u8 freeing : 1; }; struct smc_lo_dev { @@ -40,6 +42,9 @@ struct smc_lo_dev { DECLARE_BITMAP(sba_idx_mask, SMC_LODEV_MAX_DMBS); rwlock_t dmb_ht_lock; DECLARE_HASHTABLE(dmb_ht, SMC_LODEV_DMBS_HASH_BITS); + atomic_t dmb_cnt; + wait_queue_head_t dmbs_release; + wait_queue_head_t ldev_release; }; #endif