From patchwork Tue Dec 20 03:21:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 34892 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:adf:e747:0:0:0:0:0 with SMTP id c7csp2751266wrn; Mon, 19 Dec 2022 19:27:35 -0800 (PST) X-Google-Smtp-Source: AA0mqf5zxYZviPaJk9/kauyGC6LRJ0kJulL6AkYyfTQnPkz6QZR1RPoz1THXGnch1m5vzBZz9j/k X-Received: by 2002:a05:6402:3987:b0:467:e9a5:77c4 with SMTP id fk7-20020a056402398700b00467e9a577c4mr39655044edb.14.1671506855535; Mon, 19 Dec 2022 19:27:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671506855; cv=none; d=google.com; s=arc-20160816; b=MWjcQ81KOBAXAKxunF64SPQIYtfYGfRpEukI5yZB3izZHzFwbJV0VgrbFjmyxnf5as EvY9xrVq9fp4E8gqZjWpebKh/KLijGcmH5pHFjFTobPXIMNFWMcKp9AiQcAmpb3eB+R+ /2OIyffZOmAh5Oc6iKUS1C9Flt2Q+rMKPNQzD6RUgK8PehZ3RZlfCtqWFTMsWLDJSTWU OzCrzmJAT8asHg5ei4vVtRcvTGe2g+x8MAA/suOJ2ZagXyGSYslSrDfMOjen8Ludl+TF YQkTHR95p7aMLhDJc/FKi8c7aRHPy5osum7LHUV3l5ns0PVHRS42r9YJt29BB/bo3nef G4fA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=ougehpcvfo992wZt4jMV57EDC5z4jCUCs7jhMRjUBEQ=; b=ULhFlyheZ9sFqJ9TrwSvSg1u5NIRzbB7zFGBqRIJ+AchPL9fplSHB/Q3AnxGu83M4N LRHQ7DTc7flBGm3eHZneU6XZD6XdO+C0AQAVF17qRXdrrC7EDNS6IUaiSkz+kqcwihqY Pykwr7xQzwxWBFu/JjDUyL3VJ3g93Xa4YBAV2MOdK+UVdWG02Sbvca/qyFl2bP6grqUE yz1mYusAA9if0GQfCwhi3FveLKas2ZSC6D3CF8584v43/MjAXaJaIfZ8im3sw3TyLRZH jO1gaaGfYYpBC4OCAz97v6IQJjlFD12Iv8FmNJXUuuSukmXXd25nkJkMwfFG6CW6zNuA 4BWQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id z16-20020a05640240d000b00461d9b36740si11749290edb.240.2022.12.19.19.27.12; Mon, 19 Dec 2022 19:27:35 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233039AbiLTDWX (ORCPT + 99 others); Mon, 19 Dec 2022 22:22:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52868 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233053AbiLTDWC (ORCPT ); Mon, 19 Dec 2022 22:22:02 -0500 Received: from out30-8.freemail.mail.aliyun.com (out30-8.freemail.mail.aliyun.com [115.124.30.8]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D560A1A1; Mon, 19 Dec 2022 19:22:00 -0800 (PST) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R101e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045170;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=10;SR=0;TI=SMTPD_---0VXjKPrz_1671506516; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0VXjKPrz_1671506516) by smtp.aliyun-inc.com; Tue, 20 Dec 2022 11:21:58 +0800 From: Wen Gu To: kgraul@linux.ibm.com, wenjia@linux.ibm.com, jaka@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com Cc: linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH net-next v2 2/5] net/smc: choose loopback device in SMC-D communication Date: Tue, 20 Dec 2022 11:21:42 +0800 Message-Id: <1671506505-104676-3-git-send-email-guwen@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1671506505-104676-1-git-send-email-guwen@linux.alibaba.com> References: <1671506505-104676-1-git-send-email-guwen@linux.alibaba.com> X-Spam-Status: No, score=-9.9 required=5.0 tests=BAYES_00, ENV_AND_HDR_SPF_MATCH,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-getmail-retrieved-from-mailbox: =?utf-8?q?INBOX?= X-GMAIL-THRID: =?utf-8?q?1752701972618238491?= X-GMAIL-MSGID: =?utf-8?q?1752701972618238491?= This patch allows SMC-D to use loopback device. But noted that the implementation here is quiet simple and informal. Loopback device will always be firstly chosen, and fallback happens if loopback communication is impossible. It needs to be discussed how client indicates to peer that multiple SMC-D devices are available and how server picks a suitable one. Signed-off-by: Wen Gu --- net/smc/af_smc.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++------ net/smc/smc_clc.c | 4 +++- net/smc/smc_ism.c | 3 ++- 3 files changed, 54 insertions(+), 8 deletions(-) diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index 9546c02..b9884c8 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -979,6 +979,28 @@ static int smc_find_ism_device(struct smc_sock *smc, struct smc_init_info *ini) return 0; } +/* check if there is a lo device available for this connection. */ +static int smc_find_lo_device(struct smc_sock *smc, struct smc_init_info *ini) +{ + struct smcd_dev *sdev; + + mutex_lock(&smcd_dev_list.mutex); + list_for_each_entry(sdev, &smcd_dev_list.list, list) { + if (sdev->is_loopback && !sdev->going_away && + (!ini->ism_peer_gid[0] || + !smc_ism_cantalk(ini->ism_peer_gid[0], ini->vlan_id, + sdev))) { + ini->ism_dev[0] = sdev; + break; + } + } + mutex_unlock(&smcd_dev_list.mutex); + if (!ini->ism_dev[0]) + return SMC_CLC_DECL_NOSMCDDEV; + ini->ism_chid[0] = smc_ism_get_chid(ini->ism_dev[0]); + return 0; +} + /* is chid unique for the ism devices that are already determined? */ static bool smc_find_ism_v2_is_unique_chid(u16 chid, struct smc_init_info *ini, int cnt) @@ -1044,10 +1066,20 @@ static int smc_find_proposal_devices(struct smc_sock *smc, { int rc = 0; - /* check if there is an ism device available */ + /* TODO: + * How to indicate to peer if ism device and loopback + * device are both available ? + * + * The RFC patch hasn't resolved this, just simply always + * chooses loopback device first, and fallback if loopback + * communication is impossible. + * + */ + /* check if there is an ism or loopback device available */ if (!(ini->smcd_version & SMC_V1) || - smc_find_ism_device(smc, ini) || - smc_connect_ism_vlan_setup(smc, ini)) + (smc_find_lo_device(smc, ini) && + (smc_find_ism_device(smc, ini) || + smc_connect_ism_vlan_setup(smc, ini)))) ini->smcd_version &= ~SMC_V1; /* else ISM V1 is supported for this connection */ @@ -2135,9 +2167,20 @@ static void smc_find_ism_v1_device_serv(struct smc_sock *new_smc, goto not_found; ini->is_smcd = true; /* prepare ISM check */ ini->ism_peer_gid[0] = ntohll(pclc_smcd->ism.gid); - rc = smc_find_ism_device(new_smc, ini); - if (rc) - goto not_found; + + /* TODO: + * How to know that peer has both loopback and ism device ? + * + * The RFC patch hasn't resolved this, simply tries loopback + * device first, then ism device. + */ + /* find available loopback or ism device */ + if (smc_find_lo_device(new_smc, ini)) { + rc = smc_find_ism_device(new_smc, ini); + if (rc) + goto not_found; + } + ini->ism_selected = 0; rc = smc_listen_ism_init(new_smc, ini); if (!rc) diff --git a/net/smc/smc_clc.c b/net/smc/smc_clc.c index dfb9797..3887692 100644 --- a/net/smc/smc_clc.c +++ b/net/smc/smc_clc.c @@ -486,7 +486,9 @@ static int smc_clc_prfx_set4_rcu(struct dst_entry *dst, __be32 ipv4, return -ENODEV; in_dev_for_each_ifa_rcu(ifa, in_dev) { - if (!inet_ifa_match(ipv4, ifa)) + /* add loopback support */ + if (inet_addr_type(dev_net(dst->dev), ipv4) != RTN_LOCAL && + !inet_ifa_match(ipv4, ifa)) continue; prop->prefix_len = inet_mask_len(ifa->ifa_mask); prop->outgoing_subnet = ifa->ifa_address & ifa->ifa_mask; diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c index 911fe08..1d10435 100644 --- a/net/smc/smc_ism.c +++ b/net/smc/smc_ism.c @@ -227,7 +227,8 @@ static int smc_nl_handle_smcd_dev(struct smcd_dev *smcd, if (nla_put_u8(skb, SMC_NLA_DEV_IS_CRIT, use_cnt > 0)) goto errattr; memset(&smc_pci_dev, 0, sizeof(smc_pci_dev)); - smc_set_pci_values(to_pci_dev(smcd->dev.parent), &smc_pci_dev); + if (!smcd->is_loopback) + smc_set_pci_values(to_pci_dev(smcd->dev.parent), &smc_pci_dev); if (nla_put_u32(skb, SMC_NLA_DEV_PCI_FID, smc_pci_dev.pci_fid)) goto errattr; if (nla_put_u16(skb, SMC_NLA_DEV_PCI_CHID, smc_pci_dev.pci_pchid))