From patchwork Thu Jan 11 12:00:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wen Gu X-Patchwork-Id: 187350 Return-Path: Delivered-To: ouuuleilei@gmail.com Received: by 2002:a05:7300:2411:b0:101:2151:f287 with SMTP id m17csp1397686dyi; Thu, 11 Jan 2024 04:07:06 -0800 (PST) X-Google-Smtp-Source: AGHT+IGpw1jiMQSnHZxi5OIfQ0IJasnz5OkF1Hf+qtrbq82nCLmSOVecALuiyGud7bysOfcAwSa0 X-Received: by 2002:a17:903:120b:b0:1d4:5c6c:2089 with SMTP id l11-20020a170903120b00b001d45c6c2089mr859937plh.14.1704974826638; Thu, 11 Jan 2024 04:07:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704974826; cv=none; d=google.com; s=arc-20160816; b=SqR8BWNRpdtU9N+uStRPu3bMU8nW16WUilvCxRPKoN1ZN+T7aJZIVO2CxE4HygAIVk wUrm2pSigrsQgvgt8j+5mRpflQAjygcpcyiD40KDZMtVdBGVeBmf5h/cSOn3zgZAJMqc LJbT0iHjdm3o95LyX1JAym/EtvRR51hkRPwipmsFUHucbGMb3buWaZc9ue3QQT72IuPN 2PNuVjC4IaytuwvEJHZZA/VQyC7yObsM3gj44TwxZxTEf6/dIvOanIo3mMX63fiFdUCS ZfA1DUgtfAaJ6oH8MoUGpMVA/FZ5jBvVVigByA6KInsH1V1vig8TEqNdm7KOPcWzdPJ1 DHJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from; bh=27YIQJuqewpMM5elqbRB3VfdAmuW6PwcL0i+nf3Sluk=; fh=jvQSXgb9+Gwj+2SIugFJuht8/m1b8ZaGKJU/ZBOVFgM=; b=Wm+3ueVhHY4tm0Qfcfg2ruwpu05gEIuW/e4eMHjSq1VxAx2zaQYerd356mn1kJe1Xp mZRJCgiN8QPtpxRi+7m/8tCvou1JxoQSMza/WoBvqmWNSzie1MIjfUSkDsmTzFCEbker II2t2LjX79WlZJjUiwyeabuTWI/hCQte5MAfQ4fLpKgb0StBbb63ZFSkeHd5CMSFVZM1 J8zpqT8sE0/9VzsZ2//G5Im+PTl9P1SACXM6lTfWgnae28VVbNT8jCklg0dYv9Wt7M14 sRWPFTsqneS35npFwsMSUzdjKqiMS0cIFP6ZrKX6MuwiYmeI0efICG5zdehKw2G3sU54 Y8uQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-23552-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-23552-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id j8-20020a170902da8800b001d54c7aa75dsi1014730plx.282.2024.01.11.04.07.06 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Jan 2024 04:07:06 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-23552-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel+bounces-23552-ouuuleilei=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-23552-ouuuleilei=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 6431E286505 for ; Thu, 11 Jan 2024 12:07:06 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 913AC3EA7C; Thu, 11 Jan 2024 12:01:50 +0000 (UTC) Received: from out199-12.us.a.mail.aliyun.com (out199-12.us.a.mail.aliyun.com [47.90.199.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 739B83D55F; Thu, 11 Jan 2024 12:01:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046059;MF=guwen@linux.alibaba.com;NM=1;PH=DS;RN=18;SR=0;TI=SMTPD_---0W-PgOvd_1704974485; Received: from localhost(mailfrom:guwen@linux.alibaba.com fp:SMTPD_---0W-PgOvd_1704974485) by smtp.aliyun-inc.com; Thu, 11 Jan 2024 20:01:27 +0800 From: Wen Gu To: wintera@linux.ibm.com, wenjia@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, jaka@linux.ibm.com Cc: borntraeger@linux.ibm.com, svens@linux.ibm.com, alibuda@linux.alibaba.com, tonylu@linux.alibaba.com, guwen@linux.alibaba.com, linux-s390@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 10/15] net/smc: add operations to merge sndbuf with peer DMB Date: Thu, 11 Jan 2024 20:00:31 +0800 Message-Id: <20240111120036.109903-11-guwen@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20240111120036.109903-1-guwen@linux.alibaba.com> References: <20240111120036.109903-1-guwen@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-THRID: 1787795683821014372 X-GMAIL-MSGID: 1787795683821014372 In some scenarios using virtual ISM device, sndbuf can share the same physical memory region with peer DMB to avoid data copy from one side to the other. In such case the sndbuf is only a descriptor that describes the shared memory and does not actually occupy memory, it's more like a ghost buffer. +----------+ +----------+ | socket A | | socket B | +----------+ +----------+ | | +--------+ +--------+ | sndbuf | | DMB | | desc | | desc | +--------+ +--------+ | | | +----v-----+ +--------------------------> memory | +----------+ So here introduces three new SMC-D device operations to check if this feature is supported by device, and to {attach|detach} ghost sndbuf to peer DMB. For now only loopback-ism supports this. Signed-off-by: Wen Gu --- include/net/smc.h | 3 +++ net/smc/smc_ism.c | 40 ++++++++++++++++++++++++++++++++++++++++ net/smc/smc_ism.h | 4 ++++ 3 files changed, 47 insertions(+) diff --git a/include/net/smc.h b/include/net/smc.h index 6273c3a8b24a..01387631d8a6 100644 --- a/include/net/smc.h +++ b/include/net/smc.h @@ -62,6 +62,9 @@ struct smcd_ops { int (*register_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb, void *client); int (*unregister_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb); + int (*support_dmb_nocopy)(struct smcd_dev *dev); + int (*attach_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb); + int (*detach_dmb)(struct smcd_dev *dev, u64 token); int (*add_vlan_id)(struct smcd_dev *dev, u64 vlan_id); int (*del_vlan_id)(struct smcd_dev *dev, u64 vlan_id); int (*set_vlan_required)(struct smcd_dev *dev); diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c index 4065ebd2e43d..2d2781724932 100644 --- a/net/smc/smc_ism.c +++ b/net/smc/smc_ism.c @@ -246,6 +246,46 @@ int smc_ism_register_dmb(struct smc_link_group *lgr, int dmb_len, return rc; } +bool smc_ism_support_dmb_nocopy(struct smcd_dev *smcd) +{ + /* for now only loopback-ism supports + * merging sndbuf with peer DMB to avoid + * data copies between them. + */ + return (smcd->ops->support_dmb_nocopy && + smcd->ops->support_dmb_nocopy(smcd)); +} + +int smc_ism_attach_dmb(struct smcd_dev *dev, u64 token, + struct smc_buf_desc *dmb_desc) +{ + struct smcd_dmb dmb; + int rc = 0; + + if (!dev->ops->attach_dmb) + return -EINVAL; + + memset(&dmb, 0, sizeof(dmb)); + dmb.dmb_tok = token; + rc = dev->ops->attach_dmb(dev, &dmb); + if (!rc) { + dmb_desc->sba_idx = dmb.sba_idx; + dmb_desc->token = dmb.dmb_tok; + dmb_desc->cpu_addr = dmb.cpu_addr; + dmb_desc->dma_addr = dmb.dma_addr; + dmb_desc->len = dmb.dmb_len; + } + return rc; +} + +int smc_ism_detach_dmb(struct smcd_dev *dev, u64 token) +{ + if (!dev->ops->detach_dmb) + return -EINVAL; + + return dev->ops->detach_dmb(dev, token); +} + static int smc_nl_handle_smcd_dev(struct smcd_dev *smcd, struct sk_buff *skb, struct netlink_callback *cb) diff --git a/net/smc/smc_ism.h b/net/smc/smc_ism.h index 6903cd5d4d4d..8ea5ab737c6f 100644 --- a/net/smc/smc_ism.h +++ b/net/smc/smc_ism.h @@ -48,6 +48,10 @@ int smc_ism_put_vlan(struct smcd_dev *dev, unsigned short vlan_id); int smc_ism_register_dmb(struct smc_link_group *lgr, int buf_size, struct smc_buf_desc *dmb_desc); int smc_ism_unregister_dmb(struct smcd_dev *dev, struct smc_buf_desc *dmb_desc); +bool smc_ism_support_dmb_nocopy(struct smcd_dev *smcd); +int smc_ism_attach_dmb(struct smcd_dev *dev, u64 token, + struct smc_buf_desc *dmb_desc); +int smc_ism_detach_dmb(struct smcd_dev *dev, u64 token); int smc_ism_signal_shutdown(struct smc_link_group *lgr); void smc_ism_get_system_eid(u8 **eid); u16 smc_ism_get_chid(struct smcd_dev *dev);