mbox series

[RFC,net-next,v5,0/9] net/smc: Introduce SMC-D-based OS internal communication acceleration

Message ID 1682252271-2544-1-git-send-email-guwen@linux.alibaba.com
Headers
Series net/smc: Introduce SMC-D-based OS internal communication acceleration |

Message

Wen Gu April 23, 2023, 12:17 p.m. UTC 
  Hi, all

# Background

The background and previous discussion can be referred from [1]~[3].

We found SMC-D can be used to accelerate OS internal communication, such as
loopback or between two containers within the same OS instance. So this patch
set provides a kind of SMC-D dummy device (we call it the SMC-D loopback device)
to emulate an ISM device, so that SMC-D can also be used on architectures
other than s390. The SMC-D loopback device are designed as a system global
device, visible to all containers.

# Design

This patch set basically follows the design of the previous version.

Patch #1/9 ~ #3/9 attempt to decouple ISM-related structures from the SMC-D
generalized code and extract some helpers to make SMC-D protocol compatible
with devices other than s390 ISM device.

Patch #4/9 introduces a kind of loopback device, which is defined as SMC-Dv2
device and designed to provide communication between SMC sockets on the same
OS instance.

 +-------------------------------------------+
 |  +--------------+       +--------------+  |
 |  | SMC socket A |       | SMC socket B |  |
 |  +--------------+       +--------------+  |
 |       ^                         ^         |
 |       |    +----------------+   |         |
 |       |    |   SMC stack    |   |         |
 |       +--->| +------------+ |<--|         |
 |            | |   dummy    | |             |
 |            | |   device   | |             |
 |            +-+------------+-+             |
 |                   OS                      |
 +-------------------------------------------+

Patch #5/9 ~ #8/9 expand SMC-D protocol interface (smcd_ops) for scenarios where
SMC-D is used to communicate within VM (loopback here) or between VMs on the same
host (based on virtio-ism device, see [4]). What these scenarios have in common
is that the local sndbuf and peer RMB can be mapped to same physical memory region,
so the data copy between the local sndbuf and peer RMB can be omitted. Performance
improvement brought by this extension can be found in # Benchmark Test.

 +----------+                     +----------+
 | socket A |                     | socket B |
 +----------+                     +----------+
       |                               ^
       |         +---------+           |
  regard as      |         | ----------|
  local sndbuf   |  B's    |     regard as
       |         |  RMB    |     local RMB
       |-------> |         |
                 +---------+

Patch #9/9 realizes the support of loopback device for the above-mentioned expanded
SMC-D protocol interface.

# Benchmark Test

 * Test environments:
      - VM with Intel Xeon Platinum 8 core 2.50GHz, 16 GiB mem.
      - SMC sndbuf/RMB size 1MB.

 * Test object:
      - TCP lo: run on TCP loopback.
      - domain: run on UNIX domain.
      - SMC lo: run on SMC loopback device with patch #1/9 ~ #4/9.
      - SMC lo-nocpy: run on SMC loopback device with patch #1/9 ~ #9/9.

1. ipc-benchmark (see [5])

 - ./<foo> -c 1000000 -s 100

                    TCP-lo              domain              SMC-lo          SMC-lo-nocpy
Message
rate (msg/s)         79025      115736(+46.45%)    146760(+85.71%)       149800(+89.56%)

2. sockperf

 - serv: <smc_run> taskset -c <cpu> sockperf sr --tcp
 - clnt: <smc_run> taskset -c <cpu> sockperf { tp | pp } --tcp --msg-size={ 64000 for tp | 14 for pp } -i 127.0.0.1 -t 30

                    TCP-lo                  SMC-lo             SMC-lo-nocpy
Bandwidth(MBps)   4822.388        4940.918(+2.56%)         8086.67(+67.69%)
Latency(us)          6.298          3.352(-46.78%)            3.35(-46.81%)

3. iperf3

 - serv: <smc_run> taskset -c <cpu> iperf3 -s
 - clnt: <smc_run> taskset -c <cpu> iperf3 -c 127.0.0.1 -t 15

                    TCP-lo                  SMC-lo             SMC-lo-nocpy
Bitrate(Gb/s)         40.7            40.5(-0.49%)            72.4(+77.89%)

4. nginx/wrk

 - serv: <smc_run> nginx
 - clnt: <smc_run> wrk -t 8 -c 500 -d 30 http://127.0.0.1:80

                    TCP-lo                  SMC-lo             SMC-lo-nocpy
Requests/s       155994.57      214544.79(+37.53%)       215538.55(+38.17%)


v5->v4
 1. The loopback device generates SEID in the same way as the ISM devices when coexisting
    with ISM devices and uses a default fixed SEID in other cases.
 2. Ensure each DMB token of the same loopback device is unique.
 3. Fixe a crash caused by setting smcd_ops->signal_event interface to NULL.
 4. Fixe a compilation warning complained by kernel test rebot.

v4->v3
 1. Rebase to the latest net-next;
 2. Introduce SEID helper. SMC-D loopback will return SMCD_DEFAULT_V2_SEID. And if it
    coexist with ISM device, the SEID of ISM device will overwrite SMCD_DEFAULT_V2_SEID
    as smc_ism_v2_system_eid.
 3. Won't remove dmb_node from hashtable until no sndbuf attaching to it.

 Something postponed in this version
 1. Hierarchy perference of SMC-D devices when loopback and ISM devices coexist, which
    will be determinated after comparing the performance of loopback and ISM.

v3->v2
 1. Adapt new generalized interface provided by [2];
 2. Select loopback device through SMC-D v2 protocol;
 3. Split the loopback-related implementation and generic implementation into different
    patches more reasonably.

v1->v2
 1. Fix some build WARNINGs complained by kernel test rebot
    Reported-by: kernel test robot <lkp@intel.com>
 2. Add iperf3 test data.


[1] https://lore.kernel.org/netdev/1671506505-104676-1-git-send-email-guwen@linux.alibaba.com/
[2] https://lore.kernel.org/netdev/1676477905-88043-1-git-send-email-guwen@linux.alibaba.com/
[3] https://lore.kernel.org/netdev/1679887699-54797-1-git-send-email-guwen@linux.alibaba.com/
[4] https://lore.kernel.org/all/20230209033056.96657-1-xuanzhuo@linux.alibaba.com/
[5] https://github.com/goldsborough/ipc-bench



Wen Gu (9):
  net/smc: Decouple ism_dev from SMC-D device dump
  net/smc: Decouple ism_dev from SMC-D DMB registration
  net/smc: Extract v2 check helper from SMC-D device registration
  net/smc: Introduce SMC-D loopback device
  net/smc: Introduce an interface for getting DMB attribute
  net/smc: Introudce interfaces for DMB attach and detach
  net/smc: Avoid data copy from sndbuf to peer RMB in SMC-D
  net/smc: Modify cursor update logic when using mappable DMB
  net/smc: Add interface implementation of loopback device

 drivers/s390/net/ism_drv.c |   5 +-
 include/net/smc.h          |  18 +-
 net/smc/Makefile           |   2 +-
 net/smc/af_smc.c           |  26 ++-
 net/smc/smc_cdc.c          |  59 ++++--
 net/smc/smc_cdc.h          |   1 +
 net/smc/smc_core.c         |  70 ++++++-
 net/smc/smc_core.h         |   1 +
 net/smc/smc_ism.c          |  79 ++++++--
 net/smc/smc_ism.h          |   6 +
 net/smc/smc_loopback.c     | 491 +++++++++++++++++++++++++++++++++++++++++++++
 net/smc/smc_loopback.h     |  56 ++++++
 12 files changed, 777 insertions(+), 37 deletions(-)
 create mode 100644 net/smc/smc_loopback.c
 create mode 100644 net/smc/smc_loopback.h

Comments

Wen Gu May 10, 2023, 2:02 a.m. UTC | #1 
Hi Wenjia & Jan:

Does this version work fine on your platform?

And any comments on this version? :)


Best regards,
Wen Gu


On 2023/4/23 20:17, Wen Gu wrote:

> Hi, all
> 
> # Background
> 
> The background and previous discussion can be referred from [1]~[3].
> 
> We found SMC-D can be used to accelerate OS internal communication, such as
> loopback or between two containers within the same OS instance. So this patch
> set provides a kind of SMC-D dummy device (we call it the SMC-D loopback device)
> to emulate an ISM device, so that SMC-D can also be used on architectures
> other than s390. The SMC-D loopback device are designed as a system global
> device, visible to all containers.
> 
> # Design
> 
> This patch set basically follows the design of the previous version.
> 
> Patch #1/9 ~ #3/9 attempt to decouple ISM-related structures from the SMC-D
> generalized code and extract some helpers to make SMC-D protocol compatible
> with devices other than s390 ISM device.
> 
> Patch #4/9 introduces a kind of loopback device, which is defined as SMC-Dv2
> device and designed to provide communication between SMC sockets on the same
> OS instance.
> 
>   +-------------------------------------------+
>   |  +--------------+       +--------------+  |
>   |  | SMC socket A |       | SMC socket B |  |
>   |  +--------------+       +--------------+  |
>   |       ^                         ^         |
>   |       |    +----------------+   |         |
>   |       |    |   SMC stack    |   |         |
>   |       +--->| +------------+ |<--|         |
>   |            | |   dummy    | |             |
>   |            | |   device   | |             |
>   |            +-+------------+-+             |
>   |                   OS                      |
>   +-------------------------------------------+
> 
> Patch #5/9 ~ #8/9 expand SMC-D protocol interface (smcd_ops) for scenarios where
> SMC-D is used to communicate within VM (loopback here) or between VMs on the same
> host (based on virtio-ism device, see [4]). What these scenarios have in common
> is that the local sndbuf and peer RMB can be mapped to same physical memory region,
> so the data copy between the local sndbuf and peer RMB can be omitted. Performance
> improvement brought by this extension can be found in # Benchmark Test.
> 
>   +----------+                     +----------+
>   | socket A |                     | socket B |
>   +----------+                     +----------+
>         |                               ^
>         |         +---------+           |
>    regard as      |         | ----------|
>    local sndbuf   |  B's    |     regard as
>         |         |  RMB    |     local RMB
>         |-------> |         |
>                   +---------+
> 
> Patch #9/9 realizes the support of loopback device for the above-mentioned expanded
> SMC-D protocol interface.
> 
> # Benchmark Test
> 
>   * Test environments:
>        - VM with Intel Xeon Platinum 8 core 2.50GHz, 16 GiB mem.
>        - SMC sndbuf/RMB size 1MB.
> 
>   * Test object:
>        - TCP lo: run on TCP loopback.
>        - domain: run on UNIX domain.
>        - SMC lo: run on SMC loopback device with patch #1/9 ~ #4/9.
>        - SMC lo-nocpy: run on SMC loopback device with patch #1/9 ~ #9/9.
> 
> 1. ipc-benchmark (see [5])
> 
>   - ./<foo> -c 1000000 -s 100
> 
>                      TCP-lo              domain              SMC-lo          SMC-lo-nocpy
> Message
> rate (msg/s)         79025      115736(+46.45%)    146760(+85.71%)       149800(+89.56%)
> 
> 2. sockperf
> 
>   - serv: <smc_run> taskset -c <cpu> sockperf sr --tcp
>   - clnt: <smc_run> taskset -c <cpu> sockperf { tp | pp } --tcp --msg-size={ 64000 for tp | 14 for pp } -i 127.0.0.1 -t 30
> 
>                      TCP-lo                  SMC-lo             SMC-lo-nocpy
> Bandwidth(MBps)   4822.388        4940.918(+2.56%)         8086.67(+67.69%)
> Latency(us)          6.298          3.352(-46.78%)            3.35(-46.81%)
> 
> 3. iperf3
> 
>   - serv: <smc_run> taskset -c <cpu> iperf3 -s
>   - clnt: <smc_run> taskset -c <cpu> iperf3 -c 127.0.0.1 -t 15
> 
>                      TCP-lo                  SMC-lo             SMC-lo-nocpy
> Bitrate(Gb/s)         40.7            40.5(-0.49%)            72.4(+77.89%)
> 
> 4. nginx/wrk
> 
>   - serv: <smc_run> nginx
>   - clnt: <smc_run> wrk -t 8 -c 500 -d 30 http://127.0.0.1:80
> 
>                      TCP-lo                  SMC-lo             SMC-lo-nocpy
> Requests/s       155994.57      214544.79(+37.53%)       215538.55(+38.17%)
> 
> 
> v5->v4
>   1. The loopback device generates SEID in the same way as the ISM devices when coexisting
>      with ISM devices and uses a default fixed SEID in other cases.
>   2. Ensure each DMB token of the same loopback device is unique.
>   3. Fixe a crash caused by setting smcd_ops->signal_event interface to NULL.
>   4. Fixe a compilation warning complained by kernel test rebot.
> 
> v4->v3
>   1. Rebase to the latest net-next;
>   2. Introduce SEID helper. SMC-D loopback will return SMCD_DEFAULT_V2_SEID. And if it
>      coexist with ISM device, the SEID of ISM device will overwrite SMCD_DEFAULT_V2_SEID
>      as smc_ism_v2_system_eid.
>   3. Won't remove dmb_node from hashtable until no sndbuf attaching to it.
> 
>   Something postponed in this version
>   1. Hierarchy perference of SMC-D devices when loopback and ISM devices coexist, which
>      will be determinated after comparing the performance of loopback and ISM.
> 
> v3->v2
>   1. Adapt new generalized interface provided by [2];
>   2. Select loopback device through SMC-D v2 protocol;
>   3. Split the loopback-related implementation and generic implementation into different
>      patches more reasonably.
> 
> v1->v2
>   1. Fix some build WARNINGs complained by kernel test rebot
>      Reported-by: kernel test robot <lkp@intel.com>
>   2. Add iperf3 test data.
> 
> 
> [1] https://lore.kernel.org/netdev/1671506505-104676-1-git-send-email-guwen@linux.alibaba.com/
> [2] https://lore.kernel.org/netdev/1676477905-88043-1-git-send-email-guwen@linux.alibaba.com/
> [3] https://lore.kernel.org/netdev/1679887699-54797-1-git-send-email-guwen@linux.alibaba.com/
> [4] https://lore.kernel.org/all/20230209033056.96657-1-xuanzhuo@linux.alibaba.com/
> [5] https://github.com/goldsborough/ipc-bench
> 
> 
> 
> Wen Gu (9):
>    net/smc: Decouple ism_dev from SMC-D device dump
>    net/smc: Decouple ism_dev from SMC-D DMB registration
>    net/smc: Extract v2 check helper from SMC-D device registration
>    net/smc: Introduce SMC-D loopback device
>    net/smc: Introduce an interface for getting DMB attribute
>    net/smc: Introudce interfaces for DMB attach and detach
>    net/smc: Avoid data copy from sndbuf to peer RMB in SMC-D
>    net/smc: Modify cursor update logic when using mappable DMB
>    net/smc: Add interface implementation of loopback device
> 
>   drivers/s390/net/ism_drv.c |   5 +-
>   include/net/smc.h          |  18 +-
>   net/smc/Makefile           |   2 +-
>   net/smc/af_smc.c           |  26 ++-
>   net/smc/smc_cdc.c          |  59 ++++--
>   net/smc/smc_cdc.h          |   1 +
>   net/smc/smc_core.c         |  70 ++++++-
>   net/smc/smc_core.h         |   1 +
>   net/smc/smc_ism.c          |  79 ++++++--
>   net/smc/smc_ism.h          |   6 +
>   net/smc/smc_loopback.c     | 491 +++++++++++++++++++++++++++++++++++++++++++++
>   net/smc/smc_loopback.h     |  56 ++++++
>   12 files changed, 777 insertions(+), 37 deletions(-)
>   create mode 100644 net/smc/smc_loopback.c
>   create mode 100644 net/smc/smc_loopback.h
>
Jan Karcher May 10, 2023, 9:56 a.m. UTC | #2 
On 10/05/2023 04:02, Wen Gu wrote:
> Hi Wenjia & Jan:
> 
> Does this version work fine on your platform?
> 
> And any comments on this version? :)

Hi Wen Gu,

as background for the others: IBM & Alibaba are having calls regarding 
SMC since Alibaba shows a lot of interest and there is a lot to discuss 
(as you may have noticed).

As we have agreed in our calls, the information shared about SMC-D in 
specific doesn't make it easy to understand the underlying concepts 
(e.g. role of GID, EIDs, DMB tokens, etc.).
That's why one of the next calls is dedicated to explain some of the 
concepts in depth to you and see where we can improve our flow of 
information to the community in the future.
After that we can go over this RFC with a shared understanding of the 
concepts.

Thank you
- Jan


> 
> 
> Best regards,
> Wen Gu
> 
> 
> On 2023/4/23 20:17, Wen Gu wrote:
> 
>> Hi, all
>>
>> # Background
>>
>> The background and previous discussion can be referred from [1]~[3].
>>
>> We found SMC-D can be used to accelerate OS internal communication, 
>> such as
>> loopback or between two containers within the same OS instance. So 
>> this patch
>> set provides a kind of SMC-D dummy device (we call it the SMC-D 
>> loopback device)
>> to emulate an ISM device, so that SMC-D can also be used on architectures
>> other than s390. The SMC-D loopback device are designed as a system 
>> global
>> device, visible to all containers.
>>
>> # Design
>>
>> This patch set basically follows the design of the previous version.
>>
>> Patch #1/9 ~ #3/9 attempt to decouple ISM-related structures from the 
>> SMC-D
>> generalized code and extract some helpers to make SMC-D protocol 
>> compatible
>> with devices other than s390 ISM device.
>>
>> Patch #4/9 introduces a kind of loopback device, which is defined as 
>> SMC-Dv2
>> device and designed to provide communication between SMC sockets on 
>> the same
>> OS instance.
>>
>>   +-------------------------------------------+
>>   |  +--------------+       +--------------+  |
>>   |  | SMC socket A |       | SMC socket B |  |
>>   |  +--------------+       +--------------+  |
>>   |       ^                         ^         |
>>   |       |    +----------------+   |         |
>>   |       |    |   SMC stack    |   |         |
>>   |       +--->| +------------+ |<--|         |
>>   |            | |   dummy    | |             |
>>   |            | |   device   | |             |
>>   |            +-+------------+-+             |
>>   |                   OS                      |
>>   +-------------------------------------------+
>>
>> Patch #5/9 ~ #8/9 expand SMC-D protocol interface (smcd_ops) for 
>> scenarios where
>> SMC-D is used to communicate within VM (loopback here) or between VMs 
>> on the same
>> host (based on virtio-ism device, see [4]). What these scenarios have 
>> in common
>> is that the local sndbuf and peer RMB can be mapped to same physical 
>> memory region,
>> so the data copy between the local sndbuf and peer RMB can be omitted. 
>> Performance
>> improvement brought by this extension can be found in # Benchmark Test.
>>
>>   +----------+                     +----------+
>>   | socket A |                     | socket B |
>>   +----------+                     +----------+
>>         |                               ^
>>         |         +---------+           |
>>    regard as      |         | ----------|
>>    local sndbuf   |  B's    |     regard as
>>         |         |  RMB    |     local RMB
>>         |-------> |         |
>>                   +---------+
>>
>> Patch #9/9 realizes the support of loopback device for the 
>> above-mentioned expanded
>> SMC-D protocol interface.
>>
>> # Benchmark Test
>>
>>   * Test environments:
>>        - VM with Intel Xeon Platinum 8 core 2.50GHz, 16 GiB mem.
>>        - SMC sndbuf/RMB size 1MB.
>>
>>   * Test object:
>>        - TCP lo: run on TCP loopback.
>>        - domain: run on UNIX domain.
>>        - SMC lo: run on SMC loopback device with patch #1/9 ~ #4/9.
>>        - SMC lo-nocpy: run on SMC loopback device with patch #1/9 ~ #9/9.
>>
>> 1. ipc-benchmark (see [5])
>>
>>   - ./<foo> -c 1000000 -s 100
>>
>>                      TCP-lo              domain              
>> SMC-lo          SMC-lo-nocpy
>> Message
>> rate (msg/s)         79025      115736(+46.45%)    
>> 146760(+85.71%)       149800(+89.56%)
>>
>> 2. sockperf
>>
>>   - serv: <smc_run> taskset -c <cpu> sockperf sr --tcp
>>   - clnt: <smc_run> taskset -c <cpu> sockperf { tp | pp } --tcp 
>> --msg-size={ 64000 for tp | 14 for pp } -i 127.0.0.1 -t 30
>>
>>                      TCP-lo                  SMC-lo             
>> SMC-lo-nocpy
>> Bandwidth(MBps)   4822.388        4940.918(+2.56%)         
>> 8086.67(+67.69%)
>> Latency(us)          6.298          3.352(-46.78%)            
>> 3.35(-46.81%)
>>
>> 3. iperf3
>>
>>   - serv: <smc_run> taskset -c <cpu> iperf3 -s
>>   - clnt: <smc_run> taskset -c <cpu> iperf3 -c 127.0.0.1 -t 15
>>
>>                      TCP-lo                  SMC-lo             
>> SMC-lo-nocpy
>> Bitrate(Gb/s)         40.7            40.5(-0.49%)            
>> 72.4(+77.89%)
>>
>> 4. nginx/wrk
>>
>>   - serv: <smc_run> nginx
>>   - clnt: <smc_run> wrk -t 8 -c 500 -d 30 http://127.0.0.1:80
>>
>>                      TCP-lo                  SMC-lo             
>> SMC-lo-nocpy
>> Requests/s       155994.57      214544.79(+37.53%)       
>> 215538.55(+38.17%)
>>
>>
>> v5->v4
>>   1. The loopback device generates SEID in the same way as the ISM 
>> devices when coexisting
>>      with ISM devices and uses a default fixed SEID in other cases.
>>   2. Ensure each DMB token of the same loopback device is unique.
>>   3. Fixe a crash caused by setting smcd_ops->signal_event interface 
>> to NULL.
>>   4. Fixe a compilation warning complained by kernel test rebot.
>>
>> v4->v3
>>   1. Rebase to the latest net-next;
>>   2. Introduce SEID helper. SMC-D loopback will return 
>> SMCD_DEFAULT_V2_SEID. And if it
>>      coexist with ISM device, the SEID of ISM device will overwrite 
>> SMCD_DEFAULT_V2_SEID
>>      as smc_ism_v2_system_eid.
>>   3. Won't remove dmb_node from hashtable until no sndbuf attaching to 
>> it.
>>
>>   Something postponed in this version
>>   1. Hierarchy perference of SMC-D devices when loopback and ISM 
>> devices coexist, which
>>      will be determinated after comparing the performance of loopback 
>> and ISM.
>>
>> v3->v2
>>   1. Adapt new generalized interface provided by [2];
>>   2. Select loopback device through SMC-D v2 protocol;
>>   3. Split the loopback-related implementation and generic 
>> implementation into different
>>      patches more reasonably.
>>
>> v1->v2
>>   1. Fix some build WARNINGs complained by kernel test rebot
>>      Reported-by: kernel test robot <lkp@intel.com>
>>   2. Add iperf3 test data.
>>
>>
>> [1] 
>> https://lore.kernel.org/netdev/1671506505-104676-1-git-send-email-guwen@linux.alibaba.com/
>> [2] 
>> https://lore.kernel.org/netdev/1676477905-88043-1-git-send-email-guwen@linux.alibaba.com/
>> [3] 
>> https://lore.kernel.org/netdev/1679887699-54797-1-git-send-email-guwen@linux.alibaba.com/
>> [4] 
>> https://lore.kernel.org/all/20230209033056.96657-1-xuanzhuo@linux.alibaba.com/
>> [5] https://github.com/goldsborough/ipc-bench
>>
>>
>>
>> Wen Gu (9):
>>    net/smc: Decouple ism_dev from SMC-D device dump
>>    net/smc: Decouple ism_dev from SMC-D DMB registration
>>    net/smc: Extract v2 check helper from SMC-D device registration
>>    net/smc: Introduce SMC-D loopback device
>>    net/smc: Introduce an interface for getting DMB attribute
>>    net/smc: Introudce interfaces for DMB attach and detach
>>    net/smc: Avoid data copy from sndbuf to peer RMB in SMC-D
>>    net/smc: Modify cursor update logic when using mappable DMB
>>    net/smc: Add interface implementation of loopback device
>>
>>   drivers/s390/net/ism_drv.c |   5 +-
>>   include/net/smc.h          |  18 +-
>>   net/smc/Makefile           |   2 +-
>>   net/smc/af_smc.c           |  26 ++-
>>   net/smc/smc_cdc.c          |  59 ++++--
>>   net/smc/smc_cdc.h          |   1 +
>>   net/smc/smc_core.c         |  70 ++++++-
>>   net/smc/smc_core.h         |   1 +
>>   net/smc/smc_ism.c          |  79 ++++++--
>>   net/smc/smc_ism.h          |   6 +
>>   net/smc/smc_loopback.c     | 491 
>> +++++++++++++++++++++++++++++++++++++++++++++
>>   net/smc/smc_loopback.h     |  56 ++++++
>>   12 files changed, 777 insertions(+), 37 deletions(-)
>>   create mode 100644 net/smc/smc_loopback.c
>>   create mode 100644 net/smc/smc_loopback.h
>>
Wen Gu May 10, 2023, 11:59 a.m. UTC | #3 
On 2023/5/10 17:56, Jan Karcher wrote:

> On 10/05/2023 04:02, Wen Gu wrote:
>> Hi Wenjia & Jan:
>>
>> Does this version work fine on your platform?
>>
>> And any comments on this version? :)
> 
> Hi Wen Gu,
> 
> as background for the others: IBM & Alibaba are having calls regarding SMC since Alibaba shows a lot of interest and 
> there is a lot to discuss (as you may have noticed).
> 
> As we have agreed in our calls, the information shared about SMC-D in specific doesn't make it easy to understand the 
> underlying concepts (e.g. role of GID, EIDs, DMB tokens, etc.).
> That's why one of the next calls is dedicated to explain some of the concepts in depth to you and see where we can 
> improve our flow of information to the community in the future.
> After that we can go over this RFC with a shared understanding of the concepts.
> 
> Thank you
> - Jan
> 

OK, Jan, Let's follow this plan and thank you for organizing this.

Thanks,
Wen Gu