mbox series

[v3,0/2] gro: optimise redundant parsing of packets

Message ID 20230313162520.GA17199@debian
Headers
Series gro: optimise redundant parsing of packets |

Message

Richard Gobert March 13, 2023, 4:25 p.m. UTC 
  Currently the IPv6 extension headers are parsed twice: first in
ipv6_gro_receive, and then again in ipv6_gro_complete.

By using the new ->transport_proto and ->network_proto fields, and also
storing the size of the network header, we can avoid parsing a second time
during the gro complete phase.

The first commit frees up space in the GRO CB. The second commit reduces
the redundant parsing during the complete phase, using the freed CB space.

In addition, the second commit contains a fix for a potential future
problem in BIG TCP, which is detailed in the commit message itself.

Performance tests for TCP stream over IPv6 with extension headers
demonstrate rx improvement of ~0.7%.

For the benchmarks, I used 100Gbit NIC mlx5 single-core (power management
off), turboboost off.

Typical IPv6 traffic (zero extension headers):

    for i in {1..5}; do netperf -t TCP_STREAM -H 2001:db8:2:2::2 -l 90 | tail -1; done
    # before
    131072  16384  16384    90.00    16391.20
    131072  16384  16384    90.00    16403.50
    131072  16384  16384    90.00    16403.30
    131072  16384  16384    90.00    16397.84
    131072  16384  16384    90.00    16398.00

    # after
    131072  16384  16384    90.00    16399.85
    131072  16384  16384    90.00    16392.37
    131072  16384  16384    90.00    16403.06
    131072  16384  16384    90.00    16406.97
    131072  16384  16384    90.00    16406.09

IPv6 over IPv6 traffic:

    for i in {1..5}; do netperf -t TCP_STREAM -H 4001:db8:2:2::2 -l 90 | tail -1; done
    # before
    131072  16384  16384    90.00    14791.61
    131072  16384  16384    90.00    14791.66
    131072  16384  16384    90.00    14783.47
    131072  16384  16384    90.00    14810.17
    131072  16384  16384    90.00    14806.15

    # after
    131072  16384  16384    90.00    14793.49
    131072  16384  16384    90.00    14816.10
    131072  16384  16384    90.00    14818.41
    131072  16384  16384    90.00    14780.35
    131072  16384  16384    90.00    14800.48

IPv6 traffic with varying extension headers:

    for i in {1..5}; do netperf -t TCP_STREAM -H 2001:db8:2:2::2 -l 90 | tail -1; done
    # before
    131072  16384  16384    90.00    14812.37
    131072  16384  16384    90.00    14813.04
    131072  16384  16384    90.00    14802.54
    131072  16384  16384    90.00    14804.06
    131072  16384  16384    90.00    14819.08

    # after
    131072  16384  16384    90.00    14927.11
    131072  16384  16384    90.00    14910.45
    131072  16384  16384    90.00    14917.36
    131072  16384  16384    90.00    14916.53
    131072  16384  16384    90.00    14928.88

Richard Gobert (2):
  gro: decrease size of CB
  gro: optimise redundant parsing of packets

 include/net/gro.h      | 33 ++++++++++++++++++++++++---------
 net/core/gro.c         | 18 +++++++++++-------
 net/ethernet/eth.c     | 14 +++++++++++---
 net/ipv6/ip6_offload.c | 20 +++++++++++++++-----
 4 files changed, 61 insertions(+), 24 deletions(-)

Comments

Eric Dumazet March 13, 2023, 4:51 p.m. UTC | #1 
On Mon, Mar 13, 2023 at 9:46 AM Richard Gobert <richardbgobert@gmail.com> wrote:
>
> Currently the IPv6 extension headers are parsed twice: first in
> ipv6_gro_receive, and then again in ipv6_gro_complete.
>
> By using the new ->transport_proto field, and also storing the size of the
> network header, we can avoid parsing extension headers a second time in
> ipv6_gro_complete (which saves multiple memory dereferences and conditional
> checks inside ipv6_exthdrs_len for a varying amount of extension headers in
> IPv6 packets).
>
> The implementation had to handle both inner and outer layers in case of
> encapsulation (as they can't use the same field). I've applied a similar
> optimisation to Ethernet.
>
> Performance tests for TCP stream over IPv6 with a varying amount of
> extension headers demonstrate throughput improvement of ~0.7%.
>
> In addition, I fixed a potential future problem:

I would remove all this block.

We fix current problems, not future hypothetical ones.

>  - The call to skb_set_inner_network_header at the beginning of
>    ipv6_gro_complete calculates inner_network_header based on skb->data by
>    calling skb_set_inner_network_header, and setting it to point to the
>    beginning of the ip header.
>  - If a packet is going to be handled by BIG TCP, the following code block
>    is going to shift the packet header, and skb->data is going to be
>    changed as well.
>
> When the two flows are combined, inner_network_header will point to the
> wrong place - which might happen if encapsulation of BIG TCP will be
> supported in the future.
>
> The fix is to place the whole encapsulation branch after the BIG TCP code
> block. This way, if encapsulation of BIG TCP will be supported,
> inner_network_header will still be calculated with the correct value of
> skb->data.

We do not support encapsulated BIG TCP yet.
We will do this later, and whoever does it will make sure to also support GRO.

> Also, by arranging the code that way, the optimisation does not
> add an additional branch.
>
> Signed-off-by: Richard Gobert <richardbgobert@gmail.com>
> ---
>

Can you give us a good explanation of why extension headers are used exactly ?

I am not sure we want to add code to GRO for something that 99.99% of
us do not use.