Re: [PATCH 0/4] Some improvements to the tap send path

[Stefano Brivio <sbrivio@redhat.com>]

On Fri, 8 Mar 2024 16:49:20 +0100
Laurent Vivier <lvivier@redhat.com> wrote:

> On 3/8/24 09:34, Stefano Brivio wrote:
> > On Fri, 8 Mar 2024 09:18:48 +0100
> > Laurent Vivier <lvivier@redhat.com> wrote:
> >   
> >> On 3/8/24 07:53, David Gibson wrote:  
> >>> This series has a handful of small improvements to the tap send path.
> >>> See individual commit messages for the details.
> >>>
> >>> I expect this will conflict with Laurent's upcoming work.  I hope the
> >>> conflicts won't be too bad, and indeed will set us up for less
> >>> duplication there in the end.  
> >>
> >> I'm working on patch that devides TCP buffers in several buffers pointed out by an IOV
> >> arrays and then provided to tap_send_frames(). I'm going to base my patch on this series.
> >>
> >> The idea is:
> >>
> >> A frame is made with 4 iovecs:
> >>
> >> #define TCP_IOV_VNET    0
> >> #define TCP_IOV_ETH     1
> >> #define TCP_IOV_IP      2
> >> #define TCP_IOV_PAYLOAD 3
> >> #define TCP_IOV_NUM     4
> >> typedef struct iovec tap_iovec_t[TCP_IOV_NUM];  
> > 
> > Except for the typedef :) (I'm actively trying to discourage them)
> > ...this looks very neat.
> > 
> > I would suggest that as soon as you have something barely spitting out
> > pseudo-correct bytes, you give it a try with perf(1). I'm quite
> > concerned that we might end up killing throughput, especially without
> > vhost-user, even though in theory it all sounds nice and byte-saving.
> >   
> 
> Using iovec improves performance:
> 
> iperf3 -c localhost -p 10001  -t 60  -4
> 
>    iovec
> 
>      [  5]   0.00-60.00  sec  32.9 GBytes  4.72 Gbits/sec    0             sender
>      [  5]   0.00-60.06  sec  32.9 GBytes  4.71 Gbits/sec                  receiver
> 
>    buf_t
> 
>      [  5]   0.00-60.00  sec  15.9 GBytes  2.27 Gbits/sec    0             sender
>      [  5]   0.00-60.06  sec  15.9 GBytes  2.27 Gbits/sec                  receiver

!

...it must be some alignment matter. Nice. I guess it doesn't happen
with large windows (-w ...) or messages (-l 1M), but it looks promising
enough with small ones.

-- 
Stefano