Re: [PATCH 0/4] Some improvements to the tap send path

[Laurent Vivier <lvivier@redhat.com>]

On 3/8/24 09:34, Stefano Brivio wrote:
> On Fri, 8 Mar 2024 09:18:48 +0100
> Laurent Vivier <lvivier@redhat.com> wrote:
> 
>> On 3/8/24 07:53, David Gibson wrote:
>>> This series has a handful of small improvements to the tap send path.
>>> See individual commit messages for the details.
>>>
>>> I expect this will conflict with Laurent's upcoming work.  I hope the
>>> conflicts won't be too bad, and indeed will set us up for less
>>> duplication there in the end.
>>
>> I'm working on patch that devides TCP buffers in several buffers pointed out by an IOV
>> arrays and then provided to tap_send_frames(). I'm going to base my patch on this series.
>>
>> The idea is:
>>
>> A frame is made with 4 iovecs:
>>
>> #define TCP_IOV_VNET    0
>> #define TCP_IOV_ETH     1
>> #define TCP_IOV_IP      2
>> #define TCP_IOV_PAYLOAD 3
>> #define TCP_IOV_NUM     4
>> typedef struct iovec tap_iovec_t[TCP_IOV_NUM];
> 
> Except for the typedef :) (I'm actively trying to discourage them)
> ...this looks very neat.
> 
> I would suggest that as soon as you have something barely spitting out
> pseudo-correct bytes, you give it a try with perf(1). I'm quite
> concerned that we might end up killing throughput, especially without
> vhost-user, even though in theory it all sounds nice and byte-saving.
> 

Using iovec improves performance:

iperf3 -c localhost -p 10001  -t 60  -4

   iovec

     [  5]   0.00-60.00  sec  32.9 GBytes  4.72 Gbits/sec    0             sender
     [  5]   0.00-60.06  sec  32.9 GBytes  4.71 Gbits/sec                  receiver

   buf_t

     [  5]   0.00-60.00  sec  15.9 GBytes  2.27 Gbits/sec    0             sender
     [  5]   0.00-60.06  sec  15.9 GBytes  2.27 Gbits/sec                  receiver

iperf3 -c localhost -p 10001  -t 60  -6

   iovec

     [  5]   0.00-60.00  sec  26.0 GBytes  3.73 Gbits/sec    0             sender
     [  5]   0.00-60.06  sec  26.0 GBytes  3.72 Gbits/sec                  receiver

   buf_t

     [  5]   0.00-60.00  sec  15.7 GBytes  2.25 Gbits/sec    0             sender
     [  5]   0.00-60.07  sec  15.7 GBytes  2.25 Gbits/sec                  receiver

Thanks,
Laurent