Re: Problematic throughput numbers

[Stefano Brivio <sbrivio@redhat.com>]

On Wed, 25 Sep 2024 12:36:43 -0400
Jon Maloy <jmaloy@redhat.com> wrote:

> I made many runs with iperf3 ns->host, and the results are puzzling me.

This has nothing to do with the path affected by your patches, because
you're connecting to a loopback address, and that's dealt with by
spliced connections. See tcp_splice.c, and:

  https://passt.top/#pasta-pack-a-subtle-tap-abstraction
  "Handling of local traffic in pasta" in pasta(1)

for more details. If you want to try out your patches, you could run
the test suite, or use a non-loopback address from the container.

> Over and over again, I see throughput practically collapse, with a two 
> orders of magnitude decrease.
> Just to make sure this wasn't something introduced by me, I went back to 
> the master branch and disabled the SO_PEEK_OFF feature.
> The result was the same.
> The below log shows typical run, but it is highly variable.
> Sometimes almost the whole series is in the 50-70 Gb/s range, and 
> sometimes almost all in the 100-300 Mb/s range.
> When I added the kernel fix it didn't seem to make any difference.
> 
> To me this is really worrying, and should be investigated.

I can't reproduce this, and we didn't have user reports of anything of
this sort, so far.

To investigate this, I would suggest that you have a look with strace
(as root) at what pasta is doing once the throughput decreases.

Packet captures with -p / --pcap won't show anything, because we don't
actually deal with packets on the spliced path.

Using --trace and a log file might help, but you'll probably decrease
throughput enough, that way, that the "good" condition won't be
distinguishable from the "bad" one.

-- 
Stefano