Re: [PATCH v2] RFC: Benchmarking script

[David Gibson <david@gibson.dropbear.id.au>]

[-- Attachment #1: Type: text/plain, Size: 2726 bytes --]

On Fri, Apr 05, 2024 at 08:10:26PM +0200, Stefano Brivio wrote:
> On Fri,  5 Apr 2024 17:12:21 +1100
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> > Although we make some performance measurements in our regular testsuite,
> > those are designed more for getting a quick(ish) rough idea of the
> > performance, rather than a more precise measurement.
> > 
> > This patch adds a Python script in contrib/benchmark which can make more
> > detailed benchmarks.  It can test both passt & pasta themselves, but also
> > various other scenarios for comparison, such as kernel veth, qemu -net tap
> > and slirp (either qemu -net user or slirp4netns).
> 
> Hah, nice. I haven't tried or reviewed this yet, but I just realised
> one thing: iperf3 3.16 finally implements separate streams (-P) as
> multiple threads! See:
> 
>   https://github.com/esnet/iperf/pull/1591

I hadn't realised it was just a new addition, but I did notice that -P
used multiple threads

> or release notes. That also means that the whole parallel process
> nonsense in the regular suite can finally go away, I guess. I haven't
> tested that yet, though.

Yes, that would make things notably less messy.

This script doesn't yet use parallel streams - I need to make it do
that though, because I'll need those results for my talk in a week/

> By the way of that, you mentioned in the past that you had some
> throughput failures with UDP tests. Well, I looked into 3.16 changes
> because of that -- they started failing for me as well with the new
> version. I temporarily reverted back to 3.14 on my system, until we
> figure out how to adjust to the new meaning of the "-P" option.

Ok, good to know.  In any case the UDP "throughput" tests as they
stand are basically nonsense.  Testing UDP throughput needs a fancier
method, basically increasing the rate until we start dropping packets.
iperf3 doesn't do that (and tools that do seem to be kinda hard to
find).  At the moment we basically just have some hardcoded target
rates, so really all these tests are doing is saying "can we reach
this target throughput, where this target was selected with no
consideration of the capabilities of this host system".

> Another thing that occurred to me: it would probably be helpful to
> already have vhost-user cases for passt here.

Yes, it would.  And in particular those would be useful for my talk as
well.

> Anyway, I'll give this a try soon. I can also apply it right away if
> you prefer.
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]