From: Stefano Brivio <sbrivio@redhat.com>
To: passt-dev@passt.top
Subject: [PATCH 17/24] test/perf: Work-around for virtio_net hang before long streams from guest
Date: Fri, 25 Mar 2022 23:52:53 +0100 [thread overview]
Message-ID: <20220325225300.2803584-18-sbrivio@redhat.com> (raw)
In-Reply-To: <20220325225300.2803584-1-sbrivio@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 5637 bytes --]
I didn't have time to investigate the root cause for the virtio_net
TX hang yet. Add a quick work-around for the moment being.
Signed-off-by: Stefano Brivio <sbrivio(a)redhat.com>
---
test/perf/passt_tcp | 15 +++++++++++++++
test/perf/passt_udp | 15 +++++++++++++++
2 files changed, 30 insertions(+)
diff --git a/test/perf/passt_tcp b/test/perf/passt_tcp
index 99d8fb1..f13fdfc 100644
--- a/test/perf/passt_tcp
+++ b/test/perf/passt_tcp
@@ -16,6 +16,11 @@ gtools sysctl ip jq nproc seq sleep bc iperf3 tcp_rr tcp_crr # From neper
nstools sysctl ip jq nproc seq sleep bc iperf3 tcp_rr tcp_crr
htools bc head sed seq
+# In this setup, virtio_net TX queue sometimes hangs, still under investigation
+def virtio_net_workaround
+guest modprobe -r virtio_net; modprobe virtio_net napi_tx=1; dhclient; dhclient -6; sleep 3
+endef
+
test passt: throughput and latency
guest /sbin/sysctl -w net.core.rmem_max=536870912
@@ -52,18 +57,22 @@ tr TCP throughput over IPv6: guest to host
bw -
bw -
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 1280
iperf3c guest __GW6__%__IFNAME__ 100${i}2 __THREADS__ __OPTS__ -w 4M
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 1.2 1.5
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 1500
iperf3c guest __GW6__%__IFNAME__ 100${i}2 __THREADS__ __OPTS__ -w 4M
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 1.6 1.8
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 9000
iperf3c guest __GW6__%__IFNAME__ 100${i}2 __THREADS__ __OPTS__ -w 8M
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 4.0 5.0
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 65520
iperf3c guest __GW6__%__IFNAME__ 100${i}2 __THREADS__ __OPTS__ -w 16M
iperf3s BW ns 100${i}2 __THREADS__
@@ -91,26 +100,32 @@ lat __LAT__ 500 400
tr TCP throughput over IPv4: guest to host
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 256
iperf3c guest __GW__ 100${i}2 __THREADS__ __OPTS__ -w 1M
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 0.2 0.3
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 576
iperf3c guest __GW__ 100${i}2 __THREADS__ __OPTS__ -w 1M
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 0.5 0.8
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 1280
iperf3c guest __GW__ 100${i}2 __THREADS__ __OPTS__ -w 4M
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 1.2 1.5
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 1500
iperf3c guest __GW__ 100${i}2 __THREADS__ __OPTS__ -w 4M
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 1.6 1.8
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 9000
iperf3c guest __GW__ 100${i}2 __THREADS__ __OPTS__ -w 8M
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 4.0 5.0
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 65520
iperf3c guest __GW__ 100${i}2 __THREADS__ __OPTS__ -w 16M
iperf3s BW ns 100${i}2 __THREADS__
diff --git a/test/perf/passt_udp b/test/perf/passt_udp
index ff4c73a..cc5711b 100644
--- a/test/perf/passt_udp
+++ b/test/perf/passt_udp
@@ -16,6 +16,11 @@ gtools sysctl ip jq nproc sleep iperf3 udp_rr # From neper
nstools ip jq sleep iperf3 udp_rr
htools bc head sed
+# In this setup, virtio_net TX queue sometimes hangs, still under investigation
+def virtio_net_workaround
+guest modprobe -r virtio_net; modprobe virtio_net napi_tx=1; dhclient; dhclient -6; sleep 3
+endef
+
test passt: throughput and latency
guest /sbin/sysctl -w net.core.rmem_max=16777216
@@ -45,18 +50,22 @@ th MTU 256B 576B 1280B 1500B 9000B 65520B
tr UDP throughput over IPv6: guest to host
bw -
bw -
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 1280
iperf3c guest __GW6__%__IFNAME__ 100${i}2 __THREADS__ __OPTS__ -b 2G
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 0.8 1.2
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 1500
iperf3c guest __GW6__%__IFNAME__ 100${i}2 __THREADS__ __OPTS__ -b 3G
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 1.0 1.5
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 9000
iperf3c guest __GW6__%__IFNAME__ 100${i}2 __THREADS__ __OPTS__ -b 5G
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 4.0 5.0
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 65520
iperf3c guest __GW6__%__IFNAME__ 100${i}2 __THREADS__ __OPTS__ -b 7G
iperf3s BW ns 100${i}2 __THREADS__
@@ -74,26 +83,32 @@ lat __LAT__ 200 150
tr UDP throughput over IPv4: guest to host
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 256
iperf3c guest __GW__ 100${i}2 __THREADS__ __OPTS__ -b 500M
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 0.0 0.0
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 576
iperf3c guest __GW__ 100${i}2 __THREADS__ __OPTS__ -b 1G
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 0.4 0.6
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 1280
iperf3c guest __GW__ 100${i}2 __THREADS__ __OPTS__ -b 2G
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 0.8 1.2
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 1500
iperf3c guest __GW__ 100${i}2 __THREADS__ __OPTS__ -b 3G
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 1.0 1.5
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 9000
iperf3c guest __GW__ 100${i}2 __THREADS__ __OPTS__ -b 6G
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 4.0 5.0
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 65520
iperf3c guest __GW__ 100${i}2 __THREADS__ __OPTS__ -b 7G
iperf3s BW ns 100${i}2 __THREADS__
--
@@ -16,6 +16,11 @@ gtools sysctl ip jq nproc sleep iperf3 udp_rr # From neper
nstools ip jq sleep iperf3 udp_rr
htools bc head sed
+# In this setup, virtio_net TX queue sometimes hangs, still under investigation
+def virtio_net_workaround
+guest modprobe -r virtio_net; modprobe virtio_net napi_tx=1; dhclient; dhclient -6; sleep 3
+endef
+
test passt: throughput and latency
guest /sbin/sysctl -w net.core.rmem_max=16777216
@@ -45,18 +50,22 @@ th MTU 256B 576B 1280B 1500B 9000B 65520B
tr UDP throughput over IPv6: guest to host
bw -
bw -
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 1280
iperf3c guest __GW6__%__IFNAME__ 100${i}2 __THREADS__ __OPTS__ -b 2G
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 0.8 1.2
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 1500
iperf3c guest __GW6__%__IFNAME__ 100${i}2 __THREADS__ __OPTS__ -b 3G
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 1.0 1.5
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 9000
iperf3c guest __GW6__%__IFNAME__ 100${i}2 __THREADS__ __OPTS__ -b 5G
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 4.0 5.0
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 65520
iperf3c guest __GW6__%__IFNAME__ 100${i}2 __THREADS__ __OPTS__ -b 7G
iperf3s BW ns 100${i}2 __THREADS__
@@ -74,26 +83,32 @@ lat __LAT__ 200 150
tr UDP throughput over IPv4: guest to host
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 256
iperf3c guest __GW__ 100${i}2 __THREADS__ __OPTS__ -b 500M
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 0.0 0.0
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 576
iperf3c guest __GW__ 100${i}2 __THREADS__ __OPTS__ -b 1G
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 0.4 0.6
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 1280
iperf3c guest __GW__ 100${i}2 __THREADS__ __OPTS__ -b 2G
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 0.8 1.2
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 1500
iperf3c guest __GW__ 100${i}2 __THREADS__ __OPTS__ -b 3G
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 1.0 1.5
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 9000
iperf3c guest __GW__ 100${i}2 __THREADS__ __OPTS__ -b 6G
iperf3s BW ns 100${i}2 __THREADS__
bw __BW__ 4.0 5.0
+virtio_net_workaround
guest ip link set dev __IFNAME__ mtu 65520
iperf3c guest __GW__ 100${i}2 __THREADS__ __OPTS__ -b 7G
iperf3s BW ns 100${i}2 __THREADS__
--
2.35.1
next prev parent reply other threads:[~2022-03-25 22:52 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-25 22:52 [PATCH 00/24] Boundary-checked "packets", TCP timerfd timeouts, assorted fixes Stefano Brivio
2022-03-25 22:52 ` [PATCH 01/24] conf, util, tap: Implement --trace option for extra verbose logging Stefano Brivio
2022-03-25 22:52 ` [PATCH 02/24] pcap: Fix mistake in printed string Stefano Brivio
2022-03-25 22:52 ` [PATCH 03/24] util: Drop CHECK_SET_MIN_MAX{,_PROTO_FD} macros Stefano Brivio
2022-03-25 22:52 ` [PATCH 04/24] util: Use standard int types Stefano Brivio
2022-03-25 22:52 ` [PATCH 05/24] tcp: Refactor to use events instead of states, split out spliced implementation Stefano Brivio
2022-03-25 22:52 ` [PATCH 06/24] test/lib/video: Fill in href attributes of video shortcuts Stefano Brivio
2022-03-25 22:52 ` [PATCH 07/24] udp: Drop _splice from recv, send, sendto static buffer names Stefano Brivio
2022-03-25 22:52 ` [PATCH 08/24] udp: Split buffer queueing/writing parts of udp_sock_handler() Stefano Brivio
2022-03-25 22:52 ` [PATCH 09/24] dhcpv6, tap, tcp: Use IN6_ARE_ADDR_EQUAL instead of open-coded memcmp() Stefano Brivio
2022-03-25 22:52 ` [PATCH 10/24] udp: Use flags for local, loopback, and configured unicast binds Stefano Brivio
2022-03-25 22:52 ` [PATCH 11/24] Makefile: Enable a few hardening flags Stefano Brivio
2022-03-25 22:52 ` [PATCH 12/24] test: Add asciinema(1) as requirement for CI in README Stefano Brivio
2022-03-25 22:52 ` [PATCH 13/24] test, seccomp, Makefile: Switch to valgrind runs for passt functional tests Stefano Brivio
2022-03-25 22:52 ` [PATCH 14/24] tcp, udp, util: Enforce 24-bit limit on socket numbers Stefano Brivio
2022-03-25 22:52 ` [PATCH 15/24] tcp: Rework timers to use timerfd instead of periodic bitmap scan Stefano Brivio
2022-03-25 22:52 ` [PATCH 16/24] tcp_splice: Close sockets right away on high number of open files Stefano Brivio
2022-03-25 22:52 ` Stefano Brivio [this message]
2022-03-25 22:52 ` [PATCH 18/24] README: Avoid "here" links Stefano Brivio
2022-03-25 22:52 ` [PATCH 19/24] README: Update Interfaces and Availability sections Stefano Brivio
2022-03-25 22:52 ` [PATCH 20/24] tcp: Fit struct tcp_conn into a single 64-byte cacheline Stefano Brivio
2022-03-25 22:52 ` [PATCH 21/24] dhcp: Minimum option length implied by RFC 951 is 60 bytes, not 62 Stefano Brivio
2022-03-25 22:52 ` [PATCH 22/24] tcp, tcp_splice: Use less awkward syntax to swap in/out sockets from pools Stefano Brivio
2022-03-25 22:52 ` [PATCH 23/24] util: Fix function declaration style of write_pidfile() Stefano Brivio
2022-03-25 22:53 ` [PATCH 24/24] treewide: Packet abstraction with mandatory boundary checks Stefano Brivio
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220325225300.2803584-18-sbrivio@redhat.com \
--to=sbrivio@redhat.com \
--cc=passt-dev@passt.top \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://passt.top/passt
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for IMAP folder(s).