From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by passt.top (Postfix) with ESMTPS id 5B5905A0082 for ; Tue, 1 Nov 2022 01:59:35 +0100 (CET) Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4N1WrK5pRLz4xN5; Tue, 1 Nov 2022 11:59:29 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=201602; t=1667264369; bh=Rvch8TlhaBOCCS8N6D9LeXYM/92vwlrfLLFs9CFhjko=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=in+WdlZSRBlAkcrhJFgdDomPLftv4a8fJnCUEz4H+pjBcUew0BSqLiQvQoXzBrgd+ KdD0GHzBMdtz33Z2TLHJcg40JsmC5MCO1Rcl5WescTYyk9RrcW+8KEYEcfP89yfLLm gLl1ag1Vdf3nSTT6EOwU1AtqALKQc4WxXNhEGKDQ= Date: Tue, 1 Nov 2022 11:58:26 +1100 From: David Gibson To: Stefano Brivio Subject: Re: [PATCH v2 5/5] test: Add memory/passt test cases Message-ID: References: <20221031112059.170269-1-sbrivio@redhat.com> <20221031112059.170269-6-sbrivio@redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="fCFZX70FAfmUrApC" Content-Disposition: inline In-Reply-To: <20221031112059.170269-6-sbrivio@redhat.com> Message-ID-Hash: U2IUWP3VUUJPDRJLLPY5FXY4FQIMQYUK X-Message-ID-Hash: U2IUWP3VUUJPDRJLLPY5FXY4FQIMQYUK X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.3 Precedence: list List-Id: Development discussion and patches for passt Archived-At: <> Archived-At: List-Archive: <> List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --fCFZX70FAfmUrApC Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Oct 31, 2022 at 12:20:59PM +0100, Stefano Brivio wrote: > These show a summary of memory usage in kernel and userspace with > different port forwarding configurations, details of userspace usage > using 'nm' (passt only uses statically allocated memory), and details > of kernel memory from slab reporting facilities. >=20 > This adds a new test image, mbuto.mem.img, with harcoded IPv4 and > IPv6 addresses and routes, and just the tools we need to start and > stop passt, to report from /proc/slabinfo, /proc/meminfo, and to > print and parse symbol sizes using nm(1). >=20 > passt can't pivot_root() for sandboxing purposes on ramfs, so we need > to create another filesystem and chroot into it, first. Huh.. weird. > We don't want to use pane context functions, as we're checking memory > usage for sockets: resort to screen-scraping. So, we don't do it at the moment, but it should be pretty easy to make the ssh-over-vsock stuff use a persistent control socket (ControlPersist / ControlMaster). That would mean we'd just establish one connection then all the subsequent guest bound commands would go over that one initial connection. It wouldn't take zero socket memory of course, but it should be small and more importantly stable. So, if we take a diff of usage as you already do, the results should still be good. This patch is not working for me, for reasons I'm still debugging. Some more minor comments in the interim. >=20 > Signed-off-by: Stefano Brivio > --- > test/Makefile | 5 +- > test/lib/layout_ugly | 39 +++++++++ > test/lib/setup_ugly | 37 +++++++++ > test/memory/passt | 187 +++++++++++++++++++++++++++++++++++++++++++ > test/passt.mem.mbuto | 42 ++++++++++ > test/run | 4 + > 6 files changed, 313 insertions(+), 1 deletion(-) > create mode 100644 test/memory/passt > create mode 100755 test/passt.mem.mbuto >=20 > diff --git a/test/Makefile b/test/Makefile > index 91498ff..9f2bc42 100644 > --- a/test/Makefile > +++ b/test/Makefile > @@ -55,7 +55,7 @@ UBUNTU_IMGS =3D $(UBUNTU_OLD_IMGS) $(UBUNTU_NEW_IMGS) > DOWNLOAD_ASSETS =3D mbuto \ > $(DEBIAN_IMGS) $(FEDORA_IMGS) $(OPENSUSE_IMGS) $(UBUNTU_IMGS) > TESTDATA_ASSETS =3D small.bin big.bin medium.bin > -LOCAL_ASSETS =3D mbuto.img QEMU_EFI.fd \ > +LOCAL_ASSETS =3D mbuto.img mbuto.mem.img QEMU_EFI.fd \ > $(DEBIAN_IMGS:%=3Dprepared-%) $(FEDORA_IMGS:%=3Dprepared-%) \ > $(UBUNTU_NEW_IMGS:%=3Dprepared-%) \ > nsholder guest-key guest-key.pub \ > @@ -76,6 +76,9 @@ guest-key guest-key.pub: > mbuto.img: passt.mbuto mbuto guest-key.pub $(TESTDATA_ASSETS) > ./mbuto/mbuto -p ./$< -c lz4 -f $@ > =20 > +mbuto.mem.img: passt.mem.mbuto mbuto ../passt.avx2 > + ./mbuto/mbuto -p ./$< -c lz4 -f $@ > + > nsholder: nsholder.c > $(CC) $(CFLAGS) -o $@ $^ > =20 > diff --git a/test/lib/layout_ugly b/test/lib/layout_ugly > index 9397b7d..d62d337 100644 > --- a/test/lib/layout_ugly > +++ b/test/lib/layout_ugly > @@ -81,3 +81,42 @@ layout_pasta_simple() { > =20 > sleep 1 > } > + > +# layout_memory() - Screen-scraped panes for memory usage tests, big gue= st pane > +layout_memory() { > + sleep 3 > + > + tmux kill-pane -a -t 0 > + cmd_write 0 clear > + > + tmux split-window -h -l '35%' -t passt_test > + tmux split-window -v -l '15%' -t passt_test > + > + PANE_PASST=3D2 > + PANE_GUEST=3D0 > + PANE_INFO=3D1 > + > + get_info_cols > + > + tmux send-keys -l -t ${PANE_INFO} 'while cat '"$STATEBASE/log_pipe"'; d= o :; done' > + tmux send-keys -t ${PANE_INFO} -N 100 C-m > + tmux select-pane -t ${PANE_INFO} -T "test log" > + > + if context_exists guest; then > + pane_watch_contexts ${PANE_GUEST} guest guest > + else > + tmux pipe-pane -O -t ${PANE_GUEST} "cat >> ${LOGDIR}/pane_guest.log" > + tmux select-pane -t ${PANE_GUEST} -T "guest" > + fi > + > + if context_exists passt; then AFAICT you're always using a context pane for passt, so you don't need this conditional. > + pane_watch_contexts ${PANE_PASST} passt passt > + else > + tmux pipe-pane -O -t ${PANE_PASST} "cat >> ${LOGDIR}/pane_passt.log" > + tmux select-pane -t ${PANE_PASST} -T "passt" > + fi > + > + info_layout "memory usage" > + > + sleep 1 > +} > diff --git a/test/lib/setup_ugly b/test/lib/setup_ugly > index 764177e..f2e07ba 100755 > --- a/test/lib/setup_ugly > +++ b/test/lib/setup_ugly > @@ -13,6 +13,8 @@ > # Copyright (c) 2022 Red Hat GmbH > # Author: Stefano Brivio > =20 > +INITRAMFS_MEM=3D"${BASEPATH}/mbuto.mem.img" > + > # setup_distro() - Set up pane layout for distro tests > setup_distro() { > layout_host > @@ -25,6 +27,30 @@ setup_pasta_options() { > layout_pasta_simple > } > =20 > +# setup_memory() - Start qemu in guest pane, and passt in passt context > +setup_memory() { > + context_setup_host passt > + > + layout_memory > + > + context_run passt "./passt -P ${STATESETUP}/passt.pid" > + > + # pidfile isn't created until passt is listening > + wait_for [ -f "${STATESETUP}/passt.pid" ] > + > + pane_or_context_run guest './qrap 5 qemu-system-$(uname -m)' \ > + ' -machine accel=3Dkvm' \ > + ' -m '${VMEM}' -cpu host -smp '${VCPUS} \ > + ' -kernel ' "/boot/vmlinuz-$(uname -r)" \ > + ' -initrd '${INITRAMFS_MEM}' -nographic -serial stdio' \ > + ' -nodefaults' \ > + ' -append "console=3DttyS0 mitigations=3Doff apparmor=3D0 ' \ > + 'virtio-net.napi_tx=3D1"' \ > + " -device virtio-net-pci,netdev=3Dhostnet0,x-txburst=3D16384" \ > + " -netdev socket,fd=3D5,id=3Dhostnet0" \ > + " -pidfile ${STATESETUP}/qemu.pid" > +} > + > # teardown_distro() - Nothing to do, yet > teardown_distro() { > : > @@ -36,3 +62,14 @@ teardown_pasta_options() { > teardown_context_watch ${PANE_PASST} passt > } > =20 > +# teardown_passt() - Kill qemu with ^C, remove passt PID file > +teardown_memory() { > + kill $(cat "${STATESETUP}/qemu.pid") > + > + rm "${STATESETUP}/passt.pid" > + > + tmux send-keys -t ${PANE_PASST} "C-c" You shouldn't need this for the passt pane, since it is context based, the teardown_context_watch will do it. > + teardown_context_watch ${PANE_PASST} passt > + teardown_context_watch ${PANE_GUEST} qemu Likewise you don't want this for the guest pane since it is *not* context based. =20 > +} > diff --git a/test/memory/passt b/test/memory/passt > new file mode 100644 > index 0000000..fa89821 > --- /dev/null > +++ b/test/memory/passt > @@ -0,0 +1,187 @@ > +# SPDX-License-Identifier: AGPL-3.0-or-later > +# > +# PASST - Plug A Simple Socket Transport > +# for qemu/UNIX domain socket mode > +# > +# PASTA - Pack A Subtle Tap Abstraction > +# for network namespace/tap device mode > +# > +# test/memory/passt - Show memory usage of passt in kernel and userspace > +# > +# Copyright (c) 2022 Red Hat GmbH > +# Author: Stefano Brivio > + > +gtools sed cat diff nm sort kill tee head tail chroot unshare mount mkdi= r cp=20 > + > +def meminfo_row > +gout DIFF meminfo_diff /tmp/meminfo.before /tmp/meminfo.after __WHAT__ > +tl __NAME__ > +td __DIFF__ 3 0 0 > +endef > + > +def meminfo_reverse_row > +gout DIFF meminfo_diff /tmp/meminfo.after /tmp/meminfo.before __WHAT__ > +tl __NAME__ > +td __DIFF__ 3 0 0 > +endef > + > +def nm_row > +gout SIZE nm_size /tmp/nm.size __WHAT__ > +tl __WHAT__ > +td __SIZE__ 6 0 0 > +endef > + > +def slab_row > +gout COUNT slab_diff_count /tmp/slabinfo.before /tmp/slabinfo.after __WH= AT__ > +gout SIZE slab_size /tmp/slabinfo.before __WHAT__ > +gout DIFF slab_diff_size /tmp/slabinfo.before /tmp/slabinfo.after __WHAT= __ > +tl __WHAT__ > +td __COUNT__ 0 0 0 > +td __SIZE__ 0 0 0 > +td __DIFF__ 6 0 0 > +endef > + > +def start_stop_diff > +guest sed /proc/slabinfo -ne 's/^\([^ ]* *[^ ]* *[^ ]* *[^ ]*\).*/\\\1/p= ' > /tmp/slabinfo.before > +guest cat /proc/meminfo > /tmp/meminfo.before > +guest /usr/bin/passt.avx2 -l /tmp/log -s /tmp/sock -P /tmp/pid __OPTS__ = --netns-only > +sleep 2 > +guest cat /proc/meminfo > /tmp/meminfo.after > +guest sed /proc/slabinfo -ne 's/^\([^ ]* *[^ ]* *[^ ]* *[^ ]*\).*/\\\1/p= ' > /tmp/slabinfo.after > +guest kill \$(cat /tmp/pid) > +guest diff -y --suppress-common-lines /tmp/meminfo.before /tmp/meminfo.a= fter || : > +guest nm -td -Sr --size-sort -P /usr/bin/passt.avx2 | head -30 | tee /tm= p/nm.size > +guest sed /proc/slabinfo -ne 's/\(.*\).*$/\1/p' | tail -1; (dif= f -y --suppress-common-lines /tmp/slabinfo.before /tmp/slabinfo.after | sor= t -grk8) > +endef > + > +def summary > +info Memory usage summary > +info=09 > +th type MiB > +set WHAT MemFree > +set NAME used > +meminfo_reverse_row > +set WHAT AnonPages > +set NAME userspace > +meminfo_row > +set WHAT Slab > +set NAME kernel > +meminfo_row > +te > +endef > + > + > +guest mkdir /test > +guest mount -t tmpfs none /test > +guest mkdir /test/proc /test/dev /test/tmp > +guest mount -o bind /proc /test/proc > +guest mount -o bind /dev /test/dev > +guest cp -Lr /bin /lib /lib64 /usr /sbin /test/ > + > +guest ulimit -Hn 300000 > +guest unshare -rUm -R /test > +guest chroot . > + > +guest meminfo_size() { grep "^$2:" $1 | tr -s ' ' | cut -f2 -d ' '; } > +guest meminfo_diff() { echo $(( $(meminfo_size $2 $3) - $(meminfo_size $= 1 $3) )); } > + > +guest nm_size() { grep -m1 "^$2 " $1 | cut -f4 -d ' '; } > + > +guest slab_count() { grep "^$2 " $1 | tr -s ' ' | cut -f3 -d ' '; } > +guest slab_size() { grep "^$2 " $1 | tr -s ' ' | cut -f4 -d ' '; } > +guest slab_diff_count() { echo $(( $(slab_count $2 $3) - $(slab_count $1= $3) )); } > +guest slab_diff_size() { echo $(( $(slab_count $2 $3) * $(slab_size $2 $= 3) - $(slab_count $1 $3) * $(slab_size $1 $3) )); } > + > + > +test Memory usage: all TCP and UDP ports forwarded, IPv4 and IPv6 > +set OPTS -t all -u all > +start_stop_diff > +summary > + > +info Userspace memory detail > +info=09 > +th symbol MiB > +set WHAT tcp_buf_discard > +nm_row > +set WHAT tcp6_l2_buf > +nm_row > +set WHAT tcp4_l2_buf > +nm_row > +set WHAT tc > +nm_row > +set WHAT pkt_buf > +nm_row > +set WHAT udp_splice_map > +nm_row > +set WHAT udp6_l2_buf > +nm_row > +set WHAT udp4_l2_buf > +nm_row > +set WHAT udp_tap_map > +nm_row > +set WHAT icmp_id_map > +nm_row > +set WHAT udp_splice_buf > +nm_row > +set WHAT tc_hash > +nm_row > +set WHAT pool_tap6_storage > +nm_row > +set WHAT pool_tap4_storage > +nm_row > +set WHAT tap6_l4 > +nm_row > +set WHAT tap4_l4 > +nm_row > +te > + > +info Kernel memory detail > +info=09 > +th objects count size MiB > +set WHAT pid > +slab_row > +set WHAT dentry > +slab_row > +set WHAT Acpi-Parse > +slab_row > +set WHAT kmalloc-64 > +slab_row > +set WHAT kmalloc-32 > +slab_row > +set WHAT lsm_file_cache > +slab_row > +set WHAT filp > +slab_row > +set WHAT anon_vma_chain > +slab_row > +set WHAT ep_head > +slab_row > +set WHAT sock_inode_cache > +slab_row > +set WHAT signal_cache > +slab_row > +set WHAT TCPv6 > +slab_row > +set WHAT TCP > +slab_row > +set WHAT UDPv6 > +slab_row > +te > + > + > +test Memory usage: all TCP ports forwarded, IPv4 > +set OPTS -t all -4 > +start_stop_diff > +summary > + > + > +test Memory usage: all TCP and UDP ports forwarded, IPv4 > +set OPTS -t all -u all -4 > +start_stop_diff > +summary > + > + > +test Memory usage: no ports forwarded > +set OPTS -t none -u none > +start_stop_diff > +summary > diff --git a/test/passt.mem.mbuto b/test/passt.mem.mbuto > new file mode 100755 > index 0000000..1550c15 > --- /dev/null > +++ b/test/passt.mem.mbuto > @@ -0,0 +1,42 @@ > +#!/bin/sh > +# > +# SPDX-License-Identifier: AGPL-3.0-or-later > +# > +# PASST - Plug A Simple Socket Transport > +# for qemu/UNIX domain socket mode > +# > +# test/passt.mem.mbuto - mbuto (https://mbuto.sh) profile for memory usa= ge tests > +# > +# Copyright (c) 2022 Red Hat GmbH > +# Author: Stefano Brivio > + > +PROGS=3D"${PROGS:-ash,dash,bash chmod ip mount insmod mkdir ln cat chmod= modprobe > + grep mknod sed chown sleep bc ls ps mount unshare chroot cp kill = diff > + head tail sort tr tee cut nm which}" > + > +KMODS=3D"${KMODS:- virtio_net virtio_pci vmw_vsock_virtio_transport}" > + > +NODES=3D"${NODES:-console kmsg null ptmx random urandom zero}" > + > +LINKS=3D"${LINKS:- > + ash,dash,bash /init > + ash,dash,bash /bin/sh}" > + > +DIRS=3D"${DIRS} /tmp /sbin" > + > +COPIES=3D"${COPIES} ../passt.avx2,/usr/bin/passt.avx2" > + > +FIXUP=3D"${FIXUP}"' > +chmod 777 /tmp > +ip link set eth0 up > +ip address add 192.0.2.2/24 dev eth0 > +ip address add 2001:db8::2/64 dev eth0 > +ip route add default via 192.0.2.1 > +ip -6 route add default via 2001:db8::1 dev eth0 > +sleep 2 > +sh +m > +' > + > +OUTPUT=3D"KERNEL=3D__KERNEL__ > +INITRD=3D__INITRD__ > +" > diff --git a/test/run b/test/run > index c3486b9..e07513f 100755 > --- a/test/run > +++ b/test/run > @@ -79,6 +79,10 @@ run() { > test pasta_options/log_to_file > teardown pasta_options > =20 > + setup memory > + test memory/passt > + teardown memory > + > setup passt > test passt/ndp > test passt/dhcp --=20 David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson --fCFZX70FAfmUrApC Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEoULxWu4/Ws0dB+XtgypY4gEwYSIFAmNgbysACgkQgypY4gEw YSLx4BAAyn1qIkedodDeU/heIPZlT8VPxaunQ/9a/3ZsrBWJUSr9JOGK2fFb9vut vAFQuLkCY6jmgaYytEQfWGTqt2mPVRsMZF29BMxLx0+jUA+LGtWgWMOcyTqR4Gtt JOhlTzeYENzfJ/NYjgmHcGaTYdfHd3O8NRMxJTHfS4RTeb1aizxfivlTYhyUTO7K bZuLb9E04BTLYUPllesiacgeTyTt2wM83kiVhJ79NNoYsyku5+WTv0COc23QiVhe pNkXSmjoMF1Y3N+dsZonczoksnq6rXAUR7sByIimu4IJVsQ4ZYgoxgHrk1Fs5FEA lwCvGtx4tGRUqDcfzkM0cLENT22nt7noshSALUqNR63bPi6+E1mZnYwuN3gcd/eS GhwTVGzU+3TF7Ecv9cgzVV6jnDD5pGDaGdmEyz0Z2Z61q17ZZRQYq14DYFCAnW9I CucnN1uoUiQLChZbnnNF/9PCOuan/wBgr7azFyG+j2n3B6orw1wFxnEOxYgKUi5C ItvOier3NXfgR+DwcydI8KsMGK87JzHiRjuu1B4d3QQAQMKLDu/P4WJzJ2O4bGBb mNIJ2wPs6XsziehhrRcnewGQN+ZT4fDVFOhVQ5/ERe/BS77X1siTMJju1AAlGE0u VJ7xQ8vFJ0abtAyFP3NUgEuxl+Xq1UqdatKyXesVrKKJJudbJuo= =NMw4 -----END PGP SIGNATURE----- --fCFZX70FAfmUrApC--