[PATCH 00/12] Assorted test fixes, batch 7

[PATCH 00/12] Assorted test fixes, batch 7

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1557 bytes --]

Here's another bundle of test fixes, focused on making the tests
faster and more reliable.

Based on my earlier series cleaning up the port forwarding data
structures.

David Gibson (12):
  test: Add wait_for() shell helper
  test: Remove unnecessary sleeps from layout functions
  test: Remove unnecessary sleeps from shutdown tests
  test: More robust wait for pasta/passt to be ready
  test: Use --config-net for namespace setup
  test: Simplify data handling for transfer tests
  test: Remove unneccessary pane naming from layout_two_guests
  clang-tidy: Disable 'readability-identifier-length'
  cppcheck: Avoid excessive scanning due to system headers
  cppcheck: Run quietly
  Makefile: Simplify getting target triple for compiler
  cppcheck: Add target specific headers

 Makefile                  |  32 +++----
 test/.gitignore           |   1 +
 test/Makefile             |  15 ++-
 test/lib/context          |   4 +-
 test/lib/layout           |  24 -----
 test/lib/setup            |  32 +++----
 test/lib/util             |   8 ++
 test/passt.mbuto          |   6 +-
 test/passt/shutdown       |   1 -
 test/passt/tcp            |  53 +++++------
 test/passt/udp            |  29 +++---
 test/passt_in_ns/shutdown |   1 -
 test/passt_in_ns/tcp      | 187 ++++++++++++++++----------------------
 test/passt_in_ns/udp      |  93 ++++++++-----------
 test/pasta/tcp            |  79 +++++++---------
 test/pasta/udp            |  43 ++++-----
 test/two_guests/basic     |   2 +-
 17 files changed, 250 insertions(+), 360 deletions(-)

-- 
2.37.3

[PATCH 01/12] test: Add wait_for() shell helper

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1196 bytes --]

Add a shell helper function to wait for some command to succeed - typically
a test for something to be done by a background process.  Use it in the
context code which waits for the guest to respond to ssh-over-vsock
connections.

Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
---
 test/lib/context | 4 +---
 test/lib/util    | 8 ++++++++
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/test/lib/context b/test/lib/context
index 43b00dd..ee6b683 100644
--- a/test/lib/context
+++ b/test/lib/context
@@ -63,9 +63,7 @@ EOF
 	echo "ssh -F ${__ssh} ${__name}" > "${__enter}"
 
 	# Wait for the guest to be booted and accepting connections
-	while ! ssh -F "${__ssh}" "${__name}" :; do
-		sleep 0.1
-	done
+	wait_for ssh -F "${__ssh}" "${__name}" :
 }
 
 # context_teardown() - Remove a context (leave log files intact)
diff --git a/test/lib/util b/test/lib/util
index dee6c8d..72023ab 100755
--- a/test/lib/util
+++ b/test/lib/util
@@ -123,3 +123,11 @@ get_info_cols() {
 		__j=$((__j + 1))
 	done
 }
+
+# wait_for() - Retry a command until it succeeds
+# $@:	Command to run
+wait_for() {
+        while ! "$@"; do
+                sleep 0.1
+        done
+}
-- 
2.37.3

Re: [PATCH 01/12] test: Add wait_for() shell helper

[Stefano Brivio]

[-- Attachment #1: Type: text/plain, Size: 1486 bytes --]

On Fri, 23 Sep 2022 17:20:27 +1000
David Gibson <david(a)gibson.dropbear.id.au> wrote:

> Add a shell helper function to wait for some command to succeed - typically
> a test for something to be done by a background process.  Use it in the
> context code which waits for the guest to respond to ssh-over-vsock
> connections.
> 
> Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
> ---
>  test/lib/context | 4 +---
>  test/lib/util    | 8 ++++++++
>  2 files changed, 9 insertions(+), 3 deletions(-)
> 
> diff --git a/test/lib/context b/test/lib/context
> index 43b00dd..ee6b683 100644
> --- a/test/lib/context
> +++ b/test/lib/context
> @@ -63,9 +63,7 @@ EOF
>  	echo "ssh -F ${__ssh} ${__name}" > "${__enter}"
>  
>  	# Wait for the guest to be booted and accepting connections
> -	while ! ssh -F "${__ssh}" "${__name}" :; do
> -		sleep 0.1
> -	done
> +	wait_for ssh -F "${__ssh}" "${__name}" :
>  }
>  
>  # context_teardown() - Remove a context (leave log files intact)
> diff --git a/test/lib/util b/test/lib/util
> index dee6c8d..72023ab 100755
> --- a/test/lib/util
> +++ b/test/lib/util
> @@ -123,3 +123,11 @@ get_info_cols() {
>  		__j=$((__j + 1))
>  	done
>  }
> +
> +# wait_for() - Retry a command until it succeeds
> +# $@:	Command to run
> +wait_for() {
> +        while ! "$@"; do
> +                sleep 0.1

While at it, could we replace this with sleep 0.1 || sleep 1, just in
case we happen to run in an environment not supporting sleep 0.1?

-- 
Stefano

Re: [PATCH 01/12] test: Add wait_for() shell helper

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1915 bytes --]

On Sat, Sep 24, 2022 at 01:07:46AM +0200, Stefano Brivio wrote:
> On Fri, 23 Sep 2022 17:20:27 +1000
> David Gibson <david(a)gibson.dropbear.id.au> wrote:
> 
> > Add a shell helper function to wait for some command to succeed - typically
> > a test for something to be done by a background process.  Use it in the
> > context code which waits for the guest to respond to ssh-over-vsock
> > connections.
> > 
> > Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
> > ---
> >  test/lib/context | 4 +---
> >  test/lib/util    | 8 ++++++++
> >  2 files changed, 9 insertions(+), 3 deletions(-)
> > 
> > diff --git a/test/lib/context b/test/lib/context
> > index 43b00dd..ee6b683 100644
> > --- a/test/lib/context
> > +++ b/test/lib/context
> > @@ -63,9 +63,7 @@ EOF
> >  	echo "ssh -F ${__ssh} ${__name}" > "${__enter}"
> >  
> >  	# Wait for the guest to be booted and accepting connections
> > -	while ! ssh -F "${__ssh}" "${__name}" :; do
> > -		sleep 0.1
> > -	done
> > +	wait_for ssh -F "${__ssh}" "${__name}" :
> >  }
> >  
> >  # context_teardown() - Remove a context (leave log files intact)
> > diff --git a/test/lib/util b/test/lib/util
> > index dee6c8d..72023ab 100755
> > --- a/test/lib/util
> > +++ b/test/lib/util
> > @@ -123,3 +123,11 @@ get_info_cols() {
> >  		__j=$((__j + 1))
> >  	done
> >  }
> > +
> > +# wait_for() - Retry a command until it succeeds
> > +# $@:	Command to run
> > +wait_for() {
> > +        while ! "$@"; do
> > +                sleep 0.1
> 
> While at it, could we replace this with sleep 0.1 || sleep 1, just in
> case we happen to run in an environment not supporting sleep 0.1?

Good idea, I'll respin with that change.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 02/12] test: Remove unnecessary sleeps from layout functions

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 2097 bytes --]

These make sense for displaying in the demo, but not for automated
testing.

Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
---
 test/lib/layout | 20 --------------------
 1 file changed, 20 deletions(-)

diff --git a/test/lib/layout b/test/lib/layout
index cf319bf..9b16c4d 100644
--- a/test/lib/layout
+++ b/test/lib/layout
@@ -15,8 +15,6 @@
 
 # layout_host() - Simple host commands layout with info and host panes
 layout_host() {
-	sleep 3
-
 	tmux kill-pane -a -t 0
 	cmd_write 0 clear
 
@@ -39,14 +37,10 @@ layout_host() {
 	fi
 
 	info_layout "host commands only"
-
-	sleep 1
 }
 
 # layout_pasta() - Panes for host, pasta, and separate one for namespace
 layout_pasta() {
-	sleep 3
-
 	tmux kill-pane -a -t 0
 	cmd_write 0 clear
 
@@ -70,14 +64,10 @@ layout_pasta() {
 	pane_watch_contexts ${PANE_NS} "namespace" unshare ns
 
 	info_layout "single pasta instance with namespace"
-
-	sleep 1
 }
 
 # layout_passt() - Panes for host, passt, and guest
 layout_passt() {
-	sleep 3
-
 	tmux kill-pane -a -t 0
 	cmd_write 0 clear
 
@@ -101,14 +91,10 @@ layout_passt() {
 	pane_watch_contexts ${PANE_GUEST} guest qemu guest
 
 	info_layout "single passt instance with guest"
-
-	sleep 1
 }
 
 # layout_passt_in_pasta() - Host, passt within pasta, namespace and guest
 layout_passt_in_pasta() {
-	sleep 3
-
 	tmux kill-pane -a -t 0
 	cmd_write 0 clear
 
@@ -137,14 +123,10 @@ layout_passt_in_pasta() {
 	pane_watch_contexts ${PANE_PASST} "passt in pasta (namespace)" pasta passt
 
 	info_layout "passt and guest in namespace, connected by pasta"
-
-	sleep 1
 }
 
 # layout_two_guests() - Two guest panes, two passt panes, plus host and log
 layout_two_guests() {
-	sleep 3
-
 	tmux kill-pane -a -t 0
 	cmd_write 0 clear
 
@@ -180,8 +162,6 @@ layout_two_guests() {
 	pane_watch_contexts ${PANE_PASST_2} "passt #2 in namespace #2" pasta_2 passt_2
 
 	info_layout "two guests, two passt instances, in namespaces"
-
-	sleep 1
 }
 
 # layout_demo_pasta() - Four panes for pasta demo
-- 
2.37.3

Re: [PATCH 02/12] test: Remove unnecessary sleeps from layout functions

[Stefano Brivio]

[-- Attachment #1: Type: text/plain, Size: 625 bytes --]

On Fri, 23 Sep 2022 17:20:28 +1000
David Gibson <david(a)gibson.dropbear.id.au> wrote:

> These make sense for displaying in the demo, but not for automated
> testing.

Actually, they kind of do for the CI terminal captures: especially the
sleep 3 before closing panes is helpful to easily pause on the results
of the last test that was run there.

Also the sleep 1 after setting up the new layout gives me a bit more
time to realise what's going on.

I know, perhaps it's not a very common use case, but on the other hand
we're just saving 20 seconds altogether, it doesn't feel like it's that
much right now.

-- 
Stefano

Re: [PATCH 02/12] test: Remove unnecessary sleeps from layout functions

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1110 bytes --]

On Sat, Sep 24, 2022 at 01:08:01AM +0200, Stefano Brivio wrote:
> On Fri, 23 Sep 2022 17:20:28 +1000
> David Gibson <david(a)gibson.dropbear.id.au> wrote:
> 
> > These make sense for displaying in the demo, but not for automated
> > testing.
> 
> Actually, they kind of do for the CI terminal captures: especially the
> sleep 3 before closing panes is helpful to easily pause on the results
> of the last test that was run there.
> 
> Also the sleep 1 after setting up the new layout gives me a bit more
> time to realise what's going on.

Hmm, I guess.  I guess I find the idea of looking back at video of
tests rather than logs a bit alien.

> I know, perhaps it's not a very common use case, but on the other hand
> we're just saving 20 seconds altogether, it doesn't feel like it's that
> much right now.

I suppose.  It is rather more significant when restricting the tests
down to a smaller handful for debugging.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 03/12] test: Remove unnecessary sleeps from shutdown tests

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 850 bytes --]

These are hangovers from older ways of shutting down the pasta/passt
processes and no longer serve any purpose.

Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
---
 test/passt/shutdown       | 1 -
 test/passt_in_ns/shutdown | 1 -
 2 files changed, 2 deletions(-)

diff --git a/test/passt/shutdown b/test/passt/shutdown
index 34ddb64..e309ac6 100644
--- a/test/passt/shutdown
+++ b/test/passt/shutdown
@@ -16,5 +16,4 @@ test	shutdown: exit code
 
 hout	PASST_PID cat __STATESETUP__/passt.pid
 host	kill __PASST_PID__
-sleep	1
 passtw
diff --git a/test/passt_in_ns/shutdown b/test/passt_in_ns/shutdown
index b602ec2..45901df 100644
--- a/test/passt_in_ns/shutdown
+++ b/test/passt_in_ns/shutdown
@@ -16,5 +16,4 @@ test	shutdown: exit code
 
 nsout	PASST_PID cat __STATESETUP__/passt.pid
 ns	kill __PASST_PID__
-sleep	1
 passtw
-- 
2.37.3

[PATCH 04/12] test: More robust wait for pasta/passt to be ready

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 3163 bytes --]

When we start passt or pasta, it may take a short time to be ready to
handle packets, especially if running under valgrind.  We have a
number of semi-arbitrary fixed sleeps to account for this.

We can do this more robustly by exploiting the fact that pasta/passt
doesn't write its pidfile until it's ready to go, so if we wait for
the pidfile to be created, we can proceed with confidence.

Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
---
 test/lib/setup | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/test/lib/setup b/test/lib/setup
index 7e3f6c3..dee7b46 100755
--- a/test/lib/setup
+++ b/test/lib/setup
@@ -54,7 +54,9 @@ setup_passt() {
 	context_run passt "make clean"
 	context_run passt "make valgrind"
 	context_run_bg passt "valgrind --max-stackframe=$((4 * 1024 * 1024)) --trace-children=yes --vgdb=no --error-exitcode=1 --suppressions=test/valgrind.supp ./passt ${__opts} -f -t 10001 -u 10001 -P ${STATESETUP}/passt.pid"
-	sleep 5
+
+        # pidfile isn't created until passt is listening
+	wait_for [ -f "${STATESETUP}/passt.pid" ]
 
 	GUEST_CID=94557
 	context_run_bg qemu './qrap 5 qemu-system-$(uname -m)'		   \
@@ -99,7 +101,9 @@ setup_pasta() {
 	[ ${TRACE} -eq 1 ] && __opts="${__opts} --trace"
 
 	context_run_bg passt "./pasta ${__opts} -f -t 10002 -T 10003 -u 10002 -U 10003 -P ${STATESETUP}/passt.pid ${__target_pid}"
-	sleep 1
+
+        # pidfile isn't created until pasta is ready
+	wait_for [ -f "${STATESETUP}/passt.pid" ]
 }
 
 # setup_passt_in_ns() - Set up namespace (with pasta), run qemu and passt into it
@@ -129,7 +133,7 @@ setup_passt_in_ns() {
 	[ ${TRACE} -eq 1 ] && __opts="${__opts} --trace"
 
 	context_run_bg pasta "./pasta ${__opts} -t 10001,10002,10011,10012 -T 10003,10013 -u 10001,10002,10011,10012 -U 10003,10013 -P ${STATESETUP}/pasta.pid ${NSHOLDER} ${STATESETUP}/ns.hold hold"
-	sleep 1
+	wait_for [ -f "${STATESETUP}/pasta.pid" ]
 	__ns_pid=$(${NSHOLDER} ${STATESETUP}/ns.hold pid)
 
 	context_setup_nsenter qemu "-t ${__ns_pid} -U -n -p --preserve-credentials"
@@ -155,7 +159,7 @@ setup_passt_in_ns() {
 		context_run passt "make"
 		context_run_bg passt "./passt -f ${__opts} -t 10001,10011,10021,10031 -u 10001,10011,10021,10031 -P ${STATESETUP}/passt.pid"
 	fi
-	sleep 5
+	wait_for [ -f "${STATESETUP}/passt.pid" ]
 
 	GUEST_CID=94557
 	context_run_bg qemu './qrap 5 qemu-system-$(uname -m)'                \
@@ -226,7 +230,7 @@ setup_two_guests() {
 	[ ${TRACE} -eq 1 ] && __opts="${__opts} --trace"
 
 	context_run_bg passt_1 "./passt -P ${STATESETUP}/passt_1.pid -f ${__opts} -t 10001 -u 10001"
-	sleep 1
+	wait_for [ -f "${STATESETUP}/passt_1.pid" ]
 
 	__opts=
 	[ ${PCAP} -eq 1 ] && __opts="${__opts} -p ${LOGDIR}/passt_2.pcap"
@@ -234,6 +238,7 @@ setup_two_guests() {
 	[ ${TRACE} -eq 1 ] && __opts="${__opts} --trace"
 
 	context_run_bg passt_2 "./passt -P ${STATESETUP}/passt_2.pid -f ${__opts} -t 10004 -u 10004"
+	wait_for [ -f "${STATESETUP}/passt_2.pid" ]
 
 	GUEST_1_CID=94557
 	context_run_bg qemu_1 './qrap 5 qemu-system-$(uname -m)'             \
-- 
2.37.3

Re: [PATCH 04/12] test: More robust wait for pasta/passt to be ready

[Stefano Brivio]

[-- Attachment #1: Type: text/plain, Size: 1787 bytes --]

On Fri, 23 Sep 2022 17:20:30 +1000
David Gibson <david(a)gibson.dropbear.id.au> wrote:

> When we start passt or pasta, it may take a short time to be ready to
> handle packets, especially if running under valgrind.  We have a
> number of semi-arbitrary fixed sleeps to account for this.
> 
> We can do this more robustly by exploiting the fact that pasta/passt
> doesn't write its pidfile until it's ready to go, so if we wait for
> the pidfile to be created, we can proceed with confidence.
> 
> Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
> ---
>  test/lib/setup | 15 ++++++++++-----
>  1 file changed, 10 insertions(+), 5 deletions(-)
> 
> diff --git a/test/lib/setup b/test/lib/setup
> index 7e3f6c3..dee7b46 100755
> --- a/test/lib/setup
> +++ b/test/lib/setup
> @@ -54,7 +54,9 @@ setup_passt() {
>  	context_run passt "make clean"
>  	context_run passt "make valgrind"
>  	context_run_bg passt "valgrind --max-stackframe=$((4 * 1024 * 1024)) --trace-children=yes --vgdb=no --error-exitcode=1 --suppressions=test/valgrind.supp ./passt ${__opts} -f -t 10001 -u 10001 -P ${STATESETUP}/passt.pid"
> -	sleep 5
> +
> +        # pidfile isn't created until passt is listening

Here,

> +	wait_for [ -f "${STATESETUP}/passt.pid" ]
>  
>  	GUEST_CID=94557
>  	context_run_bg qemu './qrap 5 qemu-system-$(uname -m)'		   \
> @@ -99,7 +101,9 @@ setup_pasta() {
>  	[ ${TRACE} -eq 1 ] && __opts="${__opts} --trace"
>  
>  	context_run_bg passt "./pasta ${__opts} -f -t 10002 -T 10003 -u 10002 -U 10003 -P ${STATESETUP}/passt.pid ${__target_pid}"
> -	sleep 1
> +
> +        # pidfile isn't created until pasta is ready

and here, we have spaces instead of tabs. I can fix that up on merge,
unless you respin anyway.

-- 
Stefano

Re: [PATCH 04/12] test: More robust wait for pasta/passt to be ready

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 2160 bytes --]

On Sat, Sep 24, 2022 at 01:08:13AM +0200, Stefano Brivio wrote:
> On Fri, 23 Sep 2022 17:20:30 +1000
> David Gibson <david(a)gibson.dropbear.id.au> wrote:
> 
> > When we start passt or pasta, it may take a short time to be ready to
> > handle packets, especially if running under valgrind.  We have a
> > number of semi-arbitrary fixed sleeps to account for this.
> > 
> > We can do this more robustly by exploiting the fact that pasta/passt
> > doesn't write its pidfile until it's ready to go, so if we wait for
> > the pidfile to be created, we can proceed with confidence.
> > 
> > Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
> > ---
> >  test/lib/setup | 15 ++++++++++-----
> >  1 file changed, 10 insertions(+), 5 deletions(-)
> > 
> > diff --git a/test/lib/setup b/test/lib/setup
> > index 7e3f6c3..dee7b46 100755
> > --- a/test/lib/setup
> > +++ b/test/lib/setup
> > @@ -54,7 +54,9 @@ setup_passt() {
> >  	context_run passt "make clean"
> >  	context_run passt "make valgrind"
> >  	context_run_bg passt "valgrind --max-stackframe=$((4 * 1024 * 1024)) --trace-children=yes --vgdb=no --error-exitcode=1 --suppressions=test/valgrind.supp ./passt ${__opts} -f -t 10001 -u 10001 -P ${STATESETUP}/passt.pid"
> > -	sleep 5
> > +
> > +        # pidfile isn't created until passt is listening
> 
> Here,
> 
> > +	wait_for [ -f "${STATESETUP}/passt.pid" ]
> >  
> >  	GUEST_CID=94557
> >  	context_run_bg qemu './qrap 5 qemu-system-$(uname -m)'		   \
> > @@ -99,7 +101,9 @@ setup_pasta() {
> >  	[ ${TRACE} -eq 1 ] && __opts="${__opts} --trace"
> >  
> >  	context_run_bg passt "./pasta ${__opts} -f -t 10002 -T 10003 -u 10002 -U 10003 -P ${STATESETUP}/passt.pid ${__target_pid}"
> > -	sleep 1
> > +
> > +        # pidfile isn't created until pasta is ready
> 
> and here, we have spaces instead of tabs. I can fix that up on merge,
> unless you respin anyway.

Oops, fixed, and I will be respinning.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 05/12] test: Use --config-net for namespace setup

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 3947 bytes --]

The setup functions for passt_in_ns and two_guests perform some fairly slow
dhclient calls to configure the network in the namespace before starting
the guest.  This isn't really part of the tests, just necessary for the
operations later.

We can simplify and speed this up a bit by using pasta's '--config-net'
option to configure the networking for us.  As a bonus this means we have
at least a minimal test of the --config-net option itself.

Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
---
 test/lib/setup | 17 +++--------------
 1 file changed, 3 insertions(+), 14 deletions(-)

diff --git a/test/lib/setup b/test/lib/setup
index dee7b46..668fee8 100755
--- a/test/lib/setup
+++ b/test/lib/setup
@@ -132,7 +132,7 @@ setup_passt_in_ns() {
 	[ ${DEBUG} -eq 1 ] && __opts="${__opts} -d"
 	[ ${TRACE} -eq 1 ] && __opts="${__opts} --trace"
 
-	context_run_bg pasta "./pasta ${__opts} -t 10001,10002,10011,10012 -T 10003,10013 -u 10001,10002,10011,10012 -U 10003,10013 -P ${STATESETUP}/pasta.pid ${NSHOLDER} ${STATESETUP}/ns.hold hold"
+	context_run_bg pasta "./pasta ${__opts} -t 10001,10002,10011,10012 -T 10003,10013 -u 10001,10002,10011,10012 -U 10003,10013 -P ${STATESETUP}/pasta.pid --config-net ${NSHOLDER} ${STATESETUP}/ns.hold hold"
 	wait_for [ -f "${STATESETUP}/pasta.pid" ]
 	__ns_pid=$(${NSHOLDER} ${STATESETUP}/ns.hold pid)
 
@@ -140,11 +140,6 @@ setup_passt_in_ns() {
 	context_setup_nsenter ns "-t ${__ns_pid} -U -n -p --preserve-credentials"
 	context_setup_nsenter passt "-t ${__ns_pid} -U -n -p --preserve-credentials"
 
-	__ifname=$(context_run ns "ip -j link show | jq -rM '.[] | select(.link_type == \"ether\").ifname'")
-	context_run ns "/sbin/dhclient -4 --no-pid ${__ifname}"
-	sleep 2
-	context_run ns "/sbin/dhclient -6 --no-pid ${__ifname}"
-
 	__opts=
 	[ ${PCAP} -eq 1 ] && __opts="${__opts} -p ${LOGDIR}/passt_in_pasta.pcap"
 	[ ${DEBUG} -eq 1 ] && __opts="${__opts} -d"
@@ -201,7 +196,7 @@ setup_two_guests() {
 	[ ${PCAP} -eq 1 ] && __opts="${__opts} -p ${LOGDIR}/pasta_1.pcap"
 	[ ${DEBUG} -eq 1 ] && __opts="${__opts} -d"
 	[ ${TRACE} -eq 1 ] && __opts="${__opts} --trace"
-	context_run_bg pasta_1 "./pasta ${__opts} -P ${STATESETUP}/pasta_1.pid -t 10001,10002 -T 10003,10004 -u 10001,10002 -U 10003,10004 ${NSHOLDER} ${STATESETUP}/ns1.hold hold"
+	context_run_bg pasta_1 "./pasta ${__opts} -P ${STATESETUP}/pasta_1.pid -t 10001,10002 -T 10003,10004 -u 10001,10002 -U 10003,10004 --config-net ${NSHOLDER} ${STATESETUP}/ns1.hold hold"
 	__ns1_pid=$(${NSHOLDER} ${STATESETUP}/ns1.hold pid)
 	context_setup_nsenter passt_1 -U -n -p --preserve-credentials -t ${__ns1_pid}
 
@@ -209,7 +204,7 @@ setup_two_guests() {
 	[ ${PCAP} -eq 1 ] && __opts="${__opts} -p ${LOGDIR}/pasta_2.pcap"
 	[ ${DEBUG} -eq 1 ] && __opts="${__opts} -d"
 	[ ${TRACE} -eq 1 ] && __opts="${__opts} --trace"
-	context_run_bg pasta_2 "./pasta ${__opts} -P ${STATESETUP}/pasta_2.pid -t 10004,10005 -T 10003,10001 -u 10004,10005 -U 10003,10001 ${NSHOLDER} ${STATESETUP}/ns2.hold hold"
+	context_run_bg pasta_2 "./pasta ${__opts} -P ${STATESETUP}/pasta_2.pid -t 10004,10005 -T 10003,10001 -u 10004,10005 -U 10003,10001 --config-net ${NSHOLDER} ${STATESETUP}/ns2.hold hold"
 	__ns2_pid=$(${NSHOLDER} ${STATESETUP}/ns2.hold pid)
 	context_setup_nsenter passt_2 -U -n -p --preserve-credentials -t ${__ns2_pid}
 
@@ -218,12 +213,6 @@ setup_two_guests() {
 
 	__ifname="$(context_run qemu_1 "ip -j link show | jq -rM '.[] | select(.link_type == \"ether\").ifname'")"
 
-	context_run qemu_1 "/sbin/dhclient -4 --no-pid ${__ifname}"
-	context_run qemu_2 "/sbin/dhclient -4 --no-pid ${__ifname}"
-	sleep 2
-	context_run qemu_1 "/sbin/dhclient -6 --no-pid ${__ifname}"
-	context_run qemu_2 "/sbin/dhclient -6 --no-pid ${__ifname}"
-
 	__opts=
 	[ ${PCAP} -eq 1 ] && __opts="${__opts} -p ${LOGDIR}/passt_1.pcap"
 	[ ${DEBUG} -eq 1 ] && __opts="${__opts} -d"
-- 
2.37.3

[PATCH 06/12] test: Simplify data handling for transfer tests

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 43889 bytes --]

Many of our tests are based around performing transfers of sample data
across passt/pasta created links.  The data flow here can be a bit
hard to follow since, e.g. we create a file transfer it to the guest,
then transfer it back to the host across several different tests.
This also means that the test cases aren't independent of each other.

Because we don't have the original file available at both ends in some
cases, we compare them by generating md5sums at each end and comparing
them, which is a bit complicated.

Make a number of changes to simplify this:
  1. Pre-generate the sample data files as a test asset, rather than
     building them on the fly during the tests proper
  2. Include the sample data files in the mbuto guest image
  3. Because we have good copies of the original data available in all
     contexts, we can now simply use 'cmp' to check if the transfer
     has worked, avoiding md5sum complications.
  4. Similarly we can always use the original copy of the sample data
     on the send side of each transfer, meaning that the tests become
     more independent of each other.

Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
---
 test/.gitignore       |   1 +
 test/Makefile         |  15 +++-
 test/passt.mbuto      |   6 +-
 test/passt/tcp        |  53 +++++-------
 test/passt/udp        |  29 +++----
 test/passt_in_ns/tcp  | 187 +++++++++++++++++-------------------------
 test/passt_in_ns/udp  |  93 +++++++++------------
 test/pasta/tcp        |  79 +++++++-----------
 test/pasta/udp        |  43 ++++------
 test/two_guests/basic |   2 +-
 10 files changed, 214 insertions(+), 294 deletions(-)

diff --git a/test/.gitignore b/test/.gitignore
index e00b544..fbee491 100644
--- a/test/.gitignore
+++ b/test/.gitignore
@@ -5,6 +5,7 @@ QEMU_EFI.fd
 *.qcow2
 *.raw
 *.raw.xz
+*.bin
 nsholder
 guest-key
 guest-key.pub
diff --git a/test/Makefile b/test/Makefile
index cc1a818..08f0c2d 100644
--- a/test/Makefile
+++ b/test/Makefile
@@ -54,10 +54,12 @@ UBUNTU_IMGS = $(UBUNTU_OLD_IMGS) $(UBUNTU_NEW_IMGS)
 
 DOWNLOAD_ASSETS = mbuto \
 	$(DEBIAN_IMGS) $(FEDORA_IMGS) $(OPENSUSE_IMGS) $(UBUNTU_IMGS)
+TESTDATA_ASSETS = small.bin big.bin medium.bin
 LOCAL_ASSETS = mbuto.img QEMU_EFI.fd \
 	$(DEBIAN_IMGS:%=prepared-%) $(FEDORA_IMGS:%=prepared-%) \
 	$(UBUNTU_NEW_IMGS:%=prepared-%) \
-	nsholder guest-key guest-key.pub
+	nsholder guest-key guest-key.pub \
+	$(TESTDATA_ASSETS)
 
 ASSETS = $(DOWNLOAD_ASSETS) $(LOCAL_ASSETS)
 
@@ -71,7 +73,7 @@ mbuto:
 guest-key guest-key.pub:
 	ssh-keygen -f guest-key -N ''
 
-mbuto.img: passt.mbuto mbuto guest-key.pub
+mbuto.img: passt.mbuto mbuto guest-key.pub $(TESTDATA_ASSETS)
 	./mbuto/mbuto -p ./$< -c lz4 -f $@
 
 nsholder: nsholder.c
@@ -88,6 +90,15 @@ prepared-%.img: %.img ./prepare-distro-img.sh
 	qemu-img create -f qcow2 -F qcow2 -b $< $@
 	./prepare-distro-img.sh $(IMGTYPE) $@
 
+small.bin:
+	dd if=/dev/urandom bs=2k count=1 of=$@
+
+medium.bin:
+	dd if=/dev/urandom bs=1k count=5 of=$@
+
+big.bin:
+	dd if=/dev/urandom bs=1M count=10 of=$@
+
 check: assets
 	./run
 
diff --git a/test/passt.mbuto b/test/passt.mbuto
index d29f456..ac4c37b 100755
--- a/test/passt.mbuto
+++ b/test/passt.mbuto
@@ -12,8 +12,8 @@
 
 PROGS="${PROGS:-ash,dash,bash ip mount ls insmod mkdir ln cat chmod lsmod
        modprobe find grep mknod mv rm umount jq iperf3 dhclient hostname
-       sed tr chown sipcalc cut md5sum socat dd strace ping tail killall sleep
-       sysctl nproc tcp_rr tcp_crr udp_rr which tee seq bc sshd ssh-keygen}"
+       sed tr chown sipcalc cut socat dd strace ping tail killall sleep sysctl
+       nproc tcp_rr tcp_crr udp_rr which tee seq bc sshd ssh-keygen cmp}"
 
 KMODS="${KMODS:- virtio_net virtio_pci vmw_vsock_virtio_transport}"
 
@@ -23,6 +23,8 @@ LINKS="${LINKS:-
 
 DIRS="${DIRS} /tmp /sbin /usr/share /var/log /var/lib /etc/ssh /run/sshd /root/.ssh"
 
+COPIES="${COPIES} small.bin,/root/small.bin medium.bin,/root/medium.bin big.bin,/root/big.bin"
+
 FIXUP="${FIXUP}"'
 	cat > /sbin/dhclient-script << EOF
 #!/bin/sh
diff --git a/test/passt/tcp b/test/passt/tcp
index 7af9c54..67668aa 100644
--- a/test/passt/tcp
+++ b/test/passt/tcp
@@ -11,77 +11,66 @@
 # Copyright (c) 2021 Red Hat GmbH
 # Author: Stefano Brivio <sbrivio(a)redhat.com>
 
-gtools	socat ip jq md5sum cut
-htools	dd socat ip jq md5sum cut
+gtools	socat ip jq cmp
+htools	socat ip jq
+
+set	TEMP_BIG __STATEDIR__/test_big.bin
+set	TEMP_SMALL __STATEDIR__/test_small.bin
 
 test	TCP/IPv4: host to guest: big transfer
-set	TEMP_BIG __STATEDIR__/big
 guestb	socat -u TCP4-LISTEN:10001,reuseaddr OPEN:test_big.bin,create,trunc
 sleep	1
-host	dd if=/dev/urandom bs=1M count=10 > __TEMP_BIG__
-host	socat -u OPEN:__TEMP_BIG__ TCP4:127.0.0.1:10001
+host	socat -u OPEN:__BASEPATH__/big.bin TCP4:127.0.0.1:10001
 guestw
-hout	HOST_MD5_BIG md5sum __TEMP_BIG__ | cut -d' ' -f1
-gout	GUEST_MD5_BIG md5sum test_big.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5_BIG__" = "__HOST_MD5_BIG__" ]
+guest	cmp /root/big.bin test_big.bin
 
 test	TCP/IPv4: guest to host: big transfer
 hostb	socat -u TCP4-LISTEN:10003,bind=127.0.0.1,reuseaddr OPEN:__TEMP_BIG__,create,trunc
 gout	GW ip -j -4 route show|jq -rM '.[] | select(.dst == "default").gateway'
-guest	socat -u OPEN:test_big.bin TCP4:__GW__:10003
+guest	socat -u OPEN:/root/big.bin TCP4:__GW__:10003
 hostw
-hout	HOST_MD5_BIG md5sum __TEMP_BIG__ | cut -d' ' -f1
-check	[ "__GUEST_MD5_BIG__" = "__HOST_MD5_BIG__" ]
+check	cmp __BASEPATH__/big.bin __TEMP_BIG__
 
 test	TCP/IPv4: host to guest: small transfer
-set	TEMP_SMALL __STATEDIR__/small
 guestb	socat -u TCP4-LISTEN:10001,reuseaddr OPEN:test_small.bin,create,trunc
 sleep	1
-host	dd if=/dev/urandom bs=2k count=1 > __TEMP_SMALL__
-host	socat -u OPEN:__TEMP_SMALL__ TCP4:127.0.0.1:10001
+host	socat -u OPEN:__BASEPATH__/small.bin TCP4:127.0.0.1:10001
 guestw
-hout	HOST_MD5_SMALL md5sum __TEMP_SMALL__ | cut -d' ' -f1
-gout	GUEST_MD5_SMALL md5sum test_small.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5_SMALL__" = "__HOST_MD5_SMALL__" ]
+guest	cmp /root/small.bin test_small.bin
 
 test	TCP/IPv4: guest to host: small transfer
 hostb	socat -u TCP4-LISTEN:10003,bind=127.0.0.1,reuseaddr OPEN:__TEMP_SMALL__,create,trunc
 sleep	1
-guest	socat -u OPEN:test_small.bin TCP4:__GW__:10003
+guest	socat -u OPEN:/root/small.bin TCP4:__GW__:10003
 hostw
-hout	HOST_MD5_SMALL md5sum __TEMP_SMALL__ | cut -d' ' -f1
-check	[ "__GUEST_MD5_SMALL__" = "__HOST_MD5_SMALL__" ]
+check	cmp __BASEPATH__/small.bin __TEMP_SMALL__
 
 
 test	TCP/IPv6: host to guest: big transfer
 guestb	socat -u TCP6-LISTEN:10001,reuseaddr OPEN:test_big.bin,create,trunc
 sleep	1
-host	socat -u OPEN:__TEMP_BIG__ TCP6:[::1]:10001
+host	socat -u OPEN:__BASEPATH__/big.bin TCP6:[::1]:10001
 guestw
-gout	GUEST_MD5_BIG md5sum test_big.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5_BIG__" = "__HOST_MD5_BIG__" ]
+guest	cmp /root/big.bin test_big.bin
 
 test	TCP/IPv6: guest to host: big transfer
 hostb	socat -u TCP6-LISTEN:10003,bind=[::1],reuseaddr OPEN:__TEMP_BIG__,create,trunc
 gout	GW6 ip -j -6 route show|jq -rM '.[] | select(.dst == "default").gateway'
 gout	IFNAME ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname'
-guest	socat -u OPEN:test_big.bin TCP6:[__GW6__%__IFNAME__]:10003
+guest	socat -u OPEN:/root/big.bin TCP6:[__GW6__%__IFNAME__]:10003
 hostw
-hout	HOST_MD5_BIG md5sum __TEMP_BIG__ | cut -d' ' -f1
-check	[ "__GUEST_MD5_BIG__" = "__HOST_MD5_BIG__" ]
+check	cmp __BASEPATH__/big.bin __TEMP_BIG__
 
 test	TCP/IPv6: host to guest: small transfer
 guestb	socat -u TCP6-LISTEN:10001,reuseaddr OPEN:test_small.bin,create,trunc
 sleep	1
-host	socat -u OPEN:__TEMP_SMALL__ TCP6:[::1]:10001
+host	socat -u OPEN:__BASEPATH__/small.bin TCP6:[::1]:10001
 guestw
-gout	GUEST_MD5_SMALL md5sum test_small.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5_SMALL__" = "__HOST_MD5_SMALL__" ]
+guest	cmp /root/small.bin test_small.bin
 
 test	TCP/IPv6: guest to host: small transfer
 hostb	socat -u TCP6-LISTEN:10003,bind=[::1],reuseaddr OPEN:__TEMP_SMALL__,create,trunc
 sleep	1
-guest	socat -u OPEN:test_small.bin TCP6:[__GW6__%__IFNAME__]:10003
+guest	socat -u OPEN:/root/small.bin TCP6:[__GW6__%__IFNAME__]:10003
 hostw
-hout	HOST_MD5_SMALL md5sum __TEMP_SMALL__ | cut -d' ' -f1
-check	[ "__GUEST_MD5_SMALL__" = "__HOST_MD5_SMALL__" ]
+check	cmp __BASEPATH__/small.bin __TEMP_SMALL__
diff --git a/test/passt/udp b/test/passt/udp
index 7d444b8..b9533e0 100644
--- a/test/passt/udp
+++ b/test/passt/udp
@@ -11,41 +11,36 @@
 # Copyright (c) 2021 Red Hat GmbH
 # Author: Stefano Brivio <sbrivio(a)redhat.com>
 
-gtools	socat ip jq md5sum cut
-htools	dd socat jq md5sum cut
+gtools	socat ip jq cmp
+htools	socat jq
+
+set	TEMP __STATEDIR__/test.bin
 
 test	UDP/IPv4: host to guest
-set	TEMP __STATEDIR__/data
 guestb	socat -u UDP4-LISTEN:10001,null-eof OPEN:test.bin,create,trunc
 sleep	1
-host	dd if=/dev/urandom bs=1k count=5 > __TEMP__
-host	socat -u OPEN:__TEMP__ UDP4:127.0.0.1:10001,shut-null
+host	socat -u OPEN:__BASEPATH__/medium.bin UDP4:127.0.0.1:10001,shut-null
 guestw
-hout	HOST_MD5 md5sum __TEMP__ | cut -d' ' -f1
-gout	GUEST_MD5 md5sum test.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5__" = "__HOST_MD5__" ]
+guest	cmp /root/medium.bin test.bin
 
 test	UDP/IPv4: guest to host
 hostb	socat -u UDP4-LISTEN:10003,bind=127.0.0.1,null-eof OPEN:__TEMP__,create,trunc
 gout	GW ip -j -4 route show|jq -rM '.[] | select(.dst == "default").gateway'
-guest	socat -u OPEN:test.bin UDP4:__GW__:10003,shut-null
+guest	socat -u OPEN:/root/medium.bin UDP4:__GW__:10003,shut-null
 hostw
-hout	HOST_MD5 md5sum __TEMP__ | cut -d' ' -f1
-check	[ "__GUEST_MD5__" = "__HOST_MD5__" ]
+check	cmp __BASEPATH__/medium.bin __TEMP__
 
 test	UDP/IPv6: host to guest
 guestb	socat -u UDP6-LISTEN:10001,null-eof OPEN:test.bin,create,trunc
 sleep	1
-host	socat -u OPEN:__TEMP__ UDP6:[::1]:10001,shut-null
+host	socat -u OPEN:__BASEPATH__/medium.bin UDP6:[::1]:10001,shut-null
 guestw
-gout	GUEST_MD5 md5sum test.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5__" = "__HOST_MD5__" ]
+guest	cmp /root/medium.bin test.bin
 
 test	UDP/IPv6: guest to host
 hostb	socat -u UDP6-LISTEN:10003,bind=[::1],null-eof OPEN:__TEMP__,create,trunc
 gout	GW6 ip -j -6 route show|jq -rM '.[] | select(.dst == "default").gateway'
 gout	IFNAME ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname'
-guest	socat -u OPEN:test.bin UDP6:[__GW6__%__IFNAME__]:10003,shut-null
+guest	socat -u OPEN:/root/medium.bin UDP6:[__GW6__%__IFNAME__]:10003,shut-null
 hostw
-hout	HOST_MD5 md5sum __TEMP__ | cut -d' ' -f1
-check	[ "__GUEST_MD5__" = "__HOST_MD5__" ]
+check	cmp __BASEPATH__/medium.bin __TEMP__
diff --git a/test/passt_in_ns/tcp b/test/passt_in_ns/tcp
index 71fe1c1..842b345 100644
--- a/test/passt_in_ns/tcp
+++ b/test/passt_in_ns/tcp
@@ -11,281 +11,246 @@
 # Copyright (c) 2021 Red Hat GmbH
 # Author: Stefano Brivio <sbrivio(a)redhat.com>
 
-gtools	socat ip jq md5sum cut
-htools	dd socat ip jq md5sum cut
-nstools	socat ip jq md5sum cut
+gtools	socat ip jq
+htools	socat ip jq
+nstools	socat ip jq
+
+set	TEMP_BIG __STATEDIR__/test_big.bin
+set	TEMP_SMALL __STATEDIR__/test_small.bin
+set	TEMP_NS_BIG __STATEDIR__/test_ns_big.bin
+set	TEMP_NS_SMALL __STATEDIR__/test_ns_small.bin
 
 test	TCP/IPv4: host to guest: big transfer
-set	TEMP_BIG __STATEDIR__/big.img
 guestb	socat -u TCP4-LISTEN:10001 OPEN:test_big.bin,create,trunc
-host	dd if=/dev/urandom bs=1M count=10 of=__TEMP_BIG__
 sleep	1
-host	socat -u OPEN:__TEMP_BIG__ TCP4:127.0.0.1:10001
+host	socat -u OPEN:__BASEPATH__/big.bin TCP4:127.0.0.1:10001
 guestw
-hout	MD5_BIG md5sum __TEMP_BIG__ | cut -d' ' -f1
-gout	GUEST_MD5_BIG md5sum test_big.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5_BIG__" = "__MD5_BIG__" ]
+guest	cmp test_big.bin /root/big.bin
 
 test	TCP/IPv4: host to ns: big transfer
-set	TEMP_NS_BIG __STATEDIR__/big_ns.img
 nsb	socat -u TCP4-LISTEN:10002 OPEN:__TEMP_NS_BIG__,create,trunc
 sleep	1
-host	socat -u OPEN:__TEMP_BIG__ TCP4:127.0.0.1:10002
+host	socat -u OPEN:__BASEPATH__/big.bin TCP4:127.0.0.1:10002
 nsw
-nsout	NS_MD5_BIG md5sum __TEMP_NS_BIG__ | cut -d' ' -f1
-check	[ "__NS_MD5_BIG__" = "__MD5_BIG__" ]
+check	cmp __TEMP_NS_BIG__ __BASEPATH__/big.bin
 
 test	TCP/IPv4: guest to host: big transfer
 hostb	socat -u TCP4-LISTEN:10003 OPEN:__TEMP_BIG__,create,trunc
 gout	GW ip -j -4 route show|jq -rM '.[] | select(.dst == "default").gateway'
 sleep	1
-guest	socat -u OPEN:test_big.bin TCP4:__GW__:10003
+guest	socat -u OPEN:/root/big.bin TCP4:__GW__:10003
 hostw
-hout	HOST_MD5_BIG md5sum __TEMP_BIG__ | cut -d' ' -f1
-check	[ "__HOST_MD5_BIG__" = "__MD5_BIG__" ]
+check	cmp __TEMP_BIG__ __BASEPATH__/big.bin
 
 test	TCP/IPv4: guest to ns: big transfer
-nsb	socat -u TCP4-LISTEN:10002 OPEN:__TEMP_BIG__,create,trunc
+nsb	socat -u TCP4-LISTEN:10002 OPEN:__TEMP_NS_BIG__,create,trunc
 sleep	1
-guest	socat -u OPEN:test_big.bin TCP4:__GW__:10002
+guest	socat -u OPEN:/root/big.bin TCP4:__GW__:10002
 nsw
-hout	HOST_MD5_BIG md5sum __TEMP_BIG__ | cut -d' ' -f1
-check	[ "__HOST_MD5_BIG__" = "__MD5_BIG__" ]
+check	cmp __TEMP_NS_BIG__ __BASEPATH__/big.bin
 
 test	TCP/IPv4: ns to host (spliced): big transfer
 hostb	socat -u TCP4-LISTEN:10003 OPEN:__TEMP_BIG__,create,trunc
 sleep	1
-ns	socat -u OPEN:__TEMP_NS_BIG__ TCP4:127.0.0.1:10003
+ns	socat -u OPEN:__BASEPATH__/big.bin TCP4:127.0.0.1:10003
 hostw
-hout	HOST_MD5_BIG md5sum __TEMP_BIG__ | cut -d' ' -f1
-check	[ "__HOST_MD5_BIG__" = "__MD5_BIG__" ]
+check	cmp __TEMP_BIG__ __BASEPATH__/big.bin
 
 test	TCP/IPv4: ns to host (via tap): big transfer
 hostb	socat -u TCP4-LISTEN:10003 OPEN:__TEMP_BIG__,create,trunc
 sleep	1
-ns	socat -u OPEN:__TEMP_NS_BIG__ TCP4:__GW__:10003
+ns	socat -u OPEN:__BASEPATH__/big.bin TCP4:__GW__:10003
 hostw
-hout	HOST_MD5_BIG md5sum __TEMP_BIG__ | cut -d' ' -f1
-check	[ "__HOST_MD5_BIG__" = "__MD5_BIG__" ]
+check	cmp __TEMP_BIG__ __BASEPATH__/big.bin
 
 test	TCP/IPv4: ns to guest (using loopback address): big transfer
 guestb	socat -u TCP4-LISTEN:10001 OPEN:test_big.bin,create,trunc
 sleep	1
-ns	socat -u OPEN:__TEMP_NS_BIG__ TCP4:127.0.0.1:10001
+ns	socat -u OPEN:__BASEPATH__/big.bin TCP4:127.0.0.1:10001
 guestw
-gout	GUEST_MD5_BIG md5sum test_big.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5_BIG__" = "__MD5_BIG__" ]
+guest	cmp test_big.bin /root/big.bin
 
 test	TCP/IPv4: ns to guest (using namespace address): big transfer
 guestb	socat -u TCP4-LISTEN:10001 OPEN:test_big.bin,create,trunc
 nsout	IFNAME ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname'
 nsout	ADDR ip -j -4 addr show|jq -rM '.[] | select(.ifname == "__IFNAME__").addr_info[0].local'
 sleep	1
-ns	socat -u OPEN:__TEMP_NS_BIG__ TCP4:__ADDR__:10001
+ns	socat -u OPEN:__BASEPATH__/big.bin TCP4:__ADDR__:10001
 guestw
-gout	GUEST_MD5_BIG md5sum test_big.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5_BIG__" = "__MD5_BIG__" ]
+guest	cmp test_big.bin /root/big.bin
 
 test	TCP/IPv4: host to guest: small transfer
-set	TEMP_SMALL __STATEDIR__/small.img
 guestb	socat -u TCP4-LISTEN:10001 OPEN:test_small.bin,create,trunc
-host	dd if=/dev/urandom bs=2k count=100 of=__TEMP_SMALL__
 sleep	1
-host	socat -u OPEN:__TEMP_SMALL__ TCP4:127.0.0.1:10001
+host	socat -u OPEN:__BASEPATH__/small.bin TCP4:127.0.0.1:10001
 guestw
-hout	MD5_SMALL md5sum __TEMP_SMALL__ | cut -d' ' -f1
-gout	GUEST_MD5_SMALL md5sum test_small.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5_SMALL__" = "__MD5_SMALL__" ]
+guest	cmp test_small.bin /root/small.bin
 
 test	TCP/IPv4: host to ns: small transfer
-set	TEMP_NS_SMALL __STATEDIR__/small_ns.img
 nsb	socat -u TCP4-LISTEN:10002 OPEN:__TEMP_NS_SMALL__,create,trunc
 sleep	1
-host	socat -u OPEN:__TEMP_SMALL__ TCP4:127.0.0.1:10002
+host	socat -u OPEN:__BASEPATH__/small.bin TCP4:127.0.0.1:10002
 nsw
-nsout	NS_MD5_SMALL md5sum __TEMP_NS_SMALL__ | cut -d' ' -f1
-check	[ "__NS_MD5_SMALL__" = "__MD5_SMALL__" ]
+check	cmp __TEMP_NS_SMALL__ __BASEPATH__/small.bin
 
 test	TCP/IPv4: guest to host: small transfer
 hostb	socat -u TCP4-LISTEN:10003 OPEN:__TEMP_SMALL__,create,trunc
 gout	GW ip -j -4 route show|jq -rM '.[] | select(.dst == "default").gateway'
 sleep	1
-guest	socat -u OPEN:test_small.bin TCP4:__GW__:10003
+guest	socat -u OPEN:/root/small.bin TCP4:__GW__:10003
 hostw
-hout	HOST_MD5_SMALL md5sum __TEMP_SMALL__ | cut -d' ' -f1
-check	[ "__HOST_MD5_SMALL__" = "__MD5_SMALL__" ]
+check	cmp __TEMP_SMALL__ __BASEPATH__/small.bin
 
 test	TCP/IPv4: guest to ns: small transfer
-nsb	socat -u TCP4-LISTEN:10002 OPEN:__TEMP_SMALL__,create,trunc
+nsb	socat -u TCP4-LISTEN:10002 OPEN:__TEMP_NS_SMALL__,create,trunc
 sleep	1
-guest	socat -u OPEN:test_small.bin TCP4:__GW__:10002
+guest	socat -u OPEN:/root/small.bin TCP4:__GW__:10002
 nsw
-hout	HOST_MD5_SMALL md5sum __TEMP_SMALL__ | cut -d' ' -f1
-check	[ "__HOST_MD5_SMALL__" = "__MD5_SMALL__" ]
+check	cmp __TEMP_NS_SMALL__ __BASEPATH__/small.bin
 
 test	TCP/IPv4: ns to host (spliced): small transfer
 hostb	socat -u TCP4-LISTEN:10003 OPEN:__TEMP_SMALL__,create,trunc
 sleep	1
-ns	socat -u OPEN:__TEMP_NS_SMALL__ TCP4:127.0.0.1:10003
+ns	socat -u OPEN:__BASEPATH__/small.bin TCP4:127.0.0.1:10003
 hostw
-hout	HOST_MD5_SMALL md5sum __TEMP_SMALL__ | cut -d' ' -f1
-check	[ "__HOST_MD5_SMALL__" = "__MD5_SMALL__" ]
+check	cmp __TEMP_SMALL__ __BASEPATH__/small.bin
 
 test	TCP/IPv4: ns to host (via tap): small transfer
 hostb	socat -u TCP4-LISTEN:10003 OPEN:__TEMP_SMALL__,create,trunc
 sleep	1
-ns	socat -u OPEN:__TEMP_NS_SMALL__ TCP4:__GW__:10003
+ns	socat -u OPEN:__BASEPATH__/small.bin TCP4:__GW__:10003
 hostw
-hout	HOST_MD5_SMALL md5sum __TEMP_SMALL__ | cut -d' ' -f1
-check	[ "__HOST_MD5_SMALL__" = "__MD5_SMALL__" ]
+check	cmp __TEMP_SMALL__ __BASEPATH__/small.bin
 
 test	TCP/IPv4: ns to guest (using loopback address): small transfer
 guestb	socat -u TCP4-LISTEN:10001 OPEN:test_small.bin,create,trunc
 sleep	1
-ns	socat -u OPEN:__TEMP_NS_SMALL__ TCP4:127.0.0.1:10001
+ns	socat -u OPEN:__BASEPATH__/small.bin TCP4:127.0.0.1:10001
 guestw
-gout	GUEST_MD5_SMALL md5sum test_small.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5_SMALL__" = "__MD5_SMALL__" ]
+guest	cmp test_small.bin /root/small.bin
 
 test	TCP/IPv4: ns to guest (using namespace address): small transfer
 guestb	socat -u TCP4-LISTEN:10001 OPEN:test_small.bin,create,trunc
 sleep	1
-ns	socat -u OPEN:__TEMP_NS_SMALL__ TCP4:__ADDR__:10001
+ns	socat -u OPEN:__BASEPATH__/small.bin TCP4:__ADDR__:10001
 guestw
-gout	GUEST_MD5_SMALL md5sum test_small.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5_SMALL__" = "__MD5_SMALL__" ]
+guest	cmp test_small.bin /root/small.bin
 
 test	TCP/IPv6: host to guest: big transfer
 guestb	socat -u TCP6-LISTEN:10001 OPEN:test_big.bin,create,trunc
 sleep	1
-host	socat -u OPEN:__TEMP_BIG__ TCP6:[::1]:10001
+host	socat -u OPEN:__BASEPATH__/big.bin TCP6:[::1]:10001
 guestw
-gout	GUEST_MD5_BIG md5sum test_big.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5_BIG__" = "__MD5_BIG__" ]
+guest	cmp test_big.bin /root/big.bin
 
 test	TCP/IPv6: host to ns: big transfer
 nsb	socat -u TCP6-LISTEN:10002 OPEN:__TEMP_NS_BIG__,create,trunc
 sleep	1
-host	socat -u OPEN:__TEMP_BIG__ TCP6:[::1]:10002
+host	socat -u OPEN:__BASEPATH__/big.bin TCP6:[::1]:10002
 nsw
-nsout	NS_MD5_BIG md5sum __TEMP_NS_BIG__ | cut -d' ' -f1
-check	[ "__NS_MD5_BIG__" = "__MD5_BIG__" ]
+check	cmp __TEMP_NS_BIG__ __BASEPATH__/big.bin
 
 test	TCP/IPv6: guest to host: big transfer
 hostb	socat -u TCP6-LISTEN:10003 OPEN:__TEMP_BIG__,create,trunc
 gout	GW6 ip -j -6 route show|jq -rM '.[] | select(.dst == "default").gateway'
 gout	IFNAME ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname'
 sleep	1
-guest	socat -u OPEN:test_big.bin TCP6:[__GW6__%__IFNAME__]:10003
+guest	socat -u OPEN:/root/big.bin TCP6:[__GW6__%__IFNAME__]:10003
 hostw
-hout	HOST_MD5_BIG md5sum __TEMP_BIG__ | cut -d' ' -f1
-check	[ "__HOST_MD5_BIG__" = "__MD5_BIG__" ]
+check	cmp __TEMP_BIG__ __BASEPATH__/big.bin
 
 test	TCP/IPv6: guest to ns: big transfer
-nsb	socat -u TCP6-LISTEN:10002 OPEN:__TEMP_BIG__,create,trunc
+nsb	socat -u TCP6-LISTEN:10002 OPEN:__TEMP_NS_BIG__,create,trunc
 sleep	1
-guest	socat -u OPEN:test_big.bin TCP6:[__GW6__%__IFNAME__]:10002
+guest	socat -u OPEN:/root/big.bin TCP6:[__GW6__%__IFNAME__]:10002
 nsw
-hout	HOST_MD5_BIG md5sum __TEMP_BIG__ | cut -d' ' -f1
-check	[ "__HOST_MD5_BIG__" = "__MD5_BIG__" ]
+check	cmp __TEMP_NS_BIG__ __BASEPATH__/big.bin
 
 test	TCP/IPv6: ns to host (spliced): big transfer
 hostb	socat -u TCP6-LISTEN:10003 OPEN:__TEMP_BIG__,create,trunc
 sleep	1
-ns	socat -u OPEN:__TEMP_NS_BIG__ TCP6:[::1]:10003
+ns	socat -u OPEN:__BASEPATH__/big.bin TCP6:[::1]:10003
 hostw
-hout	HOST_MD5_BIG md5sum __TEMP_BIG__ | cut -d' ' -f1
-check	[ "__HOST_MD5_BIG__" = "__MD5_BIG__" ]
+check	cmp __TEMP_BIG__ __BASEPATH__/big.bin
 
 test	TCP/IPv6: ns to host (via tap): big transfer
 hostb	socat -u TCP6-LISTEN:10003 OPEN:__TEMP_BIG__,create,trunc
 nsout	IFNAME ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname'
 sleep	1
-ns	socat -u OPEN:__TEMP_NS_BIG__ TCP6:[__GW6__%__IFNAME__]:10003
+ns	socat -u OPEN:__BASEPATH__/big.bin TCP6:[__GW6__%__IFNAME__]:10003
 hostw
-hout	HOST_MD5_BIG md5sum __TEMP_BIG__ | cut -d' ' -f1
-check	[ "__HOST_MD5_BIG__" = "__MD5_BIG__" ]
+check	cmp __TEMP_BIG__ __BASEPATH__/big.bin
 
 test	TCP/IPv6: ns to guest (using loopback address): big transfer
 guestb	socat -u TCP6-LISTEN:10001 OPEN:test_big.bin,create,trunc
 sleep	1
-ns	socat -u OPEN:__TEMP_NS_BIG__ TCP6:[::1]:10001
+ns	socat -u OPEN:__BASEPATH__/big.bin TCP6:[::1]:10001
 guestw
-gout	GUEST_MD5_BIG md5sum test_big.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5_BIG__" = "__MD5_BIG__" ]
+guest	cmp test_big.bin /root/big.bin
 
 test	TCP/IPv6: ns to guest (using namespace address): big transfer
 guestb	socat -u TCP6-LISTEN:10001 OPEN:test_big.bin,create,trunc
 nsout	ADDR6 ip -j -6 addr show|jq -rM '.[] | select(.ifname == "__IFNAME__").addr_info[0].local'
 sleep	1
-ns	socat -u OPEN:__TEMP_NS_BIG__ TCP6:[__ADDR6__]:10001
+ns	socat -u OPEN:__BASEPATH__/big.bin TCP6:[__ADDR6__]:10001
 guestw
-gout	GUEST_MD5_BIG md5sum test_big.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5_BIG__" = "__MD5_BIG__" ]
+guest	cmp test_big.bin /root/big.bin
 
 test	TCP/IPv6: host to guest: small transfer
 guestb	socat -u TCP6-LISTEN:10001 OPEN:test_small.bin,create,trunc
 sleep	1
-host	socat -u OPEN:__TEMP_SMALL__ TCP6:[::1]:10001
+host	socat -u OPEN:__BASEPATH__/small.bin TCP6:[::1]:10001
 guestw
-gout	GUEST_MD5_SMALL md5sum test_small.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5_SMALL__" = "__MD5_SMALL__" ]
+guest	cmp test_small.bin /root/small.bin
 
 test	TCP/IPv6: host to ns: small transfer
 nsb	socat -u TCP6-LISTEN:10002 OPEN:__TEMP_NS_SMALL__,create,trunc
 sleep	1
-host	socat -u OPEN:__TEMP_SMALL__ TCP6:[::1]:10002
+host	socat -u OPEN:__BASEPATH__/small.bin TCP6:[::1]:10002
 nsw
-nsout	NS_MD5_SMALL md5sum __TEMP_NS_SMALL__ | cut -d' ' -f1
-check	[ "__NS_MD5_SMALL__" = "__MD5_SMALL__" ]
+check	cmp __TEMP_NS_SMALL__ __BASEPATH__/small.bin
 
 test	TCP/IPv6: guest to host: small transfer
 hostb	socat -u TCP6-LISTEN:10003 OPEN:__TEMP_SMALL__,create,trunc
 gout	GW6 ip -j -6 route show|jq -rM '.[] | select(.dst == "default").gateway'
 gout	IFNAME ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname'
 sleep	1
-guest	socat -u OPEN:test_small.bin TCP6:[__GW6__%__IFNAME__]:10003
+guest	socat -u OPEN:/root/small.bin TCP6:[__GW6__%__IFNAME__]:10003
 hostw
-hout	HOST_MD5_SMALL md5sum __TEMP_SMALL__ | cut -d' ' -f1
-check	[ "__HOST_MD5_SMALL__" = "__MD5_SMALL__" ]
+check	cmp __TEMP_SMALL__ __BASEPATH__/small.bin
 
 test	TCP/IPv6: guest to ns: small transfer
-nsb	socat -u TCP6-LISTEN:10002 OPEN:__TEMP_SMALL__
+nsb	socat -u TCP6-LISTEN:10002 OPEN:__TEMP_NS_SMALL__
 sleep	1
-guest	socat -u OPEN:test_small.bin TCP6:[__GW6__%__IFNAME__]:10002
+guest	socat -u OPEN:/root/small.bin TCP6:[__GW6__%__IFNAME__]:10002
 nsw
-hout	HOST_MD5_SMALL md5sum __TEMP_SMALL__ | cut -d' ' -f1
-check	[ "__HOST_MD5_SMALL__" = "__MD5_SMALL__" ]
+check	cmp __TEMP_NS_SMALL__ __BASEPATH__/small.bin
 
 test	TCP/IPv6: ns to host (spliced): small transfer
 hostb	socat -u TCP6-LISTEN:10003 OPEN:__TEMP_SMALL__,create,trunc
 sleep	1
-ns	socat -u OPEN:__TEMP_NS_SMALL__ TCP6:[::1]:10003
+ns	socat -u OPEN:__BASEPATH__/small.bin TCP6:[::1]:10003
 hostw
-hout	HOST_MD5_SMALL md5sum __TEMP_SMALL__ | cut -d' ' -f1
-check	[ "__HOST_MD5_SMALL__" = "__MD5_SMALL__" ]
+check	cmp __TEMP_SMALL__ __BASEPATH__/small.bin
 
 test	TCP/IPv6: ns to host (via tap): small transfer
 hostb	socat -u TCP6-LISTEN:10003 OPEN:__TEMP_SMALL__,create,trunc
 nsout	IFNAME ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname'
 sleep	1
-ns	socat -u OPEN:__TEMP_NS_SMALL__ TCP6:[__GW6__%__IFNAME__]:10003
+ns	socat -u OPEN:__BASEPATH__/small.bin TCP6:[__GW6__%__IFNAME__]:10003
 hostw
-hout	HOST_MD5_SMALL md5sum __TEMP_SMALL__ | cut -d' ' -f1
-check	[ "__HOST_MD5_SMALL__" = "__MD5_SMALL__" ]
+check	cmp __TEMP_SMALL__ __BASEPATH__/small.bin
 
 test	TCP/IPv6: ns to guest (using loopback address): small transfer
 guestb	socat -u TCP6-LISTEN:10001 OPEN:test_small.bin,create,trunc
 sleep	1
-ns	socat -u OPEN:__TEMP_NS_SMALL__ TCP6:[::1]:10001
+ns	socat -u OPEN:__BASEPATH__/small.bin TCP6:[::1]:10001
 guestw
-gout	GUEST_MD5_SMALL md5sum test_small.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5_SMALL__" = "__MD5_SMALL__" ]
+guest	cmp test_small.bin /root/small.bin
 
 test	TCP/IPv6: ns to guest (using namespace address): small transfer
 guestb	socat -u TCP6-LISTEN:10001 OPEN:test_small.bin,create,trunc
 sleep	1
-ns	socat -u OPEN:__TEMP_NS_SMALL__ TCP6:[__ADDR6__]:10001
+ns	socat -u OPEN:__BASEPATH__/small.bin TCP6:[__ADDR6__]:10001
 guestw
-gout	GUEST_MD5_SMALL md5sum test_small.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5_SMALL__" = "__MD5_SMALL__" ]
+guest	cmp test_small.bin /root/small.bin
diff --git a/test/passt_in_ns/udp b/test/passt_in_ns/udp
index b7e9cf3..d9cd85a 100644
--- a/test/passt_in_ns/udp
+++ b/test/passt_in_ns/udp
@@ -11,145 +11,128 @@
 # Copyright (c) 2021 Red Hat GmbH
 # Author: Stefano Brivio <sbrivio(a)redhat.com>
 
-gtools	socat ip jq md5sum cut
-nstools	socat ip jq md5sum cut
-htools	dd socat ip jq md5sum cut
+gtools	socat ip jq
+nstools	socat ip jq
+htools	socat ip jq
+
+set	TEMP __STATEDIR__/test.bin
+set	TEMP_NS __STATEDIR__/test_ns.bin
 
 test	UDP/IPv4: host to guest
-set	TEMP __STATEDIR__/data
 guestb	socat -u UDP4-LISTEN:10001,null-eof OPEN:test.bin,create,trunc
-host	dd if=/dev/urandom bs=1k count=5 > __TEMP__
 sleep	1
-host	socat -u OPEN:__TEMP__ UDP4:127.0.0.1:10001,shut-null
+host	socat -u OPEN:__BASEPATH__/medium.bin UDP4:127.0.0.1:10001,shut-null
 guestw
-hout	MD5 md5sum __TEMP__ | cut -d' ' -f1
-gout	GUEST_MD5 md5sum test.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5__" = "__MD5__" ]
+guest	cmp test.bin /root/medium.bin
 
 test	UDP/IPv4: host to ns
-set	TEMP_NS __STATEDIR__/data_ns
 nsb	socat -u UDP4-LISTEN:10002,null-eof OPEN:__TEMP_NS__,create,trunc
 sleep	1
-host	socat -u OPEN:__TEMP__ UDP4:127.0.0.1:10002,shut-null
+host	socat -u OPEN:__BASEPATH__/medium.bin UDP4:127.0.0.1:10002,shut-null
 nsw
-nsout	NS_MD5 md5sum __TEMP_NS__ | cut -d' ' -f1
-check	[ "__NS_MD5__" = "__MD5__" ]
+check	cmp __TEMP_NS__ __BASEPATH__/medium.bin
 
 test	UDP/IPv4: guest to host
 hostb	socat -u UDP4-LISTEN:10003,null-eof OPEN:__TEMP__,create,trunc
 gout	GW ip -j -4 route show|jq -rM '.[] | select(.dst == "default").gateway'
 sleep	1
-guest	socat -u OPEN:test.bin UDP4:__GW__:10003,shut-null
+guest	socat -u OPEN:/root/medium.bin UDP4:__GW__:10003,shut-null
 hostw
-hout	HOST_MD5 md5sum __TEMP__ | cut -d' ' -f1
-check	[ "__HOST_MD5__" = "__MD5__" ]
+check	cmp __TEMP__ __BASEPATH__/medium.bin
 
 test	UDP/IPv4: guest to ns
 nsb	socat -u UDP4-LISTEN:10002,null-eof OPEN:__TEMP_NS__,create,trunc
 sleep	1
-guest	socat -u OPEN:test.bin UDP4:__GW__:10002,shut-null
+guest	socat -u OPEN:/root/medium.bin UDP4:__GW__:10002,shut-null
 nsw
-hout	HOST_MD5 md5sum __TEMP__ | cut -d' ' -f1
-check	[ "__HOST_MD5__" = "__MD5__" ]
+check	cmp __TEMP_NS__ __BASEPATH__/medium.bin
 
 test	UDP/IPv4: ns to host (recvmmsg/sendmmsg)
 hostb	socat -u UDP4-LISTEN:10003,null-eof OPEN:__TEMP__,create,trunc
 sleep	1
-ns	socat -u OPEN:__TEMP_NS__ UDP4:127.0.0.1:10003,shut-null
+ns	socat -u OPEN:__BASEPATH__/medium.bin UDP4:127.0.0.1:10003,shut-null
 hostw
-hout	HOST_MD5 md5sum __TEMP__ | cut -d' ' -f1
-check	[ "__HOST_MD5__" = "__MD5__" ]
+check	cmp __TEMP__ __BASEPATH__/medium.bin
 
 test	UDP/IPv4: ns to host (via tap)
 hostb	socat -u UDP4-LISTEN:10003,null-eof OPEN:__TEMP__,create,trunc
 sleep	1
-ns	socat -u OPEN:__TEMP_NS__ UDP4:__GW__:10003,shut-null
+ns	socat -u OPEN:__BASEPATH__/medium.bin UDP4:__GW__:10003,shut-null
 hostw
-hout	HOST_MD5 md5sum __TEMP__ | cut -d' ' -f1
-check	[ "__HOST_MD5__" = "__MD5__" ]
+check	cmp __TEMP__ __BASEPATH__/medium.bin
 
 test	UDP/IPv4: ns to guest (using loopback address)
 guestb	socat -u UDP4-LISTEN:10001,null-eof OPEN:test.bin,create,trunc
 sleep	1
-ns	socat -u OPEN:__TEMP_NS__ UDP4:127.0.0.1:10001,shut-null
+ns	socat -u OPEN:__BASEPATH__/medium.bin UDP4:127.0.0.1:10001,shut-null
 guestw
-gout	GUEST_MD5 md5sum test.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5__" = "__MD5__" ]
+guest	cmp test.bin /root/medium.bin
 
 test	UDP/IPv4: ns to guest (using namespace address)
 guestb	socat -u UDP4-LISTEN:10001,null-eof OPEN:test.bin,create,trunc
 nsout	IFNAME ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname'
 nsout	ADDR ip -j -4 addr show|jq -rM '.[] | select(.ifname == "__IFNAME__").addr_info[0].local'
 sleep	1
-ns	socat -u OPEN:__TEMP_NS__ UDP4:__ADDR__:10001,shut-null
+ns	socat -u OPEN:__BASEPATH__/medium.bin UDP4:__ADDR__:10001,shut-null
 guestw
-gout	GUEST_MD5 md5sum test.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5__" = "__MD5__" ]
+guest	cmp test.bin /root/medium.bin
 
 test	UDP/IPv6: host to guest
 guestb	socat -u UDP6-LISTEN:10001,null-eof OPEN:test.bin,create,trunc
 sleep	1
-host	socat -u OPEN:__TEMP__ UDP6:[::1]:10001,shut-null
+host	socat -u OPEN:__BASEPATH__/medium.bin UDP6:[::1]:10001,shut-null
 guestw
-gout	GUEST_MD5 md5sum test.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5__" = "__MD5__" ]
+guest	cmp test.bin /root/medium.bin
 
 test	UDP/IPv6: host to ns
 nsb	socat -u UDP6-LISTEN:10002,null-eof OPEN:__TEMP_NS__,create,trunc
 sleep	1
-host	socat -u OPEN:__TEMP__ UDP6:[::1]:10002,shut-null
+host	socat -u OPEN:__BASEPATH__/medium.bin UDP6:[::1]:10002,shut-null
 nsw
-nsout	NS_MD5 md5sum __TEMP_NS__ | cut -d' ' -f1
-check	[ "__NS_MD5__" = "__MD5__" ]
+check	cmp __TEMP_NS__ __BASEPATH__/medium.bin
 
 test	UDP/IPv6: guest to host
 hostb	socat -u UDP6-LISTEN:10003,null-eof OPEN:__TEMP__,create,trunc
 gout	GW6 ip -j -6 route show|jq -rM '.[] | select(.dst == "default").gateway'
 gout	IFNAME ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname'
 sleep	1
-guest	socat -u OPEN:test.bin UDP6:[__GW6__%__IFNAME__]:10003,shut-null
+guest	socat -u OPEN:/root/medium.bin UDP6:[__GW6__%__IFNAME__]:10003,shut-null
 hostw
-hout	HOST_MD5 md5sum __TEMP__ | cut -d' ' -f1
-check	[ "__HOST_MD5__" = "__MD5__" ]
+check	cmp __TEMP__ __BASEPATH__/medium.bin
 
 test	UDP/IPv6: guest to ns
 nsb	socat -u UDP6-LISTEN:10002,null-eof OPEN:__TEMP_NS__,create,trunc
 sleep	1
-guest	socat -u OPEN:test.bin UDP6:[__GW6__%__IFNAME__]:10002,shut-null
+guest	socat -u OPEN:/root/medium.bin UDP6:[__GW6__%__IFNAME__]:10002,shut-null
 nsw
-hout	HOST_MD5 md5sum __TEMP__ | cut -d' ' -f1
-check	[ "__HOST_MD5__" = "__MD5__" ]
+check	cmp __TEMP_NS__ __BASEPATH__/medium.bin
 
 test	UDP/IPv6: ns to host (recvmmsg/sendmmsg)
 hostb	socat -u UDP6-LISTEN:10003,null-eof OPEN:__TEMP__,create,trunc
 sleep	1
-ns	socat -u OPEN:__TEMP_NS__ UDP6:[::1]:10003,shut-null
+ns	socat -u OPEN:__BASEPATH__/medium.bin UDP6:[::1]:10003,shut-null
 hostw
-hout	HOST_MD5 md5sum __TEMP__ | cut -d' ' -f1
-check	[ "__HOST_MD5__" = "__MD5__" ]
+check	cmp __TEMP__ __BASEPATH__/medium.bin
 
 test	UDP/IPv6: ns to host (via tap)
 hostb	socat -u UDP6-LISTEN:10003,null-eof OPEN:__TEMP__,create,trunc
 nsout	IFNAME ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname'
 sleep	1
-ns	socat -u OPEN:__TEMP_NS__ UDP6:[__GW6__%__IFNAME__]:10003,shut-null
+ns	socat -u OPEN:__BASEPATH__/medium.bin UDP6:[__GW6__%__IFNAME__]:10003,shut-null
 hostw
-hout	HOST_MD5 md5sum __TEMP__ | cut -d' ' -f1
-check	[ "__HOST_MD5__" = "__MD5__" ]
+check	cmp __TEMP__ __BASEPATH__/medium.bin
 
 test	UDP/IPv6: ns to guest (using loopback address)
 guestb	socat -u UDP6-LISTEN:10001,null-eof OPEN:test.bin,create,trunc
 sleep	1
-ns	socat -u OPEN:__TEMP_NS__ UDP6:[::1]:10001,shut-null
+ns	socat -u OPEN:__BASEPATH__/medium.bin UDP6:[::1]:10001,shut-null
 guestw
-gout	GUEST_MD5 md5sum test.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5__" = "__MD5__" ]
+guest	cmp test.bin /root/medium.bin
 
 test	UDP/IPv6: ns to guest (using namespace address)
 guestb	socat -u UDP6-LISTEN:10001,null-eof OPEN:test.bin,create,trunc
 nsout	ADDR6 ip -j -6 addr show|jq -rM '.[] | select(.ifname == "__IFNAME__").addr_info[0].local'
 sleep	1
-ns	socat -u OPEN:__TEMP_NS__ UDP6:[__ADDR6__]:10001,shut-null
+ns	socat -u OPEN:__BASEPATH__/medium.bin UDP6:[__ADDR6__]:10001,shut-null
 guestw
-gout	GUEST_MD5 md5sum test.bin | cut -d' ' -f1
-check	[ "__GUEST_MD5__" = "__MD5__" ]
+guest	cmp test.bin /root/medium.bin
diff --git a/test/pasta/tcp b/test/pasta/tcp
index d37e21c..eeda20b 100644
--- a/test/pasta/tcp
+++ b/test/pasta/tcp
@@ -11,103 +11,86 @@
 # Copyright (c) 2021 Red Hat GmbH
 # Author: Stefano Brivio <sbrivio(a)redhat.com>
 
-htools	dd socat ip jq md5sum cut
-nstools	socat ip jq md5sum cut
+htools	socat ip jq
+nstools	socat ip jq
+
+set	TEMP_BIG __STATEDIR__/test_big.bin
+set	TEMP_NS_BIG __STATEDIR__/test_ns_big.bin
+set	TEMP_SMALL __STATEDIR__/test_small.bin
+set	TEMP_NS_SMALL __STATEDIR__/test_ns_small.bin
 
 test	TCP/IPv4: host to ns: big transfer
-set	TEMP_BIG __STATEDIR__/big
-set	TEMP_NS_BIG __STATEDIR__/big_ns
 nsb	socat -u TCP4-LISTEN:10002,bind=127.0.0.1 OPEN:__TEMP_NS_BIG__,create,trunc
-host	dd if=/dev/urandom bs=1M count=10 of=__TEMP_BIG__
-host	socat -u OPEN:__TEMP_BIG__ TCP4:127.0.0.1:10002
+host	socat -u OPEN:__BASEPATH__/big.bin TCP4:127.0.0.1:10002
 nsw
-hout	MD5_BIG md5sum __TEMP_BIG__ | cut -d' ' -f1
-nsout	NS_MD5_BIG md5sum __TEMP_NS_BIG__ | cut -d' ' -f1
-check	[ "__NS_MD5_BIG__" = "__MD5_BIG__" ]
+check	cmp __BASEPATH__/big.bin __TEMP_NS_BIG__
 
 test	TCP/IPv4: ns to host (spliced): big transfer
 hostb	socat -u TCP4-LISTEN:10003,bind=127.0.0.1 OPEN:__TEMP_BIG__,create,trunc
-ns	socat -u OPEN:__TEMP_NS_BIG__ TCP4:127.0.0.1:10003
+ns	socat -u OPEN:__BASEPATH__/big.bin TCP4:127.0.0.1:10003
 hostw
-hout	HOST_MD5_BIG md5sum __TEMP_BIG__ | cut -d' ' -f1
-check	[ "__HOST_MD5_BIG__" = "__MD5_BIG__" ]
+check	cmp __BASEPATH__/big.bin __TEMP_BIG__
 
 test	TCP/IPv4: ns to host (via tap): big transfer
 hostb	socat -u TCP4-LISTEN:10003 OPEN:__TEMP_BIG__,create,trunc
 nsout	GW ip -j -4 route show|jq -rM '.[] | select(.dst == "default").gateway'
-ns	socat -u OPEN:__TEMP_NS_BIG__ TCP4:__GW__:10003
+ns	socat -u OPEN:__BASEPATH__/big.bin TCP4:__GW__:10003
 hostw
-hout	HOST_MD5_BIG md5sum __TEMP_BIG__ | cut -d' ' -f1
-check	[ "__HOST_MD5_BIG__" = "__MD5_BIG__" ]
+check	cmp __BASEPATH__/big.bin __TEMP_BIG__
 
 test	TCP/IPv4: host to ns: small transfer
-set	TEMP_SMALL __STATEDIR__/small
-set	TEMP_NS_SMALL __STATEDIR__/small_ns
 nsb	socat -u TCP4-LISTEN:10002,bind=127.0.0.1 OPEN:__TEMP_NS_SMALL__,create,trunc
-host	dd if=/dev/urandom bs=2k count=1 of=__TEMP_SMALL__
-host	socat OPEN:__TEMP_SMALL__ TCP4:127.0.0.1:10002
+host	socat OPEN:__BASEPATH__/small.bin TCP4:127.0.0.1:10002
 nsw
-hout	MD5_SMALL md5sum __TEMP_SMALL__ | cut -d' ' -f1
-nsout	NS_MD5_SMALL md5sum __TEMP_NS_SMALL__ | cut -d' ' -f1
-check	[ "__NS_MD5_SMALL__" = "__MD5_SMALL__" ]
+check	cmp __BASEPATH__/small.bin __TEMP_NS_SMALL__
 
 test	TCP/IPv4: ns to host (spliced): small transfer
 hostb	socat -u TCP4-LISTEN:10003,bind=127.0.0.1 OPEN:__TEMP_SMALL__,create,trunc
-ns	socat OPEN:__TEMP_NS_SMALL__ TCP4:127.0.0.1:10003
+ns	socat OPEN:__BASEPATH__/small.bin TCP4:127.0.0.1:10003
 hostw
-hout	HOST_MD5_SMALL md5sum __TEMP_SMALL__ | cut -d' ' -f1
-check	[ "__HOST_MD5_SMALL__" = "__MD5_SMALL__" ]
+check	cmp __BASEPATH__/small.bin __TEMP_SMALL__
 
 test	TCP/IPv4: ns to host (via tap): small transfer
 hostb	socat -u TCP4-LISTEN:10003 OPEN:__TEMP_SMALL__,create,trunc
 nsout	GW ip -j -4 route show|jq -rM '.[] | select(.dst == "default").gateway'
-ns	socat -u OPEN:__TEMP_NS_SMALL__ TCP4:__GW__:10003
+ns	socat -u OPEN:__BASEPATH__/small.bin TCP4:__GW__:10003
 hostw
-hout	HOST_MD5_SMALL md5sum __TEMP_SMALL__ | cut -d' ' -f1
-check	[ "__HOST_MD5_SMALL__" = "__MD5_SMALL__" ]
+check	cmp __BASEPATH__/small.bin __TEMP_SMALL__
 
 test	TCP/IPv6: host to ns: big transfer
 nsb	socat -u TCP6-LISTEN:10002,bind=[::1] OPEN:__TEMP_NS_BIG__,create,trunc
-host	socat -u OPEN:__TEMP_BIG__ TCP6:[::1]:10002
+host	socat -u OPEN:__BASEPATH__/big.bin TCP6:[::1]:10002
 nsw
-hout	MD5_BIG md5sum __TEMP_BIG__ | cut -d' ' -f1
-nsout	NS_MD5_BIG md5sum __TEMP_NS_BIG__ | cut -d' ' -f1
-check	[ "__NS_MD5_BIG__" = "__MD5_BIG__" ]
+check	cmp __BASEPATH__/big.bin __TEMP_NS_BIG__
 
 test	TCP/IPv6: ns to host (spliced): big transfer
 hostb	socat -u TCP6-LISTEN:10003,bind=[::1] OPEN:__TEMP_BIG__,create,trunc
-ns	socat -u OPEN:__TEMP_NS_BIG__ TCP6:[::1]:10003
+ns	socat -u OPEN:__BASEPATH__/big.bin TCP6:[::1]:10003
 hostw
-hout	HOST_MD5_BIG md5sum __TEMP_BIG__ | cut -d' ' -f1
-check	[ "__HOST_MD5_BIG__" = "__MD5_BIG__" ]
+check	cmp __BASEPATH__/big.bin __TEMP_BIG__
 
 test	TCP/IPv6: ns to host (via tap): big transfer
 hostb	socat -u TCP6-LISTEN:10003 OPEN:__TEMP_BIG__,create,trunc
 nsout	GW6 ip -j -6 route show|jq -rM '.[] | select(.dst == "default").gateway'
 nsout	IFNAME ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname'
-ns	socat -u OPEN:__TEMP_NS_BIG__ TCP6:[__GW6__%__IFNAME__]:10003
+ns	socat -u OPEN:__BASEPATH__/big.bin TCP6:[__GW6__%__IFNAME__]:10003
 hostw
-hout	HOST_MD5_BIG md5sum __TEMP_BIG__ | cut -d' ' -f1
-check	[ "__HOST_MD5_BIG__" = "__MD5_BIG__" ]
+check	cmp __BASEPATH__/big.bin __TEMP_BIG__
 
 test	TCP/IPv6: host to ns: small transfer
 nsb	socat -u TCP6-LISTEN:10002,bind=[::1] OPEN:__TEMP_NS_SMALL__,create,trunc
-host	socat -u OPEN:__TEMP_SMALL__ TCP6:[::1]:10002
+host	socat -u OPEN:__BASEPATH__/small.bin TCP6:[::1]:10002
 nsw
-hout	MD5_SMALL md5sum __TEMP_SMALL__ | cut -d' ' -f1
-nsout	NS_MD5_SMALL md5sum __TEMP_NS_SMALL__ | cut -d' ' -f1
-check	[ "__NS_MD5_SMALL__" = "__MD5_SMALL__" ]
+check	cmp __BASEPATH__/small.bin __TEMP_NS_SMALL__
 
 test	TCP/IPv6: ns to host (spliced): small transfer
 hostb	socat -u TCP6-LISTEN:10003,bind=[::1] OPEN:__TEMP_SMALL__,create,trunc
-ns	socat -u OPEN:__TEMP_NS_SMALL__ TCP6:[::1]:10003
+ns	socat -u OPEN:__BASEPATH__/small.bin TCP6:[::1]:10003
 hostw
-hout	HOST_MD5_SMALL md5sum __TEMP_SMALL__ | cut -d' ' -f1
-check	[ "__HOST_MD5_SMALL__" = "__MD5_SMALL__" ]
+check	cmp __BASEPATH__/small.bin __TEMP_SMALL__
 
 test	TCP/IPv6: ns to host (via tap): small transfer
 hostb	socat -u TCP6-LISTEN:10003 OPEN:__TEMP_SMALL__,create,trunc
-ns	socat -u OPEN:__TEMP_NS_SMALL__ TCP6:[__GW6__%__IFNAME__]:10003
+ns	socat -u OPEN:__BASEPATH__/small.bin TCP6:[__GW6__%__IFNAME__]:10003
 hostw
-hout	HOST_MD5_SMALL md5sum __TEMP_SMALL__ | cut -d' ' -f1
-check	[ "__HOST_MD5_SMALL__" = "__MD5_SMALL__" ]
+check	cmp __BASEPATH__/small.bin __TEMP_SMALL__
diff --git a/test/pasta/udp b/test/pasta/udp
index fc6bf0a..c8ff418 100644
--- a/test/pasta/udp
+++ b/test/pasta/udp
@@ -11,58 +11,49 @@
 # Copyright (c) 2021 Red Hat GmbH
 # Author: Stefano Brivio <sbrivio(a)redhat.com>
 
-nstools	socat ip jq md5sum cut
-htools	dd socat ip jq md5sum cut
+nstools	socat ip jq
+htools	dd socat ip jq
+
+set	TEMP __STATEDIR__/test.bin
+set	TEMP_NS __STATEDIR__/test_ns.bin
 
 test	UDP/IPv4: host to ns
-set	TEMP __STATEDIR__/data
-set	TEMP_NS __STATEDIR__/data_ns
 nsb	socat -u UDP4-LISTEN:10002,bind=127.0.0.1,null-eof OPEN:__TEMP_NS__,create,trunc
-host	dd if=/dev/urandom bs=1k count=5 > __TEMP__
-
-host	socat OPEN:__TEMP__ UDP4:127.0.0.1:10002,shut-null
+host	socat OPEN:__BASEPATH__/medium.bin UDP4:127.0.0.1:10002,shut-null
 nsw
-hout	MD5 md5sum __TEMP__ | cut -d' ' -f1
-nsout	NS_MD5 md5sum __TEMP_NS__ | cut -d' ' -f1
-check	[ "__NS_MD5__" = "__MD5__" ]
+check	cmp __BASEPATH__/medium.bin __TEMP_NS__
 
 test	UDP/IPv4: ns to host (recvmmsg/sendmmsg)
 hostb	socat -u UDP4-LISTEN:10003,bind=127.0.0.1,null-eof OPEN:__TEMP__,create,trunc
 sleep	1
-ns	socat OPEN:__TEMP_NS__ UDP4:127.0.0.1:10003,shut-null
+ns	socat OPEN:__BASEPATH__/medium.bin UDP4:127.0.0.1:10003,shut-null
 hostw
-hout	HOST_MD5 md5sum __TEMP__ | cut -d' ' -f1
-check	[ "__HOST_MD5__" = "__MD5__" ]
+check	cmp __BASEPATH__/medium.bin __TEMP__
 
 test	UDP/IPv4: ns to host (via tap)
 hostb	socat -u UDP4-LISTEN:10003,null-eof OPEN:__TEMP__,create,trunc
 nsout	GW ip -j -4 route show|jq -rM '.[] | select(.dst == "default").gateway'
-ns	socat -u OPEN:__TEMP_NS__ UDP4:__GW__:10003,shut-null
+ns	socat -u OPEN:__BASEPATH__/medium.bin UDP4:__GW__:10003,shut-null
 hostw
-hout	HOST_MD5 md5sum __TEMP__ | cut -d' ' -f1
-check	[ "__HOST_MD5__" = "__MD5__" ]
+check	cmp __BASEPATH__/medium.bin __TEMP__
 
 test	UDP/IPv6: host to ns
 nsb	socat -u UDP6-LISTEN:10002,bind=[::1],null-eof OPEN:__TEMP_NS__,create,trunc
-host	socat -u OPEN:__TEMP__ UDP6:[::1]:10002,shut-null
+host	socat -u OPEN:__BASEPATH__/medium.bin UDP6:[::1]:10002,shut-null
 nsw
-hout	MD5 md5sum __TEMP__ | cut -d' ' -f1
-nsout	NS_MD5 md5sum __TEMP_NS__ | cut -d' ' -f1
-check	[ "__NS_MD5__" = "__MD5__" ]
+check	cmp __BASEPATH__/medium.bin __TEMP_NS__
 
 test	UDP/IPv6: ns to host (recvmmsg/sendmmsg)
 hostb	socat -u UDP6-LISTEN:10003,bind=[::1],null-eof OPEN:__TEMP__,create,trunc
 sleep	1
-ns	socat -u OPEN:__TEMP_NS__ UDP6:[::1]:10003,shut-null
+ns	socat -u OPEN:__BASEPATH__/medium.bin UDP6:[::1]:10003,shut-null
 hostw
-hout	HOST_MD5 md5sum __TEMP__ | cut -d' ' -f1
-check	[ "__HOST_MD5__" = "__MD5__" ]
+check	cmp __BASEPATH__/medium.bin __TEMP__
 
 test	UDP/IPv6: ns to host (via tap)
 hostb	socat -u UDP6-LISTEN:10003,null-eof OPEN:__TEMP__,create,trunc
 nsout	GW6 ip -j -6 route show|jq -rM '.[] | select(.dst == "default").gateway'
 nsout	IFNAME ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname'
-ns	socat -u OPEN:__TEMP_NS__ UDP6:[__GW6__%__IFNAME__]:10003,shut-null
+ns	socat -u OPEN:__BASEPATH__/medium.bin UDP6:[__GW6__%__IFNAME__]:10003,shut-null
 hostw
-hout	HOST_MD5 md5sum __TEMP__ | cut -d' ' -f1
-check	[ "__HOST_MD5__" = "__MD5__" ]
+check	cmp __BASEPATH__/medium.bin __TEMP__
diff --git a/test/two_guests/basic b/test/two_guests/basic
index e226178..dba4b9e 100644
--- a/test/two_guests/basic
+++ b/test/two_guests/basic
@@ -13,7 +13,7 @@
 
 g1tools	ip jq dhclient socat cat
 g2tools	ip jq dhclient socat cat
-htools	ip jq md5sum cut
+htools	ip jq
 
 test	Interface names
 g1out	IFNAME1 ip -j link show | jq -rM '.[] | select(.link_type == "ether").ifname'
-- 
2.37.3

Re: [PATCH 06/12] test: Simplify data handling for transfer tests

[Stefano Brivio]

[-- Attachment #1: Type: text/plain, Size: 1865 bytes --]

On Fri, 23 Sep 2022 17:20:32 +1000
David Gibson <david(a)gibson.dropbear.id.au> wrote:

> Many of our tests are based around performing transfers of sample data
> across passt/pasta created links.  The data flow here can be a bit
> hard to follow since, e.g. we create a file transfer it to the guest,
> then transfer it back to the host across several different tests.
> This also means that the test cases aren't independent of each other.
> 
> Because we don't have the original file available at both ends in some
> cases, we compare them by generating md5sums at each end and comparing
> them, which is a bit complicated.
> 
> Make a number of changes to simplify this:
>   1. Pre-generate the sample data files as a test asset, rather than
>      building them on the fly during the tests proper
>   2. Include the sample data files in the mbuto guest image
>   3. Because we have good copies of the original data available in all
>      contexts, we can now simply use 'cmp' to check if the transfer
>      has worked, avoiding md5sum complications.
>   4. Similarly we can always use the original copy of the sample data
>      on the send side of each transfer, meaning that the tests become
>      more independent of each other.
> 
> Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
> ---
>  test/.gitignore       |   1 +
>  test/Makefile         |  15 +++-
>  test/passt.mbuto      |   6 +-
>  test/passt/tcp        |  53 +++++-------
>  test/passt/udp        |  29 +++----
>  test/passt_in_ns/tcp  | 187 +++++++++++++++++-------------------------
>  test/passt_in_ns/udp  |  93 +++++++++------------
>  test/pasta/tcp        |  79 +++++++-----------
>  test/pasta/udp        |  43 ++++------
>  test/two_guests/basic |   2 +-
>  10 files changed, 214 insertions(+), 294 deletions(-)

Uh, yes, this does clean up a lot of mess.

-- 
Stefano

[PATCH 07/12] test: Remove unneccessary pane naming from layout_two_guests

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 695 bytes --]

This loop goes through and gives a numeric label to each pane, even though
we name the panes properly shortly thereafter.  Looks like a leftover from
some earlier version.  Remove it.

Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
---
 test/lib/layout | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/test/lib/layout b/test/lib/layout
index 9b16c4d..3875dec 100644
--- a/test/lib/layout
+++ b/test/lib/layout
@@ -137,10 +137,6 @@ layout_two_guests() {
 	tmux split-window -h -l '35%' -t passt_test:1.0
 	tmux split-window -v -t passt_test:1.0
 
-	for i in `seq 0 5`; do
-		tmux select-pane -t $i -T "${i}"
-	done
-
 	PANE_GUEST_1=0
 	PANE_GUEST_2=1
 	PANE_INFO=2
-- 
2.37.3

[PATCH 08/12] clang-tidy: Disable 'readability-identifier-length'

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1219 bytes --]

This check complains about any identifier of less than 3 characters.  For
locals and parameters this is often pointlessly verbose.  Disable it.

Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
---
 Makefile | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 432ee7a..0598865 100644
--- a/Makefile
+++ b/Makefile
@@ -237,6 +237,10 @@ docs: README.md
 #
 # - concurrency-mt-unsafe
 #	TODO: check again if multithreading is implemented
+#
+# - readability-identifier-length
+#	Complains about any identifier <3 characters, reasonable for
+#	globals, pointlessly verbose for locals and parameters.
 
 clang-tidy: $(SRCS) $(HEADERS)
 	clang-tidy -checks=*,-modernize-*,\
@@ -260,7 +264,7 @@ clang-tidy: $(SRCS) $(HEADERS)
 	-bugprone-easily-swappable-parameters,\
 	-readability-function-cognitive-complexity,\
 	-altera-struct-pack-align,\
-	-concurrency-mt-unsafe \
+	-concurrency-mt-unsafe,-readability-identifier-length \
 	-config='{CheckOptions: [{key: bugprone-suspicious-string-compare.WarnOnImplicitComparison, value: "false"}]}' \
 	--warnings-as-errors=* $(SRCS) -- $(filter-out -pie,$(FLAGS) $(CFLAGS))
 
-- 
2.37.3

Re: [PATCH 08/12] clang-tidy: Disable 'readability-identifier-length'

[Stefano Brivio]

[-- Attachment #1: Type: text/plain, Size: 1458 bytes --]

On Fri, 23 Sep 2022 17:20:34 +1000
David Gibson <david(a)gibson.dropbear.id.au> wrote:

> This check complains about any identifier of less than 3 characters.  For
> locals and parameters this is often pointlessly verbose.  Disable it.
> 
> Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
> ---
>  Makefile | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/Makefile b/Makefile
> index 432ee7a..0598865 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -237,6 +237,10 @@ docs: README.md
>  #
>  # - concurrency-mt-unsafe
>  #	TODO: check again if multithreading is implemented
> +#
> +# - readability-identifier-length
> +#	Complains about any identifier <3 characters, reasonable for
> +#	globals, pointlessly verbose for locals and parameters.
>  
>  clang-tidy: $(SRCS) $(HEADERS)
>  	clang-tidy -checks=*,-modernize-*,\
> @@ -260,7 +264,7 @@ clang-tidy: $(SRCS) $(HEADERS)
>  	-bugprone-easily-swappable-parameters,\
>  	-readability-function-cognitive-complexity,\
>  	-altera-struct-pack-align,\
> -	-concurrency-mt-unsafe \
> +	-concurrency-mt-unsafe,-readability-identifier-length \

I would rather keep those (unrelated) switches on different lines, I
find it's a bit easier to follow through the comments like that.

I can change this on merge if you agree, unless you respin.

I reviewed the rest of the series, it looks great to me, I have no
further observations -- but I haven't tested it yet.

-- 
Stefano

Re: [PATCH 08/12] clang-tidy: Disable 'readability-identifier-length'

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1823 bytes --]

On Sat, Sep 24, 2022 at 01:08:55AM +0200, Stefano Brivio wrote:
> On Fri, 23 Sep 2022 17:20:34 +1000
> David Gibson <david(a)gibson.dropbear.id.au> wrote:
> 
> > This check complains about any identifier of less than 3 characters.  For
> > locals and parameters this is often pointlessly verbose.  Disable it.
> > 
> > Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
> > ---
> >  Makefile | 6 +++++-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
> > 
> > diff --git a/Makefile b/Makefile
> > index 432ee7a..0598865 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -237,6 +237,10 @@ docs: README.md
> >  #
> >  # - concurrency-mt-unsafe
> >  #	TODO: check again if multithreading is implemented
> > +#
> > +# - readability-identifier-length
> > +#	Complains about any identifier <3 characters, reasonable for
> > +#	globals, pointlessly verbose for locals and parameters.
> >  
> >  clang-tidy: $(SRCS) $(HEADERS)
> >  	clang-tidy -checks=*,-modernize-*,\
> > @@ -260,7 +264,7 @@ clang-tidy: $(SRCS) $(HEADERS)
> >  	-bugprone-easily-swappable-parameters,\
> >  	-readability-function-cognitive-complexity,\
> >  	-altera-struct-pack-align,\
> > -	-concurrency-mt-unsafe \
> > +	-concurrency-mt-unsafe,-readability-identifier-length \
> 
> I would rather keep those (unrelated) switches on different lines, I
> find it's a bit easier to follow through the comments like that.
> 
> I can change this on merge if you agree, unless you respin.

I am respinning, so I've made this change.

> I reviewed the rest of the series, it looks great to me, I have no
> further observations -- but I haven't tested it yet.
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 09/12] cppcheck: Avoid excessive scanning due to system headers

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 2475 bytes --]

make cppcheck takes a long time, because it checks a large number of
different configurations.  It's assembling this very large set of
configurations not because of conditionals in the passt code itself, but
from those in the system headers.  By adding --config-exclude directives
to stop considering those configs, make cppcheck because around 60x faster.

Similarly, any problems that are found in the system headers are not our
problem, and so we can uniformly suppress them, rather than having specific
suppressions for particular problems in particular files (which might not
be correct for all different distro / version combinations either).

Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
---
 Makefile | 20 ++++++--------------
 1 file changed, 6 insertions(+), 14 deletions(-)

diff --git a/Makefile b/Makefile
index 0598865..b6c68bd 100644
--- a/Makefile
+++ b/Makefile
@@ -268,27 +268,19 @@ clang-tidy: $(SRCS) $(HEADERS)
 	-config='{CheckOptions: [{key: bugprone-suspicious-string-compare.WarnOnImplicitComparison, value: "false"}]}' \
 	--warnings-as-errors=* $(SRCS) -- $(filter-out -pie,$(FLAGS) $(CFLAGS))
 
+SYSTEM_INCLUDES := /usr/include
 ifeq ($(shell $(CC) -v 2>&1 | grep -c "gcc version"),1)
 TARGET := $(shell ${CC} -v 2>&1 | sed -n 's/Target: \(.*\)/\1/p')
 VER := $(shell $(CC) -dumpversion)
-EXTRA_INCLUDES := /usr/lib/gcc/$(TARGET)/$(VER)/include
-EXTRA_INCLUDES_OPT := -I$(EXTRA_INCLUDES)
-else
-EXTRA_INCLUDES_OPT :=
+SYSTEM_INCLUDES += /usr/lib/gcc/$(TARGET)/$(VER)/include
 endif
 cppcheck: $(SRCS) $(HEADERS)
 	cppcheck --std=c99 --error-exitcode=1 --enable=all --force	\
 	--inconclusive --library=posix					\
-	-I/usr/include $(EXTRA_INCLUDES_OPT)				\
-									\
-	--suppress=syntaxError:/usr/include/stdlib.h			\
-	--suppress=missingIncludeSystem					\
-	--suppress="*:$(EXTRA_INCLUDES)/avx512fintrin.h"		\
-	--suppress="*:$(EXTRA_INCLUDES)/xmmintrin.h"			\
-	--suppress="*:$(EXTRA_INCLUDES)/emmintrin.h"			\
-	--suppress="*:$(EXTRA_INCLUDES)/avxintrin.h"			\
-	--suppress="*:$(EXTRA_INCLUDES)/bmiintrin.h"			\
-									\
+	$(SYSTEM_INCLUDES:%=-I%)					\
+	$(SYSTEM_INCLUDES:%=--config-exclude=%)				\
+	$(SYSTEM_INCLUDES:%=--suppress=*:%/*)				\
+	$(SYSTEM_INCLUDES:%=--suppress=unmatchedSuppression:%/*)	\
 	--suppress=objectIndex:tcp.c --suppress=objectIndex:udp.c	\
 	--suppress=va_list_usedBeforeStarted:util.c			\
 	--suppress=unusedFunction					\
-- 
2.37.3

[PATCH 10/12] cppcheck: Run quietly

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 713 bytes --]

Adding the --quiet option to cppcheck makes the actual errors and warnings
easier to find.

Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index b6c68bd..fd0f5a8 100644
--- a/Makefile
+++ b/Makefile
@@ -276,7 +276,7 @@ SYSTEM_INCLUDES += /usr/lib/gcc/$(TARGET)/$(VER)/include
 endif
 cppcheck: $(SRCS) $(HEADERS)
 	cppcheck --std=c99 --error-exitcode=1 --enable=all --force	\
-	--inconclusive --library=posix					\
+	--inconclusive --library=posix --quiet				\
 	$(SYSTEM_INCLUDES:%=-I%)					\
 	$(SYSTEM_INCLUDES:%=--config-exclude=%)				\
 	$(SYSTEM_INCLUDES:%=--suppress=*:%/*)				\
-- 
2.37.3

[PATCH 11/12] Makefile: Simplify getting target triple for compiler

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1300 bytes --]

We do some manipulation of the output of cc -v to get the target triple
for the platform, to locate headers for cppcheck.  However, we can get
this more easily with cc -dumpmachine - and in fact we do so elsewhere in
the Makefile.

Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
---
 Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index fd0f5a8..ebae81d 100644
--- a/Makefile
+++ b/Makefile
@@ -14,8 +14,9 @@ ifeq ($(RLIMIT_STACK_VAL),unlimited)
 RLIMIT_STACK_VAL := 1024
 endif
 
+TARGET := $(shell $(CC) -dumpmachine)
 # Get 'uname -m'-like architecture description for target
-TARGET_ARCH := $(shell $(CC) -dumpmachine | cut -f1 -d- | tr [a-z] [A-Z])
+TARGET_ARCH := $(shell echo $(TARGET) -dumpmachine | cut -f1 -d- | tr [a-z] [A-Z])
 TARGET_ARCH := $(shell echo $(TARGET_ARCH) | sed 's/POWERPC/PPC/')
 
 AUDIT_ARCH := $(shell echo $(TARGET_ARCH) | sed 's/^ARM.*/ARM/')
@@ -270,7 +271,6 @@ clang-tidy: $(SRCS) $(HEADERS)
 
 SYSTEM_INCLUDES := /usr/include
 ifeq ($(shell $(CC) -v 2>&1 | grep -c "gcc version"),1)
-TARGET := $(shell ${CC} -v 2>&1 | sed -n 's/Target: \(.*\)/\1/p')
 VER := $(shell $(CC) -dumpversion)
 SYSTEM_INCLUDES += /usr/lib/gcc/$(TARGET)/$(VER)/include
 endif
-- 
2.37.3

[PATCH 12/12] cppcheck: Add target specific headers

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 956 bytes --]

Debian and similar distros put target specific header files in
/usr/include/<arch-vendor-os>, rather than directly in /usr/include.  Add
this directory to the includes for cppcheck so it can find them.

Signed-off-by: David Gibson <david(a)gibson.dropbear.id.au>
---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index ebae81d..07bea79 100644
--- a/Makefile
+++ b/Makefile
@@ -269,7 +269,7 @@ clang-tidy: $(SRCS) $(HEADERS)
 	-config='{CheckOptions: [{key: bugprone-suspicious-string-compare.WarnOnImplicitComparison, value: "false"}]}' \
 	--warnings-as-errors=* $(SRCS) -- $(filter-out -pie,$(FLAGS) $(CFLAGS))
 
-SYSTEM_INCLUDES := /usr/include
+SYSTEM_INCLUDES := /usr/include $(wildcard /usr/include/$(TARGET))
 ifeq ($(shell $(CC) -v 2>&1 | grep -c "gcc version"),1)
 VER := $(shell $(CC) -dumpversion)
 SYSTEM_INCLUDES += /usr/lib/gcc/$(TARGET)/$(VER)/include
-- 
2.37.3