Re: [PATCH v4 1/3] test: Extend test scripts to allow running exeter tests.

[Stefano Brivio <sbrivio@redhat.com>]

On Thu,  7 Aug 2025 21:32:35 +1000
David Gibson <david@gibson.dropbear.id.au> wrote:

> Introduce some trivial testcases based on the exeter library.  These run
> passt and pasta with --help and --version options.  Extend our test
> scripts to run these tests.
> 
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> ---
>  test/.gitignore     |  1 +
>  test/Makefile       |  5 ++++-
>  test/lib/exeter     | 46 +++++++++++++++++++++++++++++++++++++++++++++
>  test/run            |  9 ++++++++-
>  test/smoke/smoke.sh | 27 ++++++++++++++++++++++++++
>  5 files changed, 86 insertions(+), 2 deletions(-)
>  create mode 100644 test/lib/exeter
>  create mode 100755 test/smoke/smoke.sh
> 
> diff --git a/test/.gitignore b/test/.gitignore
> index 3573444f..cf48b885 100644
> --- a/test/.gitignore
> +++ b/test/.gitignore
> @@ -11,3 +11,4 @@ nstool
>  rampstream
>  guest-key
>  guest-key.pub
> +/exeter/
> diff --git a/test/Makefile b/test/Makefile
> index bf63db87..332f3f3e 100644
> --- a/test/Makefile
> +++ b/test/Makefile
> @@ -50,7 +50,7 @@ UBUNTU_NEW_IMGS = xenial-server-cloudimg-powerpc-disk1.img \
>  	jammy-server-cloudimg-s390x.img
>  UBUNTU_IMGS = $(UBUNTU_OLD_IMGS) $(UBUNTU_NEW_IMGS)
>  
> -DOWNLOAD_ASSETS = mbuto podman \
> +DOWNLOAD_ASSETS = exeter mbuto podman \
>  	$(DEBIAN_IMGS) $(FEDORA_IMGS) $(OPENSUSE_IMGS) $(UBUNTU_IMGS)

Pre-existing, but this broke a while after you switched the image
preparation to Makefile targets, so you can't really expect people to
issue 'make' at the moment.

I ran 'make exeter' to get further. It would be nice to update
test/README.md (or fix the Makefile... or both).

>  TESTDATA_ASSETS = small.bin big.bin medium.bin \
>  	rampstream
> @@ -70,6 +70,9 @@ assets: $(ASSETS)
>  pull-%: %
>  	git -C $* pull
>  
> +exeter:
> +	git clone https://gitlab.com/dgibson/exeter.git
> +
>  mbuto:
>  	git clone git://mbuto.sh/mbuto
>  
> diff --git a/test/lib/exeter b/test/lib/exeter
> new file mode 100644
> index 00000000..4f7bcff9
> --- /dev/null
> +++ b/test/lib/exeter
> @@ -0,0 +1,46 @@
> +#!/bin/sh
> +#
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +#
> +# PASST - Plug A Simple Socket Transport
> +#  for qemu/UNIX domain socket mode
> +#
> +# PASTA - Pack A Subtle Tap Abstraction
> +#  for network namespace/tap device mode
> +#
> +# test/lib/exeter - Run exeter tests within the rest of passt's tests
> +#
> +# Copyright Red Hat
> +# Author: David Gibson <david@gibson.dropbear.id.au>
> +
> +exeter() {

It would be nice to have the usual arguments / function description
here.

> +	STATESETUP="${STATEBASE}/$1"
> +	mkdir -p "${STATESETUP}"
> +
> +	context_setup_host host
> +        layout_host

Indentation here was done with tabs, this adds spaces instead.

> +
> +        cd test
> +
> +        __ntests=$("$@" --list | wc -l)
> +        if [ $? != 0 ]; then
> +                info "Failed to get exeter manifest for $@"
> +	        pause_continue \
> +		        "Press any key to pause test session"		\
> +		        "Resuming in "					\
> +		        "Paused, press any key to continue"		\
> +		        5
> +                return
> +        fi
> +
> +        status_file_start "Exeter tests: $*" ${__ntests}
> +
> +        for __testid in $("$@" --list); do
> +	        status_test_start "${__testid}"
> +                context_run host "$@" "${__testid}" && status_test_ok || status_test_fail
> +        done
> +
> +        cd ..
> +
> +	teardown_context_watch ${PANE_HOST} host
> +}
> diff --git a/test/run b/test/run
> index f73c3119..9e183682 100755
> --- a/test/run
> +++ b/test/run
> @@ -53,6 +53,7 @@ COMMIT="$(git log --oneline --no-decorate -1)"
>  . lib/layout_ugly
>  . lib/test
>  . lib/video
> +. lib/exeter
>  
>  # cleanup() - Remove temporary files
>  cleanup() {
> @@ -67,6 +68,8 @@ run() {
>  	perf_init
>  	[ ${CI} -eq 1 ]   && video_start ci
>  
> +	exeter smoke/smoke.sh
> +
>  	setup build
>  	test build/all
>  	test build/cppcheck
> @@ -223,6 +226,10 @@ run_selected() {
>  
>  	__setup=
>  	for __test; do
> +                if "test/${__test}" --list; then
> +                        exeter "${__test}"
> +                        continue
> +                fi
>  		# HACK: the migrate tests need the setup repeated for
>  		#       each test
>  		if [ "${__test%%/*}" != "${__setup}" -o		\
> @@ -234,7 +241,7 @@ run_selected() {
>  
>  		test "${__test}"
>  	done
> -	teardown "${__setup}"
> +	[ -n "${__setup}" ] && teardown "${__setup}"
>  
>  	log "PASS: ${STATUS_PASS}, FAIL: ${STATUS_FAIL}, SKIPPED: ${STATUS_SKIPPED}"
>  
> diff --git a/test/smoke/smoke.sh b/test/smoke/smoke.sh
> new file mode 100755
> index 00000000..26a98d31
> --- /dev/null
> +++ b/test/smoke/smoke.sh
> @@ -0,0 +1,27 @@
> +#! /bin/sh

...this is desirable, I think (and, for consistency, I'd use "#!/bin/sh")
as Bash is not universally available, but:

> +#
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +#
> +# PASST - Plug A Simple Socket Transport
> +#  for qemu/UNIX domain socket mode
> +#
> +# PASTA - Pack A Subtle Tap Abstraction
> +#  for network namespace/tap device mode
> +#
> +# test/smoke/smoke.sh - Basic snoke tests

smoke

> +#
> +# Copyright Red Hat
> +# Author: David Gibson <david@gibson.dropbear.id.au>
> +
> +source $(dirname $0)/../exeter/sh/exeter.sh

this is specific to Bash, so what I get with dash is:

smoke/smoke.sh: 16: source: not found
smoke/smoke.sh: 21: exeter_register: not found
smoke/smoke.sh: 22: exeter_register: not found
smoke/smoke.sh: 24: exeter_register: not found
smoke/smoke.sh: 25: exeter_register: not found
smoke/smoke.sh: 27: exeter_main: not found
...

same for build/static_checkers.sh and build/source.sh. See also:



$ checkbashisms test/build/static_checkers.sh 
possible bashism in test/build/static_checkers.sh line 16 (should be '.', not 'source'):
source $(dirname $0)/../exeter/sh/exeter.sh


> +
> +PASST=$(dirname $0)/../../passt
> +PASTA=$(dirname $0)/../../pasta
> +
> +exeter_register passt_version $PASST --version
> +exeter_register pasta_version $PASTA --version
> +
> +exeter_register passt_help $PASST --help
> +exeter_register pasta_help $PASTA --help
> +
> +exeter_main "$@"