* [PATCH v4 1/3] test: Extend test scripts to allow running exeter tests.
2025-08-07 11:32 [PATCH v4 0/3] RFC: New proof-of-concept based exeter tests David Gibson
@ 2025-08-07 11:32 ` David Gibson
2025-08-07 11:32 ` [PATCH v4 2/3] test: Run static checkers as " David Gibson
2025-08-07 11:32 ` [PATCH v4 3/3] test: Convert build tests to exeter David Gibson
2 siblings, 0 replies; 4+ messages in thread
From: David Gibson @ 2025-08-07 11:32 UTC (permalink / raw)
To: passt-dev, Stefano Brivio; +Cc: David Gibson
Introduce some trivial testcases based on the exeter library. These run
passt and pasta with --help and --version options. Extend our test
scripts to run these tests.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
test/.gitignore | 1 +
test/Makefile | 5 ++++-
test/lib/exeter | 46 +++++++++++++++++++++++++++++++++++++++++++++
test/run | 9 ++++++++-
test/smoke/smoke.sh | 27 ++++++++++++++++++++++++++
5 files changed, 86 insertions(+), 2 deletions(-)
create mode 100644 test/lib/exeter
create mode 100755 test/smoke/smoke.sh
diff --git a/test/.gitignore b/test/.gitignore
index 3573444f..cf48b885 100644
--- a/test/.gitignore
+++ b/test/.gitignore
@@ -11,3 +11,4 @@ nstool
rampstream
guest-key
guest-key.pub
+/exeter/
diff --git a/test/Makefile b/test/Makefile
index bf63db87..332f3f3e 100644
--- a/test/Makefile
+++ b/test/Makefile
@@ -50,7 +50,7 @@ UBUNTU_NEW_IMGS = xenial-server-cloudimg-powerpc-disk1.img \
jammy-server-cloudimg-s390x.img
UBUNTU_IMGS = $(UBUNTU_OLD_IMGS) $(UBUNTU_NEW_IMGS)
-DOWNLOAD_ASSETS = mbuto podman \
+DOWNLOAD_ASSETS = exeter mbuto podman \
$(DEBIAN_IMGS) $(FEDORA_IMGS) $(OPENSUSE_IMGS) $(UBUNTU_IMGS)
TESTDATA_ASSETS = small.bin big.bin medium.bin \
rampstream
@@ -70,6 +70,9 @@ assets: $(ASSETS)
pull-%: %
git -C $* pull
+exeter:
+ git clone https://gitlab.com/dgibson/exeter.git
+
mbuto:
git clone git://mbuto.sh/mbuto
diff --git a/test/lib/exeter b/test/lib/exeter
new file mode 100644
index 00000000..4f7bcff9
--- /dev/null
+++ b/test/lib/exeter
@@ -0,0 +1,46 @@
+#!/bin/sh
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# PASST - Plug A Simple Socket Transport
+# for qemu/UNIX domain socket mode
+#
+# PASTA - Pack A Subtle Tap Abstraction
+# for network namespace/tap device mode
+#
+# test/lib/exeter - Run exeter tests within the rest of passt's tests
+#
+# Copyright Red Hat
+# Author: David Gibson <david@gibson.dropbear.id.au>
+
+exeter() {
+ STATESETUP="${STATEBASE}/$1"
+ mkdir -p "${STATESETUP}"
+
+ context_setup_host host
+ layout_host
+
+ cd test
+
+ __ntests=$("$@" --list | wc -l)
+ if [ $? != 0 ]; then
+ info "Failed to get exeter manifest for $@"
+ pause_continue \
+ "Press any key to pause test session" \
+ "Resuming in " \
+ "Paused, press any key to continue" \
+ 5
+ return
+ fi
+
+ status_file_start "Exeter tests: $*" ${__ntests}
+
+ for __testid in $("$@" --list); do
+ status_test_start "${__testid}"
+ context_run host "$@" "${__testid}" && status_test_ok || status_test_fail
+ done
+
+ cd ..
+
+ teardown_context_watch ${PANE_HOST} host
+}
diff --git a/test/run b/test/run
index f73c3119..9e183682 100755
--- a/test/run
+++ b/test/run
@@ -53,6 +53,7 @@ COMMIT="$(git log --oneline --no-decorate -1)"
. lib/layout_ugly
. lib/test
. lib/video
+. lib/exeter
# cleanup() - Remove temporary files
cleanup() {
@@ -67,6 +68,8 @@ run() {
perf_init
[ ${CI} -eq 1 ] && video_start ci
+ exeter smoke/smoke.sh
+
setup build
test build/all
test build/cppcheck
@@ -223,6 +226,10 @@ run_selected() {
__setup=
for __test; do
+ if "test/${__test}" --list; then
+ exeter "${__test}"
+ continue
+ fi
# HACK: the migrate tests need the setup repeated for
# each test
if [ "${__test%%/*}" != "${__setup}" -o \
@@ -234,7 +241,7 @@ run_selected() {
test "${__test}"
done
- teardown "${__setup}"
+ [ -n "${__setup}" ] && teardown "${__setup}"
log "PASS: ${STATUS_PASS}, FAIL: ${STATUS_FAIL}, SKIPPED: ${STATUS_SKIPPED}"
diff --git a/test/smoke/smoke.sh b/test/smoke/smoke.sh
new file mode 100755
index 00000000..26a98d31
--- /dev/null
+++ b/test/smoke/smoke.sh
@@ -0,0 +1,27 @@
+#! /bin/sh
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# PASST - Plug A Simple Socket Transport
+# for qemu/UNIX domain socket mode
+#
+# PASTA - Pack A Subtle Tap Abstraction
+# for network namespace/tap device mode
+#
+# test/smoke/smoke.sh - Basic snoke tests
+#
+# Copyright Red Hat
+# Author: David Gibson <david@gibson.dropbear.id.au>
+
+source $(dirname $0)/../exeter/sh/exeter.sh
+
+PASST=$(dirname $0)/../../passt
+PASTA=$(dirname $0)/../../pasta
+
+exeter_register passt_version $PASST --version
+exeter_register pasta_version $PASTA --version
+
+exeter_register passt_help $PASST --help
+exeter_register pasta_help $PASTA --help
+
+exeter_main "$@"
--
@@ -0,0 +1,27 @@
+#! /bin/sh
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# PASST - Plug A Simple Socket Transport
+# for qemu/UNIX domain socket mode
+#
+# PASTA - Pack A Subtle Tap Abstraction
+# for network namespace/tap device mode
+#
+# test/smoke/smoke.sh - Basic snoke tests
+#
+# Copyright Red Hat
+# Author: David Gibson <david@gibson.dropbear.id.au>
+
+source $(dirname $0)/../exeter/sh/exeter.sh
+
+PASST=$(dirname $0)/../../passt
+PASTA=$(dirname $0)/../../pasta
+
+exeter_register passt_version $PASST --version
+exeter_register pasta_version $PASTA --version
+
+exeter_register passt_help $PASST --help
+exeter_register pasta_help $PASTA --help
+
+exeter_main "$@"
--
2.50.1
^ permalink raw reply related [flat|nested] 4+ messages in thread* [PATCH v4 2/3] test: Run static checkers as exeter tests
2025-08-07 11:32 [PATCH v4 0/3] RFC: New proof-of-concept based exeter tests David Gibson
2025-08-07 11:32 ` [PATCH v4 1/3] test: Extend test scripts to allow running " David Gibson
@ 2025-08-07 11:32 ` David Gibson
2025-08-07 11:32 ` [PATCH v4 3/3] test: Convert build tests to exeter David Gibson
2 siblings, 0 replies; 4+ messages in thread
From: David Gibson @ 2025-08-07 11:32 UTC (permalink / raw)
To: passt-dev, Stefano Brivio; +Cc: David Gibson
Move the static checkers from the current DSL to exeter.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
test/build/clang_tidy | 17 -----------------
test/build/cppcheck | 17 -----------------
test/build/static_checkers.sh | 30 ++++++++++++++++++++++++++++++
test/run | 3 +--
4 files changed, 31 insertions(+), 36 deletions(-)
delete mode 100644 test/build/clang_tidy
delete mode 100644 test/build/cppcheck
create mode 100755 test/build/static_checkers.sh
diff --git a/test/build/clang_tidy b/test/build/clang_tidy
deleted file mode 100644
index 40573bfd..00000000
--- a/test/build/clang_tidy
+++ /dev/null
@@ -1,17 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-or-later
-#
-# PASST - Plug A Simple Socket Transport
-# for qemu/UNIX domain socket mode
-#
-# PASTA - Pack A Subtle Tap Abstraction
-# for network namespace/tap device mode
-#
-# test/build/clang_tidy - Run source through clang-tidy(1) linter
-#
-# Copyright (c) 2021 Red Hat GmbH
-# Author: Stefano Brivio <sbrivio@redhat.com>
-
-htools clang-tidy
-
-test Run clang-tidy
-host make clang-tidy
diff --git a/test/build/cppcheck b/test/build/cppcheck
deleted file mode 100644
index 0e1dbced..00000000
--- a/test/build/cppcheck
+++ /dev/null
@@ -1,17 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-or-later
-#
-# PASST - Plug A Simple Socket Transport
-# for qemu/UNIX domain socket mode
-#
-# PASTA - Pack A Subtle Tap Abstraction
-# for network namespace/tap device mode
-#
-# test/build/cppcheck - Run source through cppcheck(1) linter
-#
-# Copyright (c) 2021 Red Hat GmbH
-# Author: Stefano Brivio <sbrivio@redhat.com>
-
-htools cppcheck
-
-test Run cppcheck
-host make cppcheck
diff --git a/test/build/static_checkers.sh b/test/build/static_checkers.sh
new file mode 100755
index 00000000..41152c25
--- /dev/null
+++ b/test/build/static_checkers.sh
@@ -0,0 +1,30 @@
+#! /bin/sh
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# PASST - Plug A Simple Socket Transport
+# for qemu/UNIX domain socket mode
+#
+# PASTA - Pack A Subtle Tap Abstraction
+# for network namespace/tap device mode
+#
+# test/build/static_checkers.sh - Run static checkers
+#
+# Copyright Red Hat
+# Author: David Gibson <david@gibson.dropbear.id.au>
+
+source $(dirname $0)/../exeter/sh/exeter.sh
+
+cppcheck () {
+ make -C .. cppcheck
+}
+exeter_register cppcheck
+
+clang_tidy () {
+ make -C .. clang-tidy
+}
+exeter_register clang_tidy
+
+exeter_main "$@"
+
+
diff --git a/test/run b/test/run
index 9e183682..5fd02d91 100755
--- a/test/run
+++ b/test/run
@@ -69,11 +69,10 @@ run() {
[ ${CI} -eq 1 ] && video_start ci
exeter smoke/smoke.sh
+ exeter build/static_checkers.sh
setup build
test build/all
- test build/cppcheck
- test build/clang_tidy
teardown build
setup pasta
--
@@ -69,11 +69,10 @@ run() {
[ ${CI} -eq 1 ] && video_start ci
exeter smoke/smoke.sh
+ exeter build/static_checkers.sh
setup build
test build/all
- test build/cppcheck
- test build/clang_tidy
teardown build
setup pasta
--
2.50.1
^ permalink raw reply related [flat|nested] 4+ messages in thread* [PATCH v4 3/3] test: Convert build tests to exeter
2025-08-07 11:32 [PATCH v4 0/3] RFC: New proof-of-concept based exeter tests David Gibson
2025-08-07 11:32 ` [PATCH v4 1/3] test: Extend test scripts to allow running " David Gibson
2025-08-07 11:32 ` [PATCH v4 2/3] test: Run static checkers as " David Gibson
@ 2025-08-07 11:32 ` David Gibson
2 siblings, 0 replies; 4+ messages in thread
From: David Gibson @ 2025-08-07 11:32 UTC (permalink / raw)
To: passt-dev, Stefano Brivio; +Cc: David Gibson
Convert the tests in build/all to be based on exeter. The new version of
the tests is more robust than the original, since it makes a temporary copy
of the source tree so will not be affected by concurrent manual builds.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
test/build/all | 61 --------------------------------
test/build/build.py | 84 +++++++++++++++++++++++++++++++++++++++++++++
test/run | 8 ++---
3 files changed, 88 insertions(+), 65 deletions(-)
delete mode 100644 test/build/all
create mode 100755 test/build/build.py
diff --git a/test/build/all b/test/build/all
deleted file mode 100644
index 1f79e0d8..00000000
--- a/test/build/all
+++ /dev/null
@@ -1,61 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0-or-later
-#
-# PASST - Plug A Simple Socket Transport
-# for qemu/UNIX domain socket mode
-#
-# PASTA - Pack A Subtle Tap Abstraction
-# for network namespace/tap device mode
-#
-# test/build/all - Build targets, one by one, then all together, check output
-#
-# Copyright (c) 2021 Red Hat GmbH
-# Author: Stefano Brivio <sbrivio@redhat.com>
-
-htools make cc rm uname getconf mkdir cp rm man
-
-test Build passt
-host make clean
-check ! [ -e passt ]
-host CFLAGS="-Werror" make passt
-check [ -f passt ]
-
-test Build pasta
-host make clean
-check ! [ -e pasta ]
-host CFLAGS="-Werror" make pasta
-check [ -h pasta ]
-
-test Build qrap
-host make clean
-check ! [ -e qrap ]
-host CFLAGS="-Werror" make qrap
-check [ -f qrap ]
-
-test Build all
-host make clean
-check ! [ -e passt ]
-check ! [ -e pasta ]
-check ! [ -e qrap ]
-host CFLAGS="-Werror" make
-check [ -f passt ]
-check [ -h pasta ]
-check [ -f qrap ]
-
-test Install
-host mkdir __STATEDIR__/prefix
-host prefix=__STATEDIR__/prefix make install
-check [ -f __STATEDIR__/prefix/bin/passt ]
-check [ -h __STATEDIR__/prefix/bin/pasta ]
-check [ -f __STATEDIR__/prefix/bin/qrap ]
-check man -M __STATEDIR__/prefix/share/man -W passt
-check man -M __STATEDIR__/prefix/share/man -W pasta
-check man -M __STATEDIR__/prefix/share/man -W qrap
-
-test Uninstall
-host prefix=__STATEDIR__/prefix make uninstall
-check ! [ -f __STATEDIR__/prefix/bin/passt ]
-check ! [ -h __STATEDIR__/prefix/bin/pasta ]
-check ! [ -f __STATEDIR__/prefix/bin/qrap ]
-check ! man -M __STATEDIR__/prefix/share/man -W passt 2>/dev/null
-check ! man -M __STATEDIR__/prefix/share/man -W pasta 2>/dev/null
-check ! man -M __STATEDIR__/prefix/share/man -W qrap 2>/dev/null
diff --git a/test/build/build.py b/test/build/build.py
new file mode 100755
index 00000000..12bb82d8
--- /dev/null
+++ b/test/build/build.py
@@ -0,0 +1,84 @@
+#! /usr/bin/env python3
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# PASST - Plug A Simple Socket Transport
+# for qemu/UNIX domain socket mode
+#
+# PASTA - Pack A Subtle Tap Abstraction
+# for network namespace/tap device mode
+#
+# test/build/build.py - Test build and install targets
+#
+# Copyright Red Hat
+# Author: David Gibson <david@gibson.dropbear.id.au>
+
+import contextlib
+import os
+from pathlib import Path
+import subprocess
+import tempfile
+from typing import Iterable, Iterator
+
+import exeter
+
+def sh(cmd):
+ subprocess.run(cmd, shell=True)
+
+
+@contextlib.contextmanager
+def clone_sources() -> Iterator[str]:
+ os.chdir('..') # Move from test/ to repo base
+ with tempfile.TemporaryDirectory(ignore_cleanup_errors=False) as tmpdir:
+ sh(f"cp --parents -d $(git ls-files) {tmpdir}")
+ os.chdir(tmpdir)
+ yield tmpdir
+
+
+def test_make(target: str, outputs: Iterable[str]) -> None:
+ outpaths = [Path(o) for o in outputs]
+ with clone_sources():
+ for o in outpaths:
+ assert not o.exists(), f"{o} existed before make"
+ sh(f'make {target} CFLAGS="-Werror"')
+ for o in outpaths:
+ assert o.exists(), f"{o} wasn't made"
+ sh('make clean')
+ for o in outpaths:
+ assert not o.exists(), f"{o} existed after make clean"
+
+
+exeter.register('make_passt', test_make, 'passt', ['passt'])
+exeter.register('make_pasta', test_make, 'pasta', ['pasta'])
+exeter.register('make_qrap', test_make, 'qrap', ['qrap'])
+exeter.register('make_all', test_make, 'all', ['passt', 'pasta', 'qrap'])
+
+
+@exeter.test
+def test_install_uninstall() -> None:
+ with clone_sources():
+ with tempfile.TemporaryDirectory(ignore_cleanup_errors=False) \
+ as prefix:
+ bindir = Path(prefix) / 'bin'
+ mandir = Path(prefix) / 'share/man'
+ progs = ['passt', 'pasta', 'qrap']
+
+ # Install
+ sh(f'make install CFLAGS="-Werror" prefix={prefix}')
+
+ for prog in progs:
+ exe = bindir / prog
+ assert exe.is_file(), f"{exe} does not exist as a regular file"
+ sh(f'man -M {mandir} -W {prog}')
+
+ # Uninstall
+ sh(f'make uninstall prefix={prefix}')
+
+ for prog in progs:
+ exe = bindir / prog
+ assert not exe.exists(), f"{exe} exists after uninstall"
+ sh(f'! man -M {mandir} -W {prog}')
+
+
+if __name__ == '__main__':
+ exeter.main()
diff --git a/test/run b/test/run
index 5fd02d91..6a53e24b 100755
--- a/test/run
+++ b/test/run
@@ -43,6 +43,9 @@ KERNEL=${KERNEL:-"/boot/vmlinuz-$(uname -r)"}
COMMIT="$(git log --oneline --no-decorate -1)"
+# Let exeter tests written in Python find their modules
+export PYTHONPATH=${BASEPATH}/exeter/py3
+
. lib/util
. lib/context
. lib/setup
@@ -69,12 +72,9 @@ run() {
[ ${CI} -eq 1 ] && video_start ci
exeter smoke/smoke.sh
+ exeter build/build.py
exeter build/static_checkers.sh
- setup build
- test build/all
- teardown build
-
setup pasta
test pasta/ndp
test pasta/dhcp
--
@@ -43,6 +43,9 @@ KERNEL=${KERNEL:-"/boot/vmlinuz-$(uname -r)"}
COMMIT="$(git log --oneline --no-decorate -1)"
+# Let exeter tests written in Python find their modules
+export PYTHONPATH=${BASEPATH}/exeter/py3
+
. lib/util
. lib/context
. lib/setup
@@ -69,12 +72,9 @@ run() {
[ ${CI} -eq 1 ] && video_start ci
exeter smoke/smoke.sh
+ exeter build/build.py
exeter build/static_checkers.sh
- setup build
- test build/all
- teardown build
-
setup pasta
test pasta/ndp
test pasta/dhcp
--
2.50.1
^ permalink raw reply related [flat|nested] 4+ messages in thread