From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=none (p=none dis=none) header.from=gibson.dropbear.id.au Authentication-Results: passt.top; dkim=pass (2048-bit key; secure) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.a=rsa-sha256 header.s=202508 header.b=AYDyzwRL; dkim-atps=neutral Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by passt.top (Postfix) with ESMTPS id 0F2D15A0279 for ; Wed, 20 Aug 2025 13:20:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=202508; t=1755688808; bh=mljleVKAqimtssX5Z0cz85hqZSP1OrfNQriYkJKuzkA=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=AYDyzwRLXOQb0NmSkkaIW6qu0JHtyc4YMbsJS9juCmiphPwWmz5jLtJq9sLJaohQ/ ilLFa5LMJU+o5e1Ru5i/DebjzKYmd6dxZNQE4Z+B/9zgpfE+mERi5oFvZpH9aWUDOQ ufKJ5tWuedPsm9bXhh55JQy76j8MPsaYQ7EP47ifzORCIHNFwmarL3d3+I43O014CL cK0mMWEjfeoDFpeenClZEsqOyzoxswh4jPDvr1+9trCSBYU3OaaR0Cl1ALrkCIwoMB ImUQnQJF7zUPG3P74Tc99MuT6KHE+D6E5L0rY0zt67Sk9iuhWXHzPb9AWXhO3gQjgV TE/HkEyMuSJOg== Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4c6PBJ5XCpz4xQ1; Wed, 20 Aug 2025 21:20:08 +1000 (AEST) Date: Wed, 20 Aug 2025 21:10:17 +1000 From: David Gibson To: Stefano Brivio Subject: Re: [PATCH v4 1/3] test: Extend test scripts to allow running exeter tests. Message-ID: References: <20250807113237.548294-1-david@gibson.dropbear.id.au> <20250807113237.548294-2-david@gibson.dropbear.id.au> <20250819162738.36d35d52@elisabeth> <20250820115218.2cdab1ba@elisabeth> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="I5EvpRFOHc15XgYG" Content-Disposition: inline In-Reply-To: <20250820115218.2cdab1ba@elisabeth> Message-ID-Hash: O7JDTNWNTTTTV6NDX36OSAN6DXKVYRII X-Message-ID-Hash: O7JDTNWNTTTTV6NDX36OSAN6DXKVYRII X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --I5EvpRFOHc15XgYG Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 20, 2025 at 11:52:18AM +0200, Stefano Brivio wrote: > On Wed, 20 Aug 2025 12:55:58 +1000 > David Gibson wrote: >=20 > > On Tue, Aug 19, 2025 at 04:27:38PM +0200, Stefano Brivio wrote: > > > On Thu, 7 Aug 2025 21:32:35 +1000 > > > David Gibson wrote: > > > =20 > > > > Introduce some trivial testcases based on the exeter library. Thes= e run > > > > passt and pasta with --help and --version options. Extend our test > > > > scripts to run these tests. > > > >=20 > > > > Signed-off-by: David Gibson > > > > --- > > > > test/.gitignore | 1 + > > > > test/Makefile | 5 ++++- > > > > test/lib/exeter | 46 +++++++++++++++++++++++++++++++++++++++++= ++++ > > > > test/run | 9 ++++++++- > > > > test/smoke/smoke.sh | 27 ++++++++++++++++++++++++++ > > > > 5 files changed, 86 insertions(+), 2 deletions(-) > > > > create mode 100644 test/lib/exeter > > > > create mode 100755 test/smoke/smoke.sh > > > >=20 > > > > diff --git a/test/.gitignore b/test/.gitignore > > > > index 3573444f..cf48b885 100644 > > > > --- a/test/.gitignore > > > > +++ b/test/.gitignore > > > > @@ -11,3 +11,4 @@ nstool > > > > rampstream > > > > guest-key > > > > guest-key.pub > > > > +/exeter/ > > > > diff --git a/test/Makefile b/test/Makefile > > > > index bf63db87..332f3f3e 100644 > > > > --- a/test/Makefile > > > > +++ b/test/Makefile > > > > @@ -50,7 +50,7 @@ UBUNTU_NEW_IMGS =3D xenial-server-cloudimg-powerp= c-disk1.img \ > > > > jammy-server-cloudimg-s390x.img > > > > UBUNTU_IMGS =3D $(UBUNTU_OLD_IMGS) $(UBUNTU_NEW_IMGS) > > > > =20 > > > > -DOWNLOAD_ASSETS =3D mbuto podman \ > > > > +DOWNLOAD_ASSETS =3D exeter mbuto podman \ > > > > $(DEBIAN_IMGS) $(FEDORA_IMGS) $(OPENSUSE_IMGS) $(UBUNTU_IMGS) =20 > > >=20 > > > Pre-existing, but this broke a while after you switched the image > > > preparation to Makefile targets, so you can't really expect people to > > > issue 'make' at the moment. =20 > >=20 > > Sorry, I'm not quite sure what the problem is you're getting at. >=20 > This: > -- Hm, could you avoid using -- to mark quotes like this. mutt mistakes it for the sig and cuts it out; it's kind of a pain to paste it back in for repying. > $ mkdir psst > $ cd psst > $ git clone git://passt.top/passt > [...] > $ cd passt/test/ > $ make > [...] > wget -c -O debian-11-generic-ppc64el.qcow2 https://cloud.debian.org/image= s/cloud/bullseye/latest/debian-11-generic-ppc64el.qcow2 > --2025-08-20 11:27:05-- > https://cloud.debian.org/images/cloud/bullseye/latest/debian-11-generic-p= pc64el.qcow2 > Resolving cloud.debian.org (cloud.debian.org)... 2001:6b0:19::165, > 2001:6b0:19::173, 2001:6b0:19::163, ... > Connecting to cloud.debian.org > (cloud.debian.org)|2001:6b0:19::165|:443... connected. > HTTP request sent, awaiting response... 404 Not Found 2025-08-20 11:27:05= ERROR > 404: Not Found. > make: *** [Makefile:150: debian-11-generic-ppc64el.qcow2] Error 8 Ah, right. All the distro wgets are so slow from Australia that I've always avoided them by keeping the image files in place (or even copying the back from backups). So, I didn't hit this. Yeah.. we should do something about that. > The (very) old logic would make the "distro" tests fail, but it's > been a while that one can't really run 'make' under test/ anymore > (unless you have everything already prepared, that is). > On one hand, we don't have a replacement for those tests, and, while > ugly and clumsy, they were actually useful as they would have told > us > in advance about stuff such as: > https://bugs.passt.top/show_bug.cgi?id=3D121 > ...and probably much more that we're ignoring now. I don't even test > routinely on big-endian, we just have build tests from > distributions. That's bad, and it will be ages before we can > reasonably get a replacement for that. Yeah. On the other hand, we could at least remove the links that are actually broken. The idea of testing across distros seems to me more important than the exact set of distros/versions to check - that's relatively easily expanded if we have the structure. > So it would be still valuable to get them working again, and it > shouldn't be much effort. > On the other hand... you can't do 'make' there, because of tests we > don't even run. Maybe those failures should be warnings instead. If > somebody finds a "moment" it would be nice to fix those though... > > > I ran 'make exeter' to get further. It would be nice to update > > > test/README.md (or fix the Makefile... or both). > >=20 > > Hrm.. "make check" already depends on "make assets" which should > > (now) include exeter. I'm not sure why that's not working for > > you. > ...and not 'make assets', either. > > > > TESTDATA_ASSETS =3D small.bin big.bin medium.bin \ rampstream > > > > @@ -70,6 +70,9 @@ assets: $(ASSETS) pull-%: % git -C > > > > $* pull > > > > =20 > > > > +exeter: > > > > + git clone https://gitlab.com/dgibson/exeter.git + > > > > mbuto: > > > > git clone git://mbuto.sh/mbuto > > > > =20 > > > > diff --git a/test/lib/exeter b/test/lib/exeter new file mode > > > > 100644 index 00000000..4f7bcff9 --- /dev/null +++ > > > > b/test/lib/exeter @@ -0,0 +1,46 @@ +#!/bin/sh +# +# > > > > SPDX-License-Identifier: GPL-2.0-or-later +# +# PASST - Plug A > > > > Simple Socket Transport +# for qemu/UNIX domain socket mode +# > > > > +# PASTA - Pack A Subtle Tap Abstraction +# for network > > > > namespace/tap device mode +# +# test/lib/exeter - Run exeter > > > > tests within the rest of passt's tests +# +# Copyright Red Hat > > > > +# Author: David Gibson + > > > > +exeter() { > > >=20 > > > It would be nice to have the usual arguments / function > > > description here. > >=20 > > Good point, added. > >=20 > > > > + STATESETUP=3D"${STATEBASE}/$1" + mkdir -p "${STATESETUP}" + + > > > > context_setup_host host + layout_host > > >=20 > > > Indentation here was done with tabs, this adds spaces instead. > >=20 > > Oops. For some reason I'd added emacs configuration to explicitly > > *disable* tab indent mode for shell mode in the passt tree. > > Weird, and now reversed. > >=20 > > > > + + cd test + + __ntests=3D$("$@" --list | wc -l) + if [ $? !=3D 0 > > > > ]; then + info "Failed to get exeter manifest for $@" + > > > > pause_continue \ + "Press any key to pause test session" \ + > > > > "Resuming in " \ + "Paused, press any key to continue" \ + 5 + > > > > return + fi + + status_file_start "Exeter tests: $*" > > > > ${__ntests} + + for __testid in $("$@" --list); do + > > > > status_test_start "${__testid}" + context_run host "$@" > > > > "${__testid}" && status_test_ok || status_test_fail + done + + > > > > cd .. + + teardown_context_watch ${PANE_HOST} host +} diff > > > > --git a/test/run b/test/run index f73c3119..9e183682 100755 > > > > --- a/test/run +++ b/test/run @@ -53,6 +53,7 @@ COMMIT=3D"$(git > > > > log --oneline --no-decorate -1)" . lib/layout_ugly . lib/test > > > > . lib/video +. lib/exeter > > > > =20 > > > > # cleanup() - Remove temporary files cleanup() { @@ -67,6 > > > > +68,8 @@ run() { perf_init [ ${CI} -eq 1 ] && video_start ci > > > > =20 > > > > + exeter smoke/smoke.sh + setup build test build/all test > > > > build/cppcheck @@ -223,6 +226,10 @@ run_selected() { > > > > =20 > > > > __setup=3D for __test; do + if "test/${__test}" --list; > > > > then + exeter "${__test}" + continue + fi # HACK: the > > > > migrate tests need the setup repeated for # each test > > > > if [ "${__test%%/*}" !=3D "${__setup}" -o \ @@ -234,7 > > > > +241,7 @@ run_selected() { > > > > =20 > > > > test "${__test}" done - teardown "${__setup}" > > > > + [ -n "${__setup}" ] && teardown "${__setup}" > > > > =20 > > > > log "PASS: ${STATUS_PASS}, FAIL: ${STATUS_FAIL}, > > > > SKIPPED: ${STATUS_SKIPPED}" > > > > =20 > > > > diff --git a/test/smoke/smoke.sh b/test/smoke/smoke.sh new > > > > file mode 100755 index 00000000..26a98d31 --- /dev/null +++ > > > > b/test/smoke/smoke.sh @@ -0,0 +1,27 @@ +#! /bin/sh > > >=20 > > > ...this is desirable, I think (and, for consistency, I'd use "#!/bin/= sh") as Bash is not universally available, but: > >=20 > > So.. I believe "#! " (with the space) is actually the traditional > > / correct version. I did once encounter a situation where it > > mattered, although to be fair, that might have been with 25 year > > old SunOS or something equally obscure. Nonetheless I've > > habitually included the space ever since (see > > find-arm64-firmware.sh, prepare-distro-img.sh and > > rapstream-check.sh). > Oh, I didn't know. I thought it was just something that happened to > work somehow, but I didn't realise about this: > https://en.wikipedia.org/wiki/Shebang_(Unix)#Version_8_improved_shell_s= cripts > and while this has some details too: > https://www.in-ulm.de/~mascheck/various/shebang/#blankrequired > ...sure, makes sense, let's keep those. > -- Stefano --=20 David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson --I5EvpRFOHc15XgYG Content-Type: application/pgp-signature; name=signature.asc -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEO+dNsU4E3yXUXRK2zQJF27ox2GcFAmilrRMACgkQzQJF27ox 2GcB5w//R4oHBPEJ11Pfa4L5g6cnHt8aACgs2b/kbQ71WODBP46/hAb02teJKRZh 9fso+YOWDviJJsRKEnkLYkiuUJf6vharO88Hn94ZGQ2Rg7hCe1QfiQuV71vLttuv lPWr7KaH5gnhNbwiIAQ640NvLBrqs2Ea83veNk4OE59xFj722Vg7rysD3W1lk3LA 0EoEFe887yOrLRrIMsAhRdQa0AZ07sUKd1pP3I3Dk/bgJ5FmjVQdHv1AQuK0EPWD OhQv9iLgCX4d1uA0n5aOKrZdSUOz54snXte1hgLSF+CHz5d1yL5YCxWAwbR60me1 jImKjQ5EMyfaz6I7AlSZlWqsSiJfbL8IpS3HXO9TcwBcIxmS6R6gybOYCXFfFO67 taUVPBHJ81aL0wA60KoIMXYPBBGEhoin+U2CkqNIezeEq9QSZkRIEe7SHsCHStUV 06O0CDpwAP33RwQ4jbIVYGoLRT1gpo5ijUm9HP0lnvXDzOCAjfh3pe5bdhwFbw43 6jgz5IYZrJgnjIM3UnUXoKYmMK8tRkK6ZtAJMVqNBTDNPYDykSj1u9XI+jIy2kki wECCkXAsyYVxUs3ChsEm7bJLwFOCSvDxPHfm0fSuIqbfLNwYBYJSP2TG+evf2MX8 jnj037amTSbUoGf8lBrcRD1BGqmy8qS2PU2B95TtP7jRY5QswzQ= =W0c3 -----END PGP SIGNATURE----- --I5EvpRFOHc15XgYG--