From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=none (p=none dis=none) header.from=gibson.dropbear.id.au Authentication-Results: passt.top; dkim=pass (2048-bit key; secure) header.d=gibson.dropbear.id.au header.i=@gibson.dropbear.id.au header.a=rsa-sha256 header.s=202508 header.b=bJhOIGZI; dkim-atps=neutral Received: from mail.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by passt.top (Postfix) with ESMTPS id 1D3D95A027A for ; Wed, 20 Aug 2025 13:20:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gibson.dropbear.id.au; s=202508; t=1755688808; bh=Qyh2tyql+4u7OgFNImBOYdES2ZM79XoZLLeqkaIvddI=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=bJhOIGZIR7+UmaWknQXvHa82+L1sHY/db7sZcgc9MhXT9EtxAeKSIuwgpWlEn2rrQ rH/sH3m9JjFKoo2GLMxlgwybFcwPDSMlim0hG8XWRqjTKhO5Hr0P17aTk229mCL2hD RaXiEALOJu9UXCLvOYSCqYIXoAr3izk8ZnEGkAM5pv/XqThEcbjtS+DfEuEBw4hbFu wa8m38MnuI2GC+cqqjnJzOCBTNGfXxA9W5Fh/QHKieRsocEHZ5AiAUgL+OjmLtH+Eh TnXZeTTK22x4NhIAvCJ+RCyzzIwffLiRwCGBWuzbMDLeWdTdK+XsxQgXTBpZftJgQ3 8Blo5sTeOKH3Q== Received: by gandalf.ozlabs.org (Postfix, from userid 1007) id 4c6PBJ5m4xz4xQN; Wed, 20 Aug 2025 21:20:08 +1000 (AEST) Date: Wed, 20 Aug 2025 21:19:40 +1000 From: David Gibson To: Stefano Brivio Subject: Re: [PATCH v4 3/3] test: Convert build tests to exeter Message-ID: References: <20250807113237.548294-1-david@gibson.dropbear.id.au> <20250807113237.548294-4-david@gibson.dropbear.id.au> <20250819162754.0fdb4c86@elisabeth> <20250820124044.7f1bd32c@elisabeth> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="cIe68M4BgGVHuCm+" Content-Disposition: inline In-Reply-To: <20250820124044.7f1bd32c@elisabeth> Message-ID-Hash: VOVD7J6IV5LDWS2SY7F6CJEZUO5J7YCL X-Message-ID-Hash: VOVD7J6IV5LDWS2SY7F6CJEZUO5J7YCL X-MailFrom: dgibson@gandalf.ozlabs.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --cIe68M4BgGVHuCm+ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 20, 2025 at 12:40:44PM +0200, Stefano Brivio wrote: > On Wed, 20 Aug 2025 13:10:02 +1000 > David Gibson wrote: >=20 > > On Tue, Aug 19, 2025 at 04:27:54PM +0200, Stefano Brivio wrote: > > > On Thu, 7 Aug 2025 21:32:37 +1000 > > > David Gibson wrote: > > > =20 > > > > Convert the tests in build/all to be based on exeter. The new vers= ion of > > > > the tests is more robust than the original, since it makes a tempor= ary copy > > > > of the source tree so will not be affected by concurrent manual bui= lds. > > > >=20 > > > > Signed-off-by: David Gibson > > > > --- > > > > test/build/all | 61 -------------------------------- > > > > test/build/build.py | 84 +++++++++++++++++++++++++++++++++++++++++= ++++ > > > > test/run | 8 ++--- > > > > 3 files changed, 88 insertions(+), 65 deletions(-) > > > > delete mode 100644 test/build/all > > > > create mode 100755 test/build/build.py > > > >=20 > > > > diff --git a/test/build/all b/test/build/all > > > > deleted file mode 100644 > > > > index 1f79e0d8..00000000 > > > > --- a/test/build/all > > > > +++ /dev/null > > > > @@ -1,61 +0,0 @@ > > > > -# SPDX-License-Identifier: GPL-2.0-or-later > > > > -# > > > > -# PASST - Plug A Simple Socket Transport > > > > -# for qemu/UNIX domain socket mode > > > > -# > > > > -# PASTA - Pack A Subtle Tap Abstraction > > > > -# for network namespace/tap device mode > > > > -# > > > > -# test/build/all - Build targets, one by one, then all together, c= heck output > > > > -# > > > > -# Copyright (c) 2021 Red Hat GmbH > > > > -# Author: Stefano Brivio > > > > - > > > > -htools make cc rm uname getconf mkdir cp rm man > > > > - > > > > -test Build passt > > > > -host make clean > > > > -check ! [ -e passt ] > > > > -host CFLAGS=3D"-Werror" make passt > > > > -check [ -f passt ] > > > > - > > > > -test Build pasta > > > > -host make clean > > > > -check ! [ -e pasta ] > > > > -host CFLAGS=3D"-Werror" make pasta > > > > -check [ -h pasta ] > > > > - > > > > -test Build qrap > > > > -host make clean > > > > -check ! [ -e qrap ] > > > > -host CFLAGS=3D"-Werror" make qrap > > > > -check [ -f qrap ] > > > > - > > > > -test Build all > > > > -host make clean > > > > -check ! [ -e passt ] > > > > -check ! [ -e pasta ] > > > > -check ! [ -e qrap ] > > > > -host CFLAGS=3D"-Werror" make > > > > -check [ -f passt ] > > > > -check [ -h pasta ] > > > > -check [ -f qrap ] > > > > - > > > > -test Install > > > > -host mkdir __STATEDIR__/prefix > > > > -host prefix=3D__STATEDIR__/prefix make install > > > > -check [ -f __STATEDIR__/prefix/bin/passt ] > > > > -check [ -h __STATEDIR__/prefix/bin/pasta ] > > > > -check [ -f __STATEDIR__/prefix/bin/qrap ] > > > > -check man -M __STATEDIR__/prefix/share/man -W passt > > > > -check man -M __STATEDIR__/prefix/share/man -W pasta > > > > -check man -M __STATEDIR__/prefix/share/man -W qrap > > > > - > > > > -test Uninstall > > > > -host prefix=3D__STATEDIR__/prefix make uninstall > > > > -check ! [ -f __STATEDIR__/prefix/bin/passt ] > > > > -check ! [ -h __STATEDIR__/prefix/bin/pasta ] > > > > -check ! [ -f __STATEDIR__/prefix/bin/qrap ] > > > > -check ! man -M __STATEDIR__/prefix/share/man -W passt 2>/dev/null > > > > -check ! man -M __STATEDIR__/prefix/share/man -W pasta 2>/dev/null > > > > -check ! man -M __STATEDIR__/prefix/share/man -W qrap 2>/dev/null > > > > diff --git a/test/build/build.py b/test/build/build.py > > > > new file mode 100755 > > > > index 00000000..12bb82d8 > > > > --- /dev/null > > > > +++ b/test/build/build.py > > > > @@ -0,0 +1,84 @@ > > > > +#! /usr/bin/env python3 > > > > +# > > > > +# SPDX-License-Identifier: GPL-2.0-or-later > > > > +# > > > > +# PASST - Plug A Simple Socket Transport > > > > +# for qemu/UNIX domain socket mode > > > > +# > > > > +# PASTA - Pack A Subtle Tap Abstraction > > > > +# for network namespace/tap device mode > > > > +# > > > > +# test/build/build.py - Test build and install targets > > > > +# > > > > +# Copyright Red Hat > > > > +# Author: David Gibson > > > > + > > > > +import contextlib > > > > +import os > > > > +from pathlib import Path > > > > +import subprocess > > > > +import tempfile > > > > +from typing import Iterable, Iterator > > > > + > > > > +import exeter > > > > + > > > > +def sh(cmd): > > > > + subprocess.run(cmd, shell=3DTrue) > > > > + > > > > + > > > > +@contextlib.contextmanager > > > > +def clone_sources() -> Iterator[str]: > > > > + os.chdir('..') # Move from test/ to repo base > > > > + with tempfile.TemporaryDirectory(ignore_cleanup_errors=3DFalse= ) as tmpdir: =20 > > >=20 > > > I guess I see the advantage of this syntax, it's still a bit less > > > obvious to me than a mere sequence of steps and an explicit cleanup > > > function. =20 > >=20 > > I agree that it's less obvious, but I think the advantages are worth > > it. Namely that it will correctly run the cleanup in nearly all cases > > of an interrupted test (not a SIGKILL, obviously). > >=20 > > Arguably, it's not a good trade off in this simple case. However once > > we get to real network tests, it's pretty common to have multiple > > nested layers of setup, each with their own teardown. Sometimes you > > need something torn down, but it's only set up partway through the > > test, or you want something for the first part of the test but not > > after, so it needs to be torn down only if interrupted at certain > > points. Manually keeping track of that quickly becomes really > > painful, I really want to use this technique there. >=20 > Another idea (I'm not sure if it makes this useless, but we probably > want to implement it anyway) is what nft tests (and some kselftests) do > nowadays: every test runs in its own network namespace, so you don't > need an explicit teardown. The kernel already tracks things for you. Right, I'm planning to implement that in tunbridge. Unlike the earlier drafts, my intention is that all the namespaces / whatever you explicitly create will be nested inside a top-level sandboxing namespace. I don't think it obviates the use for context managers, though. For one thing I think we may have occasional need for things that won't be handled by the normal sandbox, but the contextmanager can handle those too. More widely, there are cases where you want to tear something down partway through normal test operation. The context manager will do that neatly, while also doing the teardown if interrupted before that point. > If you combine that with what pasta does (same as container runtimes > really), you could get something that's also robust to SIGKILL, by > making every test "parent" process (assuming there can be one) PID 1 in > its own PID namespace. Right, it can sometimes make things more confusing during debugging, though. > Add mount namespaces on top, with tmpfs, and we probably don't even > need to clean up after ourselves in build tests. >=20 > Yes, it could be called a container but it's probably useful to retain > tight control of what namespace we detach and when. >=20 > Given that pasta(1) is now available on any distribution where you > might reasonably find Python, by the way, you could consider using it > directly (the installed version, that is) if it helps. I don't really want to. I consider the test cases being entirely cut off from the outside internet an advantage, in most cases. > > Given that I intend to use it there, I think it makes sense to use it > > here too: after all it will be even less obvious if there are only > > hard examples of it and no easy ones. >=20 > Ah, sure, fair point. --=20 David Gibson (he or they) | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you, not the other way | around. http://www.ozlabs.org/~dgibson --cIe68M4BgGVHuCm+ Content-Type: application/pgp-signature; name=signature.asc -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEO+dNsU4E3yXUXRK2zQJF27ox2GcFAmilrz0ACgkQzQJF27ox 2GcnhRAAjlq+Vbi+xDPZ/IdRySDaQzP/Ttv7vvYLgtfjjmEBjt6kmJO+YGLTfi+6 WexDH69/tv745GHuWXRJmMAZXQsJks6+spOlgwQxncdjwxjcvu32OJCoaQpoJ53F dKZ74Yqhp/BDroy6MYRWkgsfBGWUFQC56q3Chd2o4yXApaRDU1zjutx2fu8zYpgT MKvgUkWwcc1SsfzSMEdAqP7diCewuB/Xi4vzjXUwxK4NcqHnrEKZsBwO/aoDBw/H g/HrGnyYXAiYKQWZjiUIL1nMF2P81/RTdVmppsAspMLfq+zugsvRZ7tOZ6rlQILX tIxeIHzVas3kgszYwAPUQPJQzAuhproboa2xD+DhrAzCtH896QRQG9m0Inzr7ryj 8onNMnNz7Qj3tL3rHCczuTdBwhDZG1S6vGKugGMWocDxsp4pocijcFjonIL3kspn C1/8ETXElw3lQB6zgMMpocCRQUEyxbJx19hERsFbnxYDOvEmp2znz7AX/rBnyOA2 0O6/ZylgPWFLNB9qq+eoPL/ZHduFw1UioxpHT9dsxxaQeN84zFJNWv1Urd0zfRPI Ki/w2bcr1QvE1Nk9Nwq9N7Vl8gftydnLhPE5V3WkNHcYwU3wVPvm7kXhHHQpMr6y 8pCHDRPZuS/IBEpmL+LBRaOtQa37U5Tkhue3JOVdq0k0dqgsgJU= =85LW -----END PGP SIGNATURE----- --cIe68M4BgGVHuCm+--