From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=gd1kRi64; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by passt.top (Postfix) with ESMTPS id 5F8AC5A0271 for ; Tue, 19 Aug 2025 16:28:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1755613681; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iUuaehuOJyEwiqBXbXV1LLtMIqgUbscMcBZ3fxOGT/4=; b=gd1kRi64z+pQj/ly1OB4yQXLkMby048+B8tzCaMmMs4iySuH1/u0g/PXs4ujpkYs2lsPvd LxOwIb+bjU4y66aUZOjyJqaK4a2AZcfSjh8PQaj6Wevvdsfk6JxH3DINVqgunSb/4+y6tk P6hQ9fzXXUWBH6VEIkOEfCwajzGTRIc= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-357-ssxt_5R4OvGrLJCKa5G0jQ-1; Tue, 19 Aug 2025 10:27:58 -0400 X-MC-Unique: ssxt_5R4OvGrLJCKa5G0jQ-1 X-Mimecast-MFC-AGG-ID: ssxt_5R4OvGrLJCKa5G0jQ_1755613677 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-3b9edf41d07so3665439f8f.3 for ; Tue, 19 Aug 2025 07:27:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755613677; x=1756218477; h=content-transfer-encoding:mime-version:organization:references :in-reply-to:message-id:subject:cc:to:from:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=iUuaehuOJyEwiqBXbXV1LLtMIqgUbscMcBZ3fxOGT/4=; b=ob17asr7XaxhzHFPp2tcmGBW5vYwOYlpv69lICfffpBlqXpAnYqydsBAlFG2Gg6WMe AV3wxyuy2cZZiVcDSmwi4neuwksIir7jJGx5WsB9th4b9iesZa8tM0mbUjYZ3TC12U7i swyt63TEy7M32paSr6m0h2mmV79911ERhwIEaO5divr/NpOydj58WfZg4TUExN7BeUSd lb3AHA2j5/YL0930bk0hDsr+rHn/ZF2/Sxc8fHd3CsLp7klDO0wbHVdP1h5GsYpm+pqd XlcgdL42Yzub/TQcfAFuiXe14I5PVJfynRCwfyFh1oYkVELLQ/tBUePjCMUryXQyPtSa qS8A== X-Gm-Message-State: AOJu0YwsjV78VnxBwvuOOW2R8uYL+k1hYgtPVHimtTsCx1bO6GjCHiH9 7C5ILY0imFb2CG6HwhCbT2k2oK2i8RqJhm6BE4Zz29GT//IUPF5EytkNVfZw4mNi00ywkWcfCgg hqBJb47qAbHMdPik4uhVkXGokO79WMq19aipnw8uctS0mKVq9QQCUJA== X-Gm-Gg: ASbGncsoXLh4UAHo57dtopQkuuVQDBplVoXnLSKhxS7HoIvSUFOiM4BbE1rjgn92tn0 O2QbPzFWen/mPa+4st+MxKeMYMz/AelB45PPYvOh2qatiXFkX5YtKFfl00ziLtd8hNW59YNKL66 sGFkT8BF/LTu0LUB6VMmeaiT1uLlu3WLSnA1uqE+xu1C/4qZ5kQgMrxaDWLsmYDQ3F9mQJn1mci ZPp24Rf80sJf05WBO7usvzv+C1x5lGKanxVsHscb3Fqx3QiN7hZE+L80V65eQEV5fQWZfb2LqxT H3VWiYerzNxbbjm/saZ+eoIZ6VAphhzCSYz8h0CuE3ABq96tOpA= X-Received: by 2002:a5d:5f53:0:b0:3b7:9715:75f1 with SMTP id ffacd0b85a97d-3c0ebcc6eb0mr2444624f8f.36.1755613677199; Tue, 19 Aug 2025 07:27:57 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGSd8etqEqG5ot1U7TNvQ4W/j6zKlP627QHXt3j5h9/Hb+4MRJ/l4f5H+MnUZOZ3bInrzPnIA== X-Received: by 2002:a5d:5f53:0:b0:3b7:9715:75f1 with SMTP id ffacd0b85a97d-3c0ebcc6eb0mr2444587f8f.36.1755613676564; Tue, 19 Aug 2025 07:27:56 -0700 (PDT) Received: from maya.myfinge.rs (ifcgrfdd.trafficplex.cloud. [2a10:fc81:a806:d6a9::1]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3c077c56454sm3948061f8f.58.2025.08.19.07.27.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Aug 2025 07:27:56 -0700 (PDT) Date: Tue, 19 Aug 2025 16:27:54 +0200 From: Stefano Brivio To: David Gibson Subject: Re: [PATCH v4 3/3] test: Convert build tests to exeter Message-ID: <20250819162754.0fdb4c86@elisabeth> In-Reply-To: <20250807113237.548294-4-david@gibson.dropbear.id.au> References: <20250807113237.548294-1-david@gibson.dropbear.id.au> <20250807113237.548294-4-david@gibson.dropbear.id.au> Organization: Red Hat X-Mailer: Claws Mail 4.2.0 (GTK 3.24.49; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: V41Rn4k-3W1GzD0K1CXSprh46ORPT7RiMY7ADO6NEJQ_1755613677 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Message-ID-Hash: E4KCQKJWZQFBS2Z2FNY7AKEG7PUGRPI3 X-Message-ID-Hash: E4KCQKJWZQFBS2Z2FNY7AKEG7PUGRPI3 X-MailFrom: sbrivio@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Thu, 7 Aug 2025 21:32:37 +1000 David Gibson wrote: > Convert the tests in build/all to be based on exeter. The new version of > the tests is more robust than the original, since it makes a temporary copy > of the source tree so will not be affected by concurrent manual builds. > > Signed-off-by: David Gibson > --- > test/build/all | 61 -------------------------------- > test/build/build.py | 84 +++++++++++++++++++++++++++++++++++++++++++++ > test/run | 8 ++--- > 3 files changed, 88 insertions(+), 65 deletions(-) > delete mode 100644 test/build/all > create mode 100755 test/build/build.py > > diff --git a/test/build/all b/test/build/all > deleted file mode 100644 > index 1f79e0d8..00000000 > --- a/test/build/all > +++ /dev/null > @@ -1,61 +0,0 @@ > -# SPDX-License-Identifier: GPL-2.0-or-later > -# > -# PASST - Plug A Simple Socket Transport > -# for qemu/UNIX domain socket mode > -# > -# PASTA - Pack A Subtle Tap Abstraction > -# for network namespace/tap device mode > -# > -# test/build/all - Build targets, one by one, then all together, check output > -# > -# Copyright (c) 2021 Red Hat GmbH > -# Author: Stefano Brivio > - > -htools make cc rm uname getconf mkdir cp rm man > - > -test Build passt > -host make clean > -check ! [ -e passt ] > -host CFLAGS="-Werror" make passt > -check [ -f passt ] > - > -test Build pasta > -host make clean > -check ! [ -e pasta ] > -host CFLAGS="-Werror" make pasta > -check [ -h pasta ] > - > -test Build qrap > -host make clean > -check ! [ -e qrap ] > -host CFLAGS="-Werror" make qrap > -check [ -f qrap ] > - > -test Build all > -host make clean > -check ! [ -e passt ] > -check ! [ -e pasta ] > -check ! [ -e qrap ] > -host CFLAGS="-Werror" make > -check [ -f passt ] > -check [ -h pasta ] > -check [ -f qrap ] > - > -test Install > -host mkdir __STATEDIR__/prefix > -host prefix=__STATEDIR__/prefix make install > -check [ -f __STATEDIR__/prefix/bin/passt ] > -check [ -h __STATEDIR__/prefix/bin/pasta ] > -check [ -f __STATEDIR__/prefix/bin/qrap ] > -check man -M __STATEDIR__/prefix/share/man -W passt > -check man -M __STATEDIR__/prefix/share/man -W pasta > -check man -M __STATEDIR__/prefix/share/man -W qrap > - > -test Uninstall > -host prefix=__STATEDIR__/prefix make uninstall > -check ! [ -f __STATEDIR__/prefix/bin/passt ] > -check ! [ -h __STATEDIR__/prefix/bin/pasta ] > -check ! [ -f __STATEDIR__/prefix/bin/qrap ] > -check ! man -M __STATEDIR__/prefix/share/man -W passt 2>/dev/null > -check ! man -M __STATEDIR__/prefix/share/man -W pasta 2>/dev/null > -check ! man -M __STATEDIR__/prefix/share/man -W qrap 2>/dev/null > diff --git a/test/build/build.py b/test/build/build.py > new file mode 100755 > index 00000000..12bb82d8 > --- /dev/null > +++ b/test/build/build.py > @@ -0,0 +1,84 @@ > +#! /usr/bin/env python3 > +# > +# SPDX-License-Identifier: GPL-2.0-or-later > +# > +# PASST - Plug A Simple Socket Transport > +# for qemu/UNIX domain socket mode > +# > +# PASTA - Pack A Subtle Tap Abstraction > +# for network namespace/tap device mode > +# > +# test/build/build.py - Test build and install targets > +# > +# Copyright Red Hat > +# Author: David Gibson > + > +import contextlib > +import os > +from pathlib import Path > +import subprocess > +import tempfile > +from typing import Iterable, Iterator > + > +import exeter > + > +def sh(cmd): > + subprocess.run(cmd, shell=True) > + > + > +@contextlib.contextmanager > +def clone_sources() -> Iterator[str]: > + os.chdir('..') # Move from test/ to repo base > + with tempfile.TemporaryDirectory(ignore_cleanup_errors=False) as tmpdir: I guess I see the advantage of this syntax, it's still a bit less obvious to me than a mere sequence of steps and an explicit cleanup function. In any case, it's clear enough to copy and paste if needed, and if one has the possibility to define tests as you showed in 2/3 (even just as an emergency / quick experiment thing), I think it's absolutely fine. > + sh(f"cp --parents -d $(git ls-files) {tmpdir}") > + os.chdir(tmpdir) > + yield tmpdir > + > + > +def test_make(target: str, outputs: Iterable[str]) -> None: I probably mentioned this offline but it's hard to figure out that this produces a list of executable paths as output. A comment or a different argument name (binary_output_files?) might help. A simple list might be a bit more obvious than Iterable, I suppose. > + outpaths = [Path(o) for o in outputs] > + with clone_sources(): > + for o in outpaths: > + assert not o.exists(), f"{o} existed before make" > + sh(f'make {target} CFLAGS="-Werror"') > + for o in outpaths: > + assert o.exists(), f"{o} wasn't made" > + sh('make clean') > + for o in outpaths: > + assert not o.exists(), f"{o} existed after make clean" > + > + > +exeter.register('make_passt', test_make, 'passt', ['passt']) > +exeter.register('make_pasta', test_make, 'pasta', ['pasta']) > +exeter.register('make_qrap', test_make, 'qrap', ['qrap']) > +exeter.register('make_all', test_make, 'all', ['passt', 'pasta', 'qrap']) > + > + > +@exeter.test > +def test_install_uninstall() -> None: > + with clone_sources(): > + with tempfile.TemporaryDirectory(ignore_cleanup_errors=False) \ > + as prefix: > + bindir = Path(prefix) / 'bin' > + mandir = Path(prefix) / 'share/man' > + progs = ['passt', 'pasta', 'qrap'] > + > + # Install > + sh(f'make install CFLAGS="-Werror" prefix={prefix}') > + > + for prog in progs: > + exe = bindir / prog > + assert exe.is_file(), f"{exe} does not exist as a regular file" > + sh(f'man -M {mandir} -W {prog}') > + > + # Uninstall > + sh(f'make uninstall prefix={prefix}') > + > + for prog in progs: > + exe = bindir / prog > + assert not exe.exists(), f"{exe} exists after uninstall" > + sh(f'! man -M {mandir} -W {prog}') This looks almost as simple and obvious as the shell script equivalent, which is quite remarkable. > + > + > +if __name__ == '__main__': > + exeter.main() > diff --git a/test/run b/test/run > index 5fd02d91..6a53e24b 100755 > --- a/test/run > +++ b/test/run > @@ -43,6 +43,9 @@ KERNEL=${KERNEL:-"/boot/vmlinuz-$(uname -r)"} > > COMMIT="$(git log --oneline --no-decorate -1)" > > +# Let exeter tests written in Python find their modules > +export PYTHONPATH=${BASEPATH}/exeter/py3 > + > . lib/util > . lib/context > . lib/setup > @@ -69,12 +72,9 @@ run() { > [ ${CI} -eq 1 ] && video_start ci > > exeter smoke/smoke.sh > + exeter build/build.py > exeter build/static_checkers.sh > > - setup build > - test build/all > - teardown build > - > setup pasta > test pasta/ndp > test pasta/dhcp -- Stefano