From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=K2uhhwaD; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by passt.top (Postfix) with ESMTPS id 0DFAB5A0279 for ; Wed, 20 Aug 2025 12:40:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1755686450; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LkQw1ihe1nt2O+wzW+FIC7XOE0UQkFR7SBjHCg/KkRc=; b=K2uhhwaD5k8vNqHL3RfNwk+EnfFeXQMlGKtu4gqgHFEU/nsHc934INbn4po1vp32iMpLJe +xQP2G5mRCBiZMrNvxatW2PfNqZJGGdblRfRX89tVd+6KSAmZhed8Sw6rcstsGCWYPZtXs WW1W4nKW3MGSIdAS9E9sL3Mi2+wDCN8= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-385-WUNssYg6PZKq3Q8U7LQdiA-1; Wed, 20 Aug 2025 06:40:49 -0400 X-MC-Unique: WUNssYg6PZKq3Q8U7LQdiA-1 X-Mimecast-MFC-AGG-ID: WUNssYg6PZKq3Q8U7LQdiA_1755686448 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-45a1b00352eso24678865e9.0 for ; Wed, 20 Aug 2025 03:40:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755686448; x=1756291248; h=content-transfer-encoding:mime-version:organization:references :in-reply-to:message-id:subject:cc:to:from:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=LkQw1ihe1nt2O+wzW+FIC7XOE0UQkFR7SBjHCg/KkRc=; b=Tm6L69A9dIUnW1+Bg5IWGTQUwmlxBIDVIqxQH9ZW9UsvvWsve94wuibtJvkaRbZYZN Hqq+IGsujTFjK+BlR4+y+j5i3SsUepsxMwLKLailJTQMowtuhjii1YZRqtTk9B3ANGFM bZV9diW4jlO8uJmuBwjqjCiXDlUzHUdRsiU6jM8CLF4sz5jTNUzEJYt/J7CHuVQe9Gr3 J8g872dAqyHtkQ5MzGGg2C42QdA/1QgT7YdGS1ybsNayrbPN5pDLdnQEKheMNREFcA5Z IXAbrJBcFGeb4e3VLAL//6JFHmDU1f94TUhqwrdV8p0iiutVUCWPtsfMmWUExVgRDAIH l1kg== X-Gm-Message-State: AOJu0Yycxweye3qWECKFUWcWfRXVHwKO601ATBvASdJg8WGdS1AsGAVa 9rEf8d0Fxjcpb8y4NUX6HKym5JJy7SYaUBMtWLq6cLUcgBAonlCITc1r2zVPwF7d6Z7iWp8MyxJ hU66Sd08B9n/p20imPG594YL7PZhU+8bRkwx2r6w6i15zI7SEwn8Shw== X-Gm-Gg: ASbGncuZLMrf/RZADjcDCjrCNmmvzySYfHDA6aLg/cGq/2e3dkvNpBL9MhF+82yrTql X5hwOu+5oZBiOfdl/fRPVHIfUuPTEG2sAOVR/4+WhC2U9BexcqXyCJL+PRexOJJF0L5K1+ti3D6 WPZrpDbynSvI9vzyYVpjELsoFYqeHtcvS66lj4NH7nMUlMinS8xwpVoK09ms3vvNaOxtTz4NBuo n7JCKk877oVX1EMNN9OlBtDgXdduPHheYM8f4Fldv94/Aty4fbodnBbFtjqDzpr9NZk0SS9mpkJ o4VlqtojE16E9MN/MQ74zQXTFW4s3CJs/Kty+4JW6x40p0zBpsQ= X-Received: by 2002:a05:600c:154d:b0:456:1e5a:885e with SMTP id 5b1f17b1804b1-45b4798c1bbmr17544755e9.3.1755686448260; Wed, 20 Aug 2025 03:40:48 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFBy86sFdFidQQv7YnjLPDdg14QEXV3s95nakz7C6+Hlm3x3at9WIWV2b0aKKxrpso2+vKvgQ== X-Received: by 2002:a05:600c:154d:b0:456:1e5a:885e with SMTP id 5b1f17b1804b1-45b4798c1bbmr17543585e9.3.1755686445984; Wed, 20 Aug 2025 03:40:45 -0700 (PDT) Received: from maya.myfinge.rs (ifcgrfdd.trafficplex.cloud. [2a10:fc81:a806:d6a9::1]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-45b47c29be6sm28605645e9.5.2025.08.20.03.40.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Aug 2025 03:40:45 -0700 (PDT) Date: Wed, 20 Aug 2025 12:40:44 +0200 From: Stefano Brivio To: David Gibson Subject: Re: [PATCH v4 3/3] test: Convert build tests to exeter Message-ID: <20250820124044.7f1bd32c@elisabeth> In-Reply-To: References: <20250807113237.548294-1-david@gibson.dropbear.id.au> <20250807113237.548294-4-david@gibson.dropbear.id.au> <20250819162754.0fdb4c86@elisabeth> Organization: Red Hat X-Mailer: Claws Mail 4.2.0 (GTK 3.24.49; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: BesbQT8earouq99INMIQ8bbVwEHWbKBXR6sYkl218mU_1755686448 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Message-ID-Hash: 6OVQAVWGUNHJ6V4CGA2JDUY5CZB2OGYS X-Message-ID-Hash: 6OVQAVWGUNHJ6V4CGA2JDUY5CZB2OGYS X-MailFrom: sbrivio@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Wed, 20 Aug 2025 13:10:02 +1000 David Gibson wrote: > On Tue, Aug 19, 2025 at 04:27:54PM +0200, Stefano Brivio wrote: > > On Thu, 7 Aug 2025 21:32:37 +1000 > > David Gibson wrote: > > > > > Convert the tests in build/all to be based on exeter. The new version of > > > the tests is more robust than the original, since it makes a temporary copy > > > of the source tree so will not be affected by concurrent manual builds. > > > > > > Signed-off-by: David Gibson > > > --- > > > test/build/all | 61 -------------------------------- > > > test/build/build.py | 84 +++++++++++++++++++++++++++++++++++++++++++++ > > > test/run | 8 ++--- > > > 3 files changed, 88 insertions(+), 65 deletions(-) > > > delete mode 100644 test/build/all > > > create mode 100755 test/build/build.py > > > > > > diff --git a/test/build/all b/test/build/all > > > deleted file mode 100644 > > > index 1f79e0d8..00000000 > > > --- a/test/build/all > > > +++ /dev/null > > > @@ -1,61 +0,0 @@ > > > -# SPDX-License-Identifier: GPL-2.0-or-later > > > -# > > > -# PASST - Plug A Simple Socket Transport > > > -# for qemu/UNIX domain socket mode > > > -# > > > -# PASTA - Pack A Subtle Tap Abstraction > > > -# for network namespace/tap device mode > > > -# > > > -# test/build/all - Build targets, one by one, then all together, check output > > > -# > > > -# Copyright (c) 2021 Red Hat GmbH > > > -# Author: Stefano Brivio > > > - > > > -htools make cc rm uname getconf mkdir cp rm man > > > - > > > -test Build passt > > > -host make clean > > > -check ! [ -e passt ] > > > -host CFLAGS="-Werror" make passt > > > -check [ -f passt ] > > > - > > > -test Build pasta > > > -host make clean > > > -check ! [ -e pasta ] > > > -host CFLAGS="-Werror" make pasta > > > -check [ -h pasta ] > > > - > > > -test Build qrap > > > -host make clean > > > -check ! [ -e qrap ] > > > -host CFLAGS="-Werror" make qrap > > > -check [ -f qrap ] > > > - > > > -test Build all > > > -host make clean > > > -check ! [ -e passt ] > > > -check ! [ -e pasta ] > > > -check ! [ -e qrap ] > > > -host CFLAGS="-Werror" make > > > -check [ -f passt ] > > > -check [ -h pasta ] > > > -check [ -f qrap ] > > > - > > > -test Install > > > -host mkdir __STATEDIR__/prefix > > > -host prefix=__STATEDIR__/prefix make install > > > -check [ -f __STATEDIR__/prefix/bin/passt ] > > > -check [ -h __STATEDIR__/prefix/bin/pasta ] > > > -check [ -f __STATEDIR__/prefix/bin/qrap ] > > > -check man -M __STATEDIR__/prefix/share/man -W passt > > > -check man -M __STATEDIR__/prefix/share/man -W pasta > > > -check man -M __STATEDIR__/prefix/share/man -W qrap > > > - > > > -test Uninstall > > > -host prefix=__STATEDIR__/prefix make uninstall > > > -check ! [ -f __STATEDIR__/prefix/bin/passt ] > > > -check ! [ -h __STATEDIR__/prefix/bin/pasta ] > > > -check ! [ -f __STATEDIR__/prefix/bin/qrap ] > > > -check ! man -M __STATEDIR__/prefix/share/man -W passt 2>/dev/null > > > -check ! man -M __STATEDIR__/prefix/share/man -W pasta 2>/dev/null > > > -check ! man -M __STATEDIR__/prefix/share/man -W qrap 2>/dev/null > > > diff --git a/test/build/build.py b/test/build/build.py > > > new file mode 100755 > > > index 00000000..12bb82d8 > > > --- /dev/null > > > +++ b/test/build/build.py > > > @@ -0,0 +1,84 @@ > > > +#! /usr/bin/env python3 > > > +# > > > +# SPDX-License-Identifier: GPL-2.0-or-later > > > +# > > > +# PASST - Plug A Simple Socket Transport > > > +# for qemu/UNIX domain socket mode > > > +# > > > +# PASTA - Pack A Subtle Tap Abstraction > > > +# for network namespace/tap device mode > > > +# > > > +# test/build/build.py - Test build and install targets > > > +# > > > +# Copyright Red Hat > > > +# Author: David Gibson > > > + > > > +import contextlib > > > +import os > > > +from pathlib import Path > > > +import subprocess > > > +import tempfile > > > +from typing import Iterable, Iterator > > > + > > > +import exeter > > > + > > > +def sh(cmd): > > > + subprocess.run(cmd, shell=True) > > > + > > > + > > > +@contextlib.contextmanager > > > +def clone_sources() -> Iterator[str]: > > > + os.chdir('..') # Move from test/ to repo base > > > + with tempfile.TemporaryDirectory(ignore_cleanup_errors=False) as tmpdir: > > > > I guess I see the advantage of this syntax, it's still a bit less > > obvious to me than a mere sequence of steps and an explicit cleanup > > function. > > I agree that it's less obvious, but I think the advantages are worth > it. Namely that it will correctly run the cleanup in nearly all cases > of an interrupted test (not a SIGKILL, obviously). > > Arguably, it's not a good trade off in this simple case. However once > we get to real network tests, it's pretty common to have multiple > nested layers of setup, each with their own teardown. Sometimes you > need something torn down, but it's only set up partway through the > test, or you want something for the first part of the test but not > after, so it needs to be torn down only if interrupted at certain > points. Manually keeping track of that quickly becomes really > painful, I really want to use this technique there. Another idea (I'm not sure if it makes this useless, but we probably want to implement it anyway) is what nft tests (and some kselftests) do nowadays: every test runs in its own network namespace, so you don't need an explicit teardown. The kernel already tracks things for you. If you combine that with what pasta does (same as container runtimes really), you could get something that's also robust to SIGKILL, by making every test "parent" process (assuming there can be one) PID 1 in its own PID namespace. Add mount namespaces on top, with tmpfs, and we probably don't even need to clean up after ourselves in build tests. Yes, it could be called a container but it's probably useful to retain tight control of what namespace we detach and when. Given that pasta(1) is now available on any distribution where you might reasonably find Python, by the way, you could consider using it directly (the installed version, that is) if it helps. > Given that I intend to use it there, I think it makes sense to use it > here too: after all it will be even less obvious if there are only > hard examples of it and no easy ones. Ah, sure, fair point. -- Stefano