From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: passt.top; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: passt.top; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=UufsCV1B; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by passt.top (Postfix) with ESMTPS id D85635A0279 for ; Wed, 20 Aug 2025 11:52:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1755683544; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2n81mw4PoCzjKCmSSiaXw15pPhHysMZvhUUbG/b9EZM=; b=UufsCV1BNin18JzbqZIDMgOu9Z0bCC8ERRoXgwiu/dQsgRPHLHIOdneQxLAoh3767eOWUy jvwaKfuN4p0aU0Au6v2sXq+n3G+wdQp1WPbkfbBWfU/q3JunOFPi95ATrM0tuKf3tUhje6 bDfG0S0v0ZK8vM2C7rUYZSlo4oggG9s= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-474-7UrbdqUBN_mo3pwtZqQ5dQ-1; Wed, 20 Aug 2025 05:52:23 -0400 X-MC-Unique: 7UrbdqUBN_mo3pwtZqQ5dQ-1 X-Mimecast-MFC-AGG-ID: 7UrbdqUBN_mo3pwtZqQ5dQ_1755683542 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-45a1b05d251so36219145e9.1 for ; Wed, 20 Aug 2025 02:52:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1755683542; x=1756288342; h=content-transfer-encoding:mime-version:organization:references :in-reply-to:message-id:subject:cc:to:from:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=2n81mw4PoCzjKCmSSiaXw15pPhHysMZvhUUbG/b9EZM=; b=wBvmLwQXfiX2zQnvMHuOVJdi35yyTjXkuhqgXsA949Ss3RTj6KAoIGr8pRS6abWvhH 68a5+Fvm7KgO2xiihE8RS5JRDP2NcqVGaNR/ki3oxwrtTf0WcIbjZ5n7up8Mu5/RXa18 fj7wfqM+/+OgOM92mve5EKpsUrluFBrfQYgFBtyELL9G7lmdoBT18Xgj70woNb4O6SfF Ism9h2u/dRvo9wPtrtxjYf9r7O+NB+hIQqsFyW/NbJBvPh0XYE2n3QJerINxf6k2kQSQ O4KFei5JYDBMPRNXQT+NkvGsNvhIydHpV5wzVZ4yfiLZHtSUruolVhW+UZEwUsoDfx2p 4GIg== X-Gm-Message-State: AOJu0Yw3HatGUtTAqYZmUFsv7TfHJ0YPSHCk907Xj6NR02ApPQrFUYoS 2ujZBCCMqvib4wrhjKqCKkHEm29FvfbzsC1rEcE4+jYTxSUn7J3RktOPoC/gHtWKjbrLi6BjH15 KYsfssnAbZMpTZBqNX1JPBrjrBtsU7r1cAChIC2fcygaAcfBwrS5N2Q== X-Gm-Gg: ASbGncvnOzUVbmrYDnahJmUxfpBtOhuk30RaCtiM0vRkMA5GtI30orjffsMeTQKB+ZA yR32YX32W3bsYIbeu2cjw6mnJKO0+kyD0s5n6pOQs1HAAqlnl0jyW0Jbf+hpn/PCvoV0+39KBPx 5qJHl/R7jT25c3w0ZmyYnMMNtRu1zTGNbBj2FgT5P/G9Xqi5HA/z88s7fFaKJWMIJTEjUKBa2oL pB43i+Zktcm8FiDf3wgVivbF7N/9IlEl00X0n4SAG8XgORxamevCOKzVINmGbEv5+3U1e06QPtS tA89DJ28Mg28CkaiQ3L/Btjoyc7T2jnn23De/po049vOcitshA8= X-Received: by 2002:a05:600c:4448:b0:43c:f0ae:da7 with SMTP id 5b1f17b1804b1-45b479ad3d3mr18207925e9.7.1755683541904; Wed, 20 Aug 2025 02:52:21 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHQF5cgOatVTTdbarbxS9O53W7KcUyTLZqbccxYZyLbJPcUy5kStTZCXgOAstiQz4Ex35fhzw== X-Received: by 2002:a05:600c:4448:b0:43c:f0ae:da7 with SMTP id 5b1f17b1804b1-45b479ad3d3mr18207555e9.7.1755683541222; Wed, 20 Aug 2025 02:52:21 -0700 (PDT) Received: from maya.myfinge.rs (ifcgrfdd.trafficplex.cloud. [2a10:fc81:a806:d6a9::1]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3c074d43bc6sm7004127f8f.25.2025.08.20.02.52.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Aug 2025 02:52:20 -0700 (PDT) Date: Wed, 20 Aug 2025 11:52:18 +0200 From: Stefano Brivio To: David Gibson Subject: Re: [PATCH v4 1/3] test: Extend test scripts to allow running exeter tests. Message-ID: <20250820115218.2cdab1ba@elisabeth> In-Reply-To: References: <20250807113237.548294-1-david@gibson.dropbear.id.au> <20250807113237.548294-2-david@gibson.dropbear.id.au> <20250819162738.36d35d52@elisabeth> Organization: Red Hat X-Mailer: Claws Mail 4.2.0 (GTK 3.24.49; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 0WU-xs_g_PMwr4Zu84kgLHRSz8wHav3IVamBLNir2C8_1755683542 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Message-ID-Hash: SDB5DXU3EM2E7FC6RNOT4F37DAVGW6AP X-Message-ID-Hash: SDB5DXU3EM2E7FC6RNOT4F37DAVGW6AP X-MailFrom: sbrivio@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: passt-dev@passt.top X-Mailman-Version: 3.3.8 Precedence: list List-Id: Development discussion and patches for passt Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Wed, 20 Aug 2025 12:55:58 +1000 David Gibson wrote: > On Tue, Aug 19, 2025 at 04:27:38PM +0200, Stefano Brivio wrote: > > On Thu, 7 Aug 2025 21:32:35 +1000 > > David Gibson wrote: > > > > > Introduce some trivial testcases based on the exeter library. These run > > > passt and pasta with --help and --version options. Extend our test > > > scripts to run these tests. > > > > > > Signed-off-by: David Gibson > > > --- > > > test/.gitignore | 1 + > > > test/Makefile | 5 ++++- > > > test/lib/exeter | 46 +++++++++++++++++++++++++++++++++++++++++++++ > > > test/run | 9 ++++++++- > > > test/smoke/smoke.sh | 27 ++++++++++++++++++++++++++ > > > 5 files changed, 86 insertions(+), 2 deletions(-) > > > create mode 100644 test/lib/exeter > > > create mode 100755 test/smoke/smoke.sh > > > > > > diff --git a/test/.gitignore b/test/.gitignore > > > index 3573444f..cf48b885 100644 > > > --- a/test/.gitignore > > > +++ b/test/.gitignore > > > @@ -11,3 +11,4 @@ nstool > > > rampstream > > > guest-key > > > guest-key.pub > > > +/exeter/ > > > diff --git a/test/Makefile b/test/Makefile > > > index bf63db87..332f3f3e 100644 > > > --- a/test/Makefile > > > +++ b/test/Makefile > > > @@ -50,7 +50,7 @@ UBUNTU_NEW_IMGS = xenial-server-cloudimg-powerpc-disk1.img \ > > > jammy-server-cloudimg-s390x.img > > > UBUNTU_IMGS = $(UBUNTU_OLD_IMGS) $(UBUNTU_NEW_IMGS) > > > > > > -DOWNLOAD_ASSETS = mbuto podman \ > > > +DOWNLOAD_ASSETS = exeter mbuto podman \ > > > $(DEBIAN_IMGS) $(FEDORA_IMGS) $(OPENSUSE_IMGS) $(UBUNTU_IMGS) > > > > Pre-existing, but this broke a while after you switched the image > > preparation to Makefile targets, so you can't really expect people to > > issue 'make' at the moment. > > Sorry, I'm not quite sure what the problem is you're getting at. This: -- $ mkdir psst $ cd psst $ git clone git://passt.top/passt [...] $ cd passt/test/ $ make [...] wget -c -O debian-11-generic-ppc64el.qcow2 https://cloud.debian.org/images/cloud/bullseye/latest/debian-11-generic-ppc64el.qcow2 --2025-08-20 11:27:05-- https://cloud.debian.org/images/cloud/bullseye/latest/debian-11-generic-ppc64el.qcow2 Resolving cloud.debian.org (cloud.debian.org)... 2001:6b0:19::165, 2001:6b0:19::173, 2001:6b0:19::163, ... Connecting to cloud.debian.org (cloud.debian.org)|2001:6b0:19::165|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2025-08-20 11:27:05 ERROR 404: Not Found. make: *** [Makefile:150: debian-11-generic-ppc64el.qcow2] Error 8 -- The (very) old logic would make the "distro" tests fail, but it's been a while that one can't really run 'make' under test/ anymore (unless you have everything already prepared, that is). On one hand, we don't have a replacement for those tests, and, while ugly and clumsy, they were actually useful as they would have told us in advance about stuff such as: https://bugs.passt.top/show_bug.cgi?id=121 ...and probably much more that we're ignoring now. I don't even test routinely on big-endian, we just have build tests from distributions. That's bad, and it will be ages before we can reasonably get a replacement for that. So it would be still valuable to get them working again, and it shouldn't be much effort. On the other hand... you can't do 'make' there, because of tests we don't even run. Maybe those failures should be warnings instead. If somebody finds a "moment" it would be nice to fix those though... > > I ran 'make exeter' to get further. It would be nice to update > > test/README.md (or fix the Makefile... or both). > > Hrm.. "make check" already depends on "make assets" which should (now) > include exeter. I'm not sure why that's not working for you. ...and not 'make assets', either. > > > TESTDATA_ASSETS = small.bin big.bin medium.bin \ > > > rampstream > > > @@ -70,6 +70,9 @@ assets: $(ASSETS) > > > pull-%: % > > > git -C $* pull > > > > > > +exeter: > > > + git clone https://gitlab.com/dgibson/exeter.git > > > + > > > mbuto: > > > git clone git://mbuto.sh/mbuto > > > > > > diff --git a/test/lib/exeter b/test/lib/exeter > > > new file mode 100644 > > > index 00000000..4f7bcff9 > > > --- /dev/null > > > +++ b/test/lib/exeter > > > @@ -0,0 +1,46 @@ > > > +#!/bin/sh > > > +# > > > +# SPDX-License-Identifier: GPL-2.0-or-later > > > +# > > > +# PASST - Plug A Simple Socket Transport > > > +# for qemu/UNIX domain socket mode > > > +# > > > +# PASTA - Pack A Subtle Tap Abstraction > > > +# for network namespace/tap device mode > > > +# > > > +# test/lib/exeter - Run exeter tests within the rest of passt's tests > > > +# > > > +# Copyright Red Hat > > > +# Author: David Gibson > > > + > > > +exeter() { > > > > It would be nice to have the usual arguments / function description > > here. > > Good point, added. > > > > + STATESETUP="${STATEBASE}/$1" > > > + mkdir -p "${STATESETUP}" > > > + > > > + context_setup_host host > > > + layout_host > > > > Indentation here was done with tabs, this adds spaces instead. > > Oops. For some reason I'd added emacs configuration to explicitly > *disable* tab indent mode for shell mode in the passt tree. Weird, > and now reversed. > > > > + > > > + cd test > > > + > > > + __ntests=$("$@" --list | wc -l) > > > + if [ $? != 0 ]; then > > > + info "Failed to get exeter manifest for $@" > > > + pause_continue \ > > > + "Press any key to pause test session" \ > > > + "Resuming in " \ > > > + "Paused, press any key to continue" \ > > > + 5 > > > + return > > > + fi > > > + > > > + status_file_start "Exeter tests: $*" ${__ntests} > > > + > > > + for __testid in $("$@" --list); do > > > + status_test_start "${__testid}" > > > + context_run host "$@" "${__testid}" && status_test_ok || status_test_fail > > > + done > > > + > > > + cd .. > > > + > > > + teardown_context_watch ${PANE_HOST} host > > > +} > > > diff --git a/test/run b/test/run > > > index f73c3119..9e183682 100755 > > > --- a/test/run > > > +++ b/test/run > > > @@ -53,6 +53,7 @@ COMMIT="$(git log --oneline --no-decorate -1)" > > > . lib/layout_ugly > > > . lib/test > > > . lib/video > > > +. lib/exeter > > > > > > # cleanup() - Remove temporary files > > > cleanup() { > > > @@ -67,6 +68,8 @@ run() { > > > perf_init > > > [ ${CI} -eq 1 ] && video_start ci > > > > > > + exeter smoke/smoke.sh > > > + > > > setup build > > > test build/all > > > test build/cppcheck > > > @@ -223,6 +226,10 @@ run_selected() { > > > > > > __setup= > > > for __test; do > > > + if "test/${__test}" --list; then > > > + exeter "${__test}" > > > + continue > > > + fi > > > # HACK: the migrate tests need the setup repeated for > > > # each test > > > if [ "${__test%%/*}" != "${__setup}" -o \ > > > @@ -234,7 +241,7 @@ run_selected() { > > > > > > test "${__test}" > > > done > > > - teardown "${__setup}" > > > + [ -n "${__setup}" ] && teardown "${__setup}" > > > > > > log "PASS: ${STATUS_PASS}, FAIL: ${STATUS_FAIL}, SKIPPED: ${STATUS_SKIPPED}" > > > > > > diff --git a/test/smoke/smoke.sh b/test/smoke/smoke.sh > > > new file mode 100755 > > > index 00000000..26a98d31 > > > --- /dev/null > > > +++ b/test/smoke/smoke.sh > > > @@ -0,0 +1,27 @@ > > > +#! /bin/sh > > > > ...this is desirable, I think (and, for consistency, I'd use "#!/bin/sh") > > as Bash is not universally available, but: > > So.. I believe "#! " (with the space) is actually the traditional / > correct version. I did once encounter a situation where it mattered, > although to be fair, that might have been with 25 year old SunOS or > something equally obscure. Nonetheless I've habitually included the > space ever since (see find-arm64-firmware.sh, prepare-distro-img.sh > and rapstream-check.sh). Oh, I didn't know. I thought it was just something that happened to work somehow, but I didn't realise about this: https://en.wikipedia.org/wiki/Shebang_(Unix)#Version_8_improved_shell_scripts and while this has some details too: https://www.in-ulm.de/~mascheck/various/shebang/#blankrequired ...sure, makes sense, let's keep those. -- Stefano