Re: [PATCH v4 1/3] test: Extend test scripts to allow running exeter tests.

[Stefano Brivio <sbrivio@redhat.com>]

On Wed, 20 Aug 2025 21:10:17 +1000
David Gibson <david@gibson.dropbear.id.au> wrote:

> On Wed, Aug 20, 2025 at 11:52:18AM +0200, Stefano Brivio wrote:
> > On Wed, 20 Aug 2025 12:55:58 +1000
> > David Gibson <david@gibson.dropbear.id.au> wrote:
> >   
> > > On Tue, Aug 19, 2025 at 04:27:38PM +0200, Stefano Brivio wrote:  
> > > > On Thu,  7 Aug 2025 21:32:35 +1000
> > > > David Gibson <david@gibson.dropbear.id.au> wrote:
> > > >     
> > > > > Introduce some trivial testcases based on the exeter library.  These run
> > > > > passt and pasta with --help and --version options.  Extend our test
> > > > > scripts to run these tests.
> > > > > 
> > > > > Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> > > > > ---
> > > > >  test/.gitignore     |  1 +
> > > > >  test/Makefile       |  5 ++++-
> > > > >  test/lib/exeter     | 46 +++++++++++++++++++++++++++++++++++++++++++++
> > > > >  test/run            |  9 ++++++++-
> > > > >  test/smoke/smoke.sh | 27 ++++++++++++++++++++++++++
> > > > >  5 files changed, 86 insertions(+), 2 deletions(-)
> > > > >  create mode 100644 test/lib/exeter
> > > > >  create mode 100755 test/smoke/smoke.sh
> > > > > 
> > > > > diff --git a/test/.gitignore b/test/.gitignore
> > > > > index 3573444f..cf48b885 100644
> > > > > --- a/test/.gitignore
> > > > > +++ b/test/.gitignore
> > > > > @@ -11,3 +11,4 @@ nstool
> > > > >  rampstream
> > > > >  guest-key
> > > > >  guest-key.pub
> > > > > +/exeter/
> > > > > diff --git a/test/Makefile b/test/Makefile
> > > > > index bf63db87..332f3f3e 100644
> > > > > --- a/test/Makefile
> > > > > +++ b/test/Makefile
> > > > > @@ -50,7 +50,7 @@ UBUNTU_NEW_IMGS = xenial-server-cloudimg-powerpc-disk1.img \
> > > > >  	jammy-server-cloudimg-s390x.img
> > > > >  UBUNTU_IMGS = $(UBUNTU_OLD_IMGS) $(UBUNTU_NEW_IMGS)
> > > > >  
> > > > > -DOWNLOAD_ASSETS = mbuto podman \
> > > > > +DOWNLOAD_ASSETS = exeter mbuto podman \
> > > > >  	$(DEBIAN_IMGS) $(FEDORA_IMGS) $(OPENSUSE_IMGS) $(UBUNTU_IMGS)    
> > > > 
> > > > Pre-existing, but this broke a while after you switched the image
> > > > preparation to Makefile targets, so you can't really expect people to
> > > > issue 'make' at the moment.    
> > > 
> > > Sorry, I'm not quite sure what the problem is you're getting at.  
> > 
> > This:
> > --  
> 
> Hm, could you avoid using -- to mark quotes like this.  mutt mistakes
> it for the sig and cuts it out; it's kind of a pain to paste it back
> in for repying.

Oops, weird, because that was originally "--\n", without a space:

  https://en.wikipedia.org/wiki/Signature_block#Standard_delimiter

In your quote that has now three spaces. Is it a bug in mutt? I never
hit it though (but I almost exclusively use claws-mail).

Would "---\n" avoid that?

> > $ mkdir psst
> > $ cd psst
> > $ git clone git://passt.top/passt
> > [...]
> > $ cd passt/test/
> > $ make
> > [...]
> > wget -c -O debian-11-generic-ppc64el.qcow2 https://cloud.debian.org/images/cloud/bullseye/latest/debian-11-generic-ppc64el.qcow2
> > --2025-08-20 11:27:05--
> > https://cloud.debian.org/images/cloud/bullseye/latest/debian-11-generic-ppc64el.qcow2
> > Resolving cloud.debian.org (cloud.debian.org)... 2001:6b0:19::165,
> > 2001:6b0:19::173, 2001:6b0:19::163, ...
> > Connecting to cloud.debian.org
> > (cloud.debian.org)|2001:6b0:19::165|:443... connected.
> > HTTP request sent, awaiting response... 404 Not Found 2025-08-20 11:27:05 ERROR
> > 404: Not Found.  
> 
> > make: *** [Makefile:150: debian-11-generic-ppc64el.qcow2] Error 8  
> 
> Ah, right.  All the distro wgets are so slow from Australia that I've
> always avoided them by keeping the image files in place (or even
> copying the back from backups).  So, I didn't hit this.  Yeah.. we
> should do something about that.
> 
> > The (very) old logic would make the "distro" tests fail, but it's
> > been a while that one can't really run 'make' under test/ anymore
> > (unless you have everything already prepared, that is).  
> 
> > On one hand, we don't have a replacement for those tests, and, while
> > ugly and clumsy, they were actually useful as they would have told
> > us
> > in advance about stuff such as:  
> 
> >   https://bugs.passt.top/show_bug.cgi?id=121  
> 
> > ...and probably much more that we're ignoring now. I don't even test
> > routinely on big-endian, we just have build tests from
> > distributions.  That's bad, and it will be ages before we can
> > reasonably get a replacement for that.  
> 
> Yeah.  On the other hand, we could at least remove the links that are
> actually broken.  The idea of testing across distros seems to me more
> important than the exact set of distros/versions to check - that's
> relatively easily expanded if we have the structure.

...or find alternate links / similar versions. Older versions are
important, too. For that one, this looks like a stable link:

  https://cdimage.debian.org/cdimage/cloud/bullseye/20250703-2162/debian-11-generic-ppc64el-20250703-2162.qcow2

-- 
Stefano